Proxmox 5 and 6’s version of the OVMF firmware includes two commits (2ac1730 and 147fd35) that are intended to mark the pagetables as read-only during startup. This was first seen in Proxmox 5.1. This conflicts with the OsxAptioFixDrv drivers in Clover, which expect to be able to modify the pagetables to remap memory:
https://sourceforge.net/p/cloverefiboot/tickets/439/
I’ve patched OVMF to revert the effect of these two commits, which allows macOS to boot again (I also tested it by booting Windows 10, which worked fine). If you just want to download the fixed .deb, skip to the end of the article, otherwise if you want to build it yourself, follow along with the instructions in the next section:
Building the fixed pve-edk2-firmware
Proxmox 5
Use Debian 9 (Stretch) as a template to create a new container. Inside it, run:
# Add missing locale export LC_ALL=en_US.UTF-8 echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen locale-gen # Add tools and build dependencies: apt-get update apt-get install build-essential git lintian debhelper iasl nasm \ python uuid-dev gcc-aarch64-linux-gnu bc -y # Clone my patched source git clone -b macos-support-proxmox-5.6 https://github.com/thenickdude/pve-edk2-firmware.git # Build it cd pve-edk2-firmware make
This will include my patch, which you can see here as the newest commit on EDK2:
https://github.com/thenickdude/edk2/commits/macos-support-proxmox-5.6
Proxmox 6.1
Use Debian 10 (Buster) as a template to create a new container. Inside it, run:
# Add missing locale echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen locale-gen export LC_ALL=en_US.UTF-8 # Add tools and build dependencies: apt update apt install build-essential git lintian debhelper iasl nasm \ python uuid-dev gcc-aarch64-linux-gnu bc python3-distutils -y # Clone my patched source git clone -b macos-support-proxmox-6.1 https://github.com/thenickdude/pve-edk2-firmware.git # Build it cd pve-edk2-firmware make
You’ll end up with a built .deb file ready to be installed.
Installing the fixed OVMF
Download the prebuilt pve-edk2-firmware_1.20190312-1_all.deb (Proxmox 5) or pve-edk2-firmware_2.202002-1_all.deb (Proxmox 6.1) if you didn’t build it yourself in the previous step. Upload it to Proxmox somewhere and run these commands in that directory to install it:
dpkg -i pve-edk2-firmware_*.deb # Prevent it from being replaced later by apt upgrade: apt-mark hold pve-edk2-firmware
You’re done! All future VM launches will use this new OMVF build.
Thanks, it works great!
Hello,
I’m trying to install macOS High Sierra 10.13.4 with no success.
Without this ovmf version I could not get to installer.
Now, the installer gives me an error:
https://imgur.com/a/TEePNq2
and in the installer.log
https://imgur.com/a/ly2tona
* proxmox version:
pve-manager/5.1-52/ba597a64 (running kernel: 4.13.13-6-pve)
* config
args: -device isa-applesmc,osk=”…” -smbios type=2 -cpu Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on
balloon: 0
bios: ovmf
boot: cdn
bootdisk: ide2
cores: 4
cpu: Penryn
efidisk0: local-zfs:vm-102-disk-2,size=128K
ide0: shared-data:iso/HighSierra-10.13.4-mod.iso,cache=unsafe,media=cdrom,size=5600M
ide2: shared-data:iso/clover-r4428-amd64.iso,cache=unsafe
machine: pc-q35-2.11
memory: 8192
name: osx1
net0: e1000-82545em=C6:C4:F0:42:99:01,bridge=vmbr0
numa: 0
ostype: other
sata0: local-zfs:vm-102-disk-1,cache=unsafe,size=40G
scsihw: virtio-scsi-pci
smbios1: uuid=cbcf9d9a-c152-411c-8911-349969148059
sockets: 1
vga: std
Do you have any idea ?
Thank you.
I would guess you are not using the full ISO image, or its damaged. Nick’s other post on running High Sierra mentions that some Macs used to create the ISO don’t get the full installer. It also mentions how to work around it.
This is only a guess, but it’s based on your log saying “Image not found” when looking osinstallersetup.
I used this script to generate de iso:
https://github.com/kholia/OSX-KVM/blob/master/HighSierra/create_iso_highsierra.sh
The size of iso is 5600M, and the md5sum 0e8e53f29947c39211b6f07f97f01246.
It wasn’t the 16 MB installer.
Can you confirm the checksum of the iso ?
Thanks.
Excellent! Wasn’t able to make it work with the patched Qemu package but with this is works.
I passed a GTX 770 through to the VM but i’m still stuck with the 1280*1024 resolution, even after changing it in the GUI option in the config.plist and in the OVMF boot. And installed the latest Nvidia web drivers. Any solution for that?
Are the also some recommended UEFI drivers?
Hello,
I am following the guide but when I try to start my VM I get the error:
kvm: -machine type=pc-q35-2.11: unsupported machine type
I’m on Proxmox 5.1-41
There’s a new QEMU and pve-manager version available, you need to upgrade:
pve-manager 5.1-52
pve-qemu-kvm 2.11.1-5
qemu-server 5.0-25
If you already have, you may need to restart the host to start using the new QEMU version.
Hi nick! thank you for your packages.
I’m trying to figure out another way to do this without modify the pve-firmware package, maintaining original OVMF plus the modified one for mac…
Maybe we can change in QemuServer.pm this:
”
push @$cmd, ‘-drive’, “if=pflash,unit=0,format=raw,readonly,file=$OVMF_CODE”;
push @$cmd, ‘-drive’, “if=pflash,unit=1,format=$format,id=drive-efidisk0,file=$path”;
”
to add something like this: ( I don’t know perl syntax so…)
if $conf->{ostype} && $conf->{ostype} eq ‘other’; {
push @$cmd, ‘-drive’, “if=pflash,unit=0,format=raw,readonly,file=$OVMF_CODE_MAC”;
push @$cmd, ‘-drive’, “if=pflash,unit=1,format=$format,id=drive-efidisk0,file=$OVMF_VARS_MAC”;
} else {
*ORIGINAL LINE*
What do you think?
PS:Sorry for bad english, not main language
That’d work, but there doesn’t seem to be any disadvantage in using the modified OVMF for everything.
hi Zeusin,
I was trying a cleaner way : changing /etc/pve/qemu-server/ID.conf adding the line
args: -drive if=pflash,unit=0,format=raw,readonly,file=/DIR/OVMF_CODE.fd ….
but I get the error: “drive with bus=0, unit=0 (index=-1) exists”
did you find something better?
Hello Sir!
I am trying to set up macOS High Sierra on Proxmox 5.1 by following your article but i am stuck at the point
“Install Patched version of proxmox OVMF library”
you mentioned above that i have to upload that package file on proxmox somewhere.. but the problem is that i am not able to find any place in proxmox where i could upload a package file with extension “.deb”
kind guide me that how and where do i upload prebuilt pve-edk2-firmware on proxmox ?
You can upload it anywhere using SFTP, /root is a good choice.
i am fresh to proxmox will you please explain me how to do it?
Thanks
SSH/SFTP works the same way on Proxmox as on any other Linux system, I suggest that you google for tutorials. Without this knowledge, working with Proxmox in general is going to be very difficult.
Hi Nick,
Nice article! but i am running into a problem when booting up the installation. When i am booting the installation with verbose mode i am finding out it starts hanging at:
AppleKeyStore::start: _sep_enabled = 0
Screenshot of the boot log:
http://imgur.com/VFnR0bKl.png
I just can’t figure out how to fix this problem, anybody known an solution to this?
Kindly regards,
Pascal S.
Are we going to be blessed with an update on how to install Mac OS X Mojave with Proxmox?
I can supply the beta download if you need it 🙂
Hi, I followed your guide and downloaded clover iso; also I created high sierra iso and tried to boot with clover but it does not work. Always, boot in shell uefi.
Why?
Most likely your Boot Order setting on the Options tab is not set to the Clover device, or your Clover device is not configured correctly. Post your VM conf file on pastebin.com or something and link it here.
This is configuration file vm
https://pastebin.com/VhNDGMcB
Hi,
thanks for the tutorial! I can’t install mojave because Proxmox says “Guest has not initialized the display (yes) .”
Did anyone encounter this error before?
Dominik
Try starting the VM from the Proxmox command line (qm start xxx), it may print some errors/warnings that are missed in the GUI.
Hi,
There is a new version of this package avalaible on Proxmox’s repository.
pve-edk2-firmware/stable,stable 1.20181023-1
Can you please update your patched version ?
Thanks for the good work.
Sure, I’ve updated the package now and retested it with Mojave.
Hello Nicholas,
Downloaded and installed.
Tested as well with latest Mojave, no problem.
Thanks a lot !
Hi there,
There is (again) a new version of this package available on Proxmox’s repository.
pve-edk2-firmware/stable,stable 1.20190312-1
Can you please update your patched version ?
Thanks !
I’ve updated that now!
I upgraded my other Proxmox packages too and encountered a side-effect: VMs with passthrough video that had also set “vga: std” used to be equivalent to setting “vga: none” (the emulated adapter was disabled), but this now results in the emulated adapter being enabled as the primary video adapter, with the passthrough video card as secondary.
I had to edit my VM config to set “vga: none” for my VMs that use passthrough video.
It was super weird having Clover being invisible on boot, and then both of my two hardware monitors being relegated to “secondary displays”!
I personaly don’t use the video passthrough.
Thank you Nicholas !
Does the pre-built package need to be updated for Proxmox 5.4?
Nope, looks like pve-edk2-firmware package is still at 1.20190312-1 in Proxmox 5.4.
May I have your advice? I’ve got a multi-core CPU, lots of RAM, seven disks (2 NVMe, 3 SATA SSDs and 2 SATA hard drives) and one DVD device. I’ve got two identical Vega 56 PCIe GPUs and a CPU-internal GPU. At present, I can boot Mojave (from an M.2 NVMe), Windows 10 (from another M.2 NVMe) or Windows 8.1 (from a SATA SSD). I have an empty SATA SSD on which I plan to install Proxmox 6. A USB 2.0 Bluetooth dongle connects one keyboard/mouse pair, and a USB 2.0 PS/2-to-USB wire connects another keyboard/mouse pair. I’ll want to use passthrough for each of the PCIe GPUs (one Mojave, one Windows), for the HFS+ and NTFS disks as well as for the keyboards and mice.
First, what are your overall thoughts? What (re-)configuration do you think would be optimal for concurrently running Win10 and Mojave (or Win8.1 and Mojave)? Do you see some pitfalls I should watch out for?
Then: I’d planned to pass through 3 disks, a GPU and a kbd/mouse pair to Mojave and do the same with Windows. Then I read your description of the PCIe passthrough limitations. Hmm.
I’ll obviously need to use your PCIe passthrough fix.
Do you think I can simply point Proxmox 6 to my existing Mojave installation? Or must I install Mojave a second time for Proxmox?
Moving Mojave and Win10 to bootable SATA SSDs and then devoting the two M.2 NVMe disks to Proxmox (ZFS) is one option I’ve come up with. I’m mulling over some other vague ideas.
And: I greatly appreciate your documenting your experiences. Your guides and your descriptions have already saved me considerable time.
Thanks,
Barry
Note that for PCIe disk passthrough, you’d be talking about passing through either a PCIe NVMe disk, or a whole SATA controller (you can’t PCIe passthrough an individual SATA disk). You *can* give a SATA disk to a VM, it will just have to be presented using an emulated storage device in the guest.
Same for USB, you can give an individual USB device, but this requires an emulated USB controller to be presented to the guest, and this can have compatibility issues (mostly seen for high speed devices like VR headsets). The alternative is PCIe passthrough of an entire USB controller.
You should be able to boot your existing Mojave install using the Clover ISO provided here.
Vega currently has some PCIe reset bugs – once your VM shuts down your card will likely become unusable until the host is power cycled. This is being fixed right now: https://www.reddit.com/r/VFIO/comments/ckh9hp/reset_bug_fixed_for_vega_close_on_navi/
What’s the best way to find out when/where the AMD GPU reset bug fix is released?
Thanks,
Barry
Check in here from time to time, it’ll make headlines:
https://www.reddit.com/r/VFIO/
Thanks!
Barry
Yes, passthrough’s distinct IOMMU group requirement — I understand. I’ll re-route a couple of SATA cables to better organize the disks for passthrough:
1. First M.2 NVMe
2. Second M.2 NVMe
3. MB SATA controller – 2 SSDs and 1 HDD
4. PCIe SATA adapter card – 1 SSD, 1 HDD and the optical drive
Frankly, I want to pass through the disks to Mac OS (APFS) and to Windows (NTFS) because I’m afraid of storing data on either ZFS or EXT4. What if Proxmox I/O is slower than I’d like? What happens with my files if I decide to regress back to dual-boot Windows and Mac OS?
I guess I could try Proxmox out for a while and then decide. I shouldn’t be such an old stick-in-the-mud curmudgeon. (-:
Barry
If you pass your existing drives to your VMs then you should be able to go right back to dual booting without changing anything. In that case attach your Windows license to your Microsoft account so you don’t have troubles with activation. On your macOS VM keep on using the Clover ISO to boot rather than overwriting the existing install on your disk using dd (like my tutorial suggests you do at the end).
And thanks for the heads-up on the Vega GPU reset bugs …
Great tutorial. I am trying install Mojave on Proxmox 6. I am getting stuck with not being about to boot into the Mojave installer. I think it’s because either I didn’t get this patch uploaded correctly or the step to run “echo “options kvm ignore_msrs=Y” >> /etc/modprobe.d/kvm.conf && update-initramfs -k all -u” On this second step is where I get an error. This is what I get.
Running hook script ‘zz-pve-efiboot’..
Re-executing ‘/etc/kernel/postinst.d/zz-pve-efiboot’ in new private mount namespace..
No /etc/kernel/pve-efiboot-uuids found, skipping ESP sync.
So maybe I’m not actually making this change to impact any new VMs.. Any input you might have I would appreciate.
I think that warning from the second step is that Proxmox isn’t booting in UEFI mode with a ZFS root volume, so you’re not booting using systemd-boot, so it’s telling you that there was no systemd-boot ESP partition that needed syncing. In other words it’s harmless.
What command did you run to install the pve-edk2-firmware package?
Did you get the OSK written into your VM config correctly (no stray spaces or fancy curly quotes?)
I’ll check the config to make sure I got the OSK right.
As for the pve-edk2-firmware I downloaded the one that you had built for proxmox 6. I uploaded it to my proxmox and ran it. It appeared to take. Then ran that second line to make it permanent.
The OSK looks right. I ran that bit of c code. Any other troubleshooting ideas you have i would appreciate.
Please paste your VM config here
I cannot seem to install the patched firmware. When I try to run the dpkg I get the following error.
dpkg-deb: error: ‘pve-edk2-firmware_2.20190614-1_all.deb’ is not a Debian format archive
dpkg: error processing archive pve-edk2-firmware_2.20190614-1_all.deb (–install):
dpkg-deb –control subprocess returned error exit status 2
Errors were encountered while processing:
pve-edk2-firmware_2.20190614-1_all.deb
I have tried deleting and redownloading the patched file, but it won’t seem to take.
Please run these commands on the file and post the output:
ls -l pve-edk2-firmware_2.20190614-1_all.debfile pve-edk2-firmware_2.20190614-1_all.deb
root@**:~# ls -l pve-edk2-firmware_2.20190614-1_all.deb
-rw-r–r– 1 root root 629 Oct 19 14:33 pve-edk2-firmware_2.20190614-1_all.deb
root@**:~# file pve-edk2-firmware_2.20190614-1_all.deb
pve-edk2-firmware_2.20190614-1_all.deb: HTML document, ASCII text, with very long lines, with no line terminators
The file you downloaded is for sure an error message, not the package. Run cat on it and see what it says inside.
How are you downloading it?
I’ve used curl to download it directly to proxmox and also used scp on a windows machine to transfer the file to proxmox.
Edit: I just used cat and it’s downloading a redirect html file. I’ll try following the link and downloading the file again.
Here is the website it directs me to
https://github-production-release-asset-2e65be.s3.amazonaws.com/131541939/b0476e00-a252-11e9-8592-a9e87452db95?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20191019%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191019T193323Z&X-Amz-Expires=300&X-Amz-Signature=cff013ee0a7edab8c167e661b7596c41f7b231d3d56b0160dd943b95c8ecd8dc&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dpve-edk2-firmware_2.20190614-1_all.deb&response-content-type=application%2Foctet-stream
Great, download that.
I’m having trouble installing the patched firmware. When I try to run the dpkg I get the following error.
(Reading database … 43351 files and directories currently installed.)
Preparing to unpack pve-edk2-firmware_2.20190614-1_all.deb …
Unpacking pve-edk2-firmware (2.20190614-1) over (2.20190614-1) …
dpkg-deb (subprocess): cannot copy archive member from ‘pve-edk2-firmware_2.20190614-1_all.deb’ to decompressor pipe: unexpected end of file or stream
dpkg-deb (subprocess): decompressing archive member: lzma error: unexpected end of input
dpkg-deb: error: subprocess returned error exit status 2
dpkg: error processing archive pve-edk2-firmware_2.20190614-1_all.deb (–install):
cannot copy extracted data for ‘./usr/share/pve-edk2-firmware/AAVMF_CODE.fd’ to ‘/usr/share/pve-edk2-firmware/AAVMF_CODE.fd.dpkg-new’: unexpected end of file or stream
Errors were encountered while processing:
pve-edk2-firmware_2.20190614-1_all.deb
Any Ideas?
Your download got truncated. It should be 1683496 bytes long with an MD5 sum of e6453279cf4b532fc50335ffbc6c9077.
Thanks, got it working.
Hi Nicholas,
this came on the ≤make≥
Active Platform = /root/pve-edk2-firmware/pve-edk2-firmware-2.20191002/EmulatorPkg/EmulatorPkg.dsc
Processing meta-data …. done!
build.py…
: error C0DE: Unknown fatal error when processing [/root/pve-edk2-firmware/pve-edk2-firmware-2.20191002/MdePkg/Library/BaseLib/BaseLib.inf]
(Please send email to devel@edk2.groups.io for help, attaching following call stack trace!)
(Python 3.7.3 on linux) Traceback (most recent call last):
File “/root/pve-edk2-firmware/pve-edk2-firmware-2.20191002/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py”, line 2493, in Main
MyBuild.Launch()
File “/root/pve-edk2-firmware/pve-edk2-firmware-2.20191002/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py”, line 2287, in Launch
self._BuildPlatform()
File “/root/pve-edk2-firmware/pve-edk2-firmware-2.20191002/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py”, line 1742, in _BuildPlatform
self._BuildPa(self.Target, Pa, FfsCommand=CmdListDict,PcdMaList=PcdMaList)
File “/root/pve-edk2-firmware/pve-edk2-firmware-2.20191002/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py”, line 1236, in _BuildPa
AutoGenObject.CreateCodeFile(True)
File “/root/pve-edk2-firmware/pve-edk2-firmware-2.20191002/BaseTools/Source/Python/Common/caching.py”, line 18, in CallMeHere
Value = self._function(obj, *args,**kwargs)
File “/root/pve-edk2-firmware/pve-edk2-firmware-2.20191002/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py”, line 171, in CreateCodeFile
Ma.CreateCodeFile(CreateModuleCodeFile)
File “/root/pve-edk2-firmware/pve-edk2-firmware-2.20191002/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py”, line 1847, in CreateCodeFile
LibraryAutoGen.CreateCodeFile()
File “/root/pve-edk2-firmware/pve-edk2-firmware-2.20191002/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py”, line 1892, in CreateCodeFile
with GlobalData.cache_lock:
AttributeError: __enter__
– Failed –
Build end time: 11:05:24, Dec.19 2019
Build total time: 00:00:04
make[2]: *** [debian/rules:85: override_dh_auto_clean] Error 1
make[2]: Leaving directory ‘/root/pve-edk2-firmware/pve-edk2-firmware-2.20191002’
make[1]: *** [debian/rules:35: clean] Error 2
make[1]: Leaving directory ‘/root/pve-edk2-firmware/pve-edk2-firmware-2.20191002’
dpkg-buildpackage: error: debian/rules clean subprocess returned exit status 2
make: *** [Makefile:25: pve-edk2-firmware_2.20191002-1_all.deb] Error 2
Any Ideas ?
cheers 😉
Hm, you didn’t run out of RAM or disk space did you?
Hi,
thanks for this great tutorial.
It seems that pve-edk-firmware was updated to 2.20191127-1.
I tried creating the new firmware according to your tutorial but after cloning the git from master branch i was asked to fill in some credentials wihich i dont know.
Can you pleae update your tutorial concerning this aspect?
Regards
HW
The master branch is the unmodified upstream version from Proxmox, you need to use the branch I’ve created that corresponds to your version of Proxmox instead (the instructions include the checkout of that branch).
You can just run an older version of the pve-edk2-firmware package, I’ve never had any problems doing that.
Thanks for the tutorial. Mine crashes at start time with this error when following the instructions to the letter.
root@pve:~# qm start 116
kvm: -device ide-hd,bus=ide.1,unit=1,drive=drive-ide3,id=ide3,rotation_rate=1: Can’t create IDE unit 1, bus supports only 1 units
start failed: QEMU exited with code 1
ide0: gdata-vzdump:iso/OSX-Install-Catalina.iso,cache=unsafe,size=8500M
ide2: gdata-vzdump:iso/clover-r5070.iso,cache=unsafe
ide3: gdata-dsm:116/vm-116-disk-0.raw,cache=unsafe,discard=on,size=80G,ssd=1
You can’t have a drive on ide3, move that to sata0.
Hi Nick, thank you for your great tips and tutorials for ProxmoxVE an OSX
Have you noticed of this problem in saving a defined boot order if you do not have any qemu disk bur only disk passed via sata passthrough?
In my case i set my ssd with efi inside to first disk,but at dirst reboot it starts with ipx network..entering in bios and forcing to boot from ssd is working…
Hi Nick,
could you please also provide for 6.2? Thanks a lot.
Not yet because I haven’t updated to 6.2 myself. Consider upgrading to OpenCore instead of Clover, because it doesn’t require pve-edk2-firmware to be patched:
https://www.nicksherlock.com/2020/04/installing-macos-catalina-on-proxmox-with-opencore/
You can mount your EFI disk and just replace the EFI folder with the one from my OpenCore EFI folder zip.
Failing that, I’ve never had any trouble running old pve-edk2-firmware packages with newer QEMU, although the structure may have changed in 6.2.
Sorry for resurrecting this – how do you replace the EFI disk with the OpenCore version? Can’t for the life of me figure this out and trying to avoid a complete reinstall.
Disregard the last comment – figured it out.
The manual says “Upload it to Proxmox somewhere”.
My Proxmox 7 doesn’t accept .deb files. It accepts ISO files. I really don’t know where to upload. 🙁
Why didn’t he say more specifically where to upload .deb file? 🙁
This tutorial doesn’t apply to Proxmox 7
Is there any way to install High Sierra on Proxmox 7?
Yep, you can use my newer macOS guides and boot it using OpenCore rather than Clover. You need to use the network adapter from the High Sierra tutorial rather than the new vfio one though since High Sierra predates support for that.
Thanks! Can Mojave be instaled on Proxmox 7 at all?
Or should I install Proxmox 6 first instead?
Thanks!
I managed to install Mojave on Proxmox 7 making use of OpenCore instead of Clover. See “Installing macOS Catalina 10.15 on Proxmox 6.1 or 6.2 using OpenCore”