climateprediction.net (CPDN) home page
Thread 'Running 32-bit MacOS Tasks on Linux with KVM'

Thread 'Running 32-bit MacOS Tasks on Linux with KVM'

Questions and Answers : Unix/Linux : Running 32-bit MacOS Tasks on Linux with KVM
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 · Next

AuthorMessage
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65151 - Posted: 13 Feb 2022, 17:12:25 UTC

My computers are looking like they're going to be more than a tiny bit bored here, with no Linux CPDN tasks around or on the horizon (except some resends), and with World Community Grid shutting down for a few months, they're going to be rather task starved. I might try some other projects for a bit, but, at least right now, there are plenty of CPDN tasks. Just, 32-bit MacOS only tasks. And I run 64-bit Linux.

Buuuut, virtualization is a thing - so, perhaps, we can solve this mismatch in what, if not the most efficient way, is at least better than idling - and it turns out, yes, you can do that!

https://github.com/kholia/OSX-KVM

This is a set of scripts to allow running Intel MacOS, with hardware virtualization, as a guest under Linux! And, at least so far, it seems to be working. I'm several hours into a few of the remaining Intel MacOS 32-bit tasks, and they're executing normally (not throwing bad CPU type errors as people have been seeing with 64-bit only versions of MacOS).

The last version of MacOS that supports 32-bit binaries is Mojave - so that's what I installed. This isn't a "howto" as much as a "If the steps make sense, you'll be able to follow through" guide, because it's sort of on the end of the bell curve of what's supported and isn't in the slightest bit sane, but:

- You'll need to use Disk Utility before you start the install to format the blank disk image attached, otherwise you won't see it as an install target. If you see a "locked" drive, that's the install media, and you can't install to it. Go format your actual target drive.
- You may need to change the network driver in the script from virtio to vmxnet3 - if your VM complains about not being able to contact the recovery server, it probably doesn't have a network connection. That change should get you online. Read the networking notes from the repo as well - I already have virt-manager installed where I run things, so I had the network bridge and such running. If you don't, you'll need to set that up as well.
- Once it reboots, you need to boot into the MacOS Installer again - this is the second stage of the install process, and should be a "progress bar" chewing along for some long while. If you end up back at the installer menu, you've selected the wrong boot option. Note that the mouse doesn't seem to work in the boot selector screen - use the keyboard.
- I installed BOINC 7.16.1 for Intel Macs, not the latest for Apple Silicon. I don't know if that one still supports 32-bit tasks or not, but 7.16.1 seems to work just fine.
- You'll probably want to add some RAM - edit the script and go. It looks like the default 3GB should work (the tasks are only using a few hundred meg each), but if you add more CPU cores (see also the notes to make sure it's a layout MacOS tolerates), you'll want to add some RAM.
- Disable power savings and screensaver. Qemu doesn't seem to tolerate the Mac trying to sleep, and it's not the slightest bit helpful for compute, so disable that, and get rid of the screensaver too - without GPU acceleration, that burns an awful lot of CPU.

There's also a headless script in there that might be useful if you're running this on a headless box - it just opens a vnc port for access. I haven't tried that yet, but probably will after my WCG tasks drain out after tomorrow. Seems an interesting thing to mess with.

I was getting about 1.4%/hr with a single task on my 8700K, and with four threads running, I'm getting about 1.08%/hr per task - I have four CPU cores assigned to the VM, as there's some other stuff going on with this system as well, and I didn't want to go bonkers before I verified that it worked. With tasks a few percent of the way through, it certainly seems to be working, though!

If I get super bored, I might even try it using nested virtualization with some cloud instances!
ID: 65151 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 65154 - Posted: 13 Feb 2022, 18:38:40 UTC - in response to Message 65151.  

Wow! Quite interesting. I'm not at my most of the year residence or I'd give it a try (limited computer resources here). Thanks.
ID: 65154 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65155 - Posted: 13 Feb 2022, 18:43:37 UTC - in response to Message 65154.  
Last modified: 13 Feb 2022, 19:41:42 UTC

Interesting indeed. However, I am almost certain it won't work with an AMD CPU.

Edit: I have now looked at the link. It certainly looks like it will install on my ryzen. Will over next week have a play and when my native Linux tasks have finished will have a go.
ID: 65155 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65156 - Posted: 13 Feb 2022, 20:15:27 UTC

qemu/kvm lie to the guest as needed about the actual nature of the host CPU. As long as it supports the needed extensions (of the SSE and AVX variety), it should work totally fine. A new Ryzen ought to support everything needed to do this, and I expect will rock those units right along! Looking at memory use as things run, I think 1GB per CPU core is probably sufficient - I'm under 3GB used with 4 tasks running, plus the whole OS. A 40GB root drive is probably fine too, my install is only using about 16GB total. The units seem fairly small in footprint, a couple days of compute per unit, just absolutely obscure in terms of what hardware currently supports them.
ID: 65156 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65165 - Posted: 15 Feb 2022, 20:22:58 UTC

I've got a set of tasks running on one of my servers, which has an AMD Ryzen 7 3700X - MacOS shows a "3.6GHz Unknown" processor, but the tasks run properly and are purring along nicely. So it shouldn't be an issue. I actually forgot it was an AMD box for a while... only just thought about it now.
ID: 65165 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 65166 - Posted: 15 Feb 2022, 21:53:16 UTC - in response to Message 65156.  

I think 1GB per CPU core is probably sufficient


I do know that on my Linux machine running N216 models, that they each take about 1.3GBytes of RAM to run, and near the end, they take 1.4 GBytes of RAM. This on my 64-bit Xeon processor. The numbers are the working set sizes; the maximum is slightly more.
ID: 65166 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65167 - Posted: 15 Feb 2022, 23:30:22 UTC

These aren't the N216 models, though. They're CM3 short.

Virtual memory size is about 950MB, but working set size is only around 150MB for each task. All the ones I have running right now are in the 145-146MB range.

Now I should see if I can light up a really big cloud instance to process a batch of these!
ID: 65167 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65168 - Posted: 16 Feb 2022, 4:45:52 UTC

Works on DigitalOcean too, though CPU performance is not amazing.
ID: 65168 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65169 - Posted: 16 Feb 2022, 15:13:44 UTC

Interesting. Copying OS installs around confuses BOINC, it's got some running tasks listed as "abandoned," looks like it's because I did a VM install, and then copied that install to another machine, so the same "machine" is running two sets of tasks - even though I started them separately on both machines. I've got 4 tasks listed as "abandoned" that are actually still running and sending trickles. I will fix this as soon as they're done and reinstall that VM from scratch.
ID: 65169 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65170 - Posted: 16 Feb 2022, 16:34:36 UTC

Seems to be taking a long time before the Catelina install finishes. Almost an hour and a half using a core and a bit so far and I am not on the slowest machine. Still, a few days to go before my native Linux tasks are all finished so no rush.
ID: 65170 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 65171 - Posted: 16 Feb 2022, 17:06:50 UTC - in response to Message 65170.  

Seems to be taking a long time before the Catelina install finishes. Almost an hour and a half using a core and a bit so far and I am not on the slowest machine. Still, a few days to go before my native Linux tasks are all finished so no rush.

Why Catalina? That's the one that removed 32bit. You want Mohave or High Sierra to run hadcm3s.
ID: 65171 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65172 - Posted: 16 Feb 2022, 17:30:25 UTC

Catalina won't work. Needs to be Mojave or earlier.

The install isn't particularly fast, and for some reason or another, the Mojave net-installer doesn't get you the latest version - it has a point release update. You can't run the system device updates, they fail - so just select the point release, if you care. Or ignore it, it's a compute VM.
ID: 65172 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65173 - Posted: 16 Feb 2022, 20:05:38 UTC - in response to Message 65172.  

Catalina won't work. Needs to be Mojave or earlier.

The install isn't particularly fast, and for some reason or another, the Mojave net-installer doesn't get you the latest version - it has a point release update. You can't run the system device updates, they fail - so just select the point release, if you care. Or ignore it, it's a compute VM.


Now trying Mojave!
ID: 65173 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65174 - Posted: 17 Feb 2022, 0:28:51 UTC

Progress rates as reported under Properties:

DigitalOcean 4 CPU VM: 0.720%/hr
i7-8700K, 4 threads: 1.080%/hr
i7-5775C, 4 threads: 1.080%/hr (this is an eDRAM chip, 128MB L4, so tends to do well on memory hard tasks)
Ryzen 7 3700X, 4 threads: 2.160%/hr

I think the takeaway here is kill the DigitalOcean VM when it's done, and let that Ryzen 7 rock away hard!
ID: 65174 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 65176 - Posted: 17 Feb 2022, 17:06:39 UTC

I've decided to give this a go. I've run lots of Linux VM guests on Windows hosts, but I've never set up guests on a Linux host. I've gotten up to the step where the GitHub guy said:

Create a virtual HDD image where macOS will be installed. If you change the name of the disk image from mac_hdd_ng.img to something else, the boot scripts will need to be updated to point to the new image name.

qemu-img create -f qcow2 mac_hdd_ng.img 128G

NOTE: Create this HDD image file on a fast SSD/NVMe disk for best results.


Is he creating that as his default user? When he did that, where did the HDD image go? Does it go to a default location? Or is he assuming you would modify that line to designate where on your system you want to put the image?

Where did you place the HDD image on your system?

And you said

- You'll need to use Disk Utility before you start the install to format the blank disk image attached, otherwise you won't see it as an install target. If you see a "locked" drive, that's the install media, and you can't install to it. Go format your actual target drive.


So after the command line step in the previous quote, that image will show up in a disk utility like "Disks" in Ubuntu?

Sorry for all the questions from a noob at creating VMs in Linux.
ID: 65176 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65177 - Posted: 17 Feb 2022, 18:41:27 UTC - in response to Message 65176.  

I've decided to give this a go. I've run lots of Linux VM guests on Windows hosts, but I've never set up guests on a Linux host. I've gotten up to the step where the GitHub guy said:
Where did you place the HDD image on your system?


It goes in the OSX-KVM directory you checked out, with the BaseSystem.dmg/img you downloaded/converted.

So after the command line step in the previous quote, that image will show up in a disk utility like "Disks" in Ubuntu?


No, that's in the OSX installer - there's a Disk Utility menu option you'll need to use to format the blank drive into something that the OSX installer will see and install to. Just erase the blank disk, there's a button for it. Sorry, I've installed OS X a ton over the years, so the instructions are a bit casual on that front.

Also, it's not worth trying this in nested virt on cloud providers. DigitalOcean and GCE are both pretty painful, around 0.7%/hr. I'll let those drain out and shut down, but not worth the effort to try on any cloud service.
ID: 65177 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 65179 - Posted: 18 Feb 2022, 3:06:45 UTC - in response to Message 65177.  

After a little confusion and stumbling around, Mohave is installed and updated, and boinc is installed, and I'm able to attach to projects. Thanks a lot SolarSyonyk for this interesting project.
ID: 65179 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65180 - Posted: 18 Feb 2022, 11:32:16 UTC - in response to Message 65179.  
Last modified: 18 Feb 2022, 14:05:02 UTC

After a little confusion and stumbling around, Mohave is installed and updated, and boinc is installed, and I'm able to attach to projects. Thanks a lot SolarSyonyk for this interesting project.


Still stumbling a bit here. starting again from
git clone --depth 1 --recursive https://github.com/kholia/OSX-KVM.git 


Still plenty of time to figure everything out before my native Linux tasks all finish.
Edit: Seems to get stuck with
xxx@xxx:~/OSX-KVM$ ./OpenCore-Boot.sh
QEMU 6.0.0 monitor - type 'help' for more information
(qemu) qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:EDX.nx [bit 20]
qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:EDX.nx [bit 20]
qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:EDX.nx [bit 20]
qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:EDX.nx [bit 20]
Uses just over one core, I see the quemu screen with the three icons but nothing ever changes.
ID: 65180 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65181 - Posted: 18 Feb 2022, 14:36:26 UTC - in response to Message 65180.  

Select the installer with the keyboard and hit enter - mouse doesn't work on that screen, and it doesn't auto-boot initially.
ID: 65181 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65182 - Posted: 18 Feb 2022, 16:40:00 UTC - in response to Message 65181.  

Select the installer with the keyboard and hit enter - mouse doesn't work on that screen, and it doesn't auto-boot initially.


Something flashes up on the screen too fast for me to read and that screen with the three icons is back but nothing changes. I suspect the answer to what I am doing wrong will simple but only once I know it!
ID: 65182 · Report as offensive     Reply Quote
1 · 2 · 3 · 4 · Next

Questions and Answers : Unix/Linux : Running 32-bit MacOS Tasks on Linux with KVM

©2024 cpdn.org