climateprediction.net (CPDN) home page
Thread 'Running 32-bit MacOS Tasks on Linux with KVM'

Thread 'Running 32-bit MacOS Tasks on Linux with KVM'

Questions and Answers : Unix/Linux : Running 32-bit MacOS Tasks on Linux with KVM
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65218 - Posted: 27 Feb 2022, 20:34:42 UTC - in response to Message 65217.  

Well, that was discouraging. We had a power cut this morning. Bringing up my linux PC, then the MacOS VM, 3 of the 6 tasks that were running (and 13 model months along) immediately crashed.


Still not got it running here and being out of work both with CPDN and WCG have switched to FAH.
ID: 65218 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65219 - Posted: 28 Feb 2022, 14:53:15 UTC - in response to Message 65217.  

Sounds about right, they're touchy on restore, like most of the rest.

I've actually had better luck with just killing the VM entirely than trying to suspend tasks if I need to power cycle something - suspend/resume cycles are likely to kill 40-60% of the running tasks, and a power cut kills fewer. I'm trying not to do that, though - like all the CPDN tasks, they're best left running, non-stop, until completion. I have a bunch of solar powered systems that I just suspend at night (suspend the host, not the guest), and that works fine too - they've been CPDN crunch boxes for a while and are quite reliable in that way.
ID: 65219 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 65220 - Posted: 28 Feb 2022, 17:41:59 UTC - in response to Message 65219.  

Sounds about right, they're touchy on restore, like most of the rest.

I've actually had better luck with just killing the VM entirely than trying to suspend tasks if I need to power cycle something - suspend/resume cycles are likely to kill 40-60% of the running tasks, and a power cut kills fewer. I'm trying not to do that, though - like all the CPDN tasks, they're best left running, non-stop, until completion. I have a bunch of solar powered systems that I just suspend at night (suspend the host, not the guest), and that works fine too - they've been CPDN crunch boxes for a while and are quite reliable in that way.


If you do do the suspend tasks thing and then exit boinc before shutting the vm down, check the associated slots directories. If there are files listed with a "finished" in the filenames in those directories, delete those files. When the boinc client starts up and sees those files, it will think the task is over and error out. Usually it will have some error listed in stderr like "finish file present too long" and have an exit status of "194 (0x000000C2) EXIT_ABORTED_BY_CLIENT".
ID: 65220 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65221 - Posted: 28 Feb 2022, 18:08:53 UTC
Last modified: 28 Feb 2022, 18:09:50 UTC

The ones I had error were ones like this: https://www.cpdn.org/result.php?resultid=22194275

	
<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Signal 3 received, exiting...
22:46:16 (397): called boinc_finish(193)

</stderr_txt>
]]>


This was with "suspend, shutdown VM, reboot, resume." If I just kill stuff, it doesn't seem to fail, though I've been trying to avoid any interruptions to compute that aren't sleep/resume (which doesn't bother them).
ID: 65221 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65252 - Posted: 9 Mar 2022, 16:58:33 UTC

Well, my attempt to get it running on a 4440S I have laying around wouldn't get past the installer, so that system is still working Einstein units until something else comes along.

I did set the "days of work to acquire" values up on some of my boxes, since they're actually completing WUs instead of spitting them out every hour with the CPU type errors, so... hopefully that doesn't bother anyone too much. They'll get chewed eventually, and I've figured out a way to keep my solar boxes running at night with the solar trailer hookup - assuming I've had some decent sun during the day. Should be a bit more steady, on top of the grid tied stuff in the house. Going to tweak one to let it use full hyperthreading as well, should improve throughput at the cost of latency.
ID: 65252 · Report as offensive     Reply Quote
leloft

Send message
Joined: 7 Jun 17
Posts: 23
Credit: 44,434,789
RAC: 2,600,991
Message 65260 - Posted: 10 Mar 2022, 16:22:16 UTC

Hello. Strange events: I've just set up a qemu/kvm instance of mojave, installed boinc manager, attached to the project and apparabntly successfully downloaded files, (there are four marked as in progress (computer ID: 1528682)). However, Boinc manager is empty and holds the message 'No work available to process'. I am used to managing boinc through boinctui and I wondered it there was a terminal interface available in the mac version so I can see what's going on.

Many thanks

leloft
ID: 65260 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65261 - Posted: 10 Mar 2022, 16:28:26 UTC - in response to Message 65260.  
Last modified: 10 Mar 2022, 18:39:27 UTC

Hello. Strange events: I've just set up a qemu/kvm instance of mojave, installed boinc manager, attached to the project and apparabntly successfully downloaded files, (there are four marked as in progress (computer ID: 1528682)). However, Boinc manager is empty and holds the message 'No work available to process'. I am used to managing boinc through boinctui and I wondered it there was a terminal interface available in the mac version so I can see what's going on.

Many thanks

leloft

boinccmd should work the same as in the Linux version

Edit:There may or may not be permissions issues. I have never been close enough to a Mac for long enough but if there are any, anyone capable of getting the VM set up can probably navigate them.
ID: 65261 · Report as offensive     Reply Quote
wateroakley

Send message
Joined: 6 Aug 04
Posts: 195
Credit: 28,410,489
RAC: 10,300
Message 65273 - Posted: 13 Mar 2022, 16:13:13 UTC - in response to Message 65219.  

I've actually had better luck with just killing the VM entirely than trying to suspend tasks if I need to power cycle something - suspend/resume cycles are likely to kill 40-60% of the running tasks, and a power cut kills fewer. I'm trying not to do that, though - like all the CPDN tasks, they're best left running, non-stop, until completion.
Today, this Windows host (running the mac mojave VM) froze after excel threw a wobbly. Power cycling the host and restarting the mac VM brought back all four running tasks. That's 100% better result than carefully closing down BOINC and the VM.
ID: 65273 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65277 - Posted: 13 Mar 2022, 23:11:42 UTC

Yeah... there are an awful lot of "suspending" style errors in the workunits I've poked around too. This binary doesn't seem to like suspend/resume, though that's no different than the rest of the CPDN binaries. Best handled by dedicated systems that sleep instead of shut down if needed for power reasons.

Looks like the pool of WUs is drying up. I've got my machines stuffed to the gills with as many as they can fit (143 in progress at the moment on a total of 24 cores - I normally try not to hoard WUs, but my VMs are actually running them to completion and producing valid results, unlike quite a few of the machines chewing on them) and will let them run until their individual stockpiles are done, which should be in about a week and a half for the grid tied stuff and a bit longer for the solar stuff (I've got a power trailer right now that will run those machines overnight without too much trouble, so they're doing 24/7 work in my office if I have sufficient sun during the day to recharge the power trailer).

It's been an interesting little diversion, for sure! I'll just shut the VMs down and see if there's anything else they can be useful for later - they're all set up and updated, won't be hard to get them back online if anything else comes down the pipeline. Otherwise, it'll be... well, I'm not actually sure what I'll point the cores at now, with WCG down for maintenance and CPDN emptying out yet again.
ID: 65277 · Report as offensive     Reply Quote
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 31,092,036
RAC: 15,068
Message 65278 - Posted: 13 Mar 2022, 23:35:27 UTC - in response to Message 65277.  

"with WCG down for maintenance and CPDN emptying out yet again."

Milky Way is being flaky as well at the moment so I've added Einstein on my Linux box.
ID: 65278 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65279 - Posted: 13 Mar 2022, 23:38:25 UTC

Rosetta doesn't have anything good either, something was faulty about their tasks and they're empty on anything that's not "python projects," which I think are VirtualBox only. I might mess with those if I'm bored...

I'm more interested in the tasks that are solving practical, real world problems, as opposed to "Finding larger prime numbers" or "Doing some weird corner of math." I'm sure some people find that interesting, but I just can't get excited about them.
ID: 65279 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 65280 - Posted: 13 Mar 2022, 23:55:12 UTC - in response to Message 65279.  

I'm more interested in the tasks that are solving practical, real world problems, as opposed to "Finding larger prime numbers" or "Doing some weird corner of math." I'm sure some people find that interesting, but I just can't get excited about them.


Me too. seti@home is gone, ...

The only one I found that is possibly useful is
universe@home and it seems to always have work, when their server is not down. On the other hand, they hand out way to much credit for the small amount of work they do.

https://universeathome.pl/universe/home.php
ID: 65280 · Report as offensive     Reply Quote
leloft

Send message
Joined: 7 Jun 17
Posts: 23
Credit: 44,434,789
RAC: 2,600,991
Message 65297 - Posted: 16 Mar 2022, 9:02:17 UTC - in response to Message 65261.  

boinccmd should work the same as in the Linux version

Edit:There may or may not be permissions issues. I have never been close enough to a Mac for long enough but if there are any, anyone capable of getting the VM set up can probably navigate them.


Thanks for the edit. The 'mac' terminal doesn't seem to understand the command 'boinccmd' and I'm just using Boinc manager, but it's complaining that it needs to be reinstalled. I'm not satisfied that the VM route is anything other than a distraction (although an interesting one):, it's very volatile: the machine needed a kernel update and after the reboot, the VM dropped a couple of WU that had been downloaded but not started; they are no longer visible in Boinc manager. I'd like to finish the 8 WU under computation and then close the VM. So here's the questions: how in Boinc manager do I set the equivalent of 'No New Tasks' in boinctui ? and can I abort those two dropped WU from the cpdn site so that they get re-assigned promptly? Or, can anyone direct me to where they might be stored on the 'mac' and how to re-attach them in BM?

Many thanks

leloft
ID: 65297 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65298 - Posted: 16 Mar 2022, 9:58:05 UTC

So here's the questions: how in Boinc manager do I set the equivalent of 'No New Tasks' in boinctui ?


In Manager use advanced view then click on ,projects. tab. highlight the project you want to set to no new tasks and click that from the options on the left.
ID: 65298 · Report as offensive     Reply Quote
KAMasud

Send message
Joined: 6 Oct 06
Posts: 204
Credit: 7,608,986
RAC: 0
Message 65299 - Posted: 16 Mar 2022, 12:11:47 UTC - in response to Message 65279.  
Last modified: 16 Mar 2022, 12:17:07 UTC

Rosetta doesn't have anything good either, something was faulty about their tasks and they're empty on anything that's not "python projects," which I think are VirtualBox only. I might mess with those if I'm bored...

I'm more interested in the tasks that are solving practical, real world problems, as opposed to "Finding larger prime numbers" or "Doing some weird corner of math." I'm sure some people find that interesting, but I just can't get excited about them.


I too. If an Asteroid has to hit Earth, then it has to hit. Just now I am running zero CPU tasks.
All those WU's out there and only one seems to be answering back. I wish someone would re-issue them. If wishes were horses, then beggars would ride.
ID: 65299 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65301 - Posted: 16 Mar 2022, 16:09:16 UTC - in response to Message 65297.  

I'm not satisfied that the VM route is anything other than a distraction (although an interesting one):, it's very volatile: the machine needed a kernel update and after the reboot, the VM dropped a couple of WU that had been downloaded but not started; they are no longer visible in Boinc manager.


I'm not sure it's fair to call it a distraction, as there are fewer and fewer Intel Macs on Mojave or earlier out there. I've been able to chew quite a few of the 32-bit WUs on my VMs, though I admit I have no idea why some other systems refuse to run them - hasn't been a priority. One of my machines, a 4440S, won't finish the install, so I just keep it doing other things. But as far as I can tell, by the time they drain, my VMs will have successfully chewed ~270 CM3 tasks with only a few errors (a few aborted tasks getting things working when I had colliding machine IDs, a few suspend/resume issues, and then some of the "legitimate errors" where planetary conditions go nonsensical and the model quits).

I'd like to finish the 8 WU under computation and then close the VM. So here's the questions: how in Boinc manager do I set the equivalent of 'No New Tasks' in boinctui ?


In the "Project" tab, click on the CPDN project and there should be a button on the left, "No New Tasks." That should do what you need.

No idea about the missing WUs, I wouldn't worry about one or two weird ones given how many failed due to not being supported by the OS anymore.
ID: 65301 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 65302 - Posted: 16 Mar 2022, 16:16:13 UTC - in response to Message 65297.  

and can I abort those two dropped WU from the cpdn site so that they get re-assigned promptly?
leloft

Once you are done with the tasks you can see in the task list, if you detach the computer from cpdn, the tasks should show up as abandoned in your task list, and should be available to download to other PCs. That is if these tasks didn't have task names with an _2 at the end, which means that they were the last of the 3 tasks issued from a work unit.
ID: 65302 · Report as offensive     Reply Quote
leloft

Send message
Joined: 7 Jun 17
Posts: 23
Credit: 44,434,789
RAC: 2,600,991
Message 65303 - Posted: 16 Mar 2022, 21:56:45 UTC - in response to Message 65301.  

I'm not sure it's fair to call it a distraction,

Haha, my bad. The distraction is mine: I am not used to having to be in front of any one machine to manage boinc. The machines are spread over two locations 15km apart, two labs, three classrooms and two offices, all of which have their own timetables. I manage everything over ssh via boinctui from wherever I happen to be working. Having boinc running as a service means that workunits are likely to survive a reboot on a remote host, even if with a small loss of work since the last checkpoint. I guess I'll just wait for the next batch of WU, they've got to be coming soon!
ID: 65303 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65304 - Posted: 16 Mar 2022, 22:55:52 UTC

If that's your environment, then CPDN MacOS 32-bit tasks may not be for you. ;)

All my compute boxes are quite local, and I enjoy the obscure challenges in computing stuff like this poses.

Though I may just update one VM, archive it off, instead of keeping my whole mess of compute VMs around. As long as you let one update a few cycles before starting another one, or change the MAC address, they don't end up merging. Just don't start multiple VMs with identical MAC addresses at the same time. That does break things as they can't be distinguished from each other.
ID: 65304 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 65305 - Posted: 17 Mar 2022, 0:38:11 UTC - in response to Message 65299.  

Rosetta doesn't have anything good either, something was faulty about their tasks and they're empty on anything that's not "python projects," which I think are VirtualBox only. I might mess with those if I'm bored...


Rosetta now has a huge number of non-Python jobs available. All of them crash in under a minute. This is true no matter what version of Linux or Window you may be running.

I think that they have an administration-management-quality control problem. Sad that is when WCG is down and ClimatePrediction has no work except for a few MacIntosh ones.
ID: 65305 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · Next

Questions and Answers : Unix/Linux : Running 32-bit MacOS Tasks on Linux with KVM

©2024 cpdn.org