Task 22182515

Name	hadcm3s_1j05_200012_168_926_012129515_0
Workunit	12129515
Created	4 Jan 2022, 11:51:43 UTC
Sent	5 Jan 2022, 3:04:19 UTC
Report deadline	18 Dec 2022, 8:24:19 UTC
Received	30 Jan 2022, 5:39:53 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1503934
Run time	21 days 0 hours 25 min
CPU time	16 days 9 hours 24 min 49 sec
Validate state	Invalid
Credit	3,421.44
Device peak FLOPS	3.74 GFLOPS
Application version	UK Met Office HadCM3 short v8.36 i686-apple-darwin
Peak working set size	153.05 MB
Peak swap size	1,134.77 MB
Peak disk usage	56.10 MB
Stderr	<core_client_version>7.16.11</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234)</message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:10:28 (18228): BOINC client no longer exists - exiting 19:10:28 (18228): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:57:38 (11011): BOINC client no longer exists - exiting 21:57:38 (11011): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:14:13 (76379): BOINC client no longer exists - exiting 15:14:13 (76379): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:42:21 (98836): BOINC client no longer exists - exiting 11:42:21 (98836): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:15:53 (3767): BOINC client no longer exists - exiting 16:15:53 (3767): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:30:54 (98603): BOINC client no longer exists - exiting 16:30:54 (98603): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy OPEN: File Open Failed: Operation timed out OPEN: Unable to Open File datain/ancil/sulpc_oxidants_19_A2_1990 for Reading Model crashed: INANCLA: Error opening file tmp/pipe_dummy SIGSEGV: segmentation violation built using BOINC library version 7.7.0 Machine type Intel 80486 (32-bit executable) Sun Jan 30 02:30:11 2022 0 hadcm3s_um_8.36_i686-apple-darwin 0x00572b85 PrintBacktrace + 1009 1 hadcm3s_um_8.36_i686-apple-darwin 0x00566300 boinc_catch_signal + 241 2 libsystem_platform.dylib 0xa17f6e5b _sigtramp + 43 3 ??? 0xffffffff 0x0 + 4294967295 4 hadcm3s_um_8.36_i686-apple-darwin 0x0056b438 _ZN10XML_PARSER11parse_startEPKc + 56 5 hadcm3s_um_8.36_i686-apple-darwin 0x00560c08 _Z20parse_init_data_fileP7__sFILER13APP_INIT_DATA + 118 6 hadcm3s_um_8.36_i686-apple-darwin 0x0055dab0 boinc_parse_init_data_file + 147 7 hadcm3s_um_8.36_i686-apple-darwin 0x0055d534 _Z26boinc_init_options_generalR13BOINC_OPTIONS + 401 8 hadcm3s_um_8.36_i686-apple-darwin 0x0055d2d8 boinc_init_options + 102 9 hadcm3s_um_8.36_i686-apple-darwin 0x004a8d49 main + 1073 10 hadcm3s_um_8.36_i686-apple-darwin 0x000018fd _start + 212 11 hadcm3s_um_8.36_i686-apple-darwin 0x00001828 start + 40 Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc27c edx: 0xa171491e edi: 0xbfffc2e8 esi: 0x00000000 ebp: 0xbfffc2b8 esp: 0xbfffc27c ss: 0x00000023 efl: 0x00000296 eip: 0xa171491e cs: 0x0000000b ds: 0x00000023 es: 0x00000023 fs: 0x00000000 gs: 0x0000000f Binary Images Description: 0x1000 - 0x5eafff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3s_um_8.36_i686-apple-darwin 0xa0016000 - 0xa0017fff /usr/lib/libSystem.B.dylib 0xa01ad000 - 0xa0203fff /usr/lib/libc++.1.dylib 0xa0204000 - 0xa0227fff /usr/lib/libc++abi.dylib 0xa0d10000 - 0xa10d4fff /usr/lib/libobjc.A.dylib 0xa1375000 - 0xa13c5fff /usr/lib/libstdc++.6.dylib 0xa1534000 - 0xa1538fff /usr/lib/system/libcache.dylib 0xa1539000 - 0xa1543fff /usr/lib/system/libcommonCrypto.dylib 0xa1544000 - 0xa1549fff /usr/lib/system/libcompiler_rt.dylib 0xa154a000 - 0xa1552fff /usr/lib/system/libcopyfile.dylib 0xa1553000 - 0xa15b9fff /usr/lib/system/libcorecrypto.dylib 0xa15ba000 - 0xa15e7fff /usr/lib/system/libdispatch.dylib 0xa15e8000 - 0xa15edfff /usr/lib/system/libdyld.dylib 0xa15ee000 - 0xa15eefff /usr/lib/system/libkeymgr.dylib 0xa15fc000 - 0xa15fcfff /usr/lib/system/liblaunch.dylib 0xa15fd000 - 0xa1603fff /usr/lib/system/libmacho.dylib 0xa1604000 - 0xa1606fff /usr/lib/system/libquarantine.dylib 0xa1607000 - 0xa1608fff /usr/lib/system/libremovefile.dylib 0xa1609000 - 0xa1620fff /usr/lib/system/libsystem_asl.dylib 0xa1621000 - 0xa1621fff /usr/lib/system/libsystem_blocks.dylib 0xa1622000 - 0xa16b0fff /usr/lib/system/libsystem_c.dylib 0xa16b1000 - 0xa16b4fff /usr/lib/system/libsystem_configuration.dylib 0xa16b5000 - 0xa16b8fff /usr/lib/system/libsystem_coreservices.dylib 0xa16b9000 - 0xa16d1fff /usr/lib/system/libsystem_coretls.dylib 0xa16d2000 - 0xa16d8fff /usr/lib/system/libsystem_dnssd.dylib 0xa16d9000 - 0xa1700fff /usr/lib/system/libsystem_info.dylib 0xa1701000 - 0xa1722fff /usr/lib/system/libsystem_kernel.dylib 0xa1723000 - 0xa176ffff /usr/lib/system/libsystem_m.dylib 0xa1770000 - 0xa178afff /usr/lib/system/libsystem_malloc.dylib 0xa178b000 - 0xa17e1fff /usr/lib/system/libsystem_network.dylib 0xa17e2000 - 0xa17ebfff /usr/lib/system/libsystem_networkextension.dylib 0xa17ec000 - 0xa17f4fff /usr/lib/system/libsystem_notify.dylib 0xa17f5000 - 0xa17fbfff /usr/lib/system/libsystem_platform.dylib 0xa17fc000 - 0xa1805fff /usr/lib/system/libsystem_pthread.dylib 0xa1806000 - 0xa1809fff /usr/lib/system/libsystem_sandbox.dylib 0xa180a000 - 0xa180cfff /usr/lib/system/libsystem_secinit.dylib 0xa180d000 - 0xa1814fff /usr/lib/system/libsystem_symptoms.dylib 0xa1815000 - 0xa1827fff /usr/lib/system/libsystem_trace.dylib 0xa1828000 - 0xa182efff /usr/lib/system/libunwind.dylib 0xa182f000 - 0xa1855fff /usr/lib/system/libxpc.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=94571, iMonCtr=1 Model crash detected, will try to restart... OPEN: File Open Failed: Operation timed out OPEN: Unable to Open File datain/ancil/waterfix.ancil.be.32 for Reading Model crashed: INANCLO: Error opening file tmp/pipe_dummy OPEN: File Open Failed: Operation timed out OPEN: Unable to Open File datain/ancil/waterfix.ancil.be.32 for Reading Model crashed: INANCLO: Error opening file tmp/pipe_dummy OPEN: File Open Failed: Operation timed out OPEN: Unable to Open File datain/ancil/SO2DMS_cmip6h_2000_2014_n48 for Reading Model crashed: INANCLA: Error opening file tmp/pipe_dummy Sorry, too many model crashes! :-( 18:30:20 (94571): called boinc_finish(22) </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Jan 2022 22:05:14	1503934	22182515	hadcm3s_1j05_200012_168_926_012129515_0	285,120	1,641,883	5.7586
27 Jan 2022 22:06:56	1503934	22182515	hadcm3s_1j05_200012_168_926_012129515_0	259,200	1,482,633	5.7200
25 Jan 2022 18:01:39	1503934	22182515	hadcm3s_1j05_200012_168_926_012129515_0	233,280	1,323,212	5.6722
23 Jan 2022 11:16:10	1503934	22182515	hadcm3s_1j05_200012_168_926_012129515_0	207,360	1,164,052	5.6137
20 Jan 2022 14:42:47	1503934	22182515	hadcm3s_1j05_200012_168_926_012129515_0	181,440	1,008,437	5.5580
18 Jan 2022 13:01:51	1503934	22182515	hadcm3s_1j05_200012_168_926_012129515_0	155,520	860,510	5.5331
16 Jan 2022 06:10:56	1503934	22182515	hadcm3s_1j05_200012_168_926_012129515_0	129,600	701,725	5.4145
14 Jan 2022 04:46:08	1503934	22182515	hadcm3s_1j05_200012_168_926_012129515_0	103,680	561,250	5.4133
12 Jan 2022 01:27:41	1503934	22182515	hadcm3s_1j05_200012_168_926_012129515_0	77,760	415,791	5.3471
10 Jan 2022 00:38:17	1503934	22182515	hadcm3s_1j05_200012_168_926_012129515_0	51,840	273,766	5.2810
07 Jan 2022 22:06:10	1503934	22182515	hadcm3s_1j05_200012_168_926_012129515_0	25,920	136,407	5.2626