Task 17345410

Name	hadcm3n_88gm_1980_40_008720385_2
Workunit	8866363
Created	4 Nov 2014, 13:52:04 UTC
Sent	4 Nov 2014, 13:52:16 UTC
Report deadline	3 Feb 2015, 21:19:27 UTC
Received	23 Nov 2014, 7:27:31 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1243317
Run time	5 days 8 hours 13 min 10 sec
CPU time	4 days 8 hours 46 min 3 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.81 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:41:41 (18052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:11:13 (13880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:11:14 (13880): No heartbeat from core client for 30 sec - exiting 14:11:15 (13880): No heartbeat from core client for 30 sec - exiting 14:11:16 (13880): No heartbeat from core client for 30 sec - exiting 14:11:17 (13880): No heartbeat from core client for 30 sec - exiting 14:11:18 (13880): No heartbeat from core client for 30 sec - exiting 14:11:19 (13880): No heartbeat from core client for 30 sec - exiting 14:11:20 (13880): No heartbeat from core client for 30 sec - exiting 14:11:21 (13880): No heartbeat from core client for 30 sec - exiting 14:11:22 (13880): No heartbeat from core client for 30 sec - exiting 14:11:23 (13880): No heartbeat from core client for 30 sec - exiting 14:11:24 (13880): No heartbeat from core client for 30 sec - exiting 14:11:25 (13880): No heartbeat from core client for 30 sec - exiting 14:11:26 (13880): No heartbeat from core client for 30 sec - exiting 14:11:27 (13880): No heartbeat from core client for 30 sec - exiting 14:11:28 (13880): No heartbeat from core client for 30 sec - exiting 14:11:29 (13880): No heartbeat from core client for 30 sec - exiting 14:11:30 (13880): No heartbeat from core client for 30 sec - exiting 14:11:31 (13880): No heartbeat from core client for 30 sec - exiting 14:11:32 (13880): No heartbeat from core client for 30 sec - exiting 14:11:33 (13880): No heartbeat from core client for 30 sec - exiting 14:11:34 (13880): No heartbeat from core client for 30 sec - exiting 14:11:35 (13880): No heartbeat from core client for 30 sec - exiting 14:11:36 (13880): No heartbeat from core client for 30 sec - exiting 14:11:38 (13880): No heartbeat from core client for 30 sec - exiting 14:11:39 (13880): No heartbeat from core client for 30 sec - exiting 14:11:40 (13880): No heartbeat from core client for 30 sec - exiting 14:11:41 (13880): No heartbeat from core client for 30 sec - exiting 14:11:42 (13880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:45:06 (13812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:08:09 (7964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:46:43 (12792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:14:18 (13460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:21:35 (11632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:57:44 (10220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:15:10 (13180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:19:43 (12940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:10:42 (16676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=18192, iMonCtr=1 Model crash detected, will try to restart... 12:33:58 (7644): No heartbeat from core client for 30 sec - exiting 12:33:59 (7644): No heartbeat from core client for 30 sec - exiting 12:34:00 (7644): No heartbeat from core client for 30 sec - exiting 12:34:01 (7644): No heartbeat from core client for 30 sec - exiting 12:34:02 (7644): No heartbeat from core client for 30 sec - exiting 12:34:03 (7644): No heartbeat from core client for 30 sec - exiting 12:34:04 (7644): No heartbeat from core client for 30 sec - exiting 12:34:05 (7644): No heartbeat from core client for 30 sec - exiting 12:34:06 (7644): No heartbeat from core client for 30 sec - exiting 12:34:07 (7644): No heartbeat from core client for 30 sec - exiting 12:34:08 (7644): No heartbeat from core client for 30 sec - exiting 12:34:09 (7644): No heartbeat from core client for 30 sec - exiting 12:34:10 (7644): No heartbeat from core client for 30 sec - exiting 12:34:11 (7644): No heartbeat from core client for 30 sec - exiting 12:34:12 (7644): No heartbeat from core client for 30 sec - exiting 12:34:13 (7644): No heartbeat from core client for 30 sec - exiting 12:34:14 (7644): No heartbeat from core client for 30 sec - exiting 12:34:15 (7644): No heartbeat from core client for 30 sec - exiting 12:34:16 (7644): No heartbeat from core client for 30 sec - exiting 12:34:17 (7644): No heartbeat from core client for 30 sec - exiting 12:34:18 (7644): No heartbeat from core client for 30 sec - exiting 12:34:19 (7644): No heartbeat from core client for 30 sec - exiting 12:34:20 (7644): No heartbeat from core client for 30 sec - exiting 12:34:21 (7644): No heartbeat from core client for 30 sec - exiting 12:34:22 (7644): No heartbeat from core client for 30 sec - exiting 12:34:23 (7644): No heartbeat from core client for 30 sec - exiting 12:34:24 (7644): No heartbeat from core client for 30 sec - exiting 12:34:25 (7644): No heartbeat from core client for 30 sec - exiting 12:34:26 (7644): No heartbeat from core client for 30 sec - exiting 12:34:27 (7644): No heartbeat from core client for 30 sec - exiting 12:34:28 (7644): No heartbeat from core client for 30 sec - exiting 12:34:29 (7644): No heartbeat from core client for 30 sec - exiting 12:34:30 (7644): No heartbeat from core client for 30 sec - exiting 12:34:31 (7644): No heartbeat from core client for 30 sec - exiting 12:34:32 (7644): No heartbeat from core client for 30 sec - exiting 12:34:33 (7644): No heartbeat from core client for 30 sec - exiting 12:34:34 (7644): No heartbeat from core client for 30 sec - exiting 12:34:35 (7644): No heartbeat from core client for 30 sec - exiting 12:34:36 (7644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:34:37 (7644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:11:44 (8584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:29:45 (8040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:34:45 (10504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:11:02 (10208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:20:49 (5328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/88gmko.pji4c10 Error converting file to netcdf: dataout/88gmko.pii4c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:33:39 (15284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:57:06 (15376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:16:14 (16092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:17:07 (12240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:21:14 (16084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:31:01 (13780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:32:32 (14908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:44:21 (11132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:45:07 (8808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:41:19 (9564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:42:06 (7080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77AE3AC3 read attempt to address 0x40DCDE45 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77AE3AC3 read attempt to address 0x40DCDE45 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_88gm_1980_40_008720385/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Nov 2014 02:12:44	1243317	17345410	hadcm3n_88gm_1980_40_008720385_2	259,200	370,622	1.4299
22 Nov 2014 13:25:29	1243317	17345410	hadcm3n_88gm_1980_40_008720385_2	233,280	332,179	1.4239
21 Nov 2014 20:27:50	1243317	17345410	hadcm3n_88gm_1980_40_008720385_2	207,360	294,538	1.4204
21 Nov 2014 08:05:24	1243317	17345410	hadcm3n_88gm_1980_40_008720385_2	181,440	259,088	1.4280
20 Nov 2014 20:24:16	1243317	17345410	hadcm3n_88gm_1980_40_008720385_2	155,520	221,053	1.4214
19 Nov 2014 14:04:43	1243317	17345410	hadcm3n_88gm_1980_40_008720385_2	129,600	181,917	1.4037
16 Nov 2014 12:37:30	1243317	17345410	hadcm3n_88gm_1980_40_008720385_2	103,680	145,015	1.3987
13 Nov 2014 17:50:03	1243317	17345410	hadcm3n_88gm_1980_40_008720385_2	77,760	107,873	1.3873
11 Nov 2014 12:50:23	1243317	17345410	hadcm3n_88gm_1980_40_008720385_2	51,840	72,107	1.3910
07 Nov 2014 13:05:49	1243317	17345410	hadcm3n_88gm_1980_40_008720385_2	25,920	36,931	1.4248