Task 15499838

Name	hadcm3n_3mx1_1940_40_008262485_1
Workunit	8417609
Created	22 Dec 2012, 22:22:57 UTC
Sent	22 Dec 2012, 23:15:55 UTC
Report deadline	24 Mar 2013, 6:43:06 UTC
Received	29 Jan 2013, 14:55:17 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1186987
Run time	5 days 15 hours 51 min 14 sec
CPU time	4 days 12 hours 6 min 11 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.13 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4672, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2176, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3300, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:37:15 (7340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:37:16 (7340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:12:54 (2208): No heartbeat from core client for 30 sec - exiting 08:12:55 (2208): No heartbeat from core client for 30 sec - exiting 08:12:56 (2208): No heartbeat from core client for 30 sec - exiting 08:12:57 (2208): No heartbeat from core client for 30 sec - exiting 08:12:58 (2208): No heartbeat from core client for 30 sec - exiting 08:13:00 (2208): No heartbeat from core client for 30 sec - exiting 08:13:01 (2208): No heartbeat from core client for 30 sec - exiting 08:13:02 (2208): No heartbeat from core client for 30 sec - exiting 08:13:03 (2208): No heartbeat from core client for 30 sec - exiting 08:13:04 (2208): No heartbeat from core client for 30 sec - exiting 08:13:05 (2208): No heartbeat from core client for 30 sec - exiting 08:13:06 (2208): No heartbeat from core client for 30 sec - exiting 08:13:07 (2208): No heartbeat from core client for 30 sec - exiting 08:13:08 (2208): No heartbeat from core client for 30 sec - exiting 08:13:09 (2208): No heartbeat from core client for 30 sec - exiting 08:13:10 (2208): No heartbeat from core client for 30 sec - exiting 08:13:12 (2208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5820, selfPID=5820, iMonCtr=1 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/3mx1ko.pje5c10 Error converting file to netcdf: dataout/3mx1ko.pie5c10 Error converting file to netcdf: dataout/3mx1ko.pfe5c10 Error converting file to netcdf: dataout/3mx1ka.phe5c10 CPDN Monitor - Quit request from BOINC... C05:48:26 (2984): No heartbeat from core client for 30 sec - exiting 05:48:27 (2984): No heartbeat from core client for 30 sec - exiting 05:48:28 (2984): No heartbeat from core client for 30 sec - exiting 05:48:29 (2984): No heartbeat from core client for 30 sec - exiting 05:48:30 (2984): No heartbeat from core client for 30 sec - exiting 05:48:31 (2984): No heartbeat from core client for 30 sec - exiting 05:48:32 (2984): No heartbeat from core client for 30 sec - exiting 05:48:33 (2984): No heartbeat from core client for 30 sec - exiting 05:48:34 (2984): No heartbeat from core client for 30 sec - exiting 05:48:36 (2984): No heartbeat from core client for 30 sec - exiting 05:48:37 (2984): No heartbeat from core client for 30 sec - exiting 05:48:38 (2984): No heartbeat from core client for 30 sec - exiting 05:48:39 (2984): No heartbeat from core client for 30 sec - exiting 05:48:40 (2984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:48:41 (2984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 23:57:01 (6732): No heartbeat from core client for 30 sec - exiting 23:57:02 (6732): No heartbeat from core client for 30 sec - exiting 23:57:03 (6732): No heartbeat from core client for 30 sec - exiting 23:57:04 (6732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:34:09 (3436): No heartbeat from core client for 30 sec - exiting 00:34:10 (3436): No heartbeat from core client for 30 sec - exiting 00:34:11 (3436): No heartbeat from core client for 30 sec - exiting 00:34:12 (3436): No heartbeat from core client for 30 sec - exiting 00:34:14 (3436): No heartbeat from core client for 30 sec - exiting 00:34:15 (3436): No heartbeat from core client for 30 sec - exiting 00:34:16 (3436): No heartbeat from core client for 30 sec - exiting 00:34:17 (3436): No heartbeat from core client for 30 sec - exiting 00:34:18 (3436): No heartbeat from core client for 30 sec - exiting 00:34:19 (3436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:34:20 (3436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6772, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5864, iMonCtr=1 Model crash detected, will try to restart... 02:13:54 (5284): No heartbeat from core client for 30 sec - exiting 02:13:55 (5284): No heartbeat from core client for 30 sec - exiting 02:13:57 (5284): No heartbeat from core client for 30 sec - exiting 02:13:58 (5284): No heartbeat from core client for 30 sec - exiting 02:13:59 (5284): No heartbeat from core client for 30 sec - exiting 02:14:00 (5284): No heartbeat from core client for 30 sec - exiting 02:14:01 (5284): No heartbeat from core client for 30 sec - exiting 02:14:02 (5284): No heartbeat from core client for 30 sec - exiting 02:14:03 (5284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x771EA5D5 read attempt to address 0x40293EDB Engaging BOINC Windows Runtime Debugger... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5844, selfPID=5844, iMonCtr=1 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x771E5EAB read attempt to address 0x40293EE3 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3mx1_1940_40_008262485/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Jan 2013 17:56:14	1186987	15499838	hadcm3n_3mx1_1940_40_008262485_1	259,200	380,767	1.4690
26 Jan 2013 06:59:36	1186987	15499838	hadcm3n_3mx1_1940_40_008262485_1	233,280	343,350	1.4718
24 Jan 2013 18:35:24	1186987	15499838	hadcm3n_3mx1_1940_40_008262485_1	207,360	306,433	1.4778
20 Jan 2013 20:06:35	1186987	15499838	hadcm3n_3mx1_1940_40_008262485_1	181,440	267,237	1.4729
20 Jan 2013 07:24:25	1186987	15499838	hadcm3n_3mx1_1940_40_008262485_1	155,520	229,540	1.4760
14 Jan 2013 05:43:02	1186987	15499838	hadcm3n_3mx1_1940_40_008262485_1	129,600	192,377	1.4844
13 Jan 2013 02:48:32	1186987	15499838	hadcm3n_3mx1_1940_40_008262485_1	103,680	154,275	1.4880
08 Jan 2013 06:46:35	1186987	15499838	hadcm3n_3mx1_1940_40_008262485_1	77,760	115,513	1.4855
31 Dec 2012 03:17:20	1186987	15499838	hadcm3n_3mx1_1940_40_008262485_1	51,840	77,285	1.4908
24 Dec 2012 07:02:28	1186987	15499838	hadcm3n_3mx1_1940_40_008262485_1	25,920	39,705	1.5318