Task 13020258

Name	hadcm3n_t12r_1940_40_007313966_1
Workunit	7511396
Created	28 Jun 2011, 13:11:10 UTC
Sent	28 Jun 2011, 13:19:45 UTC
Report deadline	27 Sep 2011, 20:46:56 UTC
Received	7 Nov 2011, 8:16:52 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1010448
Run time	25 days 21 hours 15 min 12 sec
CPU time	22 days 7 hours 21 min 58 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.29 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 13:20:46 (3784): No heartbeat from core client for 30 sec - exiting 13:20:47 (3784): No heartbeat from core client for 30 sec - exiting 13:20:48 (3784): No heartbeat from core client for 30 sec - exiting 13:20:49 (3784): No heartbeat from core client for 30 sec - exiting 13:20:50 (3784): No heartbeat from core client for 30 sec - exiting 13:20:51 (3784): No heartbeat from core client for 30 sec - exiting 13:20:52 (3784): No heartbeat from core client for 30 sec - exiting 13:20:53 (3784): No heartbeat from core client for 30 sec - exiting 13:20:54 (3784): No heartbeat from core client for 30 sec - exiting 13:20:55 (3784): No heartbeat from core client for 30 sec - exiting 13:20:56 (3784): No heartbeat from core client for 30 sec - exiting 13:20:57 (3784): No heartbeat from core client for 30 sec - exiting 13:20:58 (3784): No heartbeat from core client for 30 sec - exiting 13:20:59 (3784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:23:53 (4436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:01:07 (4028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3228, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3608, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2472, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4668, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4396, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3528, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4080, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B825C6 read attempt to address 0x40D2FE06 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76FF25C6 read attempt to address 0x40D2FE06 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_t12r_1940_40_007313966/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Nov 2011 12:01:39	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	1,036,800	1,926,589	1.8582
01 Nov 2011 22:40:40	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	1,010,880	1,879,598	1.8594
31 Oct 2011 19:48:14	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	984,960	1,832,694	1.8607
31 Oct 2011 19:42:51	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	959,040	1,786,856	1.8632
14 Oct 2011 14:55:15	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	933,120	1,738,306	1.8629
11 Oct 2011 10:56:24	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	907,200	1,689,525	1.8624
29 Sep 2011 11:43:20	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	881,280	1,640,891	1.8619
28 Sep 2011 07:13:54	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	855,360	1,591,730	1.8609
28 Sep 2011 07:13:53	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	829,440	1,543,840	1.8613
28 Sep 2011 07:13:53	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	803,520	1,494,656	1.8601
01 Aug 2011 17:57:03	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	777,600	1,446,295	1.8599
31 Jul 2011 08:59:36	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	751,680	1,397,473	1.8591
30 Jul 2011 05:29:25	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	725,760	1,347,907	1.8572
29 Jul 2011 11:24:21	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	699,840	1,298,379	1.8553
28 Jul 2011 09:44:19	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	673,920	1,248,542	1.8527
26 Jul 2011 22:40:48	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	648,000	1,198,622	1.8497
25 Jul 2011 23:21:42	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	622,080	1,147,805	1.8451
25 Jul 2011 22:48:09	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	596,160	1,098,595	1.8428
25 Jul 2011 22:01:50	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	570,240	1,048,171	1.8381
25 Jul 2011 20:50:56	1010448	13020258	hadcm3n_t12r_1940_40_007313966_1	544,320	999,032	1.8354