Task 15701287

Name	hadcm3n_z81n_1920_40_008333846_1
Workunit	8484707
Created	3 Apr 2013, 21:33:12 UTC
Sent	3 Apr 2013, 21:33:23 UTC
Report deadline	4 Jul 2013, 5:00:34 UTC
Received	16 May 2013, 15:24:55 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1098598
Run time	7 days 3 hours 27 min 10 sec
CPU time	7 days 2 hours 53 min 16 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.30 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 10:04:52 (4760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5504, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:53:04 (3212): No heartbeat from core client for 30 sec - exiting 10:53:05 (3212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:17:35 (4080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... 09:09:13 (3860): No heartbeat from core client for 30 sec - exiting 09:09:15 (3860): No heartbeat from core client for 30 sec - exiting 09:09:16 (3860): No heartbeat from core client for 30 sec - exiting 09:09:17 (3860): No heartbeat from core client for 30 sec - exiting 09:09:18 (3860): No heartbeat from core client for 30 sec - exiting 09:09:19 (3860): No heartbeat from core client for 30 sec - exiting 09:09:20 (3860): No heartbeat from core client for 30 sec - exiting 09:09:21 (3860): No heartbeat from core client for 30 sec - exiting 09:09:22 (3860): No heartbeat from core client for 30 sec - exiting 09:09:23 (3860): No heartbeat from core client for 30 sec - exiting 09:09:24 (3860): No heartbeat from core client for 30 sec - exiting 09:09:25 (3860): No heartbeat from core client for 30 sec - exiting 09:09:27 (3860): No heartbeat from core client for 30 sec - exiting 09:09:28 (3860): No heartbeat from core client for 30 sec - exiting 09:09:29 (3860): No heartbeat from core client for 30 sec - exiting 09:09:30 (3860): No heartbeat from core client for 30 sec - exiting 09:09:31 (3860): No heartbeat from core client for 30 sec - exiting 09:09:32 (3860): No heartbeat from core client for 30 sec - exiting 09:09:33 (3860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:10:19 (5200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:39:49 (956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:25:54 (5668): No heartbeat from core client for 30 sec - exiting 10:25:55 (5668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:06:32 (6008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:10:15 (5296): No heartbeat from core client for 30 sec - exiting 07:10:16 (5296): No heartbeat from core client for 30 sec - exiting 07:10:17 (5296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:20:11 (4852): No heartbeat from core client for 30 sec - exiting 20:20:12 (4852): No heartbeat from core client for 30 sec - exiting 20:20:13 (4852): No heartbeat from core client for 30 sec - exiting 20:20:14 (4852): No heartbeat from core client for 30 sec - exiting 20:20:15 (4852): No heartbeat from core client for 30 sec - exiting 20:20:16 (4852): No heartbeat from core client for 30 sec - exiting 20:20:17 (4852): No heartbeat from core client for 30 sec - exiting 20:20:18 (4852): No heartbeat from core client for 30 sec - exiting 20:20:19 (4852): No heartbeat from core client for 30 sec - exiting 20:20:21 (4852): No heartbeat from core client for 30 sec - exiting 20:20:22 (4852): No heartbeat from core client for 30 sec - exiting 20:20:23 (4852): No heartbeat from core client for 30 sec - exiting 20:20:24 (4852): No heartbeat from core client for 30 sec - exiting 20:20:25 (4852): No heartbeat from core client for 30 sec - exiting 20:20:26 (4852): No heartbeat from core client for 30 sec - exiting 20:20:27 (4852): No heartbeat from core client for 30 sec - exiting 20:20:28 (4852): No heartbeat from core client for 30 sec - exiting 20:20:29 (4852): No heartbeat from core client for 30 sec - exiting 20:20:30 (4852): No heartbeat from core client for 30 sec - exiting 20:20:31 (4852): No heartbeat from core client for 30 sec - exiting 20:20:33 (4852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:28:48 (5048): No heartbeat from core client for 30 sec - exiting 10:28:49 (5048): No heartbeat from core client for 30 sec - exiting 10:28:50 (5048): No heartbeat from core client for 30 sec - exiting 10:28:51 (5048): No heartbeat from core client for 30 sec - exiting 10:28:52 (5048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7916, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2952, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:23:45 (4808): No heartbeat from core client for 30 sec - exiting 18:23:46 (4808): No heartbeat from core client for 30 sec - exiting 18:23:47 (4808): No heartbeat from core client for 30 sec - exiting 18:23:48 (4808): No heartbeat from core client for 30 sec - exiting 18:23:49 (4808): No heartbeat from core client for 30 sec - exiting 18:23:50 (4808): No heartbeat from core client for 30 sec - exiting 18:23:51 (4808): No heartbeat from core client for 30 sec - exiting 18:23:52 (4808): No heartbeat from core client for 30 sec - exiting 18:23:54 (4808): No heartbeat from core client for 30 sec - exiting 18:23:55 (4808): No heartbeat from core client for 30 sec - exiting 18:23:56 (4808): No heartbeat from core client for 30 sec - exiting 18:23:57 (4808): No heartbeat from core client for 30 sec - exiting 18:23:58 (4808): No heartbeat from core client for 30 sec - exiting 18:23:59 (4808): No heartbeat from core client for 30 sec - exiting 18:24:00 (4808): No heartbeat from core client for 30 sec - exiting 18:24:01 (4808): No heartbeat from core client for 30 sec - exiting 18:24:02 (4808): No heartbeat from core client for 30 sec - exiting 18:24:03 (4808): No heartbeat from core client for 30 sec - exiting 18:24:04 (4808): No heartbeat from core client for 30 sec - exiting 18:24:06 (4808): No heartbeat from core client for 30 sec - exiting 18:24:07 (4808): No heartbeat from core client for 30 sec - exiting 18:24:08 (4808): No heartbeat from core client for 30 sec - exiting 18:24:09 (4808): No heartbeat from core client for 30 sec - exiting 18:24:10 (4808): No heartbeat from core client for 30 sec - exiting 18:24:11 (4808): No heartbeat from core client for 30 sec - exiting 18:24:12 (4808): No heartbeat from core client for 30 sec - exiting 18:24:13 (4808): No heartbeat from core client for 30 sec - exiting 18:24:14 (4808): No heartbeat from core client for 30 sec - exiting 18:24:15 (4808): No heartbeat from core client for 30 sec - exiting 18:24:16 (4808): No heartbeat from core client for 30 sec - exiting 18:24:18 (4808): No heartbeat from core client for 30 sec - exiting 18:24:19 (4808): No heartbeat from core client for 30 sec - exiting 18:24:20 (4808): No heartbeat from core client for 30 sec - exiting 18:24:21 (4808): No heartbeat from core client for 30 sec - exiting 18:24:22 (4808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:24:23 (4808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 21:29:00 (5348): No heartbeat from core client for 30 sec - exiting 21:29:01 (5348): No heartbeat from core client for 30 sec - exiting 21:29:02 (5348): No heartbeat from core client for 30 sec - exiting 21:29:03 (5348): No heartbeat from core client for 30 sec - exiting 21:29:04 (5348): No heartbeat from core client for 30 sec - exiting 21:29:05 (5348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:22:59 (3772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:23:04 (2244): No heartbeat from core client for 30 sec - exiting 09:23:06 (2244): No heartbeat from core client for 30 sec - exiting 09:23:07 (2244): No heartbeat from core client for 30 sec - exiting 09:23:08 (2244): No heartbeat from core client for 30 sec - exiting 09:23:09 (2244): No heartbeat from core client for 30 sec - exiting 09:23:10 (2244): No heartbeat from core client for 30 sec - exiting 09:23:11 (2244): No heartbeat from core client for 30 sec - exiting 09:23:12 (2244): No heartbeat from core client for 30 sec - exiting 09:23:13 (2244): No heartbeat from core client for 30 sec - exiting 09:23:14 (2244): No heartbeat from core client for 30 sec - exiting 09:23:16 (2244): No heartbeat from core client for 30 sec - exiting 09:23:17 (2244): No heartbeat from core client for 30 sec - exiting 09:23:18 (2244): No heartbeat from core client for 30 sec - exiting 09:23:19 (2244): No heartbeat from core client for 30 sec - exiting 09:23:20 (2244): No heartbeat from core client for 30 sec - exiting 09:23:21 (2244): No heartbeat from core client for 30 sec - exiting 09:23:22 (2244): No heartbeat from core client for 30 sec - exiting 09:23:23 (2244): No heartbeat from core client for 30 sec - exiting 09:23:24 (2244): No heartbeat from core client for 30 sec - exiting 09:23:25 (2244): No heartbeat from core client for 30 sec - exiting 09:23:26 (2244): No heartbeat from core client for 30 sec - exiting 09:23:28 (2244): No heartbeat from core client for 30 sec - exiting 09:23:29 (2244): No heartbeat from core client for 30 sec - exiting 09:23:30 (2244): No heartbeat from core client for 30 sec - exiting 09:23:31 (2244): No heartbeat from core client for 30 sec - exiting 09:23:32 (2244): No heartbeat from core client for 30 sec - exiting 09:23:33 (2244): No heartbeat from core client for 30 sec - exiting 09:23:34 (2244): No heartbeat from core client for 30 sec - exiting 09:23:35 (2244): No heartbeat from core client for 30 sec - exiting 09:23:36 (2244): No heartbeat from core client for 30 sec - exiting 09:23:37 (2244): No heartbeat from core client for 30 sec - exiting 09:23:39 (2244): No heartbeat from core client for 30 sec - exiting 09:23:40 (2244): No heartbeat from core client for 30 sec - exiting 09:23:41 (2244): No heartbeat from core client for 30 sec - exiting 09:23:42 (2244): No heartbeat from core client for 30 sec - exiting 09:23:43 (2244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:26:38 (3848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:03:59 (5080): No heartbeat from core client for 30 sec - exiting 21:04:00 (5080): No heartbeat from core client for 30 sec - exiting 21:04:01 (5080): No heartbeat from core client for 30 sec - exiting 21:04:02 (5080): No heartbeat from core client for 30 sec - exiting 21:04:03 (5080): No heartbeat from core client for 30 sec - exiting 21:04:04 (5080): No heartbeat from core client for 30 sec - exiting 21:04:06 (5080): No heartbeat from core client for 30 sec - exiting 21:04:07 (5080): No heartbeat from core client for 30 sec - exiting 21:04:08 (5080): No heartbeat from core client for 30 sec - exiting 21:04:09 (5080): No heartbeat from core client for 30 sec - exiting 21:04:10 (5080): No heartbeat from core client for 30 sec - exiting 21:04:11 (5080): No heartbeat from core client for 30 sec - exiting 21:04:12 (5080): No heartbeat from core client for 30 sec - exiting 21:04:13 (5080): No heartbeat from core client for 30 sec - exiting 21:04:14 (5080): No heartbeat from core client for 30 sec - exiting 21:04:15 (5080): No heartbeat from core client for 30 sec - exiting 21:04:16 (5080): No heartbeat from core client for 30 sec - exiting 21:04:18 (5080): No heartbeat from core client for 30 sec - exiting 21:04:19 (5080): No heartbeat from core client for 30 sec - exiting 21:04:20 (5080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77823AB3 read attempt to address 0x40DD7CC3 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B47373 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Cannot serialize file D:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_z81n_1920_40_008333846/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 May 2013 08:32:15	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	518,400	597,317	1.1522
15 May 2013 06:29:45	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	492,480	566,820	1.1510
10 May 2013 22:42:24	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	466,560	536,525	1.1500
09 May 2013 00:23:42	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	440,640	506,805	1.1502
06 May 2013 21:07:53	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	414,720	477,202	1.1507
04 May 2013 16:46:13	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	388,800	448,188	1.1527
03 May 2013 18:07:33	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	362,880	418,125	1.1522
01 May 2013 18:08:13	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	336,960	388,434	1.1528
30 Apr 2013 18:25:59	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	311,040	358,615	1.1530
26 Apr 2013 15:16:26	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	285,120	328,601	1.1525
23 Apr 2013 19:11:45	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	259,200	298,847	1.1530
19 Apr 2013 22:50:26	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	233,280	269,006	1.1531
18 Apr 2013 16:05:53	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	207,360	238,989	1.1525
16 Apr 2013 19:14:58	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	181,440	208,878	1.1512
14 Apr 2013 18:53:43	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	155,520	179,385	1.1535
13 Apr 2013 18:11:06	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	129,600	149,763	1.1556
12 Apr 2013 13:30:02	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	103,680	119,576	1.1533
06 Apr 2013 22:29:18	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	77,760	90,055	1.1581
05 Apr 2013 18:43:30	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	51,840	59,945	1.1563
05 Apr 2013 10:25:27	1098598	15701287	hadcm3n_z81n_1920_40_008333846_1	25,920	30,427	1.1739