Task 15671574

Name	hadcm3n_n4rk_1920_40_008321783_1
Workunit	8472918
Created	18 Mar 2013, 21:57:24 UTC
Sent	18 Mar 2013, 21:57:25 UTC
Report deadline	18 Jun 2013, 5:24:36 UTC
Received	16 Apr 2013, 5:58:01 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1271874
Run time	6 days 11 hours 49 min 33 sec
CPU time	5 days 22 hours 0 min 23 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.77 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:12:20 (6672): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:52:30 (7392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:52:34 (7392): No heartbeat from core client for 30 sec - exiting 17:52:35 (7392): No heartbeat from core client for 30 sec - exiting 17:52:36 (7392): No heartbeat from core client for 30 sec - exiting 17:52:38 (7392): No heartbeat from core client for 30 sec - exiting 17:52:39 (7392): No heartbeat from core client for 30 sec - exiting 17:52:40 (7392): No heartbeat from core client for 30 sec - exiting 17:52:41 (7392): No heartbeat from core client for 30 sec - exiting 17:52:42 (7392): No heartbeat from core client for 30 sec - exiting 17:52:43 (7392): No heartbeat from core client for 30 sec - exiting 17:52:44 (7392): No heartbeat from core client for 30 sec - exiting 17:52:45 (7392): No heartbeat from core client for 30 sec - exiting 17:52:46 (7392): No heartbeat from core client for 30 sec - exiting 17:52:47 (7392): No heartbeat from core client for 30 sec - exiting 17:52:48 (7392): No heartbeat from core client for 30 sec - exiting 17:52:49 (7392): No heartbeat from core client for 30 sec - exiting 17:52:50 (7392): No heartbeat from core client for 30 sec - exiting 17:55:50 (8324): No heartbeat from core client for 30 sec - exiting 17:55:51 (8324): No heartbeat from core client for 30 sec - exiting 17:55:52 (8324): No heartbeat from core client for 30 sec - exiting 17:55:53 (8324): No heartbeat from core client for 30 sec - exiting 17:55:54 (8324): No heartbeat from core client for 30 sec - exiting 17:55:55 (8324): No heartbeat from core client for 30 sec - exiting 17:55:56 (8324): No heartbeat from core client for 30 sec - exiting 17:55:57 (8324): No heartbeat from core client for 30 sec - exiting 17:57:38 (8324): No heartbeat from core client for 30 sec - exiting 17:57:42 (8324): No heartbeat from core client for 30 sec - exiting 17:57:43 (8324): No heartbeat from core client for 30 sec - exiting 17:57:44 (8324): No heartbeat from core client for 30 sec - exiting 17:57:45 (8324): No heartbeat from core client for 30 sec - exiting 17:57:46 (8324): No heartbeat from core client for 30 sec - exiting 17:57:47 (8324): No heartbeat from core client for 30 sec - exiting 17:57:48 (8324): No heartbeat from core client for 30 sec - exiting 17:57:50 (8324): No heartbeat from core client for 30 sec - exiting 17:57:51 (8324): No heartbeat from core client for 30 sec - exiting 17:57:52 (8324): No heartbeat from core client for 30 sec - exiting 17:57:53 (8324): No heartbeat from core client for 30 sec - exiting 17:57:54 (8324): No heartbeat from core client for 30 sec - exiting 17:57:55 (8324): No heartbeat from core client for 30 sec - exiting 17:57:56 (8324): No heartbeat from core client for 30 sec - exiting 17:57:57 (8324): No heartbeat from core client for 30 sec - exiting 17:57:58 (8324): No heartbeat from core client for 30 sec - exiting 17:57:59 (8324): No heartbeat from core client for 30 sec - exiting 17:58:00 (8324): No heartbeat from core client for 30 sec - exiting 17:58:02 (8324): No heartbeat from core client for 30 sec - exiting 17:58:03 (8324): No heartbeat from core client for 30 sec - exiting 17:58:04 (8324): No heartbeat from core client for 30 sec - exiting 17:58:05 (8324): No heartbeat from core client for 30 sec - exiting 17:58:06 (8324): No heartbeat from core client for 30 sec - exiting 17:58:08 (8324): No heartbeat from core client for 30 sec - exiting 17:58:09 (8324): No heartbeat from core client for 30 sec - exiting 17:58:10 (8324): No heartbeat from core client for 30 sec - exiting 17:58:11 (8324): No heartbeat from core client for 30 sec - exiting 17:58:12 (8324): No heartbeat from core client for 30 sec - exiting 17:58:13 (8324): No heartbeat from core client for 30 sec - exiting 17:58:14 (8324): No heartbeat from core client for 30 sec - exiting 17:58:15 (8324): No heartbeat from core client for 30 sec - exiting 17:58:16 (8324): No heartbeat from core client for 30 sec - exiting 17:58:17 (8324): No heartbeat from core client for 30 sec - exiting 17:58:18 (8324): No heartbeat from core client for 30 sec - exiting 17:58:19 (8324): No heartbeat from core client for 30 sec - exiting 17:58:20 (8324): No heartbeat from core client for 30 sec - exiting 17:58:21 (8324): No heartbeat from core client for 30 sec - exiting 17:58:22 (8324): No heartbeat from core client for 30 sec - exiting 17:58:23 (8324): No heartbeat from core client for 30 sec - exiting 17:58:24 (8324): No heartbeat from core client for 30 sec - exiting 17:58:25 (8324): No heartbeat from core client for 30 sec - exiting 17:58:26 (8324): No heartbeat from core client for 30 sec - exiting 17:58:27 (8324): No heartbeat from core client for 30 sec - exiting 17:58:28 (8324): No heartbeat from core client for 30 sec - exiting 17:58:29 (8324): No heartbeat from core client for 30 sec - exiting 17:58:30 (8324): No heartbeat from core client for 30 sec - exiting 17:58:31 (8324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:58:32 (8324): No heartbeat from core client for 30 sec - exiting 17:58:33 (8324): No heartbeat from core client for 30 sec - exiting 17:58:34 (8324): No heartbeat from core client for 30 sec - exiting 17:58:35 (8324): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3228, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3228, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3228, iMonCtr=1 Model crash detected, will try to restart... 18:01:54 (3228): No heartbeat from core client for 30 sec - exiting 18:01:55 (3228): No heartbeat from core client for 30 sec - exiting 18:01:56 (3228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77D83AB3 read attempt to address 0x40337033 Engaging BOINC Windows Runtime Debugger... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9600, selfPID=9600, iMonCtr=1 </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Apr 2013 18:51:35	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	777,600	499,887	0.6429
15 Apr 2013 15:03:57	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	751,680	482,990	0.6425
15 Apr 2013 15:03:57	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	725,760	466,077	0.6422
14 Apr 2013 13:08:04	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	699,840	448,758	0.6412
14 Apr 2013 07:26:09	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	673,920	430,765	0.6392
13 Apr 2013 17:23:44	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	648,000	412,650	0.6368
13 Apr 2013 11:24:21	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	622,080	395,759	0.6362
12 Apr 2013 20:58:44	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	596,160	379,607	0.6368
12 Apr 2013 16:16:04	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	570,240	363,413	0.6373
12 Apr 2013 11:45:24	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	544,320	347,436	0.6383
11 Apr 2013 21:14:47	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	518,400	330,970	0.6384
11 Apr 2013 08:08:49	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	492,480	314,633	0.6389
10 Apr 2013 11:59:18	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	466,560	298,275	0.6393
10 Apr 2013 07:07:39	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	440,640	281,984	0.6399
09 Apr 2013 20:12:08	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	414,720	265,586	0.6404
09 Apr 2013 20:12:08	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	388,800	248,902	0.6402
09 Apr 2013 08:44:57	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	362,880	232,120	0.6397
08 Apr 2013 16:00:50	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	336,960	215,462	0.6394
08 Apr 2013 11:12:59	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	311,040	198,796	0.6391
08 Apr 2013 06:31:16	1271874	15671574	hadcm3n_n4rk_1920_40_008321783_1	285,120	182,688	0.6407