Task 15606236

Name	hadcm3n_3l8n_1940_40_008262581_1
Workunit	8417705
Created	12 Feb 2013, 17:34:35 UTC
Sent	12 Feb 2013, 17:34:48 UTC
Report deadline	15 May 2013, 1:01:59 UTC
Received	20 Feb 2013, 20:46:11 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1268502
Run time	4 days 21 hours 22 min 22 sec
CPU time	4 days 16 hours 12 min 16 sec
Validate state	Invalid
Credit	2,488.32
Device peak FLOPS	2.68 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:05:08 (3872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:13:34 (3256): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 01:13:35 (3256): No heartbeat from core client for 30 sec - exiting 01:13:36 (3256): No heartbeat from core client for 30 sec - exiting 01:13:37 (3256): No heartbeat from core client for 30 sec - exiting 03:06:16 (2056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:18:03 (528): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 20:59:26 (1812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:59:28 (1812): No heartbeat from core client for 30 sec - exiting 21:00:22 (2004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:02:23 (3860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:09:27 (580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:09:29 (580): No heartbeat from core client for 30 sec - exiting 21:10:56 (2996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:14:27 (3784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:15:32 (948): No heartbeat from core client for 30 sec - exiting 21:15:33 (948): No heartbeat from core client for 30 sec - exiting 21:15:34 (948): No heartbeat from core client for 30 sec - exiting 21:15:35 (948): No heartbeat from core client for 30 sec - exiting 21:15:36 (948): No heartbeat from core client for 30 sec - exiting 21:15:37 (948): No heartbeat from core client for 30 sec - exiting 21:15:38 (948): No heartbeat from core client for 30 sec - exiting 21:15:39 (948): No heartbeat from core client for 30 sec - exiting 21:15:40 (948): No heartbeat from core client for 30 sec - exiting 21:15:41 (948): No heartbeat from core client for 30 sec - exiting 21:15:42 (948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:15:43 (948): No heartbeat from core client for 30 sec - exiting 21:15:44 (948): No heartbeat from core client for 30 sec - exiting 21:15:45 (948): No heartbeat from core client for 30 sec - exiting 21:15:46 (948): No heartbeat from core client for 30 sec - exiting 21:19:28 (2952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:20:29 (2148): No heartbeat from core client for 30 sec - exiting 21:20:30 (2148): No heartbeat from core client for 30 sec - exiting 21:20:31 (2148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:20:32 (2148): No heartbeat from core client for 30 sec - exiting 21:24:28 (556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:24:30 (556): No heartbeat from core client for 30 sec - exiting 21:25:33 (520): No heartbeat from core client for 30 sec - exiting 21:25:34 (520): No heartbeat from core client for 30 sec - exiting 21:25:35 (520): No heartbeat from core client for 30 sec - exiting 21:25:36 (520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:25:37 (520): No heartbeat from core client for 30 sec - exiting 21:29:28 (3192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:29:30 (3192): No heartbeat from core client for 30 sec - exiting 21:30:41 (2380): No heartbeat from core client for 30 sec - exiting 21:30:42 (2380): No heartbeat from core client for 30 sec - exiting 21:30:43 (2380): No heartbeat from core client for 30 sec - exiting 21:30:44 (2380): No heartbeat from core client for 30 sec - exiting 21:30:45 (2380): No heartbeat from core client for 30 sec - exiting 21:30:46 (2380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:39:29 (2952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:39:31 (2952): No heartbeat from core client for 30 sec - exiting 21:39:32 (2952): No heartbeat from core client for 30 sec - exiting 21:40:39 (2412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:49:30 (932)Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1268, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1268, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1268, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1268, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=508, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3840, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2980, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Feb 2013 11:48:54	1268502	15606236	hadcm3n_3l8n_1940_40_008262581_1	207,360	363,191	1.7515
18 Feb 2013 12:09:01	1268502	15606236	hadcm3n_3l8n_1940_40_008262581_1	181,440	317,763	1.7513
17 Feb 2013 22:20:48	1268502	15606236	hadcm3n_3l8n_1940_40_008262581_1	155,520	270,877	1.7418
17 Feb 2013 06:21:30	1268502	15606236	hadcm3n_3l8n_1940_40_008262581_1	129,600	224,531	1.7325
16 Feb 2013 13:45:19	1268502	15606236	hadcm3n_3l8n_1940_40_008262581_1	103,680	179,307	1.7294
14 Feb 2013 10:46:08	1268502	15606236	hadcm3n_3l8n_1940_40_008262581_1	77,760	135,501	1.7426
13 Feb 2013 21:37:36	1268502	15606236	hadcm3n_3l8n_1940_40_008262581_1	51,840	90,680	1.7492
13 Feb 2013 07:11:02	1268502	15606236	hadcm3n_3l8n_1940_40_008262581_1	25,920	45,573	1.7582