Task 13924403

Name	hadcm3n_y85x_1940_40_007682652_0
Workunit	7837739
Created	16 Jan 2012, 0:36:20 UTC
Sent	16 Jan 2012, 0:36:32 UTC
Report deadline	16 Apr 2012, 8:03:43 UTC
Received	28 Jan 2012, 17:19:53 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1079078
Run time	2 days 16 hours 14 min 1 sec
CPU time	2 days 13 hours 40 min 19 sec
Validate state	Invalid
Credit	1,244.16
Device peak FLOPS	2.29 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:56:21 (7152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:35:37 (6964): No heartbeat from core client for 30 sec - exiting 09:35:38 (6964): No heartbeat from core client for 30 sec - exiting 09:35:39 (6964): No heartbeat from core client for 30 sec - exiting 09:35:40 (6964): No heartbeat from core client for 30 sec - exiting 09:35:41 (6964): No heartbeat from core client for 30 sec - exiting 09:35:42 (6964): No heartbeat from core client for 30 sec - exiting 09:35:43 (6964): No heartbeat from core client for 30 sec - exiting 09:35:44 (6964): No heartbeat from core client for 30 sec - exiting 09:35:45 (6964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:57:40 (6268): No heartbeat from core client for 30 sec - exiting 08:57:42 (6268): No heartbeat from core client for 30 sec - exiting 08:57:43 (6268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:38:23 (5988): No heartbeat from core client for 30 sec - exiting 15:38:24 (5988): No heartbeat from core client for 30 sec - exiting 15:38:25 (5988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=468, iMonCtr=1 Model crash detected, will try to restart... 08:10:14 (6264): No heartbeat from core client for 30 sec - exiting 08:10:15 (6264): No heartbeat from core client for 30 sec - exiting 08:10:16 (6264): No heartbeat from core client for 30 sec - exiting 08:10:17 (6264): No heartbeat from core client for 30 sec - exiting 08:10:18 (6264): No heartbeat from core client for 30 sec - exiting 08:10:19 (6264): No heartbeat from core client for 30 sec - exiting 08:10:21 (6264): No heartbeat from core client for 30 sec - exiting 08:10:22 (6264): No heartbeat from core client for 30 sec - exiting 08:10:23 (6264): No heartbeat from core client for 30 sec - exiting 08:10:24 (6264): No heartbeat from core client for 30 sec - exiting 08:10:25 (6264): No heartbeat from core client for 30 sec - exiting 08:10:26 (6264): No heartbeat from core client for 30 sec - exiting 08:10:27 (6264): No heartbeat from core client for 30 sec - exiting 08:10:28 (6264): No heartbeat from core client for 30 sec - exiting 08:10:29 (6264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:51:56 (6600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:01:55 (3596): No heartbeat from core client for 30 sec - exiting 09:01:57 (3596): No heartbeat from core client for 30 sec - exiting 09:01:58 (3596): No heartbeat from core client for 30 sec - exiting 09:01:59 (3596): No heartbeat from core client for 30 sec - exiting 09:02:00 (3596): No heartbeat from core client for 30 sec - exiting 09:02:01 (3596): No heartbeat from core client for 30 sec - exiting 09:02:02 (3596): No heartbeat from core client for 30 sec - exiting 09:02:03 (3596): No heartbeat from core client for 30 sec - exiting 09:02:04 (3596): No heartbeat from core client for 30 sec - exiting 09:02:05 (3596): No heartbeat from core client for 30 sec - exiting 09:02:06 (3596): No heartbeat from core client for 30 sec - exiting 09:02:08 (3596): No heartbeat from core client for 30 sec - exiting 09:02:09 (3596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6904, iMonCtr=1 Model crash detected, will try to restart... 09:00:49 (5476): No heartbeat from core client for 30 sec - exiting 09:00:50 (5476): No heartbeat from core client for 30 sec - exiting 09:00:51 (5476): No heartbeat from core client for 30 sec - exiting 09:00:52 (5476): No heartbeat from core client for 30 sec - exiting 09:00:53 (5476): No heartbeat from core client for 30 sec - exiting 09:00:54 (5476): No heartbeat from core client for 30 sec - exiting 09:00:55 (5476): No heartbeat from core client for 30 sec - exiting 09:00:57 (5476): No heartbeat from core client for 30 sec - exiting 09:00:58 (5476): No heartbeat from core client for 30 sec - exiting 09:00:59 (5476): No heartbeat from core client for 30 sec - exiting 09:01:00 (5476): No heartbeat from core client for 30 sec - exiting 09:01:01 (5476): No heartbeat from core client for 30 sec - exiting 09:01:02 (5476): No heartbeat from core client for 30 sec - exiting 09:01:03 (5476): No heartbeat from core client for 30 sec - exiting 09:01:04 (5476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6892, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6892, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6892, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6892, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6892, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6892, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Jan 2012 01:10:10	1079078	13924403	hadcm3n_y85x_1940_40_007682652_0	103,680	198,751	1.9170
24 Jan 2012 03:08:01	1079078	13924403	hadcm3n_y85x_1940_40_007682652_0	77,760	149,605	1.9239
20 Jan 2012 18:40:17	1079078	13924403	hadcm3n_y85x_1940_40_007682652_0	51,840	99,431	1.9180
18 Jan 2012 00:22:11	1079078	13924403	hadcm3n_y85x_1940_40_007682652_0	25,920	50,713	1.9565