Task 13551816

Name	hadcm3n_y7s2_1900_40_007523061_0
Workunit	7720536
Created	28 Oct 2011, 13:20:59 UTC
Sent	31 Oct 2011, 19:21:45 UTC
Report deadline	31 Jan 2012, 2:48:56 UTC
Received	27 Nov 2011, 22:20:31 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1158377
Run time	16 days 1 hours 26 min 57 sec
CPU time	15 days 0 hours 54 min 57 sec
Validate state	Invalid
Credit	11,819.52
Device peak FLOPS	2.79 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:04:20 (4384): No heartbeat from core client for 30 sec - exiting 20:04:21 (4384): No heartbeat from core client for 30 sec - exiting 20:04:22 (4384): No heartbeat from core client for 30 sec - exiting 20:04:23 (4384): No heartbeat from core client for 30 sec - exiting 20:04:24 (4384): No heartbeat from core client for 30 sec - exiting 20:04:25 (4384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:18:19 (3884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:46:15 (2244): No heartbeat from core client for 30 sec - exiting 19:46:16 (2244): No heartbeat from core client for 30 sec - exiting 19:46:17 (2244): No heartbeat from core client for 30 sec - exiting 19:46:18 (2244): No heartbeat from core client for 30 sec - exiting 19:46:19 (2244): No heartbeat from core client for 30 sec - exiting 19:46:20 (2244): No heartbeat from core client for 30 sec - exiting 19:46:21 (2244): No heartbeat from core client for 30 sec - exiting 19:46:22 (2244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:45:29 (4432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:50:28 (4888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:50:29 (4888): No heartbeat from core client for 30 sec - exiting 16:50:30 (4888): No heartbeat from core client for 30 sec - exiting 16:50:31 (4888): No heartbeat from core client for 30 sec - exiting 16:50:32 (4888): No heartbeat from core client for 30 sec - exiting 16:50:33 (4888): No heartbeat from core client for 30 sec - exiting 16:50:34 (4888): No heartbeat from core client for 30 sec - exiting 16:50:35 (4888): No heartbeat from core client for 30 sec - exiting 16:50:36 (4888): No heartbeat from core client for 30 sec - exiting 16:50:37 (4888): No heartbeat from core client for 30 sec - exiting 16:50:38 (4888): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4632, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4632, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4632, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4632, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4632, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4632, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Nov 2011 11:41:28	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	984,960	1,292,514	1.3123
27 Nov 2011 00:53:19	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	959,040	1,258,000	1.3117
26 Nov 2011 11:15:12	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	933,120	1,223,492	1.3112
25 Nov 2011 19:04:47	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	907,200	1,189,230	1.3109
25 Nov 2011 09:06:03	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	881,280	1,155,427	1.3111
24 Nov 2011 19:56:16	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	855,360	1,121,417	1.3110
24 Nov 2011 06:46:02	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	829,440	1,087,812	1.3115
23 Nov 2011 18:56:17	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	803,520	1,054,367	1.3122
23 Nov 2011 09:23:01	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	777,600	1,020,702	1.3126
22 Nov 2011 20:39:05	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	751,680	987,178	1.3133
22 Nov 2011 11:24:24	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	725,760	954,259	1.3148
22 Nov 2011 01:57:09	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	699,840	921,279	1.3164
21 Nov 2011 12:09:22	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	673,920	887,835	1.3174
20 Nov 2011 23:46:15	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	648,000	853,990	1.3179
20 Nov 2011 12:43:43	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	622,080	819,670	1.3176
20 Nov 2011 02:26:27	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	596,160	785,652	1.3179
18 Nov 2011 22:25:34	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	570,240	750,888	1.3168
18 Nov 2011 12:20:10	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	544,320	717,033	1.3173
18 Nov 2011 02:22:29	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	518,400	682,775	1.3171
17 Nov 2011 11:24:02	1158377	13551816	hadcm3n_y7s2_1900_40_007523061_0	492,480	648,790	1.3174