Task 15940425

Name	hadcm3n_n0nv_1960_40_008371779_1
Workunit	8522638
Created	26 Aug 2013, 4:37:30 UTC
Sent	26 Aug 2013, 4:37:35 UTC
Report deadline	25 Nov 2013, 12:04:46 UTC
Received	7 Sep 2013, 0:27:13 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1290294
Run time	11 days 8 hours 21 min 3 sec
CPU time	9 days 22 hours 41 min 32 sec
Validate state	Invalid
Credit	5,598.72
Device peak FLOPS	2.81 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 18:41:51 (5640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:58:10 (1092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:25:32 (3516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:25:33 (3516): No heartbeat from core client for 30 sec - exiting 19:25:34 (3516): No heartbeat from core client for 30 sec - exiting 19:25:35 (3516): No heartbeat from core client for 30 sec - exiting 19:25:36 (3516): No heartbeat from core client for 30 sec - exiting 19:25:37 (3516): No heartbeat from core client for 30 sec - exiting 19:25:38 (3516): No heartbeat from core client for 30 sec - exiting 19:25:39 (3516): No heartbeat from core client for 30 sec - exiting 19:25:40 (3516): No heartbeat from core client for 30 sec - exiting 19:25:41 (3516): No heartbeat from core client for 30 sec - exiting 19:25:42 (3516): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9348, iMonCtr=1 Model crash detected, will try to restart... 19:36:44 (9348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:36:45 (9348): No heartbeat from core client for 30 sec - exiting 19:36:46 (9348): No heartbeat from core client for 30 sec - exiting 19:36:47 (9348): No heartbeat from core client for 30 sec - exiting 19:36:48 (9348): No heartbeat from core client for 30 sec - exiting 19:36:49 (9348): No heartbeat from core client for 30 sec - exiting 19:36:50 (9348): No heartbeat from core client for 30 sec - exiting 19:36:51 (9348): No heartbeat from core client for 30 sec - exiting 19:36:52 (9348): No heartbeat from core client for 30 sec - exiting 19:36:53 (9348): No heartbeat from core client for 30 sec - exiting 19:36:54 (9348): No heartbeat from core client for 30 sec - exiting 19:37:48 (10196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:00 (8132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:48:21 (8132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:20:07 (4368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: Access is denied. 07:48:21 (4816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:53:08 (4264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:15:45 (6992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5952, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5952, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5952, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5952, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5952, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5952, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
06 Sep 2013 09:22:18	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	466,560	814,936	1.7467
05 Sep 2013 18:11:48	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	440,640	768,038	1.7430
05 Sep 2013 02:26:40	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	414,720	720,559	1.7375
04 Sep 2013 11:27:49	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	388,800	674,110	1.7338
03 Sep 2013 20:05:34	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	362,880	627,262	1.7286
03 Sep 2013 04:08:31	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	336,960	581,233	1.7249
02 Sep 2013 11:20:12	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	311,040	535,135	1.7205
02 Sep 2013 00:30:45	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	285,120	491,443	1.7236
02 Sep 2013 00:30:45	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	259,200	454,297	1.7527
02 Sep 2013 00:30:45	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	233,280	411,826	1.7654
02 Sep 2013 00:30:45	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	207,360	366,200	1.7660
02 Sep 2013 00:30:45	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	181,440	319,813	1.7626
02 Sep 2013 00:30:45	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	155,520	273,684	1.7598
02 Sep 2013 00:30:45	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	129,600	227,838	1.7580
28 Aug 2013 23:07:11	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	103,680	182,247	1.7578
28 Aug 2013 08:03:03	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	77,760	136,221	1.7518
27 Aug 2013 17:43:43	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	51,840	90,652	1.7487
27 Aug 2013 03:59:40	1290294	15940425	hadcm3n_n0nv_1960_40_008371779_1	25,920	45,243	1.7455