Task 15460677

Name	hadcm3n_zk9d_1880_40_008246589_3
Workunit	8401713
Created	25 Nov 2012, 13:42:35 UTC
Sent	25 Nov 2012, 13:42:39 UTC
Report deadline	24 Feb 2013, 21:09:50 UTC
Received	15 Dec 2012, 14:20:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1083664
Run time	4 days 0 hours 53 min 20 sec
CPU time	3 days 21 hours 20 min 50 sec
Validate state	Invalid
Credit	1,244.16
Device peak FLOPS	1.53 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:25:07 (28552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:49:10 (45292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:15:26 (66908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 04:09:49 (86636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:47:53 (121320): No heartbeat from core client for 30 sec - exiting 22:47:54 (121320): No heartbeat from core client for 30 sec - exiting 22:47:55 (121320): No heartbeat from core client for 30 sec - exiting 22:47:56 (121320): No heartbeat from core client for 30 sec - exiting 22:47:57 (121320): No heartbeat from core client for 30 sec - exiting 22:47:58 (121320): No heartbeat from core client for 30 sec - exiting 22:47:59 (121320): No heartbeat from core client for 30 sec - exiting 22:48:00 (121320): No heartbeat from core client for 30 sec - exiting 22:48:01 (121320): No heartbeat from core client for 30 sec - exiting 22:48:02 (121320): No heartbeat from core client for 30 sec - exiting 22:48:03 (121320): No heartbeat from core client for 30 sec - exiting 22:48:04 (121320): No heartbeat from core client for 30 sec - exiting 22:48:05 (121320): No heartbeat from core client for 30 sec - exiting 22:48:06 (121320): No heartbeat from core client for 30 sec - exiting 22:48:07 (121320): No heartbeat from core client for 30 sec - exiting 22:48:08 (121320): No heartbeat from core client for 30 sec - exiting 22:48:09 (121320): No heartbeat from core client for 30 sec - exiting 22:48:10 (121320): No heartbeat from core client for 30 sec - exiting 22:48:11 (121320): No heartbeat from core client for 30 sec - exiting 22:48:12 (121320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:49:44 (125824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:54:22 (138040): No heartbeat from core client for 30 sec - exiting 18:54:24 (138040): No heartbeat from core client for 30 sec - exiting 18:54:25 (138040): No heartbeat from core client for 30 sec - exiting 18:54:26 (138040): No heartbeat from core client for 30 sec - exiting 18:54:27 (138040): No heartbeat from core client for 30 sec - exiting 18:54:28 (138040): No heartbeat from core client for 30 sec - exiting 18:54:29 (138040): No heartbeat from core client for 30 sec - exiting 18:54:30 (138040): No heartbeat from core client for 30 sec - exiting 18:54:31 (138040): No heartbeat from core client for 30 sec - exiting 18:54:32 (138040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:56:17 (137656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:59:09 (182828): No heartbeat from core client for 30 sec - exiting 13:59:10 (182828): No heartbeat from core client for 30 sec - exiting 13:59:11 (182828): No heartbeat from core client for 30 sec - exiting 13:59:12 (182828): No heartbeat from core client for 30 sec - exiting 13:59:13 (182828): No heartbeat from core client for 30 sec - exiting 13:59:14 (182828): No heartbeat from core client for 30 sec - exiting 13:59:15 (182828): No heartbeat from core client for 30 sec - exiting 13:59:16 (182828): No heartbeat from core client for 30 sec - exiting 13:59:17 (182828): No heartbeat from core client for 30 sec - exiting 13:59:18 (182828): No heartbeat from core client for 30 sec - exiting 13:59:19 (182828): No heartbeat from core client for 30 sec - exiting 13:59:20 (182828): No heartbeat from core client for 30 sec - exiting 13:59:21 (182828): No heartbeat from core client for 30 sec - exiting 13:59:22 (182828): No heartbeat from core client for 30 sec - exiting 13:59:23 (182828): No heartbeat from core client for 30 sec - exiting 13:59:24 (182828): No heartbeat from core client for 30 sec - exiting 13:59:25 (182828): No heartbeat from core client for 30 sec - exiting 13:59:26 (182828): No heartbeat from core client for 30 sec - exiting 13:59:27 (182828): No heartbeat from core client for 30 sec - exiting 13:59:28 (182828): No heartbeat from core client for 30 sec - exiting 13:59:29 (182828): No heartbeat from core client for 30 sec - exiting 13:59:30 (182828): No heartbeat from core client for 30 sec - exiting 13:59:31 (182828): No heartbeat from core client for 30 sec - exiting 13:59:32 (182828): No heartbeat from core client for 30 sec - exiting 13:59:33 (182828): No heartbeat from core client for 30 sec - exiting 13:59:34 (182828): No heartbeat from core client for 30 sec - exiting 13:59:35 (182828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:01:29 (137120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:27:01 (215716): No heartbeat from core client for 30 sec - exiting 00:27:02 (215716): No heartbeat from core client for 30 sec - exiting 00:27:03 (215716): No heartbeat from core client for 30 sec - exiting 00:27:04 (215716): No heartbeat from core client for 30 sec - exiting 00:27:05 (215716): No heartbeat from core client for 30 sec - exiting 00:27:06 (215716): No heartbeat from core client for 30 sec - exiting 00:27:07 (215716): No heartbeat from core client for 30 sec - exiting 00:27:08 (215716): No heartbeat from core client for 30 sec - exiting 00:27:09 (215716): No heartbeat from core client for 30 sec - exiting 00:27:10 (215716): No heartbeat from core client for 30 sec - exiting 00:27:11 (215716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:28:47 (216208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:11:39 (229688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:18:42 (74940): No heartbeat from core client for 30 sec - exiting 13:18:43 (74940): No heartbeat from core client for 30 sec - exiting 13:18:44 (74940): No heartbeat from core client for 30 sec - exiting 13:18:45 (74940): No heartbeat from core client for 30 sec - exiting 13:18:46 (74940): No heartbeat from core client for 30 sec - exiting 13:18:47 (74940): No heartbeat from core client for 30 sec - exiting 13:18:48 (74940): No heartbeat from core client for 30 sec - exiting 13:18:49 (74940): No heartbeat from core client for 30 sec - exiting 13:18:50 (74940): No heartbeat from core client for 30 sec - exiting 13:18:51 (74940): No heartbeat from core client for 30 sec - exiting 13:18:52 (74940): No heartbeat from core client for 30 sec - exiting 13:18:53 (74940): No heartbeat from core client for 30 sec - exiting 13:18:54 (74940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:18:55 (74940): No heartbeat from core client for 30 sec - exiting 13:20:24 (7748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:33:21 (81676): No heartbeat from core client for 30 sec - exiting 01:33:22 (81676): No heartbeat from core client for 30 sec - exiting 01:33:23 (81676): No heartbeat from core client for 30 sec - exiting 01:33:24 (81676): No heartbeat from core client for 30 sec - exiting 01:33:25 (81676): No heartbeat from core client for 30 sec - exiting 01:33:26 (81676): No heartbeat from core client for 30 sec - exiting 01:33:27 (81676): No heartbeat from core client for 30 sec - exiting 01:33:28 (81676): No heartbeat from core client for 30 sec - exiting 01:33:29 (81676): No heartbeat from core client for 30 sec - exiting 01:33:30 (81676): No heartbeat from core client for 30 sec - exiting 01:33:31 (81676): No heartbeat from core client for 30 sec - exiting 01:33:32 (81676): No heartbeat from core client for 30 sec - exiting 01:33:33 (81676): No heartbeat from core client for 30 sec - exiting 01:33:34 (81676): No heartbeat from core client for 30 sec - exiting 01:33:35 (81676): No heartbeat from core client for 30 sec - exiting 01:33:36 (81676): No heartbeat from core client for 30 sec - exiting 01:33:37 (81676): No heartbeat from core client for 30 sec - exiting 01:33:38 (81676): No heartbeat from core client for 30 sec - exiting 01:33:39 (81676): No heartbeat from core client for 30 sec - exiting 01:33:40 (81676): No heartbeat from core client for 30 sec - exiting 01:33:41 (81676): No heartbeat from core client for 30 sec - exiting 01:33:42 (81676): No heartbeat from core client for 30 sec - exiting 01:33:43 (81676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:25:46 (100440): No heartbeat from core client for 30 sec - exiting 12:25:47 (100440): No heartbeat from core client for 30 sec - exiting 12:25:48 (100440): No heartbeat from core client for 30 sec - exiting 12:25:49 (100440): No heartbeat from core client for 30 sec - exiting 12:25:50 (100440): No heartbeat from core client for 30 sec - exiting 12:25:51 (100440): No heartbeat from core client for 30 sec - exiting 12:25:52 (100440): No heartbeat from core client for 30 sec - exiting 12:25:53 (100440): No heartbeat from core client for 30 sec - exiting 12:25:54 (100440): No heartbeat from core client for 30 sec - exiting 12:25:55 (100440): No heartbeat from core client for 30 sec - exiting 12:25:56 (100440): No heartbeat from core client for 30 sec - exiting 12:25:57 (100440): No heartbeat from core client for 30 sec - exiting 12:25:58 (100440): No heartbeat from core client for 30 sec - exiting 12:25:59 (100440): No heartbeat from core client for 30 sec - exiting 12:26:00 (100440): No heartbeat from core client for 30 sec - exiting 12:26:01 (100440): No heartbeat from core client for 30 sec - exiting 12:26:02 (100440): No heartbeat from core client for 30 sec - exiting 12:26:03 (100440): No heartbeat from core client for 30 sec - exiting 12:26:04 (100440): No heartbeat from core client for 30 sec - exiting 12:26:05 (100440): No heartbeat from core client for 30 sec - exiting 12:26:06 (100440): No heartbeat from core client for 30 sec - exiting 12:26:07 (100440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:28:07 (100148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:52:55 (104784): No heartbeat from core client for 30 sec - exiting 21:52:57 (104784): No heartbeat from core client for 30 sec - exiting 21:52:58 (104784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:52:59 (104784): No heartbeat from core client for 30 sec - exiting 21:54:31 (102244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:54:01 (159460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:41:35 (165380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:52:54 (198996): No heartbeat from core client for 30 sec - exiting 04:52:55 (198996): No heartbeat from core client for 30 sec - exiting 04:52:56 (198996): No heartbeat from core client for 30 sec - exiting 04:52:57 (198996): No heartbeat from core client for 30 sec - exiting 04:52:58 (198996): No heartbeat from core client for 30 sec - exiting 04:52:59 (198996): No heartbeat from core client for 30 sec - exiting 04:53:00 (198996): No heartbeat from core client for 30 sec - exiting 04:53:01 (198996): No heartbeat from core client for 30 sec - exiting 04:53:02 (198996): No heartbeat from core client for 30 sec - exiting 04:53:03 (198996): No heartbeat from core client for 30 sec - exiting 04:53:04 (198996): No heartbeat from core client for 30 sec - exiting 04:53:05 (198996): No heartbeat from core client for 30 sec - exiting 04:53:06 (198996): No heartbeat from core client for 30 sec - exiting 04:53:07 (198996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:52:13 (207800): No heartbeat from core client for 30 sec - exiting 18:52:14 (207800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4092, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4092, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4092, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4092, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4092, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4092, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Dec 2012 14:58:02	1083664	15460677	hadcm3n_zk9d_1880_40_008246589_3	103,680	317,449	3.0618
04 Dec 2012 03:46:06	1083664	15460677	hadcm3n_zk9d_1880_40_008246589_3	77,760	239,411	3.0788
02 Dec 2012 17:20:22	1083664	15460677	hadcm3n_zk9d_1880_40_008246589_3	51,840	159,730	3.0812
28 Nov 2012 17:18:57	1083664	15460677	hadcm3n_zk9d_1880_40_008246589_3	25,920	79,267	3.0581