Task 13958167

Name	hadcm3n_o09q_1980_40_007694430_1
Workunit	7849538
Created	23 Jan 2012, 23:22:51 UTC
Sent	23 Jan 2012, 23:29:33 UTC
Report deadline	24 Apr 2012, 6:56:44 UTC
Received	8 Feb 2012, 23:36:09 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1072815
Run time	5 days 11 hours 25 min 38 sec
CPU time	5 days 7 hours 12 min 2 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.67 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> 18:57:56 (5408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4156, iMonCtr=1 Model crash detected, will try to restart... 19:10:09 (5864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:35:06 (7096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:15:21 (5844): No heartbeat from core client for 30 sec - exiting 19:15:22 (5844): No heartbeat from core client for 30 sec - exiting 19:15:23 (5844): No heartbeat from core client for 30 sec - exiting 19:15:24 (5844): No heartbeat from core client for 30 sec - exiting 19:15:25 (5844): No heartbeat from core client for 30 sec - exiting 19:15:26 (5844): No heartbeat from core client for 30 sec - exiting 19:15:27 (5844): No heartbeat from core client for 30 sec - exiting 19:15:28 (5844): No heartbeat from core client for 30 sec - exiting 19:15:29 (5844): No heartbeat from core client for 30 sec - exiting 19:15:30 (5844): No heartbeat from core client for 30 sec - exiting 19:15:31 (5844): No heartbeat from core client for 30 sec - exiting 19:15:33 (5844): No heartbeat from core client for 30 sec - exiting 19:15:34 (5844): No heartbeat from core client for 30 sec - exiting 19:15:35 (5844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:45:21 (1156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:06 (4780): No heartbeat from core client for 30 sec - exiting 19:46:07 (4780): No heartbeat from core client for 30 sec - exiting 19:46:08 (4780): No heartbeat from core client for 30 sec - exiting 19:46:09 (4780): No heartbeat from core client for 30 sec - exiting 19:46:10 (4780): No heartbeat from core client for 30 sec - exiting 19:46:11 (4780): No heartbeat from core client for 30 sec - exiting 19:46:12 (4780): No heartbeat from core client for 30 sec - exiting 19:46:13 (4780): No heartbeat from core client for 30 sec - exiting 19:46:14 (4780): No heartbeat from core client for 30 sec - exiting 19:46:15 (4780): No heartbeat from core client for 30 sec - exiting 19:46:16 (4780): No heartbeat from core client for 30 sec - exiting 19:46:17 (4780): No heartbeat from core client for 30 sec - exiting 19:46:18 (4780): No heartbeat from core client for 30 sec - exiting 19:46:19 (4780): No heartbeat from core client for 30 sec - exiting 19:46:20 (4780): No heartbeat from core client for 30 sec - exiting 19:46:21 (4780): No heartbeat from core client for 30 sec - exiting 19:46:22 (4780): No heartbeat from core client for 30 sec - exiting 19:46:23 (4780): No heartbeat from core client for 30 sec - exiting 19:46:24 (4780): No heartbeat from core client for 30 sec - exiting 19:46:25 (4780): No heartbeat from core client for 30 sec - exiting 19:46:26 (4780): No heartbeat from core client for 30 sec - exiting 19:46:27 (4780): No heartbeat from core client for 30 sec - exiting 19:46:28 (4780): No heartbeat from core client for 30 sec - exiting 19:46:29 (4780): No heartbeat from core client for 30 sec - exiting 19:46:30 (4780): No heartbeat from core client for 30 sec - exiting 19:46:31 (4780): No heartbeat from core client for 30 sec - exiting 19:46:32 (4780): No heartbeat from core client for 30 sec - exiting 19:46:33 (4780): No heartbeat from core client for 30 sec - exiting 19:46:34 (4780): No heartbeat from core client for 30 sec - exiting 19:46:35 (4780): No heartbeat from core client for 30 sec - exiting 19:46:36 (4780): No heartbeat from core client for 30 sec - exiting 19:46:37 (4780): No heartbeat from core client for 30 sec - exiting 19:46:38 (4780): No heartbeat from core client for 30 sec - exiting 19:46:39 (4780): No heartbeat from core client for 30 sec - exiting 19:46:40 (4780): No heartbeat from core client for 30 sec - exiting 19:46:41 (4780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:05:10 (3328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:01:33 (224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:11:40 (3524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:11:42 (3524): No heartbeat from core client for 30 sec - exiting 15:11:43 (3524): No heartbeat from core client for 30 sec - exiting 15:11:44 (3524): No heartbeat from core client for 30 sec - exiting 15:11:45 (3524): No heartbeat from core client for 30 sec - exiting 15:12:36 (1240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6716, iMonCtr=1 Model crash detected, will try to restart... 12:21:00 (3336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:15:06 (3744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:29:55 (2936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4304, iMonCtr=1 Model crash detected, will try to restart... 19:49:15 (5932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:49:16 (5932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 23:54:07 (6528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:13:12 (6104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:32:19 (5952): No heartbeat from core client for 30 sec - exiting 19:32:20 (5952): No heartbeat from core client for 30 sec - exiting 19:32:21 (5952): No heartbeat from core client for 30 sec - exiting 19:32:22 (5952): No heartbeat from core client for 30 sec - exiting 19:32:23 (5952): No heartbeat from core client for 30 sec - exiting 19:32:24 (5952): No heartbeat from core client for 30 sec - exiting 19:32:25 (5952): No heartbeat from core client for 30 sec - exiting 19:32:26 (5952): No heartbeat from core client for 30 sec - exiting 19:32:27 (5952): No heartbeat from core client for 30 sec - exiting 19:32:28 (5952): No heartbeat from core client for 30 sec - exiting 19:32:29 (5952): No heartbeat from core client for 30 sec - exiting 19:32:30 (5952): No heartbeat from core client for 30 sec - exiting 19:32:31 (5952): No heartbeat from core client for 30 sec - exiting 19:32:32 (5952): No heartbeat from core client for 30 sec - exiting 19:32:33 (5952): No heartbeat from core client for 30 sec - exiting 19:32:34 (5952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:32:35 (5952): No heartbeat from core client for 30 sec - exiting 19:32:36 (5952): No heartbeat from core client for 30 sec - exiting 19:32:37 (5952): No heartbeat from core client for 30 sec - exiting 19:32:38 (5952): No heartbeat from core client for 30 sec - exiting 21:10:19 (5692): No heartbeat from core client for 30 sec - exiting 21:10:20 (5692): No heartbeat from core client for 30 sec - exiting 21:10:21 (5692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:05:30 (5172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:06:13 (5084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:08:45 (5916): No heartbeat from core client for 30 sec - exiting 19:08:46 (5916): No heartbeat from core client for 30 sec - exiting 19:08:48 (5916): No heartbeat from core client for 30 sec - exiting 19:08:49 (5916): No heartbeat from core client for 30 sec - exiting 19:08:50 (5916): No heartbeat from core client for 30 sec - exiting 19:08:51 (5916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:08:52 (5916): No heartbeat from core client for 30 sec - exiting 01:39:11 (5944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:39:12 (5944): No heartbeat from core client for 30 sec - exiting 01:39:13 (5944): No heartbeat from core client for 30 sec - exiting 01:39:59 (1212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5940, iMonCtr=1 Model crash detected, will try to restart... 19:07:06 (5408): No heartbeat from core client for 30 sec - exiting 19:07:07 (5408): No heartbeat from core client for 30 sec - exiting 19:07:08 (5408): No heartbeat from core client for 30 sec - exiting 19:07:09 (5408): No heartbeat from core client for 30 sec - exiting 19:07:11 (5408): No heartbeat from core client for 30 sec - exiting 19:07:12 (5408): No heartbeat from core client for 30 sec - exiting 19:07:13 (5408): No heartbeat from core client for 30 sec - exiting 19:07:14 (5408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:19:43 (5104): No heartbeat from core client for 30 sec - exiting 19:19:44 (5104): No heartbeat from core client for 30 sec - exiting 19:19:45 (5104): No heartbeat from core client for 30 sec - exiting 19:19:46 (5104): No heartbeat from core client for 30 sec - exiting 19:19:47 (5104): No heartbeat from core client for 30 sec - exiting 19:19:48 (5104): No heartbeat from core client for 30 sec - exiting 19:19:50 (5104): No heartbeat from core client for 30 sec - exiting 19:19:51 (5104): No heartbeat from core client for 30 sec - exiting 19:19:52 (5104): No heartbeat from core client for 30 sec - exiting 19:19:53 (5104): No heartbeat from core client for 30 sec - exiting 19:19:54 (5104): No heartbeat from core client for 30 sec - exiting 19:19:55 (5104): No heartbeat from core client for 30 sec - exiting 19:19:56 (5104): No heartbeat from core client for 30 sec - exiting 19:19:57 (5104): No heartbeat from core client for 30 sec - exiting 19:19:58 (5104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:19:59 (5104): No heartbeat from core client for 30 sec - exiting 19:35:06 (6016): No heartbeat from core client for 30 sec - exiting 19:35:07 (6016): No heartbeat from core client for 30 sec - exiting 19:35:08 (6016): No heartbeat from core client for 30 sec - exiting 19:35:09 (6016): No heartbeat from core client for 30 sec - exiting 19:35:10 (6016): No heartbeat from core client for 30 sec - exiting 19:35:11 (6016): No heartbeat from core client for 30 sec - exiting 19:35:12 (6016): No heartbeat from core client for 30 sec - exiting 19:35:13 (6016): No heartbeat from core client for 30 sec - exiting 19:35:14 (6016): No heartbeat from core client for 30 sec - exiting 19:35:15 (6016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5248, iMonCtr=1 Model crash detected, will try to restart... 11:09:31 (3912): No heartbeat from core client for 30 sec - exiting 11:09:33 (3912): No heartbeat from core client for 30 sec - exiting 11:09:34 (3912): No heartbeat from core client for 30 sec - exiting 11:09:35 (3912): No heartbeat from core client for 30 sec - exiting 11:09:36 (3912): No heartbeat from core client for 30 sec - exiting 11:09:37 (3912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:17:45 (5108): No heartbeat from core client for 30 sec - exiting 12:17:46 (5108): No heartbeat from core client for 30 sec - exiting 12:17:47 (5108): No heartbeat from core client for 30 sec - exiting 12:17:48 (5108): No heartbeat from core client for 30 sec - exiting 12:17:49 (5108): No heartbeat from core client for 30 sec - exiting 12:17:50 (5108): No heartbeat from core client for 30 sec - exiting 12:17:51 (5108): No heartbeat from core client for 30 sec - exiting 12:17:52 (5108): No heartbeat from core client for 30 sec - exiting 12:17:54 (5108): No heartbeat from core client for 30 sec - exiting 12:17:55 (5108): No heartbeat from core client for 30 sec - exiting 12:17:56 (5108): No heartbeat from core client for 30 sec - exiting 12:17:57 (5108): No heartbeat from core client for 30 sec - exiting 12:17:58 (5108): No heartbeat from core client for 30 sec - exiting 12:17:59 (5108): No heartbeat from core client for 30 sec - exiting 12:18:00 (5108): No heartbeat from core client for 30 sec - exiting 12:18:01 (5108): No heartbeat from core client for 30 sec - exiting 12:18:02 (5108): No heartbeat from core client for 30 sec - exiting 12:18:03 (5108): No heartbeat from core client for 30 sec - exiting 12:18:04 (5108): No heartbeat from core client for 30 sec - exiting 12:18:06 (5108): No heartbeat from core client for 30 sec - exiting 12:18:07 (5108): No heartbeat from core client for 30 sec - exiting 12:18:08 (5108): No heartbeat from core client for 30 sec - exiting 12:18:09 (5108): No heartbeat from core client for 30 sec - exiting 12:18:10 (5108): No heartbeat from core client for 30 sec - exiting 12:18:11 (5108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5648, iMonCtr=1 Model crash detected, will try to restart... 11:29:13 (5828): No heartbeat from core client for 30 sec - exiting 11:29:14 (5828): No heartbeat from core client for 30 sec - exiting 11:29:15 (5828): No heartbeat from core client for 30 sec - exiting 11:29:17 (5828): No heartbeat from core client for 30 sec - exiting 11:29:18 (5828): No heartbeat from core client for 30 sec - exiting 11:29:19 (5828): No heartbeat from core client for 30 sec - exiting 11:29:20 (5828): No heartbeat from core client for 30 sec - exiting 11:29:21 (5828): No heartbeat from core client for 30 sec - exiting 11:29:22 (5828): No heartbeat from core client for 30 sec - exiting 11:29:23 (5828): No heartbeat from core client for 30 sec - exiting 11:29:24 (5828): No heartbeat from core client for 30 sec - exiting 11:29:25 (5828): No heartbeat from core client for 30 sec - exiting 11:29:26 (5828): No heartbeat from core client for 30 sec - exiting 11:29:27 (5828): No heartbeat from core client for 30 sec - exiting 11:29:29 (5828): No heartbeat from core client for 30 sec - exiting 11:29:30 (5828): No heartbeat from core client for 30 sec - exiting 11:29:31 (5828): No heartbeat from core client for 30 sec - exiting 11:29:32 (5828): No heartbeat from core client for 30 sec - exiting 11:29:33 (5828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:11:25 (6108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:11:26 (6108): No heartbeat from core client for 30 sec - exiting 12:12:12 (5000): No heartbeat from core client for 30 sec - exiting 12:12:13 (5000): No heartbeat from core client for 30 sec - exiting 12:12:14 (5000): No heartbeat from core client for 30 sec - exiting 12:12:15 (5000): No heartbeat from core client for 30 sec - exiting 12:12:16 (5000): No heartbeat from core client for 30 sec - exiting 12:12:17 (5000): No heartbeat from core client for 30 sec - exiting 12:12:18 (5000): No heartbeat from core client for 30 sec - exiting 12:12:19 (5000): No heartbeat from core client for 30 sec - exiting 12:12:21 (5000): No heartbeat from core client for 30 sec - exiting 12:12:22 (5000): No heartbeat from core client for 30 sec - exiting 12:12:23 (5000): No heartbeat from core client for 30 sec - exiting 12:12:24 (5000): No heartbeat from core client for 30 sec - exiting 12:12:25 (5000): No heartbeat from core client for 30 sec - exiting 12:12:26 (5000): No heartbeat from core client for 30 sec - exiting 12:12:27 (5000): No heartbeat from core client for 30 sec - exiting 12:12:28 (5000): No heartbeat from core client for 30 sec - exiting 12:12:29 (5000): No heartbeat from core client for 30 sec - exiting 12:12:30 (5000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:26:21 (7728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:27:30 (6308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:36:10 (5452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:31:37 (3028): No heartbeat from core client for 30 sec - exiting 19:31:38 (3028): No heartbeat from core client for 30 sec - exiting 19:31:39 (3028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:31:40 (3028): No heartbeat from core client for 30 sec - exiting 19:31:41 (3028): No heartbeat from core client for 30 sec - exiting 19:31:42 (3028): No heartbeat from core client for 30 sec - exiting 19:31:43 (3028): No heartbeat from core client for 30 sec - exiting 19:31:44 (3028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5776, iMonCtr=1 Model crash detected, will try to restart... 19:26:31 (5760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:26:32 (5760): No heartbeat from core client for 30 sec - exiting 19:26:33 (5760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:03:24 (3148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Feb 2012 23:40:58	1072815	13958167	hadcm3n_o09q_1980_40_007694430_1	259,200	457,915	1.7666
07 Feb 2012 00:12:19	1072815	13958167	hadcm3n_o09q_1980_40_007694430_1	233,280	411,877	1.7656
05 Feb 2012 15:26:36	1072815	13958167	hadcm3n_o09q_1980_40_007694430_1	207,360	366,078	1.7654
04 Feb 2012 16:28:58	1072815	13958167	hadcm3n_o09q_1980_40_007694430_1	181,440	319,627	1.7616
03 Feb 2012 00:54:29	1072815	13958167	hadcm3n_o09q_1980_40_007694430_1	155,520	273,828	1.7607
01 Feb 2012 00:15:47	1072815	13958167	hadcm3n_o09q_1980_40_007694430_1	129,600	227,418	1.7548
29 Jan 2012 21:41:29	1072815	13958167	hadcm3n_o09q_1980_40_007694430_1	103,680	181,974	1.7552
28 Jan 2012 21:40:29	1072815	13958167	hadcm3n_o09q_1980_40_007694430_1	77,760	136,628	1.7570
27 Jan 2012 23:08:27	1072815	13958167	hadcm3n_o09q_1980_40_007694430_1	51,840	91,791	1.7707
25 Jan 2012 23:27:41	1072815	13958167	hadcm3n_o09q_1980_40_007694430_1	25,920	45,741	1.7647