Task 17149866

Name	hadcm3s_3ci6_1993_2_009066113_0
Workunit	9196449
Created	7 Oct 2014, 16:10:16 UTC
Sent	10 Oct 2014, 16:18:28 UTC
Report deadline	9 Jan 2015, 23:45:39 UTC
Received	29 Oct 2014, 5:45:04 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	9 (0x00000009) Unknown error code
Computer ID	976228
Run time	5 days 11 hours 25 min 18 sec
CPU time	3 days 10 hours 27 min 40 sec
Validate state	Invalid
Credit	622.08
Device peak FLOPS	2.84 GFLOPS
Application version	UK Met Office HadCM3 short v7.24 i686-apple-darwin
Stderr	<core_client_version>6.6.29</core_client_version> <![CDATA[ <message> process exited with code 9 (0x9, -247) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:14:15 (7618): No heartbeat from client for 30 sec - exiting 00:14:15 (7618): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:17:52 (18959): No heartbeat from client for 30 sec - exiting 01:17:52 (18959): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:24:00 (19430): No heartbeat from client for 30 sec - exiting 01:24:00 (19430): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:30:01 (19476): No heartbeat from client for 30 sec - exiting 01:30:01 (19476): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:35:56 (19549): No heartbeat from client for 30 sec - exiting 01:35:56 (19549): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:41:56 (19604): No heartbeat from client for 30 sec - exiting 01:41:56 (19604): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:47:55 (19673): No heartbeat from client for 30 sec - exiting 01:47:55 (19673): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:04:54 (19729): No heartbeat from client for 30 sec - exiting 02:04:55 (19729): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:06:19 (19871): No heartbeat from client for 30 sec - exiting 00:06:19 (19871): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:13:31 (14181): No heartbeat from client for 30 sec - exiting 00:13:31 (14181): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:32:04 (14243): No heartbeat from client for 30 sec - exiting 01:32:04 (14243): timer handler: client dead, exiting 01:32:05 (14243): No heartbeat from client for 30 sec - exiting 01:32:05 (14243): timer handler: client dead, exiting 01:32:06 (14243): No heartbeat from client for 30 sec - exiting 01:32:06 (14243): timer handler: client dead, exiting 01:32:07 (14243): No heartbeat from client for 30 sec - exiting 01:32:07 (14243): timer handler: client dead, exiting 01:32:08 (14243): No heartbeat from client for 30 sec - exiting 01:32:08 (14243): timer handler: client dead, exiting 01:32:09 (14243): No heartbeat from client for 30 sec - exiting 01:32:09 (14243): timer handler: client dead, exiting 01:32:10 (14243): No heartbeat from client for 30 sec - exiting 01:32:10 (14243): timer handler: client dead, exiting 01:32:11 (14243): No heartbeat from client for 30 sec - exiting 01:32:11 (14243): timer handler: client dead, exiting 01:32:12 (14243): No heartbeat from client for 30 sec - exiting 01:32:12 (14243): timer handler: client dead, exiting 01:32:13 (14243): No heartbeat from client for 30 sec - exiting 01:32:13 (14243): timer handler: client dead, exiting 01:32:14 (14243): No heartbeat from client for 30 sec - exiting 01:32:14 (14243): timer handler: client dead, exiting 01:32:15 (14243): No heartbeat from client for 30 sec - exiting 01:32:15 (14243): timer handler: client dead, exiting 01:32:16 (14243): No heartbeat from client for 30 sec - exiting 01:32:16 (14243): timer handler: client dead, exiting 01:32:17 (14243): No heartbeat from client for 30 sec - exiting 01:32:17 (14243): timer handler: client dead, exiting 01:32:18 (14243): No heartbeat from client for 30 sec - exiting 01:32:18 (14243): timer handler: client dead, exiting 01:32:19 (14243): No heartbeat from client for 30 sec - exiting 01:32:19 (14243): timer handler: client dead, exiting 01:32:20 (14243): No heartbeat from client for 30 sec - exiting 01:32:20 (14243): timer handler: client dead, exiting 01:32:21 (14243): No heartbeat from client for 30 sec - exiting 01:32:21 (14243): timer handler: client dead, exiting 01:32:22 (14243): No heartbeat from client for 30 sec - exiting 01:32:22 (14243): timer handler: client dead, exiting 01:32:23 (14243): No heartbeat from client for 30 sec - exiting 01:32:23 (14243): timer handler: client dead, exiting 01:32:24 (14243): No heartbeat from client for 30 sec - exiting 01:32:24 (14243): timer handler: client dead, exiting 01:32:25 (14243): No heartbeat from client for 30 sec - exiting 01:32:25 (14243): timer handler: client dead, exiting 01:32:26 (14243): No heartbeat from client for 30 sec - exiting 01:32:26 (14243): timer handler: client dead, exiting 01:32:27 (14243): No heartbeat from client for 30 sec - exiting 01:32:27 (14243): timer handler: client dead, exiting 01:32:28 (14243): No heartbeat from client for 30 sec - exiting 01:32:28 (14243): timer handler: client dead, exiting 01:32:29 (14243): No heartbeat from client for 30 sec - exiting 01:32:29 (14243): timer handler: client dead, exiting 01:32:30 (14243): No heartbeat from client for 30 sec - exiting 01:32:30 (14243): timer handler: client dead, exiting 01:32:31 (14243): No heartbeat from client for 30 sec - exiting 01:32:31 (14243): timer handler: client dead, exiting 01:32:32 (14243): No heartbeat from client for 30 sec - exiting 01:32:32 (14243): timer handler: client dead, exiting 01:32:33 (14243): No heartbeat from client for 30 sec - exiting 01:32:33 (14243): timer handler: client dead, exiting 01:32:34 (14243): No heartbeat from client for 30 sec - exiting 01:32:34 (14243): timer handler: client dead, exiting 01:32:35 (14243): No heartbeat from client for 30 sec - exiting 01:32:35 (14243): timer handler: client dead, exiting 01:32:36 (14243): No heartbeat from client for 30 sec - exiting 01:32:36 (14243): timer handler: client dead, exiting 01:32:37 (14243): No heartbeat from client for 30 sec - exiting 01:32:37 (14243): timer handler: client dead, exiting 01:32:38 (14243): No heartbeat from client for 30 sec - exiting 01:32:38 (14243): timer handler: client dead, exiting 01:32:39 (14243): No heartbeat from client for 30 sec - exiting 01:32:39 (14243): timer handler: client dead, exiting 01:32:40 (14243): No heartbeat from client for 30 sec - exiting 01:32:40 (14243): timer handler: client dead, exiting 01:32:41 (14243): No heartbeat from client for 30 sec - exiting 01:32:41 (14243): timer handler: client dead, exiting 01:32:42 (14243): No heartbeat from client for 30 sec - exiting 01:32:42 (14243): timer handler: client dead, exiting 01:32:43 (14243): No heartbeat from client for 30 sec - exiting 01:32:43 (14243): timer handler: client dead, exiting 01:32:44 (14243): No heartbeat from client for 30 sec - exiting 01:32:44 (14243): timer handler: client dead, exiting 01:32:45 (14243): No heartbeat from client for 30 sec - exiting 01:32:45 (14243): timer handler: client dead, exiting 01:32:46 (14243): No heartbeat from client for 30 sec - exiting 01:32:46 (14243): timer handler: client dead, exiting 01:32:47 (14243): No heartbeat from client for 30 sec - exiting 01:32:47 (14243): timer handler: client dead, exiting 01:32:48 (14243): No heartbeat from client for 30 sec - exiting 01:32:48 (14243): timer handler: client dead, exiting 01:32:49 (14243): No heartbeat from client for 30 sec - exiting 01:32:49 (14243): timer handler: client dead, exiting 01:32:50 (14243): No heartbeat from client for 30 sec - exiting 01:32:50 (14243): timer handler: client dead, exiting 01:33:32 (14243): No heartbeat from client for 30 sec - exiting 01:33:32 (14243): timer handler: client dead, exiting 01:33:33 (14243): No heartbeat from client for 30 sec - exiting 01:33:33 (14243): timer handler: client dead, exiting 01:33:34 (14243): No heartbeat from client for 30 sec - exiting 01:33:34 (14243): timer handler: client dead, exiting 01:33:35 (14243): No heartbeat from client for 30 sec - exiting 01:33:35 (14243): timer handler: client dead, exiting 01:33:36 (14243): No heartbeat from client for 30 sec - exiting 01:33:36 (14243): timer handler: client dead, exiting 01:33:37 (14243): No heartbeat from client for 30 sec - exiting 01:33:37 (14243): timer handler: client dead, exiting 01:33:38 (14243): No heartbeat from client for 30 sec - exiting 01:33:38 (14243): timer handler: client dead, exiting 01:33:39 (14243): No heartbeat from client for 30 sec - exiting 01:33:39 (14243): timer handler: client dead, exiting 01:33:40 (14243): No heartbeat from client for 30 sec - exiting 01:33:40 (14243): timer handler: client dead, exiting 01:33:41 (14243): No heartbeat from client for 30 sec - exiting 01:33:41 (14243): timer handler: client dead, exiting 01:33:42 (14243): No heartbeat from client for 30 sec - exiting 01:33:42 (14243): timer handler: client dead, exiting 01:33:43 (14243): No heartbeat from client for 30 sec - exiting 01:33:43 (14243): timer handler: client dead, exiting 01:33:44 (14243): No heartbeat from client for 30 sec - exiting 01:33:44 (14243): timer handler: client dead, exiting 01:33:45 (14243): No heartbeat from client for 30 sec - exiting 01:33:45 (14243): timer handler: client dead, exiting 01:33:46 (14243): No heartbeat from client for 30 sec - exiting 01:33:46 (14243): timer handler: client dead, exiting 01:33:47 (14243): No heartbeat from client for 30 sec - exiting 01:33:47 (14243): timer handler: client dead, exiting 01:33:48 (14243): No heartbeat from client for 30 sec - exiting 01:33:48 (14243): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:36:09 (14831): No heartbeat from client for 30 sec - exiting 01:36:09 (14831): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:38:09 (14843): No heartbeat from client for 30 sec - exiting 01:38:10 (14843): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:39:27 (14894): No heartbeat from client for 30 sec - exiting 01:39:28 (14894): timer handler: client dead, exiting 01:39:29 (14894): No heartbeat from client for 30 sec - exiting 01:39:29 (14894): timer handler: client dead, exiting 01:39:30 (14894): No heartbeat from client for 30 sec - exiting 01:39:30 (14894): timer handler: client dead, exiting 01:39:31 (14894): No heartbeat from client for 30 sec - exiting 01:39:31 (14894): timer handler: client dead, exiting 01:39:32 (14894): No heartbeat from client for 30 sec - exiting 01:39:32 (14894): timer handler: client dead, exiting 01:39:33 (14894): No heartbeat from client for 30 sec - exiting 01:39:33 (14894): timer handler: client dead, exiting 01:39:34 (14894): No heartbeat from client for 30 sec - exiting 01:39:34 (14894): timer handler: client dead, exiting 01:39:35 (14894): No heartbeat from client for 30 sec - exiting 01:39:35 (14894): timer handler: client dead, exiting 01:39:36 (14894): No heartbeat from client for 30 sec - exiting 01:39:36 (14894): timer handler: client dead, exiting 01:39:37 (14894): No heartbeat from client for 30 sec - exiting 01:39:37 (14894): timer handler: client dead, exiting 01:39:38 (14894): No heartbeat from client for 30 sec - exiting 01:39:38 (14894): timer handler: client dead, exiting 01:39:39 (14894): No heartbeat from client for 30 sec - exiting 01:39:39 (14894): timer handler: client dead, exiting 01:39:40 (14894): No heartbeat from client for 30 sec - exiting 01:39:40 (14894): timer handler: client dead, exiting 01:39:41 (14894): No heartbeat from client for 30 sec - exiting 01:39:41 (14894): timer handler: client dead, exiting 01:39:42 (14894): No heartbeat from client for 30 sec - exiting 01:39:42 (14894): timer handler: client dead, exiting 01:39:43 (14894): No heartbeat from client for 30 sec - exiting 01:39:43 (14894): timer handler: client dead, exiting 01:39:44 (14894): No heartbeat from client for 30 sec - exiting 01:39:44 (14894): timer handler: client dead, exiting 01:39:45 (14894): No heartbeat from client for 30 sec - exiting 01:39:45 (14894): timer handler: client dead, exiting 01:39:46 (14894): No heartbeat from client for 30 sec - exiting 01:39:46 (14894): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:41:57 (14912): No heartbeat from client for 30 sec - exiting 01:41:58 (14912): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:44:07 (14956): No heartbeat from client for 30 sec - exiting 01:44:07 (14956): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:45:58 (14981): No heartbeat from client for 30 sec - exiting 01:45:58 (14981): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:48:53 (15031): No heartbeat from client for 30 sec - exiting 01:48:53 (15031): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:50:55 (15053): No heartbeat from client for 30 sec - exiting 01:50:55 (15053): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:52:44 (15102): No heartbeat from client for 30 sec - exiting 01:52:44 (15102): timer handler: client dead, exiting 01:52:45 (15102): No heartbeat from client for 30 sec - exiting 01:52:45 (15102): timer handler: client dead, exiting 01:52:46 (15102): No heartbeat from client for 30 sec - exiting 01:52:46 (15102): timer handler: client dead, exiting 01:52:47 (15102): No heartbeat from client for 30 sec - exiting 01:52:47 (15102): timer handler: client dead, exiting 01:52:48 (15102): No heartbeat from client for 30 sec - exiting 01:52:48 (15102): timer handler: client dead, exiting 01:52:49 (15102): No heartbeat from client for 30 sec - exiting 01:52:49 (15102): timer handler: client dead, exiting 01:52:50 (15102): No heartbeat from client for 30 sec - exiting 01:52:50 (15102): timer handler: client dead, exiting 01:52:51 (15102): No heartbeat from client for 30 sec - exiting 01:52:51 (15102): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:55:05 (15125): No heartbeat from client for 30 sec - exiting 01:55:05 (15125): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:57:08 (15173): No heartbeat from client for 30 sec - exiting 01:57:08 (15173): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:59:20 (15196): No heartbeat from client for 30 sec - exiting 01:59:20 (15196): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:00:48 (15256): No heartbeat from client for 30 sec - exiting 02:00:48 (15256): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:02:00 (15272): No heartbeat from client for 30 sec - exiting 02:02:00 (15272): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:03:48 (15321): No heartbeat from client for 30 sec - exiting 02:03:48 (15321): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:11:04 (15340): No heartbeat from client for 30 sec - exiting 02:11:04 (15340): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:11:06 (37852): No heartbeat from client for 30 sec - exiting 00:11:06 (37852): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:23:50 (39246): No heartbeat from client for 30 sec - exiting 01:23:50 (39246): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:29:50 (39776): No heartbeat from client for 30 sec - exiting 01:29:50 (39776): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:35:54 (39833): No heartbeat from client for 30 sec - exiting 01:35:54 (39833): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:42:00 (39903): No heartbeat from client for 30 sec - exiting 01:42:01 (39903): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:48:04 (39957): No heartbeat from client for 30 sec - exiting 01:48:05 (39957): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:06:21 (40025): No heartbeat from client for 30 sec - exiting 02:06:21 (40025): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:09:52 (40155): No heartbeat from client for 30 sec - exiting 02:09:52 (40155): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:15:07 (40227): No heartbeat from client for 30 sec - exiting 00:15:07 (40227): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:15:08 (40227): No heartbeat from client for 30 sec - exiting 00:15:08 (40227): timer handler: client dead, exiting Suspended CPDN Monitor - Suspend request from BOINC... 01:14:39 (49655): No heartbeat from client for 30 sec - exiting 01:14:39 (49655): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:21:26 (50113): No heartbeat from client for 30 sec - exiting 01:21:26 (50113): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:27:26 (50186): No heartbeat from client for 30 sec - exiting 01:27:26 (50186): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:33:29 (50241): No heartbeat from client for 30 sec - exiting 01:33:29 (50241): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:39:30 (50317): No heartbeat from client for 30 sec - exiting 01:39:30 (50317): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:45:32 (50371): No heartbeat from client for 30 sec - exiting 01:45:32 (50371): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Oct 2014 08:32:26	976228	17149866	hadcm3s_3ci6_1993_2_009066113_0	51,840	148,914	2.8726
24 Oct 2014 10:35:48	976228	17149866	hadcm3s_3ci6_1993_2_009066113_0	25,920	74,044	2.8566