Task 15667377

Name	hadcm3n_zck5_1920_40_008281668_3
Workunit	8432803
Created	14 Mar 2013, 16:43:21 UTC
Sent	14 Mar 2013, 16:43:48 UTC
Report deadline	14 Jun 2013, 0:10:59 UTC
Received	31 Mar 2013, 19:00:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1233009
Run time	9 days 15 hours 50 min 41 sec
CPU time	9 days 8 hours 35 min 39 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	1.29 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:33:01 (9704): No heartbeat from core client for 30 sec - exiting 03:33:02 (9704): No heartbeat from core client for 30 sec - exiting 03:33:03 (9704): No heartbeat from core client for 30 sec - exiting 03:33:04 (9704): No heartbeat from core client for 30 sec - exiting 03:33:05 (9704): No heartbeat from core client for 30 sec - exiting 03:33:06 (9704): No heartbeat from core client for 30 sec - exiting 03:33:07 (9704): No heartbeat from core client for 30 sec - exiting 03:33:08 (9704): No heartbeat from core client for 30 sec - exiting 03:33:09 (9704): No heartbeat from core client for 30 sec - exiting 03:33:10 (9704): No heartbeat from core client for 30 sec - exiting 03:33:12 (9704): No heartbeat from core client for 30 sec - exiting 03:33:13 (9704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:33:14 (9704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:41:56 (3188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:06:10 (296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 01:58:11 (7224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 04:13:01 (4592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:13:37 (8132): No heartbeat from core client for 30 sec - exiting 07:13:38 (8132): No heartbeat from core client for 30 sec - exiting 07:13:40 (8132): No heartbeat from core client for 30 sec - exiting 07:13:41 (8132): No heartbeat from core client for 30 sec - exiting 07:13:42 (8132): No heartbeat from core client for 30 sec - exiting 07:13:43 (8132): No heartbeat from core client for 30 sec - exiting 07:13:44 (8132): No heartbeat from core client for 30 sec - exiting 07:13:45 (8132): No heartbeat from core client for 30 sec - exiting 07:13:46 (8132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:06:14 (9412): No heartbeat from core client for 30 sec - exiting 21:06:15 (9412): No heartbeat from core client for 30 sec - exiting 21:06:16 (9412): No heartbeat from core client for 30 sec - exiting 21:06:18 (9412): No heartbeat from core client for 30 sec - exiting 21:06:19 (9412): No heartbeat from core client for 30 sec - exiting 21:06:20 (9412): No heartbeat from core client for 30 sec - exiting 21:06:21 (9412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:16:47 (9020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:25:02 (6292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:29:01 (9944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:34:51 (3160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 06:50:34 (6152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6088, iMonCtr=1 Model crash detected, will try to restart... 10:01:36 (4544): No heartbeat from core client for 30 sec - exiting 10:01:37 (4544): No heartbeat from core client for 30 sec - exiting 10:01:38 (4544): No heartbeat from core client for 30 sec - exiting 10:01:39 (4544): No heartbeat from core client for 30 sec - exiting 10:01:40 (4544): No heartbeat from core client for 30 sec - exiting 10:01:41 (4544): No heartbeat from core client for 30 sec - exiting 10:01:42 (4544): No heartbeat from core client for 30 sec - exiting 10:01:43 (4544): No heartbeat from core client for 30 sec - exiting 10:01:44 (4544): No heartbeat from core client for 30 sec - exiting 10:01:45 (4544): No heartbeat from core client for 30 sec - exiting 10:01:47 (4544): No heartbeat from core client for 30 sec - exiting 10:01:48 (4544): No heartbeat from core client for 30 sec - exiting 10:01:49 (4544): No heartbeat from core client for 30 sec - exiting 10:01:50 (4544): No heartbeat from core client for 30 sec - exiting 10:01:51 (4544): No heartbeat from core client for 30 sec - exiting 10:01:52 (4544): No heartbeat from core client for 30 sec - exiting 10:01:53 (4544): No heartbeat from core client for 30 sec - exiting 10:01:54 (4544): No heartbeat from core client for 30 sec - exiting 10:01:55 (4544): No heartbeat from core client for 30 sec - exiting 10:01:56 (4544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:02:50 (5972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:16:37 (8968): No heartbeat from core client for 30 sec - exiting 07:16:38 (8968): No heartbeat from core client for 30 sec - exiting 07:16:39 (8968): No heartbeat from core client for 30 sec - exiting 07:16:40 (8968): No heartbeat from core client for 30 sec - exiting 07:16:42 (8968): No heartbeat from core client for 30 sec - exiting 07:16:43 (8968): No heartbeat from core client for 30 sec - exiting 07:16:44 (8968): No heartbeat from core client for 30 sec - exiting 07:16:45 (8968): No heartbeat from core client for 30 sec - exiting 07:16:46 (8968): No heartbeat from core client for 30 sec - exiting 07:16:47 (8968): No heartbeat from core client for 30 sec - exiting 07:16:48 (8968): No heartbeat from core client for 30 sec - exiting 07:16:49 (8968): No heartbeat from core client for 30 sec - exiting 07:16:50 (8968): No heartbeat from core client for 30 sec - exiting 07:16:51 (8968): No heartbeat from core client for 30 sec - exiting 07:16:52 (8968): No heartbeat from core client for 30 sec - exiting 07:16:54 (8968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:16:55 (8968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 07:23:30 (7620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:23:31 (7620): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7856, iMonCtr=1 Model crash detected, will try to restart... 08:11:58 (4796): No heartbeat from core client for 30 sec - exiting 08:12:00 (4796): No heartbeat from core client for 30 sec - exiting 08:12:01 (4796): No heartbeat from core client for 30 sec - exiting 08:12:02 (4796): No heartbeat from core client for 30 sec - exiting 08:12:03 (4796): No heartbeat from core client for 30 sec - exiting 08:12:04 (4796): No heartbeat from core client for 30 sec - exiting 08:12:05 (4796): No heartbeat from core client for 30 sec - exiting 08:12:06 (4796): No heartbeat from core client for 30 sec - exiting 08:12:07 (4796): No heartbeat from core client for 30 sec - exiting 08:12:08 (4796): No heartbeat from core client for 30 sec - exiting 08:12:09 (4796): No heartbeat from core client for 30 sec - exiting 08:12:10 (4796): No heartbeat from core client for 30 sec - exiting 08:12:11 (4796): No heartbeat from core client for 30 sec - exiting 08:12:12 (4796): No heartbeat from core client for 30 sec - exiting 08:12:13 (4796): No heartbeat from core client for 30 sec - exiting 08:12:15 (4796): No heartbeat from core client for 30 sec - exiting 08:12:16 (4796): No heartbeat from core client for 30 sec - exiting 08:12:17 (4796): No heartbeat from core client for 30 sec - exiting 08:12:18 (4796): No heartbeat from core client for 30 sec - exiting 08:12:19 (4796): No heartbeat from core client for 30 sec - exiting 08:12:20 (4796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:11:43 (9012): No heartbeat from core client for 30 sec - exiting 03:11:44 (9012): No heartbeat from core client for 30 sec - exiting 03:11:45 (9012): No heartbeat from core client for 30 sec - exiting 03:11:46 (9012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:03:12 (10892): No heartbeat from core client for 30 sec - exiting 07:03:14 (10892): No heartbeat from core client for 30 sec - exiting 07:03:15 (10892): No heartbeat from core client for 30 sec - exiting 07:03:16 (10892): No heartbeat from core client for 30 sec - exiting 07:03:17 (10892): No heartbeat from core client for 30 sec - exiting 07:03:18 (10892): No heartbeat from core client for 30 sec - exiting 07:03:19 (10892): No heartbeat from core client for 30 sec - exiting 07:03:20 (10892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:03:21 (10892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 10:27:31 (7648): No heartbeat from core client for 30 sec - exiting 10:27:32 (7648): No heartbeat from core client for 30 sec - exiting 10:27:33 (7648): No heartbeat from core client for 30 sec - exiting 10:27:34 (7648): No heartbeat from core client for 30 sec - exiting 10:27:35 (7648): No heartbeat from core client for 30 sec - exiting 10:27:36 (7648): No heartbeat from core client for 30 sec - exiting 10:27:37 (7648): No heartbeat from core client for 30 sec - exiting 10:27:38 (7648): No heartbeat from core client for 30 sec - exiting 10:27:39 (7648): No heartbeat from core client for 30 sec - exiting 10:27:40 (7648): No heartbeat from core client for 30 sec - exiting 10:27:41 (7648): No heartbeat from core client for 30 sec - exiting 10:27:43 (7648): No heartbeat from core client for 30 sec - exiting 10:27:44 (7648): No heartbeat from core client for 30 sec - exiting 10:27:45 (7648): No heartbeat from core client for 30 sec - exiting 10:27:46 (7648): No heartbeat from core client for 30 sec - exiting 10:27:47 (7648): No heartbeat from core client for 30 sec - exiting 10:27:48 (7648): No heartbeat from core client for 30 sec - exiting 10:27:49 (7648): No heartbeat from core client for 30 sec - exiting 10:27:50 (7648): No heartbeat from core client for 30 sec - exiting 10:27:51 (7648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:58:08 (11548): No heartbeat from core client for 30 sec - exiting 02:58:09 (11548): No heartbeat from core client for 30 sec - exiting 02:58:10 (11548): No heartbeat from core client for 30 sec - exiting 02:58:11 (11548): No heartbeat from core client for 30 sec - exiting 02:58:12 (11548): No heartbeat from core client for 30 sec - exiting 02:58:14 (11548): No heartbeat from core client for 30 sec - exiting 02:58:15 (11548): No heartbeat from core client for 30 sec - exiting 02:58:16 (11548): No heartbeat from core client for 30 sec - exiting 02:58:17 (11548): No heartbeat from core client for 30 sec - exiting 02:58:18 (11548): No heartbeat from core client for 30 sec - exiting 02:58:19 (11548): No heartbeat from core client for 30 sec - exiting 02:58:20 (11548): No heartbeat from core client for 30 sec - exiting 02:58:21 (11548): No heartbeat from core client for 30 sec - exiting 02:58:22 (11548): No heartbeat from core client for 30 sec - exiting 02:58:23 (11548): No heartbeat from core client for 30 sec - exiting 02:58:24 (11548): No heartbeat from core client for 30 sec - exiting 02:58:26 (11548): No heartbeat from core client for 30 sec - exiting 02:58:27 (11548): No heartbeat from core client for 30 sec - exiting 02:58:28 (11548): No heartbeat from core client for 30 sec - exiting 02:58:29 (11548): No heartbeat from core client for 30 sec - exiting 02:58:30 (11548): No heartbeat from core client for 30 sec - exiting 02:58:31 (11548): No heartbeat from core client for 30 sec - exiting 02:58:32 (11548): No heartbeat from core client for 30 sec - exiting 02:58:33 (11548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:07:39 (5944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:49:54 (11124): No heartbeat from core client for 30 sec - exiting 01:49:55 (11124): No heartbeat from core client for 30 sec - exiting 01:49:56 (11124): No heartbeat from core client for 30 sec - exiting 01:49:57 (11124): No heartbeat from core client for 30 sec - exiting 01:49:59 (11124): No heartbeat from core client for 30 sec - exiting 01:50:00 (11124): No heartbeat from core client for 30 sec - exiting 01:50:01 (11124): No heartbeat from core client for 30 sec - exiting 01:50:02 (11124): No heartbeat from core client for 30 sec - exiting 01:50:03 (11124): No heartbeat from core client for 30 sec - exiting 01:50:04 (11124): No heartbeat from core client for 30 sec - exiting 01:50:05 (11124): No heartbeat from core client for 30 sec - exiting 01:50:06 (11124): No heartbeat from core client for 30 sec - exiting 01:50:07 (11124): No heartbeat from core client for 30 sec - exiting 01:50:08 (11124): No heartbeat from core client for 30 sec - exiting 01:50:10 (11124): No heartbeat from core client for 30 sec - exiting 01:50:11 (11124): No heartbeat from core client for 30 sec - exiting 01:50:12 (11124): No heartbeat from core client for 30 sec - exiting 01:50:13 (11124): No heartbeat from core client for 30 sec - exiting 01:50:14 (11124): No heartbeat from core client for 30 sec - exiting 01:50:15 (11124): No heartbeat from core client for 30 sec - exiting 01:50:16 (11124): No heartbeat from core client for 30 sec - exiting 01:50:17 (11124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:50:18 (11124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 07:00:42 (7696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:18:23 (12588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:06:20 (10072): No heartbeat from core client for 30 sec - exiting 15:06:22 (10072): No heartbeat from core client for 30 sec - exiting 15:06:23 (10072): No heartbeat from core client for 30 sec - exiting 15:06:24 (10072): No heartbeat from core client for 30 sec - exiting 15:06:25 (10072): No heartbeat from core client for 30 sec - exiting 15:06:26 (10072): No heartbeat from core client for 30 sec - exiting 15:06:27 (10072): No heartbeat from core client for 30 sec - exiting 15:06:28 (10072): No heartbeat from core client for 30 sec - exiting 15:06:29 (10072): No heartbeat from core client for 30 sec - exiting 15:06:31 (10072): No heartbeat from core client for 30 sec - exiting 15:06:32 (10072): No heartbeat from core client for 30 sec - exiting 15:06:33 (10072): No heartbeat from core client for 30 sec - exiting 15:06:34 (10072): No heartbeat from core client for 30 sec - exiting 15:06:35 (10072): No heartbeat from core client for 30 sec - exiting 15:06:36 (10072): No heartbeat from core client for 30 sec - exiting 15:06:37 (10072): No heartbeat from core client for 30 sec - exiting 15:06:38 (10072): No heartbeat from core client for 30 sec - exiting 15:06:39 (10072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 06:59:12 (3864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 06:46:33 (11788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:59:13 (10324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:20:10 (3480): No heartbeat from core client for 30 sec - exiting 11:20:12 (3480): No heartbeat from core client for 30 sec - exiting 11:20:13 (3480): No heartbeat from core client for 30 sec - exiting 11:20:14 (3480): No heartbeat from core client for 30 sec - exiting 11:20:15 (3480): No heartbeat from core client for 30 sec - exiting 11:20:16 (3480): No heartbeat from core client for 30 sec - exiting 11:20:17 (3480): No heartbeat from core client for 30 sec - exiting 11:20:18 (3480): No heartbeat from core client for 30 sec - exiting 11:20:19 (3480): No heartbeat from core client for 30 sec - exiting 11:20:20 (3480): No heartbeat from core client for 30 sec - exiting 11:20:21 (3480): No heartbeat from core client for 30 sec - exiting 11:20:23 (3480): No heartbeat from core client for 30 sec - exiting 11:20:24 (3480): No heartbeat from core client for 30 sec - exiting 11:20:25 (3480): No heartbeat from core client for 30 sec - exiting 11:20:26 (3480): No heartbeat from core client for 30 sec - exiting 11:20:27 (3480): No heartbeat from core client for 30 sec - exiting 11:20:28 (3480): No heartbeat from core client for 30 sec - exiting 11:20:29 (3480): No heartbeat from core client for 30 sec - exiting 11:20:30 (3480): No heartbeat from core client for 30 sec - exiting 11:20:31 (3480): No heartbeat from core client for 30 sec - exiting 11:20:32 (3480): No heartbeat from core client for 30 sec - exiting 11:20:33 (3480): No heartbeat from core client for 30 sec - exiting 11:20:35 (3480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:23:48 (11616): No heartbeat from core client for 30 sec - exiting 18:23:49 (11616): No heartbeat from core client for 30 sec - exiting 18:23:50 (11616): No heartbeat from core client for 30 sec - exiting 18:23:51 (11616): No heartbeat from core client for 30 sec - exiting 18:23:52 (11616): No heartbeat from core client for 30 sec - exiting 18:23:53 (11616): No heartbeat from core client for 30 sec - exiting 18:23:54 (11616): No heartbeat from core client for 30 sec - exiting 18:23:55 (11616): No heartbeat from core client for 30 sec - exiting 18:23:56 (11616): No heartbeat from core client for 30 sec - exiting 18:23:57 (11616): No heartbeat from core client for 30 sec - exiting 18:23:59 (11616): No heartbeat from core client for 30 sec - exiting 18:24:00 (11616): No heartbeat from core client for 30 sec - exiting 18:24:01 (11616): No heartbeat from core client for 30 sec - exiting 18:24:02 (11616): No heartbeat from core client for 30 sec - exiting 18:24:03 (11616): No heartbeat from core client for 30 sec - exiting 18:24:04 (11616): No heartbeat from core client for 30 sec - exiting 18:24:05 (11616): No heartbeat from core client for 30 sec - exiting 18:24:06 (11616): No heartbeat from core client for 30 sec - exiting 18:24:07 (11616): No heartbeat from core client for 30 sec - exiting 18:24:08 (11616): No heartbeat from core client for 30 sec - exiting 18:24:09 (11616): No heartbeat from core client for 30 sec - exiting 18:24:11 (11616): No heartbeat from core client for 30 sec - exiting 18:24:12 (11616): No heartbeat from core client for 30 sec - exiting 18:24:13 (11616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Mar 2013 19:02:02	1233009	15667377	hadcm3n_zck5_1920_40_008281668_3	259,200	808,534	3.1193
29 Mar 2013 20:23:12	1233009	15667377	hadcm3n_zck5_1920_40_008281668_3	233,280	727,172	3.1172
28 Mar 2013 09:43:58	1233009	15667377	hadcm3n_zck5_1920_40_008281668_3	207,360	646,646	3.1185
26 Mar 2013 15:25:26	1233009	15667377	hadcm3n_zck5_1920_40_008281668_3	181,440	566,746	3.1236
25 Mar 2013 00:55:13	1233009	15667377	hadcm3n_zck5_1920_40_008281668_3	155,520	486,471	3.1280
23 Mar 2013 11:09:13	1233009	15667377	hadcm3n_zck5_1920_40_008281668_3	129,600	404,618	3.1221
21 Mar 2013 19:26:01	1233009	15667377	hadcm3n_zck5_1920_40_008281668_3	103,680	323,821	3.1233
20 Mar 2013 08:47:02	1233009	15667377	hadcm3n_zck5_1920_40_008281668_3	77,760	242,746	3.1217
18 Mar 2013 13:04:58	1233009	15667377	hadcm3n_zck5_1920_40_008281668_3	51,840	161,110	3.1078
16 Mar 2013 07:27:45	1233009	15667377	hadcm3n_zck5_1920_40_008281668_3	25,920	80,578	3.1087