Task 15458875

Name	hadcm3n_z9v7_1880_40_008252999_2
Workunit	8408123
Created	23 Nov 2012, 18:22:06 UTC
Sent	23 Nov 2012, 18:22:39 UTC
Report deadline	23 Feb 2013, 1:49:50 UTC
Received	17 Dec 2012, 10:45:48 UTC
Server state	Over
Outcome	Computation error
Client state	Done
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1254955
Run time	8 days 18 hours 36 min 37 sec
CPU time	8 days 18 hours 36 min 37 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	1.77 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>5.10.45</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 21:31:20 (4720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:41:35 (9120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:42:41 (6536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:42:42 (6536): No heartbeat from core client for 30 sec - exiting 00:42:43 (6536): No heartbeat from core client for 30 sec - exiting 00:42:44 (6536): No heartbeat from core client for 30 sec - exiting 00:42:45 (6536): No heartbeat from core client for 30 sec - exiting 00:42:46 (6536): No heartbeat from core client for 30 sec - exiting 00:42:47 (6536): No heartbeat from core client for 30 sec - exiting 00:42:48 (6536): No heartbeat from core client for 30 sec - exiting 00:42:50 (6536): No heartbeat from core client for 30 sec - exiting 00:42:51 (6536): No heartbeat from core client for 30 sec - exiting 00:42:52 (6536): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 00:37:17 (4128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:37:32 (4128): No heartbeat from core client for 30 sec - exiting 00:37:33 (4128): No heartbeat from core client for 30 sec - exiting 00:38:10 (5716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:38:12 (5716): No heartbeat from core client for 30 sec - exiting 00:38:13 (5716): No heartbeat from core client for 30 sec - exiting 00:38:14 (5716): No heartbeat from core client for 30 sec - exiting 00:38:15 (5716): No heartbeat from core client for 30 sec - exiting 00:38:16 (5716): No heartbeat from core client for 30 sec - exiting 00:38:17 (5716): No heartbeat from core client for 30 sec - exiting 00:38:18 (5716): No heartbeat from core client for 30 sec - exiting 00:38:56 (5692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:39:40 (5936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:34:28 (5516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:34:39 (5516): No heartbeat from core client for 30 sec - exiting 00:34:40 (5516): No heartbeat from core client for 30 sec - exiting 00:35:16 (5728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:35:17 (5728): No heartbeat from core client for 30 sec - exiting 00:35:19 (5728): No heartbeat from core client for 30 sec - exiting 00:35:20 (5728): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 18:25:08 (5096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:54:02 (664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:54:15 (664): No heartbeat from core client for 30 sec - exiting 19:54:16 (664): No heartbeat from core client for 30 sec - exiting 19:54:17 (664): No heartbeat from core client for 30 sec - exiting 19:54:50 (7732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:50:33 (7564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 02:13:27 (5852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:13:39 (5852): No heartbeat from core client for 30 sec - exiting 02:13:40 (5852): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 02:23:50 (8900): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 23:56:01 (1824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:56:50 (4216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:46:04 (7752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:46:08 (7752): No heartbeat from core client for 30 sec - exiting 01:46:09 (7752): No heartbeat from core client for 30 sec - exiting 01:46:10 (7752): No heartbeat from core client for 30 sec - exiting 01:46:11 (7752): No heartbeat from core client for 30 sec - exiting 01:46:12 (7752): No heartbeat from core client for 30 sec - exiting 01:52:40 (9624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:53:44 (9224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:35:01 (608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:35:18 (608): No heartbeat from core client for 30 sec - exiting 02:35:19 (608): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 02:54:41 (9416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:54:42 (9416): No heartbeat from core client for 30 sec - exiting 02:54:43 (9416): No heartbeat from core client for 30 sec - exiting 02:54:44 (9416): No heartbeat from core client for 30 sec - exiting 02:54:45 (9416): No heartbeat from core client for 30 sec - exiting 02:54:46 (9416): No heartbeat from core client for 30 sec - exiting 02:54:47 (9416): No heartbeat from core client for 30 sec - exiting 02:54:48 (9416): No heartbeat from core client for 30 sec - exiting 02:54:49 (9416): No heartbeat from core client for 30 sec - exiting 02:54:50 (9416): No heartbeat from core client for 30 sec - exiting 02:54:51 (9416): No heartbeat from core client for 30 sec - exiting forrtl: Access is denied. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9992, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 16:34:27 (8100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:34:30 (8100): No heartbeat from core client for 30 sec - exiting 16:34:31 (8100): No heartbeat from core client for 30 sec - exiting 16:34:32 (8100): No heartbeat from core client for 30 sec - exiting 16:34:33 (8100): No heartbeat from core client for 30 sec - exiting 16:34:34 (8100): No heartbeat from core client for 30 sec - exiting 16:34:35 (8100): No heartbeat from core client for 30 sec - exiting 16:34:36 (8100): No heartbeat from core client for 30 sec - exiting 16:34:37 (8100): No heartbeat from core client for 30 sec - exiting 16:36:20 (7204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:36:21 (7204): No heartbeat from core client for 30 sec - exiting 16:36:22 (7204): No heartbeat from core client for 30 sec - exiting 16:36:23 (7204): No heartbeat from core client for 30 sec - exiting 16:36:24 (7204): No heartbeat from core client for 30 sec - exiting 16:36:25 (7204): No heartbeat from core client for 30 sec - exiting 16:36:26 (7204): No heartbeat from core client for 30 sec - exiting 16:36:27 (7204): No heartbeat from core client for 30 sec - exiting 16:36:28 (7204): No heartbeat from core client for 30 sec - exiting 16:36:29 (7204): No heartbeat from core client for 30 sec - exiting 16:36:30 (7204): No heartbeat from core client for 30 sec - exiting 16:37:12 (5832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:37:47 (3484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:46:36 (6160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:40:55 (6744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:41:12 (6744): No heartbeat from core client for 30 sec - exiting 19:41:13 (6744): No heartbeat from core client for 30 sec - exiting 19:41:14 (6744): No heartbeat from core client for 30 sec - exiting 19:41:15 (6744): No heartbeat from core client for 30 sec - exiting 19:41:16 (6744): No heartbeat from core client for 30 sec - exiting 19:41:17 (6744): No heartbeat from core client for 30 sec - exiting 19:41:18 (6744): No heartbeat from core client for 30 sec - exiting 19:41:19 (6744): No heartbeat from core client for 30 sec - exiting 19:50:10 (7332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:50:36 (7332): No heartbeat from core client for 30 sec - exiting 19:50:37 (7332): No heartbeat from core client for 30 sec - exiting 19:50:38 (7332): No heartbeat from core client for 30 sec - exiting 19:50:39 (7332): No heartbeat from core client for 30 sec - exiting 19:50:40 (7332): No heartbeat from core client for 30 sec - exiting 19:55:49 (7592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:26:04 (5204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:19:20 (4756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:02:37 (8432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:03:21 (8252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:04:25 (7100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 10:37:51 (9324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:38:34 (9324): No heartbeat from core client for 30 sec - exiting 10:38:35 (9324): No heartbeat from core client for 30 sec - exiting 10:38:36 (9324): No heartbeat from core client for 30 sec - exiting 10:38:37 (9324): No heartbeat from core client for 30 sec - exiting 10:38:38 (9324): No heartbeat from core client for 30 sec - exiting 10:38:39 (9324): No heartbeat from core client for 30 sec - exiting 10:38:40 (9324): No heartbeat from core client for 30 sec - exiting 10:38:41 (9324): No heartbeat from core client for 30 sec - exiting 11:08:46 (7612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:12:17 (4296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:15:08 (7796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:15:12 (7796): No heartbeat from core client for 30 sec - exiting 11:15:13 (7796): No heartbeat from core client for 30 sec - exiting 11:15:48 (6304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:30:28 (7296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:31:05 (6680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:56:02 (3608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:56:09 (3608): No heartbeat from core client for 30 sec - exiting 04:56:10 (3608): No heartbeat from core client for 30 sec - exiting 04:56:11 (3608): No heartbeat from core client for 30 sec - exiting 04:56:12 (3608): No heartbeat from core client for 30 sec - exiting 04:56:13 (3608): No heartbeat from core client for 30 sec - exiting 13:54:22 (2788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:26:12 (3292): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Dec 2012 19:41:50	1254955	15458875	hadcm3n_z9v7_1880_40_008252999_2	259,200	758,199	2.9252
05 Dec 2012 20:55:07	1254955	15458875	hadcm3n_z9v7_1880_40_008252999_2	233,280	683,749	2.9310
05 Dec 2012 20:55:07	1254955	15458875	hadcm3n_z9v7_1880_40_008252999_2	207,360	606,704	2.9258
05 Dec 2012 20:55:07	1254955	15458875	hadcm3n_z9v7_1880_40_008252999_2	181,440	529,932	2.9207
05 Dec 2012 20:55:07	1254955	15458875	hadcm3n_z9v7_1880_40_008252999_2	155,520	454,715	2.9238
05 Dec 2012 20:55:07	1254955	15458875	hadcm3n_z9v7_1880_40_008252999_2	129,600	377,875	2.9157
05 Dec 2012 20:55:07	1254955	15458875	hadcm3n_z9v7_1880_40_008252999_2	103,680	302,011	2.9129
05 Dec 2012 20:55:07	1254955	15458875	hadcm3n_z9v7_1880_40_008252999_2	77,760	227,027	2.9196
27 Nov 2012 16:55:24	1254955	15458875	hadcm3n_z9v7_1880_40_008252999_2	51,840	152,821	2.9479
27 Nov 2012 16:55:24	1254955	15458875	hadcm3n_z9v7_1880_40_008252999_2	25,920	76,692	2.9588