Task 14617157

Name	hadcm3n_1052_1940_40_007954803_1
Workunit	8109915
Created	2 May 2012, 11:08:57 UTC
Sent	2 May 2012, 11:10:22 UTC
Report deadline	1 Aug 2012, 18:37:33 UTC
Received	12 May 2012, 14:46:21 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1184097
Run time	5 days 11 hours 41 min 43 sec
CPU time	5 days 1 hours 0 min 21 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.50 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5412, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6924, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4872, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5096, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:28:37 (1252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:28:38 (1252): No heartbeat from core client for 30 sec - exiting 12:28:39 (1252): No heartbeat from core client for 30 sec - exiting CSuspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=772, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 11:49:38 (1552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:49:39 (1552): No heartbeat from core client for 30 sec - exiting 11:49:40 (1552): No heartbeat from core client for 30 sec - exiting 11:49:41 (1552): No heartbeat from core client for 30 sec - exiting 11:49:42 (1552): No heartbeat from core client for 30 sec - exiting 11:49:43 (1552): No heartbeat from core client for 30 sec - exiting 11:51:05 (1052): No heartbeat from core client for 30 sec - exiting 11:51:06 (1052): No heartbeat from core client for 30 sec - exiting 11:51:07 (1052): No heartbeat from core client for 30 sec - exiting 11:51:08 (1052): No heartbeat from core client for 30 sec - exiting 11:51:09 (1052): No heartbeat from core client for 30 sec - exiting 11:51:10 (1052): No heartbeat from core client for 30 sec - exiting 11:51:11 (1052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:54:51 (5712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:54:52 (5712): No heartbeat from core client for 30 sec - exiting 11:54:53 (5712): No heartbeat from core client for 30 sec - exiting 11:54:54 (5712): No heartbeat from core client for 30 sec - exiting 11:54:55 (5712): No heartbeat from core client for 30 sec - exiting 11:54:56 (5712): No heartbeat from core client for 30 sec - exiting 11:54:57 (5712): No heartbeat from core client for 30 sec - exiting 11:54:58 (5712): No heartbeat from core client for 30 sec - exiting 11:54:59 (5712): No heartbeat from core client for 30 sec - exiting 11:55:00 (5712): No heartbeat from core client for 30 sec - exiting 11:55:01 (5712): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4728, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3692, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... COcean Restart file copy failed on 1052ko.dae8bk0 Suspended CPDN Monitor - Suspend request from BOINC... 15:43:29 (3724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:43:30 (3724): No heartbeat from core client for 30 sec - exiting 15:43:31 (3724): No heartbeat from core client for 30 sec - exiting 15:43:32 (3724): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CSuspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:50:34 (1848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4712, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2220, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2596, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:02:00 (5588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:02:02 (5588): No heartbeat from core client for 30 sec - exiting 17:02:03 (5588): No heartbeat from core client for 30 sec - exiting 17:02:04 (5588): No heartbeat from core client for 30 sec - exiting 17:02:05 (5588): No heartbeat from core client for 30 sec - exiting 17:02:06 (5588): No heartbeat from core client for 30 sec - exiting 17:02:07 (5588): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 May 2012 13:22:10	1184097	14617157	hadcm3n_1052_1940_40_007954803_1	259,200	435,619	1.6806
11 May 2012 10:34:38	1184097	14617157	hadcm3n_1052_1940_40_007954803_1	233,280	392,675	1.6833
09 May 2012 12:54:06	1184097	14617157	hadcm3n_1052_1940_40_007954803_1	207,360	346,908	1.6730
07 May 2012 14:40:10	1184097	14617157	hadcm3n_1052_1940_40_007954803_1	181,440	295,107	1.6265
06 May 2012 07:36:27	1184097	14617157	hadcm3n_1052_1940_40_007954803_1	155,520	252,385	1.6228
05 May 2012 04:48:07	1184097	14617157	hadcm3n_1052_1940_40_007954803_1	129,600	210,909	1.6274
04 May 2012 16:24:37	1184097	14617157	hadcm3n_1052_1940_40_007954803_1	103,680	169,442	1.6343
04 May 2012 04:20:05	1184097	14617157	hadcm3n_1052_1940_40_007954803_1	77,760	128,150	1.6480
03 May 2012 13:00:06	1184097	14617157	hadcm3n_1052_1940_40_007954803_1	51,840	84,223	1.6247
02 May 2012 23:45:24	1184097	14617157	hadcm3n_1052_1940_40_007954803_1	25,920	41,694	1.6086