Task 13366553

Name	hadcm3n_ylhi_1940_40_007450328_1
Workunit	7647831
Created	10 Sep 2011, 5:36:51 UTC
Sent	13 Sep 2011, 17:04:03 UTC
Report deadline	14 Dec 2011, 0:31:14 UTC
Received	2 Nov 2011, 19:42:35 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1052964
Run time	8 days 4 hours 34 min 32 sec
CPU time	7 days 11 hours 52 min 21 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.49 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 09:02:10 (6820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:09:54 (1640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:09:55 (1640): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 10:10:23 (6292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4224, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:16:24 (6492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:07:01 (8040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:07:02 (8040): No heartbeat from core client for 30 sec - exiting 09:12:15 (8064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:13:05 (1216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:31:25 (6920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:35:39 (4860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:54:53 (4500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:57:29 (10188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 13:51:51 (3144): No heartbeat from core client for 30 sec - exiting 13:51:53 (3144): No heartbeat from core client for 30 sec - exiting 13:51:54 (3144): No heartbeat from core client for 30 sec - exiting 13:51:55 (3144): No heartbeat from core client for 30 sec - exiting 13:51:56 (3144): No heartbeat from core client for 30 sec - exiting 13:51:57 (3144): No heartbeat from core client for 30 sec - exiting 13:51:58 (3144): No heartbeat from core client for 30 sec - exiting 13:51:59 (3144): No heartbeat from core client for 30 sec - exiting 13:52:00 (3144): No heartbeat from core client for 30 sec - exiting 13:52:01 (3144): No heartbeat from core client for 30 sec - exiting 13:52:02 (3144): No heartbeat from core client for 30 sec - exiting 13:52:03 (3144): No heartbeat from core client for 30 sec - exiting 13:52:05 (3144): No heartbeat from core client for 30 sec - exiting 13:52:06 (3144): No heartbeat from core client for 30 sec - exiting 13:52:07 (3144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1256, iMonCtr=1 Model crash detected, will try to restart... 05:49:42 (4300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:37:42 (2480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:15:58 (5212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:48:03 (3768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:20:25 (6304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:34:47 (1624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:17:30 (696): No heartbeat from core client for 30 sec - exiting 14:17:31 (696): No heartbeat from core client for 30 sec - exiting 14:17:32 (696): No heartbeat from core client for 30 sec - exiting 14:17:33 (696): No heartbeat from core client for 30 sec - exiting 14:17:34 (696): No heartbeat from core client for 30 sec - exiting 14:17:36 (696): No heartbeat from core client for 30 sec - exiting 14:17:37 (696): No heartbeat from core client for 30 sec - exiting 14:17:38 (696): No heartbeat from core client for 30 sec - exiting 14:17:39 (696): No heartbeat from core client for 30 sec - exiting 14:17:40 (696): No heartbeat from core client for 30 sec - exiting 14:17:41 (696): No heartbeat from core client for 30 sec - exiting 14:17:42 (696): No heartbeat from core client for 30 sec - exiting 14:17:43 (696): No heartbeat from core client for 30 sec - exiting 14:17:44 (696): No heartbeat from core client for 30 sec - exiting 14:17:45 (696): No heartbeat from core client for 30 sec - exiting 14:17:47 (696): No heartbeat from core client for 30 sec - exiting 14:17:48 (696): No heartbeat from core client for 30 sec - exiting 14:17:49 (696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:26:45 (8096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:26:47 (8096): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 12:55:05 (9100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 15:35:53 (5844): No heartbeat from core client for 30 sec - exiting 15:35:54 (5844): No heartbeat from core client for 30 sec - exiting 15:35:55 (5844): No heartbeat from core client for 30 sec - exiting 15:35:56 (5844): No heartbeat from core client for 30 sec - exiting 15:35:57 (5844): No heartbeat from core client for 30 sec - exiting 15:35:59 (5844): No heartbeat from core client for 30 sec - exiting 15:36:00 (5844): No heartbeat from core client for 30 sec - exiting 15:36:01 (5844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6132, iMonCtr=1 Model crash detected, will try to restart... 06:04:20 (6528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:31:16 (7608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:17:20 (3356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:43:04 (5636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6780, iMonCtr=1 Model crash detected, will try to restart... 10:15:03 (6452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:30:37 (6560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:04:24 (1564): No heartbeat from core client for 30 sec - exiting 22:04:25 (1564): No heartbeat from core client for 30 sec - exiting 22:04:26 (1564): No heartbeat from core client for 30 sec - exiting 22:04:27 (1564): No heartbeat from core client for 30 sec - exiting 22:04:31 (1564): No heartbeat from core client for 30 sec - exiting 22:04:32 (1564): No heartbeat from core client for 30 sec - exiting 22:04:33 (1564): No heartbeat from core client for 30 sec - exiting 22:04:34 (1564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:36:24 (7504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:32:12 (4244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:32:36 (5272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 11:36:31 (5888): No heartbeat from core client for 30 sec - exiting 11:36:32 (5888): No heartbeat from core client for 30 sec - exiting 11:36:33 (5888): No heartbeat from core client for 30 sec - exiting 11:36:34 (5888): No heartbeat from core client for 30 sec - exiting 11:36:35 (5888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:00:05 (7628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:22:19 (7500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:17:53 (6316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:18:45 (7744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:01:30 (3276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:04:35 (5112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:06:55 (1404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:56:25 (6004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6396, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5500, iMonCtr=1 Model crash detected, will try to restart... 13:39:07 (6120): No heartbeat from core client for 30 sec - exiting 13:39:08 (6120): No heartbeat from core client for 30 sec - exiting 13:39:09 (6120): No heartbeat from core client for 30 sec - exiting 13:39:10 (6120): No heartbeat from core client for 30 sec - exiting 13:39:11 (6120): No heartbeat from core client for 30 sec - exiting 13:39:12 (6120): No heartbeat from core client for 30 sec - exiting 13:39:13 (6120): No heartbeat from core client for 30 sec - exiting 13:39:14 (6120): No heartbeat from core client for 30 sec - exiting 13:39:15 (6120): No heartbeat from core client for 30 sec - exiting 13:39:16 (6120): No heartbeat from core client for 30 sec - exiting 13:39:18 (6120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:52:05 (7724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:29:08 (5792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:29:50 (7908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:30:47 (7088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:39:08 (2300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:56:18 (6708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Nov 2011 09:39:24	1052964	13366553	hadcm3n_ylhi_1940_40_007450328_1	259,200	647,519	2.4981
31 Oct 2011 18:26:49	1052964	13366553	hadcm3n_ylhi_1940_40_007450328_1	233,280	579,393	2.4837
31 Oct 2011 15:05:50	1052964	13366553	hadcm3n_ylhi_1940_40_007450328_1	207,360	521,375	2.5143
31 Oct 2011 15:05:50	1052964	13366553	hadcm3n_ylhi_1940_40_007450328_1	181,440	465,275	2.5643
17 Oct 2011 16:04:40	1052964	13366553	hadcm3n_ylhi_1940_40_007450328_1	155,520	400,452	2.5749
04 Oct 2011 19:45:45	1052964	13366553	hadcm3n_ylhi_1940_40_007450328_1	129,600	333,852	2.5760
29 Sep 2011 05:20:02	1052964	13366553	hadcm3n_ylhi_1940_40_007450328_1	103,680	270,369	2.6077
26 Sep 2011 05:37:49	1052964	13366553	hadcm3n_ylhi_1940_40_007450328_1	77,760	199,408	2.5644
22 Sep 2011 05:40:47	1052964	13366553	hadcm3n_ylhi_1940_40_007450328_1	51,840	138,654	2.6747
19 Sep 2011 12:55:55	1052964	13366553	hadcm3n_ylhi_1940_40_007450328_1	25,920	71,789	2.7696