Task 15810980

Name	hadcm3n_o3xg_1940_40_008379976_0
Workunit	8530835
Created	31 May 2013, 20:07:44 UTC
Sent	17 Jun 2013, 19:01:20 UTC
Report deadline	17 Sep 2013, 2:28:31 UTC
Received	14 Aug 2013, 19:05:25 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1246737
Run time	10 days 2 hours 49 min 56 sec
CPU time	9 days 7 hours 7 min 26 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.22 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:50:25 (372): No heartbeat from core client for 30 sec - exiting 14:50:26 (372): No heartbeat from core client for 30 sec - exiting 14:50:27 (372): No heartbeat from core client for 30 sec - exiting 14:50:28 (372): No heartbeat from core client for 30 sec - exiting 14:50:29 (372): No heartbeat from core client for 30 sec - exiting 14:50:30 (372): No heartbeat from core client for 30 sec - exiting 14:50:31 (372): No heartbeat from core client for 30 sec - exiting 14:50:32 (372): No heartbeat from core client for 30 sec - exiting 14:50:33 (372): No heartbeat from core client for 30 sec - exiting 14:50:34 (372): No heartbeat from core client for 30 sec - exiting 14:50:35 (372): No heartbeat from core client for 30 sec - exiting 14:50:36 (372): No heartbeat from core client for 30 sec - exiting 14:50:37 (372): No heartbeat from core client for 30 sec - exiting 14:50:38 (372): No heartbeat from core client for 30 sec - exiting 14:50:39 (372): No heartbeat from core client for 30 sec - exiting 14:50:40 (372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:50:41 (372): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:30:58 (4216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:52:37 (4008): No heartbeat from core client for 30 sec - exiting 09:52:39 (4008): No heartbeat from core client for 30 sec - exiting 09:52:40 (4008): No heartbeat from core client for 30 sec - exiting 09:52:42 (4008): No heartbeat from core client for 30 sec - exiting 09:52:43 (4008): No heartbeat from core client for 30 sec - exiting 09:52:44 (4008): No heartbeat from core client for 30 sec - exiting 09:52:45 (4008): No heartbeat from core client for 30 sec - exiting 09:52:46 (4008): No heartbeat from core client for 30 sec - exiting 09:52:47 (4008): No heartbeat from core client for 30 sec - exiting 09:52:48 (4008): No heartbeat from core client for 30 sec - exiting 09:52:49 (4008): No heartbeat from core client for 30 sec - exiting 09:52:50 (4008): No heartbeat from core client for 30 sec - exiting 09:52:51 (4008): No heartbeat from core client for 30 sec - exiting 09:52:52 (4008): No heartbeat from core client for 30 sec - exiting 09:52:54 (4008): No heartbeat from core client for 30 sec - exiting 09:52:55 (4008): No heartbeat from core client for 30 sec - exiting 09:52:56 (4008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... 10:03:29 (4560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:03:31 (4560): No heartbeat from core client for 30 sec - exiting 10:03:32 (4560): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:07:54 (5048): No heartbeat from core client for 30 sec - exiting 16:07:55 (5048): No heartbeat from core client for 30 sec - exiting 16:07:56 (5048): No heartbeat from core client for 30 sec - exiting 16:07:57 (5048): No heartbeat from core client for 30 sec - exiting 16:07:59 (5048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:39:46 (3356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:39:47 (3356): No heartbeat from core client for 30 sec - exiting 09:40:24 (2520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:36:31 (2492): No heartbeat from core client for 30 sec - exiting 12:36:33 (2492): No heartbeat from core client for 30 sec - exiting 12:36:34 (2492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:36:35 (2492): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... C07:35:43 (2296): No heartbeat from core client for 30 sec - exiting 07:35:45 (2296): No heartbeat from core client for 30 sec - exiting 07:35:46 (2296): No heartbeat from core client for 30 sec - exiting 07:35:47 (2296): No heartbeat from core client for 30 sec - exiting 07:35:48 (2296): No heartbeat from core client for 30 sec - exiting 07:35:49 (2296): No heartbeat from core client for 30 sec - exiting 07:35:50 (2296): No heartbeat from core client for 30 sec - exiting 07:35:51 (2296): No heartbeat from core client for 30 sec - exiting 07:35:52 (2296): No heartbeat from core client for 30 sec - exiting 07:35:54 (2296): No heartbeat from core client for 30 sec - exiting 07:35:55 (2296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:36:56 (1300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:42:54 (3144): No heartbeat from core client for 30 sec - exiting 15:42:55 (3144): No heartbeat from core client for 30 sec - exiting 15:42:56 (3144): No heartbeat from core client for 30 sec - exiting 15:42:57 (3144): No heartbeat from core client for 30 sec - exiting 15:42:59 (3144): No heartbeat from core client for 30 sec - exiting 15:43:00 (3144): No heartbeat from core client for 30 sec - exiting 15:43:01 (3144): No heartbeat from core client for 30 sec - exiting 15:43:02 (3144): No heartbeat from core client for 30 sec - exiting 15:43:03 (3144): No heartbeat from core client for 30 sec - exiting 15:43:04 (3144): No heartbeat from core client for 30 sec - exiting 15:43:05 (3144): No heartbeat from core client for 30 sec - exiting 15:43:06 (3144): No heartbeat from core client for 30 sec - exiting 15:43:07 (3144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:43:08 (3144): No heartbeat from core client for 30 sec - exiting 15:43:09 (3144): No heartbeat from core client for 30 sec - exiting 15:43:11 (3144): No heartbeat from core client for 30 sec - exiting 15:43:12 (3144): No heartbeat from core client for 30 sec - exiting 15:43:13 (3144): No heartbeat from core client for 30 sec - exiting 15:43:14 (3144): No heartbeat from core client for 30 sec - exiting 15:44:04 (4100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:35:40 (3744): Can't acquire lockfile (32) - waiting 35s 16:35:50 (3956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:03:51 (3816): No heartbeat from core client for 30 sec - exiting 09:03:53 (3816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:20:53 (4012): No heartbeat from core client for 30 sec - exiting 08:20:54 (4012): No heartbeat from core client for 30 sec - exiting 08:20:55 (4012): No heartbeat from core client for 30 sec - exiting 08:20:56 (4012): No heartbeat from core client for 30 sec - exiting 08:20:57 (4012): No heartbeat from core client for 30 sec - exiting 08:20:58 (4012): No heartbeat from core client for 30 sec - exiting 08:20:59 (4012): No heartbeat from core client for 30 sec - exiting 08:21:01 (4012): No heartbeat from core client for 30 sec - exiting 08:21:02 (4012): No heartbeat from core client for 30 sec - exiting 08:21:03 (4012): No heartbeat from core client for 30 sec - exiting 08:21:04 (4012): No heartbeat from core client for 30 sec - exiting 08:21:05 (4012): No heartbeat from core client for 30 sec - exiting 08:21:06 (4012): No heartbeat from core client for 30 sec - exiting 08:21:07 (4012): No heartbeat from core client for 30 sec - exiting 08:21:08 (4012): No heartbeat from core client for 30 sec - exiting 08:21:09 (4012): No heartbeat from core client for 30 sec - exiting 08:21:10 (4012): No heartbeat from core client for 30 sec - exiting 08:21:11 (4012): No heartbeat from core client for 30 sec - exiting 08:21:13 (4012): No heartbeat from core client for 30 sec - exiting 08:21:14 (4012): No heartbeat from core client for 30 sec - exiting 08:21:15 (4012): No heartbeat from core client for 30 sec - exiting 08:21:16 (4012): No heartbeat from core client for 30 sec - exiting 08:21:17 (4012): No heartbeat from core client for 30 sec - exiting 08:21:18 (4012): No heartbeat from core client for 30 sec - exiting 08:21:19 (4012): No heartbeat from core client for 30 sec - exiting 08:21:20 (4012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:21:21 (4012): No heartbeat from core client for 30 sec - exiting 08:21:22 (4012): No heartbeat from core client for 30 sec - exiting 08:22:11 (3232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4056, iMonCtr=1 Model crash detected, will try to restart... 12:29:46 (4084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4780, iMonCtr=1 Model crash detected, will try to restart... 10:36:10 (3260): No heartbeat from core client for 30 sec - exiting 10:36:11 (3260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:36:12 (3260): No heartbeat from core client for 30 sec - exiting 10:37:04 (3784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4376, iMonCtr=1 Model crash detected, will try to restart... 10:49:54 (4372): No heartbeat from core client for 30 sec - exiting 10:49:56 (4372): No heartbeat from core client for 30 sec - exiting 10:49:58 (4372): No heartbeat from core client for 30 sec - exiting 10:49:59 (4372): No heartbeat from core client for 30 sec - exiting 10:50:00 (4372): No heartbeat from core client for 30 sec - exiting 10:50:01 (4372): No heartbeat from core client for 30 sec - exiting 10:50:02 (4372): No heartbeat from core client for 30 sec - exiting 10:50:03 (4372): No heartbeat from core client for 30 sec - exiting 10:50:04 (4372): No heartbeat from core client for 30 sec - exiting 10:50:05 (4372): No heartbeat from core client for 30 sec - exiting 10:50:06 (4372): No heartbeat from core client for 30 sec - exiting 10:50:07 (4372): No heartbeat from core client for 30 sec - exiting 10:50:09 (4372): No heartbeat from core client for 30 sec - exiting 10:50:10 (4372): No heartbeat from core client for 30 sec - exiting 10:50:11 (4372): No heartbeat from core client for 30 sec - exiting 10:50:12 (4372): No heartbeat from core client for 30 sec - exiting 10:50:13 (4372): No heartbeat from core client for 30 sec - exiting 10:50:14 (4372): No heartbeat from core client for 30 sec - exiting 10:50:15 (4372): No heartbeat from core client for 30 sec - exiting 10:50:16 (4372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:50:17 (4372): No heartbeat from core client for 30 sec - exiting 10:50:18 (4372): No heartbeat from core client for 30 sec - exiting 10:50:19 (4372): No heartbeat from core client for 30 sec - exiting 10:50:21 (4372): No heartbeat from core client for 30 sec - exiting 10:50:22 (4372): No heartbeat from core client for 30 sec - exiting 10:50:23 (4372): No heartbeat from core client for 30 sec - exiting 10:50:24 (4372): No heartbeat from core client for 30 sec - exiting 10:50:25 (4372): No heartbeat from core client for 30 sec - exiting 10:50:26 (4372): No heartbeat from core client for 30 sec - exiting 10:50:27 (4372): No heartbeat from core client for 30 sec - exiting 10:51:41 (2652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:09:11 (4132): No heartbeat from core client for 30 sec - exiting 08:09:12 (4132): No heartbeat from core client for 30 sec - exiting 08:09:13 (4132): No heartbeat from core client for 30 sec - exiting 08:09:14 (4132): No heartbeat from core client for 30 sec - exiting 08:09:15 (4132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:09:16 (4132): No heartbeat from core client for 30 sec - exiting 08:09:17 (4132): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3416, iMonCtr=1 Model crash detected, will try to restart... 08:26:40 (4776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:28:22 (3520): No heartbeat from core client for 30 sec - exiting 14:28:23 (3520): No heartbeat from core client for 30 sec - exiting 14:28:24 (3520): No heartbeat from core client for 30 sec - exiting 14:28:25 (3520): No heartbeat from core client for 30 sec - exiting 14:28:26 (3520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 09:21:29 (3412): No heartbeat from core client for 30 sec - exiting 09:21:30 (3412): No heartbeat from core client for 30 sec - exiting 09:21:31 (3412): No heartbeat from core client for 30 sec - exiting 09:21:32 (3412): No heartbeat from core client for 30 sec - exiting 09:21:34 (3412): No heartbeat from core client for 30 sec - exiting 09:21:35 (3412): No heartbeat from core client for 30 sec - exiting 09:21:36 (3412): No heartbeat from core client for 30 sec - exiting 09:21:37 (3412): No heartbeat from core client for 30 sec - exiting 09:21:38 (3412): No heartbeat from core client for 30 sec - exiting 09:21:39 (3412): No heartbeat from core client for 30 sec - exiting 09:21:40 (3412): No heartbeat from core client for 30 sec - exiting 09:21:41 (3412): No heartbeat from core client for 30 sec - exiting 09:21:42 (3412): No heartbeat from core client for 30 sec - exiting 09:21:44 (3412): No heartbeat from core client for 30 sec - exiting 09:21:45 (3412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:21:46 (3412): No heartbeat from core client for 30 sec - exiting 16:13:10 (2100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:13:11 (2100): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2784, iMonCtr=1 Model crash detected, will try to restart... 09:15:11 (3988): No heartbeat from core client for 30 sec - exiting 09:15:13 (3988): No heartbeat from core client for 30 sec - exiting 09:15:14 (3988): No heartbeat from core client for 30 sec - exiting 09:15:15 (3988): No heartbeat from core client for 30 sec - exiting 09:15:16 (3988): No heartbeat from core client for 30 sec - exiting 09:15:17 (3988): No heartbeat from core client for 30 sec - exiting 09:15:18 (3988): No heartbeat from core client for 30 sec - exiting 09:15:20 (3988): No heartbeat from core client for 30 sec - exiting 09:15:21 (3988): No heartbeat from core client for 30 sec - exiting 09:15:22 (3988): No heartbeat from core client for 30 sec - exiting 09:15:23 (3988): No heartbeat from core client for 30 sec - exiting 09:15:24 (3988): No heartbeat from core client for 30 sec - exiting 09:15:25 (3988): No heartbeat from core client for 30 sec - exiting 09:15:26 (3988): No heartbeat from core client for 30 sec - exiting 09:15:27 (3988): No heartbeat from core client for 30 sec - exiting 09:15:28 (3988): No heartbeat from core client for 30 sec - exiting 09:15:29 (3988): No heartbeat from core client for 30 sec - exiting 09:15:30 (3988): No heartbeat from core client for 30 sec - exiting 09:15:32 (3988): No heartbeat from core client for 30 sec - exiting 09:15:33 (3988): No heartbeat from core client for 30 sec - exiting 09:15:34 (3988): No heartbeat from core client for 30 sec - exiting 09:15:35 (3988): No heartbeat from core client for 30 sec - exiting 09:15:36 (3988): No heartbeat from core client for 30 sec - exiting 09:15:37 (3988): No heartbeat from core client for 30 sec - exiting 09:15:38 (3988): No heartbeat from core client for 30 sec - exiting 09:15:39 (3988): No heartbeat from core client for 30 sec - exiting 09:15:40 (3988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4324, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Aug 2013 19:08:45	1246737	15810980	hadcm3n_o3xg_1940_40_008379976_0	259,200	803,238	3.0989
14 Aug 2013 19:08:45	1246737	15810980	hadcm3n_o3xg_1940_40_008379976_0	233,280	716,827	3.0728
14 Aug 2013 19:08:45	1246737	15810980	hadcm3n_o3xg_1940_40_008379976_0	207,360	637,741	3.0755
14 Aug 2013 19:08:45	1246737	15810980	hadcm3n_o3xg_1940_40_008379976_0	181,440	557,845	3.0745
14 Aug 2013 19:08:45	1246737	15810980	hadcm3n_o3xg_1940_40_008379976_0	155,520	472,684	3.0394
14 Aug 2013 19:08:45	1246737	15810980	hadcm3n_o3xg_1940_40_008379976_0	129,600	388,194	2.9953
30 Jul 2013 10:06:40	1246737	15810980	hadcm3n_o3xg_1940_40_008379976_0	103,680	309,508	2.9852
30 Jul 2013 09:23:46	1246737	15810980	hadcm3n_o3xg_1940_40_008379976_0	77,760	232,242	2.9867
26 Jul 2013 14:59:31	1246737	15810980	hadcm3n_o3xg_1940_40_008379976_0	51,840	151,446	2.9214
23 Jul 2013 23:44:54	1246737	15810980	hadcm3n_o3xg_1940_40_008379976_0	25,920	75,945	2.9300