Task 16014899

Name	hadcm3n_o50k_1940_40_008381830_2
Workunit	8532689
Created	13 Sep 2013, 14:05:49 UTC
Sent	13 Sep 2013, 14:16:21 UTC
Report deadline	13 Dec 2013, 21:43:32 UTC
Received	18 Oct 2013, 21:06:40 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1196485
Run time	8 days 15 hours 35 min 10 sec
CPU time	7 days 23 hours 53 min 59 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.42 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4104, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7036, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2768, iMonCtr=1 Model crash detected, will try to restart... 08:07:26 (2320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2808, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2688, iMonCtr=1 Model crash detected, will try to restart... 07:36:18 (5860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:04:53 (5328): No heartbeat from core client for 30 sec - exiting 08:04:54 (5328): No heartbeat from core client for 30 sec - exiting 08:04:55 (5328): No heartbeat from core client for 30 sec - exiting 08:04:56 (5328): No heartbeat from core client for 30 sec - exiting 08:04:58 (5328): No heartbeat from core client for 30 sec - exiting 08:04:59 (5328): No heartbeat from core client for 30 sec - exiting 08:05:00 (5328): No heartbeat from core client for 30 sec - exiting 08:05:01 (5328): No heartbeat from core client for 30 sec - exiting 08:05:02 (5328): No heartbeat from core client for 30 sec - exiting 08:05:03 (5328): No heartbeat from core client for 30 sec - exiting 08:05:04 (5328): No heartbeat from core client for 30 sec - exiting 08:05:05 (5328): No heartbeat from core client for 30 sec - exiting 08:05:06 (5328): No heartbeat from core client for 30 sec - exiting 08:05:07 (5328): No heartbeat from core client for 30 sec - exiting 08:05:08 (5328): No heartbeat from core client for 30 sec - exiting 08:05:10 (5328): No heartbeat from core client for 30 sec - exiting 08:05:11 (5328): No heartbeat from core client for 30 sec - exiting 08:05:12 (5328): No heartbeat from core client for 30 sec - exiting 08:05:13 (5328): No heartbeat from core client for 30 sec - exiting 08:05:14 (5328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:01:17 (3548): No heartbeat from core client for 30 sec - exiting 11:01:18 (3548): No heartbeat from core client for 30 sec - exiting 11:01:19 (3548): No heartbeat from core client for 30 sec - exiting 11:01:20 (3548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6896, iMonCtr=1 Model crash detected, will try to restart... 09:55:39 (7580): No heartbeat from core client for 30 sec - exiting 09:55:40 (7580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6024, iMonCtr=1 Model crash detected, will try to restart... 08:24:39 (8152): No heartbeat from core client for 30 sec - exiting 08:24:40 (8152): No heartbeat from core client for 30 sec - exiting 08:24:41 (8152): No heartbeat from core client for 30 sec - exiting 08:24:42 (8152): No heartbeat from core client for 30 sec - exiting 08:24:43 (8152): No heartbeat from core client for 30 sec - exiting 08:24:44 (8152): No heartbeat from core client for 30 sec - exiting 08:24:45 (8152): No heartbeat from core client for 30 sec - exiting 08:24:46 (8152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:16:43 (9076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:44:23 (6008): No heartbeat from core client for 30 sec - exiting 07:44:25 (6008): No heartbeat from core client for 30 sec - exiting 07:44:26 (6008): No heartbeat from core client for 30 sec - exiting 07:44:27 (6008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:15:30 (3828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:15:29 (3996): No heartbeat from core client for 30 sec - exiting 08:15:30 (3996): No heartbeat from core client for 30 sec - exiting 08:15:31 (3996): No heartbeat from core client for 30 sec - exiting 08:15:32 (3996): No heartbeat from core client for 30 sec - exiting 08:15:33 (3996): No heartbeat from core client for 30 sec - exiting 08:15:34 (3996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:15:35 (3996): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:17:53 (2104): No heartbeat from core client for 30 sec - exiting 10:17:54 (2104): No heartbeat from core client for 30 sec - exiting 10:17:56 (2104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:20:12 (4796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:37:34 (3576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:03:44 (1052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3224, iMonCtr=1 Model crash detected, will try to restart... 10:35:59 (5824): No heartbeat from core client for 30 sec - exiting 10:36:00 (5824): No heartbeat from core client for 30 sec - exiting 10:36:01 (5824): No heartbeat from core client for 30 sec - exiting 10:36:02 (5824): No heartbeat from core client for 30 sec - exiting 10:36:03 (5824): No heartbeat from core client for 30 sec - exiting 10:36:04 (5824): No heartbeat from core client for 30 sec - exiting 10:36:05 (5824): No heartbeat from core client for 30 sec - exiting 10:36:06 (5824): No heartbeat from core client for 30 sec - exiting 10:36:07 (5824): No heartbeat from core client for 30 sec - exiting 10:36:09 (5824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:46:07 (1452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:22:56 (5620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2196, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Oct 2013 20:10:33	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	518,400	690,832	1.3326
17 Oct 2013 19:09:37	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	492,480	657,388	1.3349
16 Oct 2013 15:17:20	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	466,560	625,597	1.3409
16 Oct 2013 04:13:13	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	440,640	590,126	1.3392
14 Oct 2013 17:16:35	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	414,720	559,386	1.3488
10 Oct 2013 16:35:45	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	388,800	527,007	1.3555
08 Oct 2013 19:45:58	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	362,880	493,861	1.3609
07 Oct 2013 17:54:01	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	336,960	463,432	1.3753
04 Oct 2013 19:27:42	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	311,040	431,152	1.3862
03 Oct 2013 13:54:10	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	285,120	398,446	1.3975
01 Oct 2013 20:53:25	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	259,200	364,654	1.4068
30 Sep 2013 16:57:16	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	233,280	326,999	1.4017
27 Sep 2013 21:29:58	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	207,360	295,122	1.4232
26 Sep 2013 17:21:08	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	181,440	258,499	1.4247
25 Sep 2013 13:07:28	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	155,520	224,318	1.4424
23 Sep 2013 18:07:02	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	129,600	186,132	1.4362
20 Sep 2013 15:17:40	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	103,680	147,379	1.4215
18 Sep 2013 19:38:26	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	77,760	108,803	1.3992
17 Sep 2013 17:47:59	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	51,840	70,949	1.3686
16 Sep 2013 16:39:05	1196485	16014899	hadcm3n_o50k_1940_40_008381830_2	25,920	35,799	1.3811