Task 15790837

Name	hadcm3n_u4mv_2020_40_008369719_0
Workunit	8520578
Created	20 May 2013, 15:10:59 UTC
Sent	20 May 2013, 15:11:19 UTC
Report deadline	19 Aug 2013, 22:38:30 UTC
Received	2 Jun 2013, 17:46:50 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1072815
Run time	5 days 7 hours 9 min 35 sec
CPU time	5 days 2 hours 23 min 19 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.68 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 18:36:14 (5164): No heartbeat from core client for 30 sec - exiting 18:36:15 (5164): No heartbeat from core client for 30 sec - exiting 18:36:16 (5164): No heartbeat from core client for 30 sec - exiting 18:36:17 (5164): No heartbeat from core client for 30 sec - exiting 18:36:18 (5164): No heartbeat from core client for 30 sec - exiting 18:36:19 (5164): No heartbeat from core client for 30 sec - exiting 18:36:20 (5164): No heartbeat from core client for 30 sec - exiting 18:36:21 (5164): No heartbeat from core client for 30 sec - exiting 18:36:22 (5164): No heartbeat from core client for 30 sec - exiting 18:36:23 (5164): No heartbeat from core client for 30 sec - exiting 18:36:24 (5164): No heartbeat from core client for 30 sec - exiting 18:36:26 (5164): No heartbeat from core client for 30 sec - exiting 18:36:27 (5164): No heartbeat from core client for 30 sec - exiting 18:36:28 (5164): No heartbeat from core client for 30 sec - exiting 18:36:29 (5164): No heartbeat from core client for 30 sec - exiting 18:36:30 (5164): No heartbeat from core client for 30 sec - exiting 18:36:31 (5164): No heartbeat from core client for 30 sec - exiting 18:36:32 (5164): No heartbeat from core client for 30 sec - exiting 18:36:33 (5164): No heartbeat from core client for 30 sec - exiting 18:36:34 (5164): No heartbeat from core client for 30 sec - exiting 18:36:35 (5164): No heartbeat from core client for 30 sec - exiting 18:36:36 (5164): No heartbeat from core client for 30 sec - exiting 18:36:38 (5164): No heartbeat from core client for 30 sec - exiting 18:36:39 (5164): No heartbeat from core client for 30 sec - exiting 18:36:40 (5164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:36:41 (5164): No heartbeat from core client for 30 sec - exiting 18:36:42 (5164): No heartbeat from core client for 30 sec - exiting 18:36:43 (5164): No heartbeat from core client for 30 sec - exiting 18:36:44 (5164): No heartbeat from core client for 30 sec - exiting 18:36:45 (5164): No heartbeat from core client for 30 sec - exiting 18:36:46 (5164): No heartbeat from core client for 30 sec - exiting 18:36:47 (5164): No heartbeat from core client for 30 sec - exiting 18:36:48 (5164): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6776, iMonCtr=1 Model crash detected, will try to restart... 19:22:23 (5808): No heartbeat from core client for 30 sec - exiting 19:22:24 (5808): No heartbeat from core client for 30 sec - exiting 19:22:26 (5808): No heartbeat from core client for 30 sec - exiting 19:22:27 (5808): No heartbeat from core client for 30 sec - exiting 19:22:28 (5808): No heartbeat from core client for 30 sec - exiting 19:22:29 (5808): No heartbeat from core client for 30 sec - exiting 19:22:30 (5808): No heartbeat from core client for 30 sec - exiting 19:22:31 (5808): No heartbeat from core client for 30 sec - exiting 19:22:32 (5808): No heartbeat from core client for 30 sec - exiting 19:22:33 (5808): No heartbeat from core client for 30 sec - exiting 19:22:34 (5808): No heartbeat from core client for 30 sec - exiting 19:22:35 (5808): No heartbeat from core client for 30 sec - exiting 19:22:36 (5808): No heartbeat from core client for 30 sec - exiting 19:22:38 (5808): No heartbeat from core client for 30 sec - exiting 19:22:39 (5808): No heartbeat from core client for 30 sec - exiting 19:22:40 (5808): No heartbeat from core client for 30 sec - exiting 19:22:41 (5808): No heartbeat from core client for 30 sec - exiting 19:22:42 (5808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:22:43 (5808): No heartbeat from core client for 30 sec - exiting 19:22:44 (5808): No heartbeat from core client for 30 sec - exiting 19:22:45 (5808): No heartbeat from core client for 30 sec - exiting 19:22:46 (5808): No heartbeat from core client for 30 sec - exiting 19:22:47 (5808): No heartbeat from core client for 30 sec - exiting 19:22:49 (5808): No heartbeat from core client for 30 sec - exiting 19:22:50 (5808): No heartbeat from core client for 30 sec - exiting 19:22:51 (5808): No heartbeat from core client for 30 sec - exiting 19:22:52 (5808): No heartbeat from core client for 30 sec - exiting 18:46:31 (6124): No heartbeat from core client for 30 sec - exiting 18:46:32 (6124): No heartbeat from core client for 30 sec - exiting 18:46:34 (6124): No heartbeat from core client for 30 sec - exiting 18:46:35 (6124): No heartbeat from core client for 30 sec - exiting 18:46:36 (6124): No heartbeat from core client for 30 sec - exiting 18:46:37 (6124): No heartbeat from core client for 30 sec - exiting 18:46:38 (6124): No heartbeat from core client for 30 sec - exiting 18:46:39 (6124): No heartbeat from core client for 30 sec - exiting 18:46:40 (6124): No heartbeat from core client for 30 sec - exiting 18:46:41 (6124): No heartbeat from core client for 30 sec - exiting 18:46:42 (6124): No heartbeat from core client for 30 sec - exiting 18:46:43 (6124): No heartbeat from core client for 30 sec - exiting 18:46:44 (6124): No heartbeat from core client for 30 sec - exiting 18:46:46 (6124): No heartbeat from core client for 30 sec - exiting 18:46:47 (6124): No heartbeat from core client for 30 sec - exiting 18:46:48 (6124): No heartbeat from core client for 30 sec - exiting 18:46:49 (6124): No heartbeat from core client for 30 sec - exiting 18:46:50 (6124): No heartbeat from core client for 30 sec - exiting 18:46:51 (6124): No heartbeat from core client for 30 sec - exiting 18:46:52 (6124): No heartbeat from core client for 30 sec - exiting 18:46:53 (6124): No heartbeat from core client for 30 sec - exiting 18:46:54 (6124): No heartbeat from core client for 30 sec - exiting 18:46:55 (6124): No heartbeat from core client for 30 sec - exiting 18:46:57 (6124): No heartbeat from core client for 30 sec - exiting 18:46:58 (6124): No heartbeat from core client for 30 sec - exiting 18:46:59 (6124): No heartbeat from core client for 30 sec - exiting 18:47:00 (6124): No heartbeat from core client for 30 sec - exiting 18:47:01 (6124): No heartbeat from core client for 30 sec - exiting 18:47:02 (6124): No heartbeat from core client for 30 sec - exiting 18:47:03 (6124): No heartbeat from core client for 30 sec - exiting 18:47:04 (6124): No heartbeat from core client for 30 sec - exiting 18:47:05 (6124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:47:06 (6124): No heartbeat from core client for 30 sec - exiting 18:47:41 (5888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:38:53 (1152): No heartbeat from core client for 30 sec - exiting 08:38:54 (1152): No heartbeat from core client for 30 sec - exiting 08:38:55 (1152): No heartbeat from core client for 30 sec - exiting 08:38:56 (1152): No heartbeat from core client for 30 sec - exiting 08:38:57 (1152): No heartbeat from core client for 30 sec - exiting 08:38:58 (1152): No heartbeat from core client for 30 sec - exiting 08:38:59 (1152): No heartbeat from core client for 30 sec - exiting 08:39:00 (1152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:39:02 (1152): No heartbeat from core client for 30 sec - exiting 08:39:03 (1152): No heartbeat from core client for 30 sec - exiting 08:39:04 (1152): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:28:48 (2172): No heartbeat from core client for 30 sec - exiting 19:28:49 (2172): No heartbeat from core client for 30 sec - exiting 19:28:50 (2172): No heartbeat from core client for 30 sec - exiting 19:28:51 (2172): No heartbeat from core client for 30 sec - exiting 19:28:52 (2172): No heartbeat from core client for 30 sec - exiting 19:28:53 (2172): No heartbeat from core client for 30 sec - exiting 19:28:54 (2172): No heartbeat from core client for 30 sec - exiting 19:28:55 (2172): No heartbeat from core client for 30 sec - exiting 19:28:56 (2172): No heartbeat from core client for 30 sec - exiting 19:28:57 (2172): No heartbeat from core client for 30 sec - exiting 19:28:58 (2172): No heartbeat from core client for 30 sec - exiting 19:28:59 (2172): No heartbeat from core client for 30 sec - exiting 19:29:00 (2172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:29:01 (2172): No heartbeat from core client for 30 sec - exiting 19:29:02 (2172): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6516, iMonCtr=1 Model crash detected, will try to restart... 19:29:29 (6132): No heartbeat from core client for 30 sec - exiting 19:29:30 (6132): No heartbeat from core client for 30 sec - exiting 19:29:31 (6132): No heartbeat from core client for 30 sec - exiting 19:29:32 (6132): No heartbeat from core client for 30 sec - exiting 19:29:33 (6132): No heartbeat from core client for 30 sec - exiting 19:29:34 (6132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:33:34 (5196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:53:29 (4852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:54:40 (6920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:44:35 (5668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:52:47 (4612): No heartbeat from core client for 30 sec - exiting 17:52:48 (4612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:52:49 (4612): No heartbeat from core client for 30 sec - exiting 17:52:50 (4612): No heartbeat from core client for 30 sec - exiting 17:52:51 (4612): No heartbeat from core client for 30 sec - exiting 17:52:53 (4612): No heartbeat from core client for 30 sec - exiting 17:52:54 (4612): No heartbeat from core client for 30 sec - exiting 17:52:55 (4612): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:40:36 (5456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2812, iMonCtr=1 Model crash detected, will try to restart... 11:57:24 (4040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:49:18 (7052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:49:19 (7052): No heartbeat from core client for 30 sec - exiting 12:49:20 (7052): No heartbeat from core client for 30 sec - exiting 12:49:21 (7052): No heartbeat from core client for 30 sec - exiting 12:49:22 (7052): No heartbeat from core client for 30 sec - exiting 12:49:23 (7052): No heartbeat from core client for 30 sec - exiting 12:49:24 (7052): No heartbeat from core client for 30 sec - exiting 12:49:25 (7052): No heartbeat from core client for 30 sec - exiting 12:49:27 (7052): No heartbeat from core client for 30 sec - exiting 12:49:28 (7052): No heartbeat from core client for 30 sec - exiting 12:49:29 (7052): No heartbeat from core client for 30 sec - exiting 12:50:15 (3088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:56:50 (6660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:57:06 (6660): No heartbeat from core client for 30 sec - exiting 12:57:07 (6660): No heartbeat from core client for 30 sec - exiting 12:57:08 (6660): No heartbeat from core client for 30 sec - exiting 12:57:09 (6660): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 12:59:45 (2472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:59:47 (2472): No heartbeat from core client for 30 sec - exiting 12:59:48 (2472): No heartbeat from core client for 30 sec - exiting 12:59:49 (2472): No heartbeat from core client for 30 sec - exiting 12:59:50 (2472): No heartbeat from core client for 30 sec - exiting 12:59:51 (2472): No heartbeat from core client for 30 sec - exiting 12:59:52 (2472): No heartbeat from core client for 30 sec - exiting 12:59:53 (2472): No heartbeat from core client for 30 sec - exiting 12:59:54 (2472): No heartbeat from core client for 30 sec - exiting 12:59:55 (2472): No heartbeat from core client for 30 sec - exiting 12:59:56 (2472): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold Suspended CPDN Monitor - Suspend request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Jun 2013 17:47:15	1072815	15790837	hadcm3n_u4mv_2020_40_008369719_0	259,200	440,592	1.6998
31 May 2013 22:40:04	1072815	15790837	hadcm3n_u4mv_2020_40_008369719_0	233,280	396,296	1.6988
31 May 2013 09:41:38	1072815	15790837	hadcm3n_u4mv_2020_40_008369719_0	207,360	351,940	1.6972
29 May 2013 18:45:23	1072815	15790837	hadcm3n_u4mv_2020_40_008369719_0	181,440	308,125	1.6982
27 May 2013 18:56:07	1072815	15790837	hadcm3n_u4mv_2020_40_008369719_0	155,520	263,725	1.6958
26 May 2013 11:00:48	1072815	15790837	hadcm3n_u4mv_2020_40_008369719_0	129,600	219,937	1.6970
25 May 2013 12:53:30	1072815	15790837	hadcm3n_u4mv_2020_40_008369719_0	103,680	176,039	1.6979
24 May 2013 14:24:41	1072815	15790837	hadcm3n_u4mv_2020_40_008369719_0	77,760	132,405	1.7027
23 May 2013 19:51:51	1072815	15790837	hadcm3n_u4mv_2020_40_008369719_0	51,840	88,703	1.7111
21 May 2013 20:42:39	1072815	15790837	hadcm3n_u4mv_2020_40_008369719_0	25,920	44,087	1.7009