Task 16644511

Name	hadam3p_anz_r2oj_2012_1_008733089_2
Workunit	8879067
Created	20 May 2014, 2:09:03 UTC
Sent	20 May 2014, 2:29:24 UTC
Report deadline	2 May 2015, 7:49:24 UTC
Received	15 Jun 2014, 4:57:14 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1302497
Run time	9 days 9 hours 6 min 11 sec
CPU time	8 days 10 hours 13 min
Validate state	Invalid
Credit	3,490.64
Device peak FLOPS	1.86 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:37:54 (3576): No heartbeat from core client for 30 sec - exiting 10:37:55 (3576): No heartbeat from core client for 30 sec - exiting 10:37:56 (3576): No heartbeat from core client for 30 sec - exiting 10:37:57 (3576): No heartbeat from core client for 30 sec - exiting 10:37:58 (3576): No heartbeat from core client for 30 sec - exiting 10:37:59 (3576): No heartbeat from core client for 30 sec - exiting 10:38:00 (3576): No heartbeat from core client for 30 sec - exiting 10:38:01 (3576): No heartbeat from core client for 30 sec - exiting 10:38:02 (3576): No heartbeat from core client for 30 sec - exiting 10:38:03 (3576): No heartbeat from core client for 30 sec - exiting 10:38:04 (3576): No heartbeat from core client for 30 sec - exiting 10:38:05 (3576): No heartbeat from core client for 30 sec - exiting 10:38:06 (3576): No heartbeat from core client for 30 sec - exiting 10:38:07 (3576): No heartbeat from core client for 30 sec - exiting 10:38:08 (3576): No heartbeat from core client for 30 sec - exiting 10:38:09 (3576): No heartbeat from core client for 30 sec - exiting 10:38:10 (3576): No heartbeat from core client for 30 sec - exiting 10:38:11 (3576): No heartbeat from core client for 30 sec - exiting 10:38:12 (3576): No heartbeat from core client for 30 sec - exiting 10:38:13 (3576): No heartbeat from core client for 30 sec - exiting 10:38:14 (3576): No heartbeat from core client for 30 sec - exiting 10:38:15 (3576): No heartbeat from core client for 30 sec - exiting 10:38:16 (3576): No heartbeat from core client for 30 sec - exiting 10:38:47 (3576): No heartbeat from core client for 30 sec - exiting 10:38:48 (3576): No heartbeat from core client for 30 sec - exiting 10:38:49 (3576): No heartbeat from core client for 30 sec - exiting 10:38:50 (3576): No heartbeat from core client for 30 sec - exiting 10:38:51 (3576): No heartbeat from core client for 30 sec - exiting 10:38:52 (3576): No heartbeat from core client for 30 sec - exiting 10:38:53 (3576): No heartbeat from core client for 30 sec - exiting 10:38:54 (3576): No heartbeat from core client for 30 sec - exiting 10:38:55 (3576): No heartbeat from core client for 30 sec - exiting 10:38:56 (3576): No heartbeat from core client for 30 sec - exiting 10:38:57 (3576): No heartbeat from core client for 30 sec - exiting 10:38:58 (3576): No heartbeat from core client for 30 sec - exiting 10:38:59 (3576): No heartbeat from core client for 30 sec - exiting 10:39:00 (3576): No heartbeat from core client for 30 sec - exiting 10:39:01 (3576): No heartbeat from core client for 30 sec - exiting 10:39:02 (3576): No heartbeat from core client for 30 sec - exiting 10:39:03 (3576): No heartbeat from core client for 30 sec - exiting 10:39:04 (3576): No heartbeat from core client for 30 sec - exiting 10:39:05 (3576): No heartbeat from core client for 30 sec - exiting 10:39:06 (3576): No heartbeat from core client for 30 sec - exiting 10:39:07 (3576): No heartbeat from core client for 30 sec - exiting 10:39:08 (3576): No heartbeat from core client for 30 sec - exiting 10:39:09 (3576): No heartbeat from core client for 30 sec - exiting 10:39:10 (3576): No heartbeat from core client for 30 sec - exiting 10:39:11 (3576): No heartbeat from core client for 30 sec - exiting 10:39:12 (3576): No heartbeat from core client for 30 sec - exiting 10:39:13 (3576): No heartbeat from core client for 30 sec - exiting 10:39:14 (3576): No heartbeat from core client for 30 sec - exiting 10:39:15 (3576): No heartbeat from core client for 30 sec - exiting 10:39:16 (3576): No heartbeat from core client for 30 sec - exiting 10:39:17 (3576): No heartbeat from core client for 30 sec - exiting 10:39:18 (3576): No heartbeat from core client for 30 sec - exiting 10:39:19 (3576): No heartbeat from core client for 30 sec - exiting 10:39:20 (3576): No heartbeat from core client for 30 sec - exiting 10:39:21 (3576): No heartbeat from core client for 30 sec - exiting 10:39:22 (3576): No heartbeat from core client for 30 sec - exiting 10:39:23 (3576): No heartbeat from core client for 30 sec - exiting 10:39:24 (3576): No heartbeat from core client for 30 sec - exiting 10:39:25 (3576): No heartbeat from core client for 30 sec - exiting 10:39:26 (3576): No heartbeat from core client for 30 sec - exiting 10:39:27 (3576): No heartbeat from core client for 30 sec - exiting 10:39:28 (3576): No heartbeat from core client for 30 sec - exiting 10:39:29 (3576): No heartbeat from core client for 30 sec - exiting 10:39:30 (3576): No heartbeat from core client for 30 sec - exiting 10:39:31 (3576): No heartbeat from core client for 30 sec - exiting 10:39:32 (3576): No heartbeat from core client for 30 sec - exiting 10:39:33 (3576): No heartbeat from core client for 30 sec - exiting 10:39:34 (3576): No heartbeat from core client for 30 sec - exiting 10:39:35 (3576): No heartbeat from core client for 30 sec - exiting 10:39:36 (3576): No heartbeat from core client for 30 sec - exiting 10:39:37 (3576): No heartbeat from core client for 30 sec - exiting 10:39:38 (3576): No heartbeat from core client for 30 sec - exiting 10:39:39 (3576): No heartbeat from core client for 30 sec - exiting 10:39:40 (3576): No heartbeat from core client for 30 sec - exiting 10:39:41 (3576): No heartbeat from core client for 30 sec - exiting 10:39:42 (3576): No heartbeat from core client for 30 sec - exiting 10:39:43 (3576): No heartbeat from core client for 30 sec - exiting 10:39:44 (3576): No heartbeat from core client for 30 sec - exiting 10:39:45 (3576): No heartbeat from core client for 30 sec - exiting 10:39:46 (3576): No heartbeat from core client for 30 sec - exiting 10:39:47 (3576): No heartbeat from core client for 30 sec - exiting 10:39:48 (3576): No heartbeat from core client for 30 sec - exiting 10:39:49 (3576): No heartbeat from core client for 30 sec - exiting 10:39:50 (3576): No heartbeat from core client for 30 sec - exiting 10:39:51 (3576): No heartbeat from core client for 30 sec - exiting 10:39:52 (3576): No heartbeat from core client for 30 sec - exiting 10:39:53 (3576): No heartbeat from core client for 30 sec - exiting 10:39:54 (3576): No heartbeat from core client for 30 sec - exiting 10:39:55 (3576): No heartbeat from core client for 30 sec - exiting 10:39:56 (3576): No heartbeat from core client for 30 sec - exiting 10:39:57 (3576): No heartbeat from core client for 30 sec - exiting 10:39:58 (3576): No heartbeat from core client for 30 sec - exiting 10:39:59 (3576): No heartbeat from core client for 30 sec - exiting 10:40:00 (3576): No heartbeat from core client for 30 sec - exiting 10:40:01 (3576): No heartbeat from core client for 30 sec - exiting 10:40:02 (3576): No heartbeat from core client for 30 sec - exiting 10:40:03 (3576): No heartbeat from core client for 30 sec - exiting 10:40:04 (3576): No heartbeat from core client for 30 sec - exiting 10:40:05 (3576): No heartbeat from core client for 30 sec - exiting 10:40:06 (3576): No heartbeat from core client for 30 sec - exiting 10:40:07 (3576): No heartbeat from core client for 30 sec - exiting 10:40:08 (3576): No heartbeat from core client for 30 sec - exiting 10:40:09 (3576): No heartbeat from core client for 30 sec - exiting 10:40:10 (3576): No heartbeat from core client for 30 sec - exiting 10:40:11 (3576): No heartbeat from core client for 30 sec - exiting 10:40:12 (3576): No heartbeat from core client for 30 sec - exiting 10:40:13 (3576): No heartbeat from core client for 30 sec - exiting 10:40:14 (3576): No heartbeat from core client for 30 sec - exiting 10:40:15 (3576): No heartbeat from core client for 30 sec - exiting 10:40:16 (3576): No heartbeat from core client for 30 sec - exiting 10:40:17 (3576): No heartbeat from core client for 30 sec - exiting 10:40:18 (3576): No heartbeat from core client for 30 sec - exiting 10:40:19 (3576): No heartbeat from core client for 30 sec - exiting 10:40:20 (3576): No heartbeat from core client for 30 sec - exiting 10:40:21 (3576): No heartbeat from core client for 30 sec - exiting 10:40:22 (3576): No heartbeat from core client for 30 sec - exiting 10:40:23 (3576): No heartbeat from core client for 30 sec - exiting 10:40:24 (3576): No heartbeat from core client for 30 sec - exiting 10:40:25 (3576): No heartbeat from core client for 30 sec - exiting 10:40:26 (3576): No heartbeat from core client for 30 sec - exiting 10:40:27 (3576): No heartbeat from core client for 30 sec - exiting 10:40:28 (3576): No heartbeat from core client for 30 sec - exiting 10:40:29 (3576): No heartbeat from core client for 30 sec - exiting 10:40:30 (3576): No heartbeat from core client for 30 sec - exiting 10:40:31 (3576): No heartbeat from core client for 30 sec - exiting 10:40:32 (3576): No heartbeat from core client for 30 sec - exiting 10:40:33 (3576): No heartbeat from core client for 30 sec - exiting 10:40:34 (3576): No heartbeat from core client for 30 sec - exiting 10:40:35 (3576): No heartbeat from core client for 30 sec - exiting 10:40:36 (3576): No heartbeat from core client for 30 sec - exiting 10:40:37 (3576): No heartbeat from core client for 30 sec - exiting 10:40:38 (3576): No heartbeat from core client for 30 sec - exiting 10:40:39 (3576): No heartbeat from core client for 30 sec - exiting 10:40:40 (3576): No heartbeat from core client for 30 sec - exiting 10:40:41 (3576): No heartbeat from core client for 30 sec - exiting 10:40:42 (3576): No heartbeat from core client for 30 sec - exiting 10:40:43 (3576): No heartbeat from core client for 30 sec - exiting 10:40:44 (3576): No heartbeat from core client for 30 sec - exiting 10:40:45 (3576): No heartbeat from core client for 30 sec - exiting 10:40:46 (3576): No heartbeat from core client for 30 sec - exiting 10:40:47 (3576): No heartbeat from core client for 30 sec - exiting 10:40:48 (3576): No heartbeat from core client for 30 sec - exiting 10:40:49 (3576): No heartbeat from core client for 30 sec - exiting 10:40:50 (3576): No heartbeat from core client for 30 sec - exiting 10:40:51 (3576): No heartbeat from core client for 30 sec - exiting 10:40:52 (3576): No heartbeat from core client for 30 sec - exiting 10:40:53 (3576): No heartbeat from core client for 30 sec - exiting 10:40:54 (3576): No heartbeat from core client for 30 sec - exiting 10:40:55 (3576): No heartbeat from core client for 30 sec - exiting 10:40:56 (3576): No heartbeat from core client for 30 sec - exiting 10:40:57 (3576): No heartbeat from core client for 30 sec - exiting 10:40:58 (3576): No heartbeat from core client for 30 sec - exiting 10:40:59 (3576): No heartbeat from core client for 30 sec - exiting 10:41:00 (3576): No heartbeat from core client for 30 sec - exiting 10:41:01 (3576): No heartbeat from core client for 30 sec - exiting 10:41:02 (3576): No heartbeat from core client for 30 sec - exiting 10:41:03 (3576): No heartbeat from core client for 30 sec - exiting 10:41:04 (3576): No heartbeat from core client for 30 sec - exiting 10:41:05 (3576): No heartbeat from core client for 30 sec - exiting 10:41:06 (3576): No heartbeat from core client for 30 sec - exiting 10:41:07 (3576): No heartbeat from core client for 30 sec - exiting 10:41:08 (3576): No heartbeat from core client for 30 sec - exiting 10:41:09 (3576): No heartbeat from core client for 30 sec - exiting 10:41:10 (3576): No heartbeat from core client for 30 sec - exiting 10:41:11 (3576): No heartbeat from core client for 30 sec - exiting 10:41:12 (3576): No heartbeat from core client for 30 sec - exiting 10:41:13 (3576): No heartbeat from core client for 30 sec - exiting 10:41:14 (3576): No heartbeat from core client for 30 sec - exiting 10:41:15 (3576): No heartbeat from core client for 30 sec - exiting 10:41:16 (3576): No heartbeat from core client for 30 sec - exiting 10:41:17 (3576): No heartbeat from core client for 30 sec - exiting 10:41:18 (3576): No heartbeat from core client for 30 sec - exiting 10:41:19 (3576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:41:20 (3576): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=16548, selfPID=20872, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10724, iMonCtr=2 09:32:07 (5692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12252, selfPID=12252, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 09:12:33 (13824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:39:45 (12140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:15:51 (8384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:15:52 (8384): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:35:06 (10512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=16336, selfPID=16336, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN proceCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5988, selfPID=14564, iMonCtr=1 Model crash detected, will try to restart... 01:52:09 (5884): No heartbeat from core client for 30 sec - exiting 01:52:10 (5884): No heartbeat from core client for 30 sec - exiting 01:52:11 (5884): No heartbeat from core client for 30 sec - exiting 01:52:12 (5884): No heartbeat from core client for 30 sec - exiting 01:52:13 (5884): No heartbeat from core client for 30 sec - exiting 01:52:14 (5884): No heartbeat from core client for 30 sec - exiting 01:52:15 (5884): No heartbeat from core client for 30 sec - exiting 01:52:16 (5884): No heartbeat from core client for 30 sec - exiting 01:52:17 (5884): No heartbeat from core client for 30 sec - exiting 01:52:18 (5884): No heartbeat from core client for 30 sec - exiting 01:52:19 (5884): No heartbeat from core client for 30 sec - exiting 01:52:20 (5884): No heartbeat from core client for 30 sec - exiting 01:52:21 (5884): No heartbeat from core client for 30 sec - exiting 01:52:22 (5884): No heartbeat from core client for 30 sec - exiting 01:52:23 (5884): No heartbeat from core client for 30 sec - exiting 01:52:24 (5884): No heartbeat from core client for 30 sec - exiting 01:52:25 (5884): No heartbeat from core client for 30 sec - exiting 01:52:26 (5884): No heartbeat from core client for 30 sec - exiting 01:52:27 (5884): No heartbeat from core client for 30 sec - exiting 01:52:28 (5884): No heartbeat from core client for 30 sec - exiting 01:52:29 (5884): No heartbeat from core client for 30 sec - exiting 01:52:30 (5884): No heartbeat from core client for 30 sec - exiting 01:52:31 (5884): No heartbeat from core client for 30 sec - exiting 01:52:32 (5884): No heartbeat from core client for 30 sec - exiting 01:52:33 (5884): No heartbeat from core client for 30 sec - exiting 01:52:34 (5884): No heartbeat from core client for 30 sec - exiting 01:52:35 (5884): No heartbeat from core client for 30 sec - exiting 01:52:36 (5884): No heartbeat from core client for 30 sec - exiting 01:52:37 (5884): No heartbeat from core client for 30 sec - exiting 01:52:38 (5884): No heartbeat from core client for 30 sec - exiting 01:52:39 (5884): No heartbeat from core client for 30 sec - exiting 01:52:40 (5884): No heartbeat from core client for 30 sec - exiting 01:52:41 (5884): No heartbeat from core client for 30 sec - exiting 01:52:42 (5884): No heartbeat from core client for 30 sec - exiting 01:52:43 (5884): No heartbeat from core client for 30 sec - exiting 01:52:44 (5884): No heartbeat from core client for 30 sec - exiting 01:52:45 (5884): No heartbeat from core client for 30 sec - exiting 01:52:46 (5884): No heartbeat from core client for 30 sec - exiting 01:52:47 (5884): No heartbeat from core client for 30 sec - exiting 01:52:48 (5884): No heartbeat from core client for 30 sec - exiting 01:52:49 (5884): No heartbeat from core client for 30 sec - exiting 01:52:50 (5884): No heartbeat from core client for 30 sec - exiting 01:52:51 (5884): No heartbeat from core client for 30 sec - exiting 01:52:52 (5884): No heartbeat from core client for 30 sec - exiting 01:52:53 (5884): No heartbeat from core client for 30 sec - exiting 01:52:54 (5884): No heartbeat from core client for 30 sec - exiting 01:52:55 (5884): No heartbeat from core client for 30 sec - exiting 01:52:56 (5884): No heartbeat from core client for 30 sec - exiting 01:52:57 (5884): No heartbeat from core client for 30 sec - exiting 01:52:58 (5884): No heartbeat from core client for 30 sec - exiting 01:52:59 (5884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:53:41 (7200): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12224, selfPID=6572, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14948, selfPID=14948, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=15184, selfPID=15184, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13344, selfPID=13344, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6064, selfPID=6064, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9816, selfPID=9816, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6476, selfPID=6476, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6476, selfPID=12028, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_r2oj_2012_1_008733089_2_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_r2oj_2012_1_008733089_2_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_r2oj_2012_1_008733089_2_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_r2oj_2012_1_008733089_2_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_r2oj_2012_1_008733089_2_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Jun 2014 09:04:05	1302497	16644511	hadam3p_anz_r2oj_2012_1_008733089_2	80,939	702,465	8.6789
07 Jun 2014 11:21:55	1302497	16644511	hadam3p_anz_r2oj_2012_1_008733089_2	69,419	608,309	8.7629
04 Jun 2014 12:01:35	1302497	16644511	hadam3p_anz_r2oj_2012_1_008733089_2	57,899	513,611	8.8708
31 May 2014 18:35:10	1302497	16644511	hadam3p_anz_r2oj_2012_1_008733089_2	46,379	419,766	9.0508
27 May 2014 13:27:29	1302497	16644511	hadam3p_anz_r2oj_2012_1_008733089_2	34,859	318,417	9.1344
25 May 2014 04:15:42	1302497	16644511	hadam3p_anz_r2oj_2012_1_008733089_2	23,339	212,008	9.0839
22 May 2014 10:45:31	1302497	16644511	hadam3p_anz_r2oj_2012_1_008733089_2	11,819	107,840	9.1243