Task 16494757

Name	hadam3p_anz_o8np_2012_1_008628003_1
Workunit	8774515
Created	14 Apr 2014, 11:32:34 UTC
Sent	14 Apr 2014, 11:45:40 UTC
Report deadline	27 Mar 2015, 17:05:40 UTC
Received	23 May 2014, 11:02:45 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1305003
Run time	7 days 9 hours 50 min 28 sec
CPU time	6 days 10 hours 6 min 25 sec
Validate state	Invalid
Credit	2,000.18
Device peak FLOPS	1.32 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4116, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4376, selfPID=5792, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3132, selfPID=2444, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3568, selfPID=4024, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1720, selfPID=3736, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3368, selfPID=3844, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5884, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1552, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3516, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3604, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5940, selfPID=3600, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3224, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3376, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5436, iMonCtr=2 15:36:18 (2768): No heartbeat from core client for 30 sec - exiting 15:36:19 (2768): No heartbeat from core client for 30 sec - exiting 15:36:20 (2768): No heartbeat from core client for 30 sec - exiting 15:36:54 (2768): No heartbeat from core client for 30 sec - exiting 15:36:55 (2768): No heartbeat from core client for 30 sec - exiting 15:36:56 (2768): No heartbeat from core client for 30 sec - exiting 15:36:58 (2768): No heartbeat from core client for 30 sec - exiting 15:36:59 (2768): No heartbeat from core client for 30 sec - exiting 15:37:00 (2768): No heartbeat from core client for 30 sec - exiting 15:37:01 (2768): No heartbeat from core client for 30 sec - exiting 15:37:02 (2768): No heartbeat from core client for 30 sec - exiting 15:37:03 (2768): No heartbeat from core client for 30 sec - exiting 15:37:04 (2768): No heartbeat from core client for 30 sec - exiting 15:37:05 (2768): No heartbeat from core client for 30 sec - exiting 15:37:06 (2768): No heartbeat from core client for 30 sec - exiting 15:37:07 (2768): No heartbeat from core client for 30 sec - exiting 15:37:08 (2768): No heartbeat from core client for 30 sec - exiting 15:37:10 (2768): No heartbeat from core client for 30 sec - exiting 15:37:11 (2768): No heartbeat from core client for 30 sec - exiting 15:37:12 (2768): No heartbeat from core client for 30 sec - exiting 15:37:13 (2768): No heartbeat from core client for 30 sec - exiting 15:37:14 (2768): No heartbeat from core client for 30 sec - exiting 15:37:15 (2768): No heartbeat from core client for 30 sec - exiting 15:37:16 (2768): No heartbeat from core client for 30 sec - exiting 15:37:17 (2768): No heartbeat from core client for 30 sec - exiting 15:37:18 (2768): No heartbeat from core client for 30 sec - exiting 15:37:19 (2768): No heartbeat from core client for 30 sec - exiting 15:37:21 (2768): No heartbeat from core client for 30 sec - exiting 15:37:22 (2768): No heartbeat from core client for 30 sec - exiting 15:37:23 (2768): No heartbeat from core client for 30 sec - exiting 15:37:24 (2768): No heartbeat from core client for 30 sec - exiting 15:37:25 (2768): No heartbeat from core client for 30 sec - exiting 15:37:26 (2768): No heartbeat from core client for 30 sec - exiting 15:37:27 (2768): No heartbeat from core client for 30 sec - exiting 15:37:28 (2768): No heartbeat from core client for 30 sec - exiting 15:37:29 (2768): No heartbeat from core client for 30 sec - exiting 15:37:30 (2768): No heartbeat from core client for 30 sec - exiting 15:37:31 (2768): No heartbeat from core client for 30 sec - exiting 15:37:33 (2768): No heartbeat from core client for 30 sec - exiting 15:37:34 (2768): No heartbeat from core client for 30 sec - exiting 15:37:35 (2768): No heartbeat from core client for 30 sec - exiting 15:37:36 (2768): No heartbeat from core client for 30 sec - exiting 15:37:37 (2768): No heartbeat from core client for 30 sec - exiting 15:37:38 (2768): No heartbeat from core client for 30 sec - exiting 15:37:39 (2768): No heartbeat from core client for 30 sec - exiting 15:37:40 (2768): No heartbeat from core client for 30 sec - exiting 15:37:41 (2768): No heartbeat from core client for 30 sec - exiting 15:37:42 (2768): No heartbeat from core client for 30 sec - exiting 15:37:43 (2768): No heartbeat from core client for 30 sec - exiting 15:37:45 (2768): No heartbeat from core client for 30 sec - exiting 15:37:46 (2768): No heartbeat from core client for 30 sec - exiting 15:37:47 (2768): No heartbeat from core client for 30 sec - exiting 15:37:48 (2768): No heartbeat from core client for 30 sec - exiting 15:37:49 (2768): No heartbeat from core client for 30 sec - exiting 15:37:50 (2768): No heartbeat from core client for 30 sec - exiting 15:37:51 (2768): No heartbeat from core client for 30 sec - exiting 15:37:52 (2768): No heartbeat from core client for 30 sec - exiting 15:37:53 (2768): No heartbeat from core client for 30 sec - exiting 15:37:54 (2768): No heartbeat from core client for 30 sec - exiting 15:37:55 (2768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4512, selfPID=3356, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4060, selfPID=1992, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:31:18 (3464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:35:49 (2192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:46:18 (9224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: 21:55:51 (4576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:30:35 (5560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:32:42 (5232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:35:06 (10216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6336, selfPID=6336, iMonCtr=2 10:35:56 (6312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:38:54 (7028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:58:52 (9628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:02:21 (6352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:03:00 (6908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:14:14 (8604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:18:54 (7608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:26:47 (3428): No heartbeat from core client for 30 sec - exiting 11:26:48 (3428): No heartbeat from core client for 30 sec - exiting 11:26:49 (3428): No heartbeat from core client for 30 sec - exiting 11:26:50 (3428): No heartbeat from core client for 30 sec - exiting 11:26:52 (3428): No heartbeat from core client for 30 sec - exiting 11:26:53 (3428): No heartbeat from core client for 30 sec - exiting 11:26:54 (3428): No heartbeat from core client for 30 sec - exiting 11:26:55 (3428): No heartbeat from core client for 30 sec - exiting 11:26:56 (3428): No heartbeat from core client for 30 sec - exiting 11:26:57 (3428): No heartbeat from core client for 30 sec - exiting 11:26:58 (3428): No heartbeat from core client for 30 sec - exiting 11:26:59 (3428): No heartbeat from core client for 30 sec - exiting 11:27:00 (3428): No heartbeat from core client for 30 sec - exiting 11:27:01 (3428): No heartbeat from core client for 30 sec - exiting 11:27:02 (3428): No heartbeat from core client for 30 sec - exiting 11:27:04 (3428): No heartbeat from core client for 30 sec - exiting 11:27:05 (3428): No heartbeat from core client for 30 sec - exiting 11:27:06 (3428): No heartbeat from core client for 30 sec - exiting 11:27:07 (3428): No heartbeat from core client for 30 sec - exiting 11:27:08 (3428): No heartbeat from core client for 30 sec - exiting 11:27:09 (3428): No heartbeat from core client for 30 sec - exiting 11:27:10 (3428): No heartbeat from core client for 30 sec - exiting 11:27:11 (3428): No heartbeat from core client for 30 sec - exiting 11:27:12 (3428): No heartbeat from core client for 30 sec - exiting 11:27:13 (3428): No heartbeat from core client for 30 sec - exiting 11:27:14 (3428): No heartbeat from core client for 30 sec - exiting 11:27:16 (3428): No heartbeat from core client for 30 sec - exiting 11:27:17 (3428): No heartbeat from core client for 30 sec - exiting 11:27:18 (3428): No heartbeat from core client for 30 sec - exiting 11:27:19 (3428): No heartbeat from core client for 30 sec - exiting 11:27:20 (3428): No heartbeat from core client for 30 sec - exiting 11:27:21 (3428): No heartbeat from core client for 30 sec - exiting 11:27:22 (3428): No heartbeat from core client for 30 sec - exiting 11:27:23 (3428): No heartbeat from core client for 30 sec - exiting 11:27:24 (3428): No heartbeat from core client for 30 sec - exiting 11:27:25 (3428): No heartbeat from core client for 30 sec - exiting 11:27:26 (3428): No heartbeat from core client for 30 sec - exiting 11:27:28 (3428): No heartbeat from core client for 30 sec - exiting 11:27:29 (3428): No heartbeat from core client for 30 sec - exiting 11:27:30 (3428): No heartbeat from core client for 30 sec - exiting 11:27:31 (3428): No heartbeat from core client for 30 sec - exiting 11:27:32 (3428): No heartbeat from core client for 30 sec - exiting 11:27:33 (3428): No heartbeat from core client for 30 sec - exiting 11:27:34 (3428): No heartbeat from core client for 30 sec - exiting 11:27:35 (3428): No heartbeat from core client for 30 sec - exiting 11:27:36 (3428): No heartbeat from core client for 30 sec - exiting 11:27:37 (3428): No heartbeat from core client for 30 sec - exiting 11:27:39 (3428): No heartbeat from core client for 30 sec - exiting 11:27:40 (3428): No heartbeat from core client for 30 sec - exiting 11:27:41 (3428): No heartbeat from core client for 30 sec - exiting 11:27:42 (3428): No heartbeat from core client for 30 sec - exiting 11:27:43 (3428): No heartbeat from core client for 30 sec - exiting 11:27:44 (3428): No heartbeat from core client for 30 sec - exiting 11:27:45 (3428): No heartbeat from core client for 30 sec - exiting 11:27:46 (3428): No heartbeat from core client for 30 sec - exiting 11:27:47 (3428): No heartbeat from core client for 30 sec - exiting 11:27:48 (3428): No heartbeat from core client for 30 sec - exiting 11:27:49 (3428): No heartbeat from core client for 30 sec - exiting 11:27:51 (3428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1356, iMonCtr=2 08:49:52 (3264): No heartbeat from core client for 30 sec - exiting 08:49:54 (3264): No heartbeat from core client for 30 sec - exiting 08:49:55 (3264): No heartbeat from core client for 30 sec - exiting 08:49:56 (3264): No heartbeat from core client for 30 sec - exiting 08:49:57 (3264): No heartbeat from core client for 30 sec - exiting 08:49:58 (3264): No heartbeat from core client for 30 sec - exiting 08:49:59 (3264): No heartbeat from core client for 30 sec - exiting 08:50:00 (3264): No heartbeat from core client for 30 sec - exiting 08:50:01 (3264): No heartbeat from core client for 30 sec - exiting 08:50:02 (3264): No heartbeat from core client for 30 sec - exiting 08:50:03 (3264): No heartbeat from core client for 30 sec - exiting 08:50:04 (3264): No heartbeat from core client for 30 sec - exiting 08:50:06 (3264): No heartbeat from core client for 30 sec - exiting 08:50:07 (3264): No heartbeat from core client for 30 sec - exiting 08:50:08 (3264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3588, selfPID=5276, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5636, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5652, selfPID=3428, iMonCtr=1 Model crash detected, will try to restart... GSuspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1344, selfPID=3360, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:39:02 (3488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5564, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4208, selfPID=3272, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5916, selfPID=3080, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5644, selfPID=3860, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:05:10 (3464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:12:52 (3624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1988, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1376, selfPID=4172, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5644, selfPID=3284, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3996, selfPID=3280, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5116, selfPID=2072, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:31:08 (4388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:31:56 (3968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5228, selfPID=5228, iMonCtr=2 20:33:41 (2476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5144, iMonCtr=2 16:36:44 (3824): No heartbeat from core client for 30 sec - exiting 16:36:45 (3824): No heartbeat from core client for 30 sec - exiting 16:36:46 (3824): No heartbeat from core client for 30 sec - exiting 16:36:47 (3824): No heartbeat from core client for 30 sec - exiting 16:36:48 (3824): No heartbeat from core client for 30 sec - exiting 16:36:49 (3824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:31:41 (3948): No heartbeat from core client for 30 sec - exiting 17:31:42 (3948): No heartbeat from core client for 30 sec - exiting 17:31:43 (3948): No heartbeat from core client for 30 sec - exiting 17:31:44 (3948): No heartbeat from core client for 30 sec - exiting 17:31:45 (3948): No heartbeat from core client for 30 sec - exiting 17:31:46 (3948): No heartbeat from core client for 30 sec - exiting 17:31:47 (3948): No heartbeat from core client for 30 sec - exiting 17:31:48 (3948): No heartbeat from core client for 30 sec - exiting 17:31:49 (3948): No heartbeat from core client for 30 sec - exiting 17:31:50 (3948): No heartbeat from core client for 30 sec - exiting 17:31:51 (3948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5032, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5992, iMonCtr=2 Model crash detected, will try to restart... 19:42:20 (1276): No heartbeat from core client for 30 sec - exiting 19:42:21 (1276): No heartbeat from core client for 30 sec - exiting 19:42:22 (1276): No heartbeat from core client for 30 sec - exiting 19:42:23 (1276): No heartbeat from core client for 30 sec - exiting 19:42:24 (1276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:03:37 (5800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:49:06 (4048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2004, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3488, selfPID=1432, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:27:11 (5068): No heartbeat from core client for 30 sec - exiting 17:27:12 (5068): No heartbeat from core client for 30 sec - exiting 17:27:13 (5068): No heartbeat from core client for 30 sec - exiting 17:27:14 (5068): No heartbeat from core client for 30 sec - exiting 17:27:15 (5068): No heartbeat from core client for 30 sec - exiting 17:27:16 (5068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:53:01 (5228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:56:20 (1444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:56:59 (6984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:00:19 (5488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6012, selfPID=6012, iMonCtr=2 20:02:36 (5228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:14:58 (6736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:15:35 (4896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:22:37 (1424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6124, selfPID=6124, iMonCtr=2 20:42:15 (7044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:45:02 (6876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:53:25 (1156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3884, selfPID=3884, iMonCtr=2 20:54:40 (1788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:12:44 (5984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:13:23 (7128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:14:56 (3152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=4140, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1300, selfPID=5008, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_o8np_2012_1_008628003_1_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_o8np_2012_1_008628003_1_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_o8np_2012_1_008628003_1_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_o8np_2012_1_008628003_1_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_o8np_2012_1_008628003_1_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_o8np_2012_1_008628003_1_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_o8np_2012_1_008628003_1_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_o8np_2012_1_008628003_1_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 May 2014 18:23:53	1305003	16494757	hadam3p_anz_o8np_2012_1_008628003_1	46,379	508,120	10.9558
01 May 2014 12:58:47	1305003	16494757	hadam3p_anz_o8np_2012_1_008628003_1	34,859	381,842	10.9539
25 Apr 2014 17:02:27	1305003	16494757	hadam3p_anz_o8np_2012_1_008628003_1	23,339	253,765	10.8730
19 Apr 2014 19:09:35	1305003	16494757	hadam3p_anz_o8np_2012_1_008628003_1	11,819	128,084	10.8371