Task 14328061

Name	hadam3p_pnw_8ncl_2004_1_007703217_1
Workunit	7858325
Created	28 Mar 2012, 18:27:33 UTC
Sent	28 Mar 2012, 18:27:37 UTC
Report deadline	10 Mar 2013, 23:47:37 UTC
Received	20 Jul 2012, 17:17:26 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1131474
Run time	5 days 8 hours 24 min 22 sec
CPU time	4 days 20 hours 3 min 23 sec
Validate state	Invalid
Credit	1,754.30
Device peak FLOPS	1.64 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5428, selfPID=2056, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3324, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5576, selfPID=4256, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:44:41 (4496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4412, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6456, selfPID=4104, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4828, selfPID=4768, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5128, selfPID=6104, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4204, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5576, selfPID=3768, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... GlController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5876, selfPID=4732, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:06:12 (5104): No heartbeat from core client for 30 sec - exiting 21:06:13 (5104): No heartbeat from core client for 30 sec - exiting 21:06:14 (5104): No heartbeat from core client for 30 sec - exiting 21:06:15 (5104): No heartbeat from core client for 30 sec - exiting 21:06:16 (5104): No heartbeat from core client for 30 sec - exiting 21:06:17 (5104): No heartbeat from core client for 30 sec - exiting 21:06:18 (5104): No heartbeat from core client for 30 sec - exiting 21:06:19 (5104): No heartbeat from core client for 30 sec - exiting 21:06:20 (5104): No heartbeat from core client for 30 sec - exiting 21:06:21 (5104): No heartbeat from core client for 30 sec - exiting 21:06:22 (5104): No heartbeat from core client for 30 sec - exiting 21:06:23 (5104): No heartbeat from core client for 30 sec - exiting 21:06:24 (5104): No heartbeat from core client for 30 sec - exiting 21:06:25 (5104): No heartbeat from core client for 30 sec - exiting 21:06:26 (5104): No heartbeat from core client for 30 sec - exiting 21:06:27 (5104): No heartbeat from core client for 30 sec - exiting 21:06:28 (5104): No heartbeat from core client for 30 sec - exiting 21:06:29 (5104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:15:13 (7056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:53:07 (5640): No heartbeat from core client for 30 sec - exiting 10:53:08 (5640): No heartbeat from core client for 30 sec - exiting 10:53:09 (5640): No heartbeat from core client for 30 sec - exiting 10:53:10 (5640): No heartbeat from core client for 30 sec - exiting 10:53:11 (5640): No heartbeat from core client for 30 sec - exiting 10:53:12 (5640): No heartbeat from core client for 30 sec - exiting 10:53:13 (5640): No heartbeat from core client for 30 sec - exiting 10:53:14 (5640): No heartbeat from core client for 30 sec - exiting 10:53:15 (5640): No heartbeat from core client for 30 sec - exiting 10:53:16 (5640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:29:32 (3676): No heartbeat from core client for 30 sec - exiting 13:29:33 (3676): No heartbeat from core client for 30 sec - exiting 13:29:34 (3676): No heartbeat from core client for 30 sec - exiting 13:29:35 (3676): No heartbeat from core client for 30 sec - exiting 13:29:36 (3676): No heartbeat from core client for 30 sec - exiting 13:29:37 (3676): No heartbeat from core client for 30 sec - exiting 13:29:38 (3676): No heartbeat from core client for 30 sec - exiting 13:29:39 (3676): No heartbeat from core client for 30 sec - exiting 13:29:40 (3676): No heartbeat from core client for 30 sec - exiting 13:29:41 (3676): No heartbeat from core client for 30 sec - exiting 13:29:42 (3676): No heartbeat from core client for 30 sec - exiting 13:29:43 (3676): No heartbeat from core client for 30 sec - exiting 13:29:44 (3676): No heartbeat from core client for 30 sec - exiting 13:29:45 (3676): No heartbeat from core client for 30 sec - exiting 13:29:46 (3676): No heartbeat from core client for 30 sec - exiting 13:29:47 (3676): No heartbeat from core client for 30 sec - exiting 13:29:48 (3676): No heartbeat from core client for 30 sec - exiting 13:29:49 (3676): No heartbeat from core client for 30 sec - exiting 13:29:50 (3676): No heartbeat from core client for 30 sec - exiting 13:29:51 (3676): No heartbeat from core client for 30 sec - exiting 13:29:52 (3676): No heartbeat from core client for 30 sec - exiting 13:29:53 (3676): No heartbeat from core client for 30 sec - exiting 13:29:54 (3676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:55:52 (2236): No heartbeat from core client for 30 sec - exiting 18:55:53 (2236): No heartbeat from core client for 30 sec - exiting 18:55:54 (2236): No heartbeat from core client for 30 sec - exiting 18:55:55 (2236): No heartbeat from core client for 30 sec - exiting 18:55:56 (2236): No heartbeat from core client for 30 sec - exiting 18:55:57 (2236): No heartbeat from core client for 30 sec - exiting 18:55:58 (2236): No heartbeat from core client for 30 sec - exiting 18:55:59 (2236): No heartbeat from core client for 30 sec - exiting 18:56:00 (2236): No heartbeat from core client for 30 sec - exiting 18:56:01 (2236): No heartbeat from core client for 30 sec - exiting 18:56:02 (2236): No heartbeat from core client for 30 sec - exiting 18:56:03 (2236): No heartbeat from core client for 30 sec - exiting 18:56:04 (2236): No heartbeat from core client for 30 sec - exiting 18:56:05 (2236): No heartbeat from core client for 30 sec - exiting 18:56:06 (2236): No heartbeat from core client for 30 sec - exiting 18:56:07 (2236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5908, selfPID=2068, iMonCtr=1 Model crash detected, will try to restart... 19:05:31 (1996): No heartbeat from core client for 30 sec - exiting 19:05:32 (1996): No heartbeat from core client for 30 sec - exiting 19:05:33 (1996): No heartbeat from core client for 30 sec - exiting 19:05:34 (1996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:11:50 (5100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5876, selfPID=5224, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 4 19:41:16 (5556): No heartbeat from core client for 30 sec - exiting 19:41:17 (5556): No heartbeat from core client for 30 sec - exiting 19:41:18 (5556): No heartbeat from core client for 30 sec - exiting 19:41:19 (5556): No heartbeat from core client for 30 sec - exiting 19:41:20 (5556): No heartbeat from core client for 30 sec - exiting 19:41:21 (5556): No heartbeat from core client for 30 sec - exiting 19:41:22 (5556): No heartbeat from core client for 30 sec - exiting 19:41:23 (5556): No heartbeat from core client for 30 sec - exiting 19:41:24 (5556): No heartbeat from core client for 30 sec - exiting 19:41:25 (5556): No heartbeat from core client for 30 sec - exiting 19:41:26 (5556): No heartbeat from core client for 30 sec - exiting 19:41:27 (5556): No heartbeat from core client for 30 sec - exiting 19:41:28 (5556): No heartbeat from core client for 30 sec - exiting 19:41:29 (5556): No heartbeat from core client for 30 sec - exiting 19:41:30 (5556): No heartbeat from core client for 30 sec - exiting 19:41:31 (5556): No heartbeat from core client for 30 sec - exiting 19:41:32 (5556): No heartbeat from core client for 30 sec - exiting 19:41:33 (5556): No heartbeat from core client for 30 sec - exiting 19:41:34 (5556): No heartbeat from core client for 30 sec - exiting 19:41:35 (5556): No heartbeat from core client for 30 sec - exiting 19:41:36 (5556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:42:38 (5372): No heartbeat from core client for 30 sec - exiting 00:42:39 (5372): No heartbeat from core client for 30 sec - exiting 00:42:40 (5372): No heartbeat from core client for 30 sec - exiting 00:42:41 (5372): No heartbeat from core client for 30 sec - exiting 00:42:42 (5372): No heartbeat from core client for 30 sec - exiting 00:42:43 (5372): No heartbeat from core client for 30 sec - exiting 00:42:44 (5372): No heartbeat from core client for 30 sec - exiting 00:42:45 (5372): No heartbeat from core client for 30 sec - exiting 00:42:46 (5372): No heartbeat from core client for 30 sec - exiting 00:42:47 (5372): No heartbeat from core client for 30 sec - exiting 00:42:48 (5372): No heartbeat from core client for 30 sec - exiting 00:42:49 (5372): No heartbeat from core client for 30 sec - exiting 00:42:50 (5372): No heartbeat from core client for 30 sec - exiting 00:42:51 (5372): No heartbeat from core client for 30 sec - exiting 00:42:52 (5372): No heartbeat from core client for 30 sec - exiting 00:42:53 (5372): No heartbeat from core client for 30 sec - exiting 00:42:54 (5372): No heartbeat from core client for 30 sec - exiting 00:42:55 (5372): No heartbeat from core client for 30 sec - exiting 00:42:56 (5372): No heartbeat from core client for 30 sec - exiting 00:42:57 (5372): No heartbeat from core client for 30 sec - exiting 00:42:58 (5372): No heartbeat from core client for 30 sec - exiting 00:42:59 (5372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:51:35 (5256): No heartbeat from core client for 30 sec - exiting 20:51:36 (5256): No heartbeat from core client for 30 sec - exiting 20:51:37 (5256): No heartbeat from core client for 30 sec - exiting 20:51:38 (5256): No heartbeat from core client for 30 sec - exiting 20:51:39 (5256): No heartbeat from core client for 30 sec - exiting 20:51:40 (5256): No heartbeat from core client for 30 sec - exiting 20:51:41 (5256): No heartbeat from core client for 30 sec - exiting 20:51:42 (5256): No heartbeat from core client for 30 sec - exiting 20:51:43 (5256): No heartbeat from core client for 30 sec - exiting 20:51:44 (5256): No heartbeat from core client for 30 sec - exiting 20:51:45 (5256): No heartbeat from core client for 30 sec - exiting 20:51:46 (5256): No heartbeat from core client for 30 sec - exiting 20:51:47 (5256): No heartbeat from core client for 30 sec - exiting 20:51:48 (5256): No heartbeat from core client for 30 sec - exiting 20:51:49 (5256): No heartbeat from core client for 30 sec - exiting 20:51:50 (5256): No heartbeat from core client for 30 sec - exiting 20:51:51 (5256): No heartbeat from core client for 30 sec - exiting 20:51:52 (5256): No heartbeat from core client for 30 sec - exiting 20:51:53 (5256): No heartbeat from core client for 30 sec - exiting 20:51:54 (5256): No heartbeat from core client for 30 sec - exiting 20:51:55 (5256): No heartbeat from core client for 30 sec - exiting 20:51:56 (5256): No heartbeat from core client for 30 sec - exiting 20:51:57 (5256): No heartbeat from core client for 30 sec - exiting 20:51:58 (5256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4904, iMonCtr=2 Model crash detected, will try to restart... 10:33:22 (4468): No heartbeat from core client for 30 sec - exiting 10:33:23 (4468): No heartbeat from core client for 30 sec - exiting 10:33:24 (4468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2056, selfPID=4796, iMonCtr=1 Model crash detected, will try to restart... 16:27:45 (5048): No heartbeat from core client for 30 sec - exiting 16:27:47 (5048): No heartbeat from core client for 30 sec - exiting 16:27:48 (5048): No heartbeat from core client for 30 sec - exiting 16:27:49 (5048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:27:00 (2468): No heartbeat from core client for 30 sec - exiting 13:27:01 (2468): No heartbeat from core client for 30 sec - exiting 13:27:02 (2468): No heartbeat from core client for 30 sec - exiting 13:27:03 (2468): No heartbeat from core client for 30 sec - exiting 13:27:04 (2468): No heartbeat from core client for 30 sec - exiting 13:27:05 (2468): No heartbeat from core client for 30 sec - exiting 13:27:06 (2468): No heartbeat from core client for 30 sec - exiting 13:27:07 (2468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6768, iMonCtr=2 Model crash detected, will try to restart... 08:44:22 (4452): No heartbeat from core client for 30 sec - exiting 08:44:23 (4452): No heartbeat from core client for 30 sec - exiting 08:44:24 (4452): No heartbeat from core client for 30 sec - exiting 08:44:25 (4452): No heartbeat from core client for 30 sec - exiting 08:44:26 (4452): No heartbeat from core client for 30 sec - exiting 08:44:27 (4452): No heartbeat from core client for 30 sec - exiting 08:44:28 (4452): No heartbeat from core client for 30 sec - exiting 08:44:29 (4452): No heartbeat from core client for 30 sec - exiting 08:44:30 (4452): No heartbeat from core client for 30 sec - exiting 08:44:31 (4452): No heartbeat from core client for 30 sec - exiting 08:44:32 (4452): No heartbeat from core client for 30 sec - exiting 08:44:33 (4452): No heartbeat from core client for 30 sec - exiting 08:44:34 (4452): No heartbeat from core client for 30 sec - exiting 08:44:35 (4452): No heartbeat from core client for 30 sec - exiting 08:44:36 (4452): No heartbeat from core client for 30 sec - exiting 08:44:37 (4452): No heartbeat from core client for 30 sec - exiting 08:44:38 (4452): No heartbeat from core client for 30 sec - exiting 08:44:39 (4452): No heartbeat from core client for 30 sec - exiting 08:44:40 (4452): No heartbeat from core client for 30 sec - exiting 08:44:41 (4452): No heartbeat from core client for 30 sec - exiting 08:44:42 (4452): No heartbeat from core client for 30 sec - exiting 08:44:43 (4452): No heartbeat from core client for 30 sec - exiting 08:44:44 (4452): No heartbeat from core client for 30 sec - exiting 08:44:45 (4452): No heartbeat from core client for 30 sec - exiting 08:44:46 (4452): No heartbeat from core client for 30 sec - exiting 08:44:47 (4452): No heartbeat from core client for 30 sec - exiting 08:44:48 (4452): No heartbeat from core client for 30 sec - exiting 08:44:49 (4452): No heartbeat from core client for 30 sec - exiting 08:44:50 (4452): No heartbeat from core client for 30 sec - exiting 08:44:51 (4452): No heartbeat from core client for 30 sec - exiting 08:44:52 (4452): No heartbeat from core client for 30 sec - exiting 08:44:53 (4452): No heartbeat from core client for 30 sec - exiting 08:44:54 (4452): No heartbeat from core client for 30 sec - exiting 08:44:55 (4452): No heartbeat from core client for 30 sec - exiting 08:44:56 (4452): No heartbeat from core client for 30 sec - exiting 08:44:57 (4452): No heartbeat from core client for 30 sec - exiting 08:44:58 (4452): No heartbeat from core client for 30 sec - exiting 08:44:59 (4452): No heartbeat from core client for 30 sec - exiting 08:45:00 (4452): No heartbeat from core client for 30 sec - exiting 08:45:01 (4452): No heartbeat from core client for 30 sec - exiting 08:45:02 (4452): No heartbeat from core client for 30 sec - exiting 08:45:03 (4452): No heartbeat from core client for 30 sec - exiting 08:45:04 (4452): No heartbeat from core client for 30 sec - exiting 08:45:05 (4452): No heartbeat from core client for 30 sec - exiting 08:45:06 (4452): No heartbeat from core client for 30 sec - exiting 08:45:07 (4452): No heartbeat from core client for 30 sec - exiting 08:45:08 (4452): No heartbeat from core client for 30 sec - exiting 08:45:09 (4452): No heartbeat from core client for 30 sec - exiting 08:45:10 (4452): No heartbeat from core client for 30 sec - exiting 08:45:11 (4452): No heartbeat from core client for 30 sec - exiting 08:45:12 (4452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:09:47 (3640): No heartbeat from core client for 30 sec - exiting 20:09:48 (3640): No heartbeat from core client for 30 sec - exiting 20:09:49 (3640): No heartbeat from core client for 30 sec - exiting 20:09:50 (3640): No heartbeat from core client for 30 sec - exiting 20:09:51 (3640): No heartbeat from core client for 30 sec - exiting 20:09:52 (3640): No heartbeat from core client for 30 sec - exiting 20:09:53 (3640): No heartbeat from core client for 30 sec - exiting 20:09:54 (3640): No heartbeat from core client for 30 sec - exiting 20:09:55 (3640): No heartbeat from core client for 30 sec - exiting 20:09:56 (3640): No heartbeat from core client for 30 sec - exiting 20:09:57 (3640): No heartbeat from core client for 30 sec - exiting 20:09:58 (3640): No heartbeat from core client for 30 sec - exiting 20:09:59 (3640): No heartbeat from core client for 30 sec - exiting 20:10:00 (3640): No heartbeat from core client for 30 sec - exiting 20:10:01 (3640): No heartbeat from core client for 30 sec - exiting 20:10:02 (3640): No heartbeat from core client for 30 sec - exiting 20:10:03 (3640): No heartbeat from core client for 30 sec - exiting 20:10:04 (3640): No heartbeat from core client for 30 sec - exiting 20:10:05 (3640): No heartbeat from core client for 30 sec - exiting 20:10:06 (3640): No heartbeat from core client for 30 sec - exiting 20:10:07 (3640): No heartbeat from core client for 30 sec - exiting 20:10:08 (3640): No heartbeat from core client for 30 sec - exiting 20:10:09 (3640): No heartbeat from core client for 30 sec - exiting 20:10:10 (3640): No heartbeat from core client for 30 sec - exiting 20:10:11 (3640): No heartbeat from core client for 30 sec - exiting 20:10:12 (3640): No heartbeat from core client for 30 sec - exiting 20:10:13 (3640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:12:21 (3204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4200, selfPID=4200, iMonCtr=2 13:35:35 (4188): No heartbeat from core client for 30 sec - exiting 13:35:36 (4188): No heartbeat from core client for 30 sec - exiting 13:35:37 (4188): No heartbeat from core client for 30 sec - exiting 13:35:38 (4188): No heartbeat from core client for 30 sec - exiting 13:35:39 (4188): No heartbeat from core client for 30 sec - exiting 13:35:40 (4188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:23:39 (5412): No heartbeat from core client for 30 sec - exiting 16:23:40 (5412): No heartbeat from core client for 30 sec - exiting 16:23:41 (5412): No heartbeat from core client for 30 sec - exiting 16:23:42 (5412): No heartbeat from core client for 30 sec - exiting 16:23:43 (5412): No heartbeat from core client for 30 sec - exiting 16:23:44 (5412): No heartbeat from core client for 30 sec - exiting 16:23:45 (5412): No heartbeat from core client for 30 sec - exiting 16:23:46 (5412): No heartbeat from core client for 30 sec - exiting 16:23:47 (5412): No heartbeat from core client for 30 sec - exiting 16:23:48 (5412): No heartbeat from core client for 30 sec - exiting 16:23:49 (5412): No heartbeat from core client for 30 sec - exiting 16:23:50 (5412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:11:32 (4912): No heartbeat from core client for 30 sec - exiting 16:11:33 (4912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:37:34 (4308): No heartbeat from core client for 30 sec - exiting 18:37:35 (4308): No heartbeat from core client for 30 sec - exiting 18:37:36 (4308): No heartbeat from core client for 30 sec - exiting 18:37:37 (4308): No heartbeat from core client for 30 sec - exiting 18:37:38 (4308): No heartbeat from core client for 30 sec - exiting 18:37:39 (4308): No heartbeat from core client for 30 sec - exiting 18:37:40 (4308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:31:33 (4356): No heartbeat from core client for 30 sec - exiting 00:31:34 (4356): No heartbeat from core client for 30 sec - exiting 00:31:35 (4356): No heartbeat from core client for 30 sec - exiting 00:31:36 (4356): No heartbeat from core client for 30 sec - exiting 00:31:37 (4356): No heartbeat from core client for 30 sec - exiting 00:31:38 (4356): No heartbeat from core client for 30 sec - exiting 00:31:39 (4356): No heartbeat from core client for 30 sec - exiting 00:31:40 (4356): No heartbeat from core client for 30 sec - exiting 00:31:41 (4356): No heartbeat from core client for 30 sec - exiting 00:31:42 (4356): No heartbeat from core client for 30 sec - exiting 00:31:43 (4356): No heartbeat from core client for 30 sec - exiting 00:31:44 (4356): No heartbeat from core client for 30 sec - exiting 00:31:45 (4356): No heartbeat from core client for 30 sec - exiting 00:31:46 (4356): No heartbeat from core client for 30 sec - exiting 00:31:47 (4356): No heartbeat from core client for 30 sec - exiting 00:31:48 (4356): No heartbeat from core client for 30 sec - exiting 00:31:49 (4356): No heartbeat from core client for 30 sec - exiting 00:31:50 (4356): No heartbeat from core client for 30 sec - exiting 00:31:51 (4356): No heartbeat from core client for 30 sec - exiting 00:31:52 (4356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:03:46 (5184): No heartbeat from core client for 30 sec - exiting 13:03:47 (5184): No heartbeat from core client for 30 sec - exiting 13:03:48 (5184): No heartbeat from core client for 30 sec - exiting 13:03:49 (5184): No heartbeat from core client for 30 sec - exiting 13:03:50 (5184): No heartbeat from core client for 30 sec - exiting 13:03:51 (5184): No heartbeat from core client for 30 sec - exiting 13:03:52 (5184): No heartbeat from core client for 30 sec - exiting 13:03:53 (5184): No heartbeat from core client for 30 sec - exiting 13:03:54 (5184): No heartbeat from core client for 30 sec - exiting 13:03:55 (5184): No heartbeat from core client for 30 sec - exiting 13:03:56 (5184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3080, iMonCtr=2 18:58:13 (4924): No heartbeat from core client for 30 sec - exiting 18:58:14 (4924): No heartbeat from core client for 30 sec - exiting 18:58:15 (4924): No heartbeat from core client for 30 sec - exiting 18:58:16 (4924): No heartbeat from core client for 30 sec - exiting 18:58:17 (4924): No heartbeat from core client for 30 sec - exiting 18:58:18 (4924): No heartbeat from core client for 30 sec - exiting 18:58:19 (4924): No heartbeat from core client for 30 sec - exiting 18:58:20 (4924): No heartbeat from core client for 30 sec - exiting 18:58:21 (4924): No heartbeat from core client for 30 sec - exiting 18:58:22 (4924): No heartbeat from core client for 30 sec - exiting 18:58:23 (4924): No heartbeat from core client for 30 sec - exiting 18:58:24 (4924): No heartbeat from core client for 30 sec - exiting 18:58:25 (4924): No heartbeat from core client for 30 sec - exiting 18:58:26 (4924): No heartbeat from core client for 30 sec - exiting 18:58:27 (4924): No heartbeat from core client for 30 sec - exiting 18:58:28 (4924): No heartbeat from core client for 30 sec - exiting 18:58:29 (4924): No heartbeat from core client for 30 sec - exiting 18:58:30 (4924): No heartbeat from core client for 30 sec - exiting 18:58:31 (4924): No heartbeat from core client for 30 sec - exiting 18:58:32 (4924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:55:54 (5460): No heartbeat from core client for 30 sec - exiting 13:55:55 (5460): No heartbeat from core client for 30 sec - exiting 13:55:56 (5460): No heartbeat from core client for 30 sec - exiting 13:55:57 (5460): No heartbeat from core client for 30 sec - exiting 13:55:58 (5460): No heartbeat from core client for 30 sec - exiting 13:55:59 (5460): No heartbeat from core client for 30 sec - exiting 13:56:00 (5460): No heartbeat from core client for 30 sec - exiting 13:56:01 (5460): No heartbeat from core client for 30 sec - exiting 13:56:02 (5460): No heartbeat from core client for 30 sec - exiting 13:56:03 (5460): No heartbeat from core client for 30 sec - exiting 13:56:04 (5460): No heartbeat from core client for 30 sec - exiting 13:56:05 (5460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5576, iMonCtr=2 20:50:51 (4948): No heartbeat from core client for 30 sec - exiting 20:50:52 (4948): No heartbeat from core client for 30 sec - exiting 20:50:53 (4948): No heartbeat from core client for 30 sec - exiting 20:50:54 (4948): No heartbeat from core client for 30 sec - exiting 20:50:55 (4948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6088, selfPID=4864, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 6 Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2916, selfPID=5108, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3744, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3984, selfPID=3156, iMonCtr=1 Model crash detected, will try to restart... 14:20:08 (5372): No heartbeat from core client for 30 sec - exiting 14:20:09 (5372): No heartbeat from core client for 30 sec - exiting 14:20:10 (5372): No heartbeat from core client for 30 sec - exiting 14:20:11 (5372): No heartbeat from core client for 30 sec - exiting 14:20:12 (5372): No heartbeat from core client for 30 sec - exiting 14:20:13 (5372): No heartbeat from core client for 30 sec - exiting 14:20:14 (5372): No heartbeat from core client for 30 sec - exiting 14:20:15 (5372): No heartbeat from core client for 30 sec - exiting 14:20:16 (5372): No heartbeat from core client for 30 sec - exiting 14:20:17 (5372): No heartbeat from core client for 30 sec - exiting 14:20:18 (5372): No heartbeat from core client for 30 sec - exiting 14:20:19 (5372): No heartbeat from core client for 30 sec - exiting 14:20:20 (5372): No heartbeat from core client for 30 sec - exiting 14:20:21 (5372): No heartbeat from core client for 30 sec - exiting 14:20:22 (5372): No heartbeat from core client for 30 sec - exiting 14:20:23 (5372): No heartbeat from core client for 30 sec - exiting 14:20:24 (5372): No heartbeat from core client for 30 sec - exiting 14:20:25 (5372): No heartbeat from core client for 30 sec - exiting 14:20:26 (5372): No heartbeat from core client for 30 sec - exiting 14:20:27 (5372): No heartbeat from core client for 30 sec - exiting 14:20:28 (5372): No heartbeat from core client for 30 sec - exiting 14:20:29 (5372): No heartbeat from core client for 30 sec - exiting 14:20:30 (5372): No heartbeat from core client for 30 sec - exiting 14:20:31 (5372): No heartbeat from core client for 30 sec - exiting 14:20:32 (5372): No heartbeat from core client for 30 sec - exiting 14:20:33 (5372): No heartbeat from core client for 30 sec - exiting 14:20:34 (5372): No heartbeat from core client for 30 sec - exiting 14:20:35 (5372): No heartbeat from core client for 30 sec - exiting 14:20:36 (5372): No heartbeat from core client for 30 sec - exiting 14:20:37 (5372): No heartbeat from core client for 30 sec - exiting 14:20:38 (5372): No heartbeat from core client for 30 sec - exiting 14:20:39 (5372): No heartbeat from core client for 30 sec - exiting 14:20:40 (5372): No heartbeat from core client for 30 sec - exiting 14:20:41 (5372): No heartbeat from core client for 30 sec - exiting 14:20:42 (5372): No heartbeat from core client for 30 sec - exiting 14:20:43 (5372): No heartbeat from core client for 30 sec - exiting 14:20:44 (5372): No heartbeat from core client for 30 sec - exiting 14:20:45 (5372): No heartbeat from core client for 30 sec - exiting 14:20:46 (5372): No heartbeat from core client for 30 sec - exiting 14:20:47 (5372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5184, selfPID=4164, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5340, selfPID=4124, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 7 forrtl: severe (24): end-of-file during read, unit 9, file C:\ProgramData\BOINC\projects\climateprediction.net\hadam3p_pnw_8ncl_2004_1_007703217\tmp\xaakg.namelists Image PC Routine Line Source hadrm3p_pnw_um_6. 00FBC52A Unknown Unknown Unknown hadrm3p_pnw_um_6. 00F64460 Unknown Unknown Unknown hadrm3p_pnw_um_6. 00F6362A Unknown Unknown Unknown hadrm3p_pnw_um_6. 00F42469 Unknown Unknown Unknown hadrm3p_pnw_um_6. 00E466EB Unknown Unknown Unknown hadrm3p_pnw_um_6. 00EE2AE2 Unknown Unknown Unknown hadrm3p_pnw_um_6. 00EE35AF Unknown Unknown Unknown hadrm3p_pnw_um_6. 00C89860 Unknown Unknown Unknown hadrm3p_pnw_um_6. 00FA0893 Unknown Unknown Unknown kernel32.dll 75C9339A Unknown Unknown Unknown ntdll.dll 773C9EF2 Unknown Unknown Unknown ntdll.dll 773C9EC5 Unknown Unknown Unknown forrtl: severe (24): end-of-file during read, unit 9, file C:\ProgramData\BOINC\projects\climateprediction.net\hadam3p_pnw_8ncl_2004_1_007703217\tmp\xaakm.namelists Image PC Routine Line Source hadam3p_pnw_um_6. 0167A39A Unknown Unknown Unknown hadam3p_pnw_um_6. 01622CD0 Unknown Unknown Unknown hadam3p_pnw_um_6. 01621E9A Unknown Unknown Unknown hadam3p_pnw_um_6. 01602819 Unknown Unknown Unknown hadam3p_pnw_um_6. 01502287 Unknown Unknown Unknown hadam3p_pnw_um_6. 0159E7B2 Unknown Unknown Unknown hadam3p_pnw_um_6. 0159F2DA Unknown Unknown Unknown hadam3p_pnw_um_6. 01319BD2 Unknown Unknown Unknown hadam3p_pnw_um_6. 0165E638 Unknown Unknown Unknown kernel32.dll 75C9339A Unknown Unknown Unknown ntdll.dll 773C9EF2 Unknown Unknown Unknown ntdll.dll 773C9EC5 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6420, selfPID=6188, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 4 Called boinc_finish </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_pnw_8ncl_2004_1_007703217_1_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_8ncl_2004_1_007703217_1_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_8ncl_2004_1_007703217_1_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_8ncl_2004_1_007703217_1_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_8ncl_2004_1_007703217_1_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Jul 2012 17:24:17	1131474	14328061	hadam3p_pnw_8ncl_2004_1_007703217_1	80,736	389,048	4.8188
03 Jul 2012 12:08:39	1131474	14328061	hadam3p_pnw_8ncl_2004_1_007703217_1	69,223	328,211	4.7414
03 Jul 2012 11:08:28	1131474	14328061	hadam3p_pnw_8ncl_2004_1_007703217_1	69,217	327,651	4.7337
02 Jul 2012 22:36:10	1131474	14328061	hadam3p_pnw_8ncl_2004_1_007703217_1	69,216	327,107	4.7259
22 Jun 2012 09:36:54	1131474	14328061	hadam3p_pnw_8ncl_2004_1_007703217_1	57,696	269,055	4.6633
06 Jun 2012 22:01:16	1131474	14328061	hadam3p_pnw_8ncl_2004_1_007703217_1	46,176	211,869	4.5883
06 May 2012 22:25:20	1131474	14328061	hadam3p_pnw_8ncl_2004_1_007703217_1	34,656	156,005	4.5015
29 Apr 2012 12:37:19	1131474	14328061	hadam3p_pnw_8ncl_2004_1_007703217_1	23,136	102,660	4.4372
16 Apr 2012 20:23:38	1131474	14328061	hadam3p_pnw_8ncl_2004_1_007703217_1	11,617	51,153	4.4033
16 Apr 2012 11:57:53	1131474	14328061	hadam3p_pnw_8ncl_2004_1_007703217_1	11,616	50,644	4.3598