Task 12161394

Name	hadam3p_saf_0zf2_1973_1_006888678_0
Workunit	7091994
Created	19 Nov 2010, 16:37:28 UTC
Sent	6 Apr 2011, 7:59:31 UTC
Report deadline	18 Mar 2012, 13:19:31 UTC
Received	21 May 2011, 18:56:42 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	893865
Run time	5 days 0 hours 8 min 5 sec
CPU time	4 days 1 hours 38 min 44 sec
Validate state	Invalid
Credit	1,309.70
Device peak FLOPS	2.21 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Southern Africa v6.09 windows_intelx86
Stderr	<core_client_version>6.12.26</core_client_version> <![CDATA[ <stderr_txt> 22:52:43 (5892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:41:30 (4660): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 03:41:32 (4660): No heartbeat from core client for 30 sec - exiting 03:41:33 (4660): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1428, iMonCtr=2 08:56:26 (6304): No heartbeat from core client for 30 sec - exiting 08:56:27 (6304): No heartbeat from core client for 30 sec - exiting 08:56:28 (6304): No heartbeat from core client for 30 sec - exiting 08:56:29 (6304): No heartbeat from core client for 30 sec - exiting 08:56:30 (6304): No heartbeat from core client for 30 sec - exiting 08:56:31 (6304): No heartbeat from core client for 30 sec - exiting 08:56:32 (6304): No heartbeat from core client for 30 sec - exiting 08:56:33 (6304): No heartbeat from core client for 30 sec - exiting 08:56:34 (6304): No heartbeat from core client for 30 sec - exiting 08:56:35 (6304): No heartbeat from core client for 30 sec - exiting 08:56:36 (6304): No heartbeat from core client for 30 sec - exiting 08:56:37 (6304): No heartbeat from core client for 30 sec - exiting 08:56:38 (6304): No heartbeat from core client for 30 sec - exiting 08:56:39 (6304): No heartbeat from core client for 30 sec - exiting 08:56:40 (6304): No heartbeat from core client for 30 sec - exiting 08:56:41 (6304): No heartbeat from core client for 30 sec - exiting 08:56:42 (6304): No heartbeat from core client for 30 sec - exiting 08:56:43 (6304): No heartbeat from core client for 30 sec - exiting 08:56:44 (6304): No heartbeat from core client for 30 sec - exiting 08:56:45 (6304): No heartbeat from core client for 30 sec - exiting 08:56:46 (6304): No heartbeat from core client for 30 sec - exiting 08:56:47 (6304): No heartbeat from core client for 30 sec - exiting 08:56:48 (6304): No heartbeat from core client for 30 sec - exiting 08:56:49 (6304): No heartbeat from core client for 30 sec - exiting 08:56:50 (6304): No heartbeat from core client for 30 sec - exiting 08:56:51 (6304): No heartbeat from core client for 30 sec - exiting 08:56:52 (6304): No heartbeat from core client for 30 sec - exiting 08:56:53 (6304): No heartbeat from core client for 30 sec - exiting 08:56:54 (6304): No heartbeat from core client for 30 sec - exiting 08:56:55 (6304): No heartbeat from core client for 30 sec - exiting 08:56:56 (6304): No heartbeat from core client for 30 sec - exiting 08:56:57 (6304): No heartbeat from core client for 30 sec - exiting 08:56:58 (6304): No heartbeat from core client for 30 sec - exiting 08:56:59 (6304): No heartbeat from core client for 30 sec - exiting 08:57:00 (6304): No heartbeat from core client for 30 sec - exiting 08:57:01 (6304): No heartbeat from core client for 30 sec - exiting 08:57:02 (6304): No heartbeat from core client for 30 sec - exiting 08:57:03 (6304): No heartbeat from core client for 30 sec - exiting 08:57:04 (6304): No heartbeat from core client for 30 sec - exiting 08:57:05 (6304): No heartbeat from core client for 30 sec - exiting 08:57:06 (6304): No heartbeat from core client for 30 sec - exiting 08:57:07 (6304): No heartbeat from core client for 30 sec - exiting 08:57:08 (6304): No heartbeat from core client for 30 sec - exiting 08:57:09 (6304): No heartbeat from core client for 30 sec - exiting 08:57:10 (6304): No heartbeat from core client for 30 sec - exiting 08:57:11 (6304): No heartbeat from core client for 30 sec - exiting 08:57:12 (6304): No heartbeat from core client for 30 sec - exiting 08:57:13 (6304): No heartbeat from core client for 30 sec - exiting 08:57:14 (6304): No heartbeat from core client for 30 sec - exiting 08:57:15 (6304): No heartbeat from core client for 30 sec - exiting 08:57:16 (6304): No heartbeat from core client for 30 sec - exiting 08:57:17 (6304): No heartbeat from core client for 30 sec - exiting 08:57:18 (6304): No heartbeat from core client for 30 sec - exiting 08:57:19 (6304): No heartbeat from core client for 30 sec - exiting 08:57:20 (6304): No heartbeat from core client for 30 sec - exiting 08:57:21 (6304): No heartbeat from core client for 30 sec - exiting 08:57:22 (6304): No heartbeat from core client for 30 sec - exiting 08:57:23 (6304): No heartbeat from core client for 30 sec - exiting 08:57:24 (6304): No heartbeat from core client for 30 sec - exiting 08:57:25 (6304): No heartbeat from core client for 30 sec - exiting 08:57:26 (6304): No heartbeat from core client for 30 sec - exiting 08:57:27 (6304): No heartbeat from core client for 30 sec - exiting 08:57:28 (6304): No heartbeat from core client for 30 sec - exiting 08:57:29 (6304): No heartbeat from core client for 30 sec - exiting 08:57:30 (6304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:57:32 (6304): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5648, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1512, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 19:45:37 (5152): No heartbeat from core client for 30 sec - exiting 19:45:38 (5152): No heartbeat from core client for 30 sec - exiting 19:45:39 (5152): No heartbeat from core client for 30 sec - exiting 19:45:40 (5152): No heartbeat from core client for 30 sec - exiting 19:45:41 (5152): No heartbeat from core client for 30 sec - exiting 19:45:42 (5152): No heartbeat from core client for 30 sec - exiting 19:45:43 (5152): No heartbeat from core client for 30 sec - exiting 19:45:44 (5152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:11:14 (9216): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 04:11:15 (9216): No heartbeat from core client for 30 sec - exiting 04:11:16 (9216): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 23:34:40 (5504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:34:41 (5504): No heartbeat from core client for 30 sec - exiting 23:34:42 (5504): No heartbeat from core client for 30 sec - exiting 23:34:43 (5504): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4300, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=524, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:50:21 (7324): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 03:50:23 (7324): No heartbeat from core client for 30 sec - exiting 03:50:24 (7324): No heartbeat from core client for 30 sec - exiting 03:50:25 (7324): No heartbeat from core client for 30 sec - exiting 03:50:26 (7324): No heartbeat from core client for 30 sec - exiting 10:35:33 (6240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:35:35 (6240): No heartbeat from core client for 30 sec - exiting 10:35:36 (6240): No heartbeat from core client for 30 sec - exiting 10:35:37 (6240): No heartbeat from core client for 30 sec - exiting 10:35:38 (6240): No heartbeat from core client for 30 sec - exiting 10:35:39 (6240): No heartbeat from core client for 30 sec - exiting 10:35:40 (6240): No heartbeat from core client for 30 sec - exiting 10:35:41 (6240): No heartbeat from core client for 30 sec - exiting 10:35:42 (6240): No heartbeat from core client for 30 sec - exiting 10:35:43 (6240): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4092, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3052, selfPID=7420, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:41:49 (4044): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 03:41:50 (4044): No heartbeat from core client for 30 sec - exiting 04:21:20 (4332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7676, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 07:34:34 (7716): No heartbeat from core client for 30 sec - exiting 07:34:35 (7716): No heartbeat from core client for 30 sec - exiting 07:34:36 (7716): No heartbeat from core client for 30 sec - exiting 07:34:37 (7716): No heartbeat from core client for 30 sec - exiting 07:34:38 (7716): No heartbeat from core client for 30 sec - exiting 07:34:39 (7716): No heartbeat from core client for 30 sec - exiting 07:34:40 (7716): No heartbeat from core client for 30 sec - exiting 07:34:41 (7716): No heartbeat from core client for 30 sec - exiting 07:34:42 (7716): No heartbeat from core client for 30 sec - exiting 07:34:43 (7716): No heartbeat from core client for 30 sec - exiting 07:34:44 (7716): No heartbeat from core client for 30 sec - exiting 07:34:45 (7716): No heartbeat from core client for 30 sec - exiting 07:34:46 (7716): No heartbeat from core client for 30 sec - exiting 07:34:47 (7716): No heartbeat from core client for 30 sec - exiting 07:34:48 (7716): No heartbeat from core client for 30 sec - exiting 07:34:49 (7716): No heartbeat from core client for 30 sec - exiting 07:34:50 (7716): No heartbeat from core client for 30 sec - exiting 07:34:51 (7716): No heartbeat from core client for 30 sec - exiting 07:34:52 (7716): No heartbeat from core client for 30 sec - exiting 07:34:53 (7716): No heartbeat from core client for 30 sec - exiting 07:34:54 (7716): No heartbeat from core client for 30 sec - exiting 07:34:55 (7716): No heartbeat from core client for 30 sec - exiting 07:34:56 (7716): No heartbeat from core client for 30 sec - exiting 07:34:57 (7716): No heartbeat from core client for 30 sec - exiting 07:34:58 (7716): No heartbeat from core client for 30 sec - exiting 07:34:59 (7716): No heartbeat from core client for 30 sec - exiting 07:35:00 (7716): No heartbeat from core client for 30 sec - exiting 07:35:01 (7716): No heartbeat from core client for 30 sec - exiting 07:35:02 (7716): No heartbeat from core client for 30 sec - exiting 07:35:03 (7716): No heartbeat from core client for 30 sec - exiting 07:35:04 (7716): No heartbeat from core client for 30 sec - exiting 07:35:05 (7716): No heartbeat from core client for 30 sec - exiting 07:35:06 (7716): No heartbeat from core client for 30 sec - exiting 07:35:07 (7716): No heartbeat from core client for 30 sec - exiting 07:35:08 (7716): No heartbeat from core client for 30 sec - exiting 07:35:09 (7716): No heartbeat from core client for 30 sec - exiting 07:35:10 (7716): No heartbeat from core client for 30 sec - exiting 07:35:11 (7716): No heartbeat from core client for 30 sec - exiting 07:35:12 (7716): No heartbeat from core client for 30 sec - exiting 07:35:45 (7716): No heartbeat from core client for 30 sec - exiting 07:35:46 (7716): No heartbeat from core client for 30 sec - exiting 07:35:48 (7716): No heartbeat from core client for 30 sec - exiting 07:35:49 (7716): No heartbeat from core client for 30 sec - exiting 07:35:50 (7716): No heartbeat from core client for 30 sec - exiting 07:35:51 (7716): No heartbeat from core client for 30 sec - exiting 07:35:52 (7716): No heartbeat from core client for 30 sec - exiting 07:35:53 (7716): No heartbeat from core client for 30 sec - exiting 07:35:54 (7716): No heartbeat from core client for 30 sec - exiting 07:35:55 (7716): No heartbeat from core client for 30 sec - exiting 07:35:56 (7716): No heartbeat from core client for 30 sec - exiting 07:35:57 (7716): No heartbeat from core client for 30 sec - exiting 07:35:58 (7716): No heartbeat from core client for 30 sec - exiting 07:35:59 (7716): No heartbeat from core client for 30 sec - exiting 07:36:00 (7716): No heartbeat from core client for 30 sec - exiting 07:36:01 (7716): No heartbeat from core client for 30 sec - exiting 07:36:02 (7716): No heartbeat from core client for 30 sec - exiting 07:36:03 (7716): No heartbeat from core client for 30 sec - exiting 07:36:04 (7716): No heartbeat from core client for 30 sec - exiting 07:36:05 (7716): No heartbeat from core client for 30 sec - exiting 07:36:06 (7716): No heartbeat from core client for 30 sec - exiting 07:36:07 (7716): No heartbeat from core client for 30 sec - exiting 07:36:08 (7716): No heartbeat from core client for 30 sec - exiting 07:36:09 (7716): No heartbeat from core client for 30 sec - exiting 07:36:10 (7716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:37:52 (7996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:44:05 (5400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6284, selfPID=6284, iMonCtr=2 22:54:46 (5600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:59:24 (500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7460, selfPID=7460, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3856, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 10:05:32 (7020): No heartbeat from core client for 30 sec - exiting 10:05:34 (7020): No heartbeat from core client for 30 sec - exiting 10:05:35 (7020): No heartbeat from core client for 30 sec - exiting 10:05:36 (7020): No heartbeat from core client for 30 sec - exiting 10:05:37 (7020): No heartbeat from core client for 30 sec - exiting 10:05:38 (7020): No heartbeat from core client for 30 sec - exiting 10:05:40 (7020): No heartbeat from core client for 30 sec - exiting 10:05:41 (7020): No heartbeat from core client for 30 sec - exiting 10:05:42 (7020): No heartbeat from core client for 30 sec - exiting 10:05:43 (7020): No heartbeat from core client for 30 sec - exiting 10:05:44 (7020): No heartbeat from core client for 30 sec - exiting 10:05:45 (7020): No heartbeat from core client for 30 sec - exiting 10:05:46 (7020): No heartbeat from core client for 30 sec - exiting 10:05:47 (7020): No heartbeat from core client for 30 sec - exiting 10:05:48 (7020): No heartbeat from core client for 30 sec - exiting 10:05:49 (7020): No heartbeat from core client for 30 sec - exiting 10:05:50 (7020): No heartbeat from core client for 30 sec - exiting 10:05:51 (7020): No heartbeat from core client for 30 sec - exiting 10:05:52 (7020): No heartbeat from core client for 30 sec - exiting 10:05:53 (7020): No heartbeat from core client for 30 sec - exiting 10:05:54 (7020): No heartbeat from core client for 30 sec - exiting 10:05:55 (7020): No heartbeat from core client for 30 sec - exiting 10:05:56 (7020): No heartbeat from core client for 30 sec - exiting 10:05:57 (7020): No heartbeat from core client for 30 sec - exiting 10:05:58 (7020): No heartbeat from core client for 30 sec - exiting 10:05:59 (7020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:07:55 (4304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:08:31 (7464): No heartbeat from core client for 30 sec - exiting 23:08:32 (7464): No heartbeat from core client for 30 sec - exiting 23:08:33 (7464): No heartbeat from core client for 30 sec - exiting 23:08:34 (7464): No heartbeat from core client for 30 sec - exiting 23:08:35 (7464): No heartbeat from core client for 30 sec - exiting 23:08:36 (7464): No heartbeat from core client for 30 sec - exiting 23:08:37 (7464): No heartbeat from core client for 30 sec - exiting 23:08:38 (7464): No heartbeat from core client for 30 sec - exiting 23:08:39 (7464): No heartbeat from core client for 30 sec - exiting 23:08:40 (7464): No heartbeat from core client for 30 sec - exiting 23:08:41 (7464): No heartbeat from core client for 30 sec - exiting 23:08:42 (7464): No heartbeat from core client for 30 sec - exiting 23:08:43 (7464): No heartbeat from core client for 30 sec - exiting 23:08:44 (7464): No heartbeat from core client for 30 sec - exiting 23:08:45 (7464): No heartbeat from core client for 30 sec - exiting 23:08:46 (7464): No heartbeat from core client for 30 sec - exiting 23:08:47 (7464): No heartbeat from core client for 30 sec - exiting 23:08:48 (7464): No heartbeat from core client for 30 sec - exiting 23:08:49 (7464): No heartbeat from core client for 30 sec - exiting 23:08:50 (7464): No heartbeat from core client for 30 sec - exiting 23:08:51 (7464): No heartbeat from core client for 30 sec - exiting 23:08:52 (7464): No heartbeat from core client for 30 sec - exiting 23:08:53 (7464): No heartbeat from core client for 30 sec - exiting 23:08:54 (7464): No heartbeat from core client for 30 sec - exiting 23:08:55 (7464): No heartbeat from core client for 30 sec - exiting 23:08:56 (7464): No heartbeat from core client for 30 sec - exiting 23:08:57 (7464): No heartbeat from core client for 30 sec - exiting 23:08:58 (7464): No heartbeat from core client for 30 sec - exiting 23:08:59 (7464): No heartbeat from core client for 30 sec - exiting 23:09:00 (7464): No heartbeat from core client for 30 sec - exiting 23:09:01 (7464): No heartbeat from core client for 30 sec - exiting 23:09:02 (7464): No heartbeat from core client for 30 sec - exiting 23:09:03 (7464): No heartbeat from core client for 30 sec - exiting 23:09:04 (7464): No heartbeat from core client for 30 sec - exiting 23:09:05 (7464): No heartbeat from core client for 30 sec - exiting 23:09:06 (7464): No heartbeat from core client for 30 sec - exiting 23:09:07 (7464): No heartbeat from core client for 30 sec - exiting 23:09:08 (7464): No heartbeat from core client for 30 sec - exiting 23:09:09 (7464): No heartbeat from core client for 30 sec - exiting 23:09:10 (7464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:09:11 (7464): No heartbeat from core client for 30 sec - exiting 23:45:58 (5960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:55:17 (7472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:05:23 (1048): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:47:14 (3960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:47:15 (3960): No heartbeat from core client for 30 sec - exiting 09:47:16 (3960): No heartbeat from core client for 30 sec - exiting 09:47:17 (3960): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:22:20 (5352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:23:13 (2560): No heartbeat from core client for 30 sec - exiting 15:23:14 (2560): No heartbeat from core client for 30 sec - exiting 15:23:15 (2560): No heartbeat from core client for 30 sec - exiting 15:23:16 (2560): No heartbeat from core client for 30 sec - exiting 15:23:17 (2560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:38:11 (1044): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 16:38:12 (1044): No heartbeat from core client for 30 sec - exiting 16:38:13 (1044): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5520, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 15:50:01 (4964): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 15:50:02 (4964): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:23:43 (8184): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 23:23:45 (8184): No heartbeat from core client for 30 sec - exiting 23:23:46 (8184): No heartbeat from core client for 30 sec - exiting 23:23:47 (8184): No heartbeat from core client for 30 sec - exiting 23:23:48 (8184): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:48:30 (8104): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 23:48:31 (8104): No heartbeat from core client for 30 sec - exiting 23:48:32 (8104): No heartbeat from core client for 30 sec - exiting 23:48:33 (8104): No heartbeat from core client for 30 sec - exiting 23:48:34 (8104): No heartbeat from core client for 30 sec - exiting 23:48:35 (8104): No heartbeat from core client for 30 sec - exiting 23:48:36 (8104): No heartbeat from core client for 30 sec - exiting 23:48:37 (8104): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6748, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2816, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:08:52 (8016): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 11:08:56 (8016): No heartbeat from core client for 30 sec - exiting 11:08:57 (8016): No heartbeat from core client for 30 sec - exiting 11:08:58 (8016): No heartbeat from core client for 30 sec - exiting 11:08:59 (8016): No heartbeat from core client for 30 sec - exiting 11:09:00 (8016): No heartbeat from core client for 30 sec - exiting 11:09:01 (8016): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:57:27 (6748): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4688, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:11:59 (4724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:12:00 (4724): No heartbeat from core client for 30 sec - exiting 23:12:01 (4724): No heartbeat from core client for 30 sec - exiting 23:12:02 (4724): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 00:12:57 (5280): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 00:12:58 (5280): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 04:52:41 (6248): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 04:52:42 (6248): No heartbeat from core client for 30 sec - exiting 04:52:43 (6248): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 14:21:44 (1320): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 14:21:45 (1320): No heartbeat from core client for 30 sec - exiting 14:21:46 (1320): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:47:40 (6344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4980, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 07:11:16 (4980): No heartbeat from core client for 30 sec - exiting 07:11:17 (4980): No heartbeat from core client for 30 sec - exiting 07:11:18 (4980): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7508, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:10:01 (7180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:10:03 (7180): No heartbeat from core client for 30 sec - exiting 09:10:04 (7180): No heartbeat from core client for 30 sec - exiting 09:10:05 (7180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:28:39 (5216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:28:42 (5216): No heartbeat from core client for 30 sec - exiting 22:28:43 (5216): No heartbeat from core client for 30 sec - exiting 22:28:44 (5216): No heartbeat from core client for 30 sec - exiting 22:28:45 (5216): No heartbeat from core client for 30 sec - exiting 22:28:46 (5216): No heartbeat from core client for 30 sec - exiting 22:28:47 (5216): No heartbeat from core client for 30 sec - exiting 22:28:48 (5216): No heartbeat from core client for 30 sec - exiting 22:28:49 (5216): No heartbeat from core client for 30 sec - exiting 22:44:36 (4020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:44:38 (4020): No heartbeat from core client for 30 sec - exiting 22:44:39 (4020): No heartbeat from core client for 30 sec - exiting 22:44:40 (4020): No heartbeat from core client for 30 sec - exiting 22:48:03 (8092): No heartbeat from core client for 30 sec - exiting 22:48:05 (8092): No heartbeat from core client for 30 sec - exiting 22:48:06 (8092): No heartbeat from core client for 30 sec - exiting 22:48:07 (8092): No heartbeat from core client for 30 sec - exiting 22:48:08 (8092): No heartbeat from core client for 30 sec - exiting 22:48:09 (8092): No heartbeat from core client for 30 sec - exiting 22:48:10 (8092): No heartbeat from core client for 30 sec - exiting 22:48:11 (8092): No heartbeat from core client for 30 sec - exiting 22:48:12 (8092): No heartbeat from core client for 30 sec - exiting 22:48:13 (8092): No heartbeat from core client for 30 sec - exiting 22:48:14 (8092): No heartbeat from core client for 30 sec - exiting 22:48:15 (8092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7716, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_saf_0zf2_1973_1_006888678_0_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_0zf2_1973_1_006888678_0_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_0zf2_1973_1_006888678_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_0zf2_1973_1_006888678_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_0zf2_1973_1_006888678_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 May 2011 03:49:42	893865	12161394	hadam3p_saf_0zf2_1973_1_006888678_0	80,736	320,079	3.9645
14 May 2011 05:57:31	893865	12161394	hadam3p_saf_0zf2_1973_1_006888678_0	69,216	273,314	3.9487
08 May 2011 01:55:05	893865	12161394	hadam3p_saf_0zf2_1973_1_006888678_0	57,696	227,778	3.9479
30 Apr 2011 17:12:04	893865	12161394	hadam3p_saf_0zf2_1973_1_006888678_0	46,176	182,544	3.9532
23 Apr 2011 14:57:37	893865	12161394	hadam3p_saf_0zf2_1973_1_006888678_0	34,656	137,016	3.9536
21 Apr 2011 12:00:28	893865	12161394	hadam3p_saf_0zf2_1973_1_006888678_0	23,151	92,286	3.9863
21 Apr 2011 12:00:28	893865	12161394	hadam3p_saf_0zf2_1973_1_006888678_0	23,136	91,616	3.9599
10 Apr 2011 13:44:52	893865	12161394	hadam3p_saf_0zf2_1973_1_006888678_0	11,616	45,668	3.9315