Task 14983383

Name	hadcm3n_2014_1980_40_008085725_1
Workunit	8240839
Created	23 Jul 2012, 12:04:10 UTC
Sent	23 Jul 2012, 18:34:27 UTC
Report deadline	23 Oct 2012, 2:01:38 UTC
Received	30 Sep 2012, 23:54:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1178110
Run time	44 days 22 hours 35 min 51 sec
CPU time	43 days 5 hours 23 min 50 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	1.40 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:16:08 (5032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:20:23 (3804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4836, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4172, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:42:26 (2124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:37:54 (1256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:37:55 (1256): No heartbeat from core client for 30 sec - exiting 07:37:57 (1256): No heartbeat from core client for 30 sec - exiting 07:37:58 (1256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:02:04 (3984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:03:03 (5948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:27:51 (4848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:27:52 (4848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:46:05 (4928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:45:15 (5068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:45:16 (5068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:26:11 (1712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:05:20 (5604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:05:54 (5800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:13:55 (4716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:14:51 (5772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:10:39 (5324): No heartbeat from core client for 30 sec - exiting 17:10:40 (5324): No heartbeat from core client for 30 sec - exiting 17:10:41 (5324): No heartbeat from core client for 30 sec - exiting 17:10:42 (5324): No heartbeat from core client for 30 sec - exiting 17:10:43 (5324): No heartbeat from core client for 30 sec - exiting 17:10:45 (5324): No heartbeat from core client for 30 sec - exiting 17:10:46 (5324): No heartbeat from core client for 30 sec - exiting 17:10:47 (5324): No heartbeat from core client for 30 sec - exiting 17:10:48 (5324): No heartbeat from core client for 30 sec - exiting 17:10:49 (5324): No heartbeat from core client for 30 sec - exiting 17:10:50 (5324): No heartbeat from core client for 30 sec - exiting 17:10:51 (5324): No heartbeat from core client for 30 sec - exiting 17:10:52 (5324): No heartbeat from core client for 30 sec - exiting 17:10:53 (5324): No heartbeat from core client for 30 sec - exiting 17:10:54 (5324): No heartbeat from core client for 30 sec - exiting 17:10:55 (5324): No heartbeat from core client for 30 sec - exiting 17:10:57 (5324): No heartbeat from core client for 30 sec - exiting 17:10:58 (5324): No heartbeat from core client for 30 sec - exiting 17:10:59 (5324): No heartbeat from core client for 30 sec - exiting 17:11:00 (5324): No heartbeat from core client for 30 sec - exiting 17:11:01 (5324): No heartbeat from core client for 30 sec - exiting 17:11:02 (5324): No heartbeat from core client for 30 sec - exiting 17:11:03 (5324): No heartbeat from core client for 30 sec - exiting 17:11:04 (5324): No heartbeat from core client for 30 sec - exiting 17:11:05 (5324): No heartbeat from core client for 30 sec - exiting 17:11:06 (5324): No heartbeat from core client for 30 sec - exiting 17:11:08 (5324): No heartbeat from core client for 30 sec - exiting 17:11:09 (5324): No heartbeat from core client for 30 sec - exiting 17:11:10 (5324): No heartbeat from core client for 30 sec - exiting 17:11:11 (5324): No heartbeat from core client for 30 sec - exiting 17:11:12 (5324): No heartbeat from core client for 30 sec - exiting 17:11:13 (5324): No heartbeat from core client for 30 sec - exiting 17:11:14 (5324): No heartbeat from core client for 30 sec - exiting 17:11:15 (5324): No heartbeat from core client for 30 sec - exiting 17:11:16 (5324): No heartbeat from core client for 30 sec - exiting 17:11:17 (5324): No heartbeat from core client for 30 sec - exiting 17:11:18 (5324): No heartbeat from core client for 30 sec - exiting 17:11:20 (5324): No heartbeat from core client for 30 sec - exiting 17:11:21 (5324): No heartbeat from core client for 30 sec - exiting 17:11:22 (5324): No heartbeat from core client for 30 sec - exiting 17:11:23 (5324): No heartbeat from core client for 30 sec - exiting 17:11:24 (5324): No heartbeat from core client for 30 sec - exiting 17:11:25 (5324): No heartbeat from core client for 30 sec - exiting 17:11:26 (5324): No heartbeat from core client for 30 sec - exiting 17:11:27 (5324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:13:06 (5460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6372, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:41:02 (6904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:41:53 (5556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:48:15 (5124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:49:01 (2508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 06:10:19 (5736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:13:25 (2924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:11:32 (4204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:12:17 (5700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:58:30 (3960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5920, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5920, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5920, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5920, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3500, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3500, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3500, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish 00:04:59 (3500): No heartbeat from core client for 30 sec - exiting 00:05:00 (3500): No heartbeat from core client for 30 sec - exiting 00:06:46 (4084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:07:26 (5532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:07:26 (1820): Can't acquire lockfile (32) - waiting 35s 00:08:47 (1820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:09:46 (3968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:10:57 (2152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:47:17 (5860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:47:18 (5860): No heartbeat from core client for 30 sec - exiting 12:47:19 (5860): No heartbeat from core client for 30 sec - exiting 12:48:12 (2960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:48:14 (2960): No heartbeat from core client for 30 sec - exiting 12:48:15 (2960): No heartbeat from core client for 30 sec - exiting 12:48:16 (2960): No heartbeat from core client for 30 sec - exiting 12:48:17 (2960): No heartbeat from core client for 30 sec - exiting 12:48:18 (2960): No heartbeat from core client for 30 sec - exiting 12:50:21 (3136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:51:36 (1568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:52:43 (2844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:52:44 (2844): No heartbeat from core client for 30 sec - exiting 12:52:45 (2844): No heartbeat from core client for 30 sec - exiting 12:52:46 (2844): No heartbeat from core client for 30 sec - exiting 12:52:47 (2844): No heartbeat from core client for 30 sec - exiting 12:52:48 (2844): No heartbeat from core client for 30 sec - exiting 12:52:49 (2844): No heartbeat from core client for 30 sec - exiting 12:52:50 (2844): No heartbeat from core client for 30 sec - exiting 12:52:51 (2844): No heartbeat from core client for 30 sec - exiting 12:52:53 (2844): No heartbeat from core client for 30 sec - exiting 12:52:54 (2844): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4884, iMonCtr=1 Model crash detected, will try to restart... 13:03:12 (4984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:03:52 (5640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:04:52 (5232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:05:49 (5800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:26:45 (4868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:26:46 (4868): No heartbeat from core client for 30 sec - exiting 13:26:47 (4868): No heartbeat from core client for 30 sec - exiting 13:26:48 (4868): No heartbeat from core client for 30 sec - exiting 13:26:49 (4868): No heartbeat from core client for 30 sec - exiting 13:26:51 (4868): No heartbeat from core client for 30 sec - exiting 13:26:52 (4868): No heartbeat from core client for 30 sec - exiting 13:26:53 (4868): No heartbeat from core client for 30 sec - exiting 13:26:54 (4868): No heartbeat from core client for 30 sec - exiting 13:26:55 (4868): No heartbeat from core client for 30 sec - exiting 13:26:56 (4868): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4972, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 16:07:00 (3036): No heartbeat from core client for 30 sec - exiting 16:07:01 (3036): No heartbeat from core client for 30 sec - exiting 16:07:02 (3036): No heartbeat from core client for 30 sec - exiting 16:07:03 (3036): No heartbeat from core client for 30 sec - exiting 16:07:04 (3036): No heartbeat from core client for 30 sec - exiting 16:07:05 (3036): No heartbeat from core client for 30 sec - exiting 16:07:06 (3036): No heartbeat from core client for 30 sec - exiting 16:07:07 (3036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:13:54 (5000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:16:10 (5620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:16:12 (5620): No heartbeat from core client for 30 sec - exiting 23:16:13 (5620): No heartbeat from core client for 30 sec - exiting 23:16:14 (5620): No heartbeat from core client for 30 sec - exiting 23:16:15 (5620): No heartbeat from core client for 30 sec - exiting 23:16:16 (5620): No heartbeat from core client for 30 sec - exiting 23:16:17 (5620): No heartbeat from core client for 30 sec - exiting 23:16:18 (5620): No heartbeat from core client for 30 sec - exiting 23:16:19 (5620): No heartbeat from core client for 30 sec - exiting 23:16:20 (5620): No heartbeat from core client for 30 sec - exiting 23:16:21 (5620): No heartbeat from core client for 30 sec - exiting 16:50:42 (3832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:50:43 (3832): No heartbeat from core client for 30 sec - exiting 16:50:44 (3832): No heartbeat from core client for 30 sec - exiting 16:50:45 (3832): No heartbeat from core client for 30 sec - exiting 16:50:46 (3832): No heartbeat from core client for 30 sec - exiting 16:50:47 (3832): No heartbeat from core client for 30 sec - exiting 16:50:48 (3832): No heartbeat from core client for 30 sec - exiting 16:50:49 (3832): No heartbeat from core client for 30 sec - exiting 16:50:50 (3832): No heartbeat from core client for 30 sec - exiting 16:50:51 (3832): No heartbeat from core client for 30 sec - exiting 16:50:53 (3832): No heartbeat from core client for 30 sec - exiting 16:52:34 (4076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:58:43 (4948): No heartbeat from core client for 30 sec - exiting 22:58:44 (4948): No heartbeat from core client for 30 sec - exiting 22:58:45 (4948): No heartbeat from core client for 30 sec - exiting 22:58:46 (4948): No heartbeat from core client for 30 sec - exiting 22:58:48 (4948): No heartbeat from core client for 30 sec - exiting 22:58:49 (4948): No heartbeat from core client for 30 sec - exiting 22:58:50 (4948): No heartbeat from core client for 30 sec - exiting 22:58:51 (4948): No heartbeat from core client for 30 sec - exiting 22:58:52 (4948): No heartbeat from core client for 30 sec - exiting 22:58:53 (4948): No heartbeat from core client for 30 sec - exiting 22:58:54 (4948): No heartbeat from core client for 30 sec - exiting 22:58:55 (4948): No heartbeat from core client for 30 sec - exiting 22:58:56 (4948): No heartbeat from core client for 30 sec - exiting 22:58:57 (4948): No heartbeat from core client for 30 sec - exiting 22:58:59 (4948): No heartbeat from core client for 30 sec - exiting 22:59:00 (4948): No heartbeat from core client for 30 sec - exiting 22:59:01 (4948): No heartbeat from core client for 30 sec - exiting 22:59:02 (4948): No heartbeat from core client for 30 sec - exiting 22:59:03 (4948): No heartbeat from core client for 30 sec - exiting 22:59:04 (4948): No heartbeat from core client for 30 sec - exiting 22:59:05 (4948): No heartbeat from core client for 30 sec - exiting 22:59:06 (4948): No heartbeat from core client for 30 sec - exiting 22:59:07 (4948): No heartbeat from core client for 30 sec - exiting 22:59:08 (4948): No heartbeat from core client for 30 sec - exiting 22:59:09 (4948): No heartbeat from core client for 30 sec - exiting 22:59:11 (4948): No heartbeat from core client for 30 sec - exiting 22:59:12 (4948): No heartbeat from core client for 30 sec - exiting 22:59:13 (4948): No heartbeat from core client for 30 sec - exiting 22:59:14 (4948): No heartbeat from core client for 30 sec - exiting 22:59:15 (4948): No heartbeat from core client for 30 sec - exiting 22:59:16 (4948): No heartbeat from core client for 30 sec - exiting 22:59:17 (4948): No heartbeat from core client for 30 sec - exiting 22:59:18 (4948): No heartbeat from core client for 30 sec - exiting 22:59:19 (4948): No heartbeat from core client for 30 sec - exiting 22:59:20 (4948): No heartbeat from core client for 30 sec - exiting 22:59:21 (4948): No heartbeat from core client for 30 sec - exiting 22:59:23 (4948): No heartbeat from core client for 30 sec - exiting 22:59:24 (4948): No heartbeat from core client for 30 sec - exiting 22:59:25 (4948): No heartbeat from core client for 30 sec - exiting 22:59:26 (4948): No heartbeat from core client for 30 sec - exiting 22:59:27 (4948): No heartbeat from core client for 30 sec - exiting 22:59:28 (4948): No heartbeat from core client for 30 sec - exiting 22:59:29 (4948): No heartbeat from core client for 30 sec - exiting 22:59:30 (4948): No heartbeat from core client for 30 sec - exiting 22:59:31 (4948): No heartbeat from core client for 30 sec - exiting 22:59:32 (4948): No heartbeat from core client for 30 sec - exiting 22:59:33 (4948): No heartbeat from core client for 30 sec - exiting 22:59:35 (4948): No heartbeat from core client for 30 sec - exiting 22:59:36 (4948): No heartbeat from core client for 30 sec - exiting 22:59:37 (4948): No heartbeat from core client for 30 sec - exiting 22:59:38 (4948): No heartbeat from core client for 30 sec - exiting 22:59:39 (4948): No heartbeat from core client for 30 sec - exiting 22:59:40 (4948): No heartbeat from core client for 30 sec - exiting 22:59:41 (4948): No heartbeat from core client for 30 sec - exiting 22:59:42 (4948): No heartbeat from core client for 30 sec - exiting 22:59:43 (4948): No heartbeat from core client for 30 sec - exiting 22:59:44 (4948): No heartbeat from core client for 30 sec - exiting 22:59:45 (4948): No heartbeat from core client for 30 sec - exiting 22:59:47 (4948): No heartbeat from core client for 30 sec - exiting 22:59:48 (4948): No heartbeat from core client for 30 sec - exiting 22:59:49 (4948): No heartbeat from core client for 30 sec - exiting 22:59:50 (4948): No heartbeat from core client for 30 sec - exiting 22:59:51 (4948): No heartbeat from core client for 30 sec - exiting 22:59:52 (4948): No heartbeat from core client for 30 sec - exiting 22:59:53 (4948): No heartbeat from core client for 30 sec - exiting 22:59:54 (4948): No heartbeat from core client for 30 sec - exiting 22:59:55 (4948): No heartbeat from core client for 30 sec - exiting 22:59:56 (4948): No heartbeat from core client for 30 sec - exiting 22:59:57 (4948): No heartbeat from core client for 30 sec - exiting 22:59:59 (4948): No heartbeat from core client for 30 sec - exiting 23:00:00 (4948): No heartbeat from core client for 30 sec - exiting 23:00:01 (4948): No heartbeat from core client for 30 sec - exiting 23:00:02 (4948): No heartbeat from core client for 30 sec - exiting 23:00:03 (4948): No heartbeat from core client for 30 sec - exiting 23:00:04 (4948): No heartbeat from core client for 30 sec - exiting 23:00:05 (4948): No heartbeat from core client for 30 sec - exiting 23:00:06 (4948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:42:41 (5252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:48:54 (4324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:01:59 (3156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:22:33 (4284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:28:48 (568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77A33AB3 read attempt to address 0x40C7AA75 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77A33AB3 read attempt to address 0x40C7AA75 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_2014_1980_40_008085725/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
30 Sep 2012 00:54:59	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	518,400	3,657,343	7.0551
28 Sep 2012 11:31:55	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	492,480	3,550,495	7.2094
27 Sep 2012 08:19:02	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	466,560	3,458,697	7.4132
26 Sep 2012 04:46:48	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	440,640	3,367,647	7.6426
31 Aug 2012 03:24:11	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	414,720	1,573,673	3.7945
29 Aug 2012 22:23:16	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	388,800	1,473,892	3.7909
28 Aug 2012 10:16:19	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	362,880	1,372,523	3.7823
26 Aug 2012 16:32:18	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	336,960	1,270,039	3.7691
25 Aug 2012 05:34:07	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	311,040	1,182,361	3.8013
23 Aug 2012 19:52:03	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	285,120	1,088,944	3.8192
22 Aug 2012 12:07:09	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	259,200	1,002,731	3.8686
21 Aug 2012 00:38:02	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	233,280	906,603	3.8863
19 Aug 2012 14:20:34	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	207,360	814,130	3.9262
17 Aug 2012 10:45:16	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	181,440	720,860	3.9730
14 Aug 2012 10:21:47	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	155,520	613,979	3.9479
12 Aug 2012 08:45:25	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	129,600	508,202	3.9213
10 Aug 2012 12:42:24	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	103,680	404,953	3.9058
03 Aug 2012 10:56:47	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	77,760	291,728	3.7516
31 Jul 2012 12:58:25	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	51,840	199,002	3.8388
26 Jul 2012 14:06:33	1178110	14983383	hadcm3n_2014_1980_40_008085725_1	25,920	109,688	4.2318