Task 13113301

Name	hadcm3n_ygv5_1900_40_007354715_1
Workunit	7552145
Created	6 Jul 2011, 14:35:27 UTC
Sent	10 Jul 2011, 23:53:34 UTC
Report deadline	10 Oct 2011, 7:20:45 UTC
Received	26 Sep 2011, 7:18:52 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1109026
Run time	10 days 15 hours 16 min 49 sec
CPU time	9 days 15 hours 27 min 52 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.25 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2876, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 00:19:32 (1308): No heartbeat from core client for 30 sec - exiting 00:19:33 (1308): No heartbeat from core client for 30 sec - exiting 00:19:34 (1308): No heartbeat from core client for 30 sec - exiting 00:19:35 (1308): No heartbeat from core client for 30 sec - exiting 00:19:37 (1308): No heartbeat from core client for 30 sec - exiting 00:19:38 (1308): No heartbeat from core client for 30 sec - exiting 00:19:39 (1308): No heartbeat from core client for 30 sec - exiting 00:19:40 (1308): No heartbeat from core client for 30 sec - exiting 00:19:41 (1308): No heartbeat from core client for 30 sec - exiting 00:19:42 (1308): No heartbeat from core client for 30 sec - exiting 00:19:43 (1308): No heartbeat from core client for 30 sec - exiting 00:19:44 (1308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3676, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:59:19 (3708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:07:58 (3864): No heartbeat from core client for 30 sec - exiting 20:07:59 (3864): No heartbeat from core client for 30 sec - exiting 20:08:00 (3864): No heartbeat from core client for 30 sec - exiting 20:08:01 (3864): No heartbeat from core client for 30 sec - exiting 20:08:02 (3864): No heartbeat from core client for 30 sec - exiting 20:08:03 (3864): No heartbeat from core client for 30 sec - exiting 20:08:04 (3864): No heartbeat from core client for 30 sec - exiting 20:08:05 (3864): No heartbeat from core client for 30 sec - exiting 20:08:06 (3864): No heartbeat from core client for 30 sec - exiting 20:08:08 (3864): No heartbeat from core client for 30 sec - exiting 20:08:09 (3864): No heartbeat from core client for 30 sec - exiting 20:08:10 (3864): No heartbeat from core client for 30 sec - exiting 20:08:11 (3864): No heartbeat from core client for 30 sec - exiting 20:08:12 (3864): No heartbeat from core client for 30 sec - exiting 20:08:13 (3864): No heartbeat from core client for 30 sec - exiting 20:08:14 (3864): No heartbeat from core client for 30 sec - exiting 20:08:15 (3864): No heartbeat from core client for 30 sec - exiting 20:08:16 (3864): No heartbeat from core client for 30 sec - exiting 20:08:17 (3864): No heartbeat from core client for 30 sec - exiting 20:08:18 (3864): No heartbeat from core client for 30 sec - exiting 20:08:20 (3864): No heartbeat from core client for 30 sec - exiting 20:08:21 (3864): No heartbeat from core client for 30 sec - exiting 20:08:22 (3864): No heartbeat from core client for 30 sec - exiting 20:08:23 (3864): No heartbeat from core client for 30 sec - exiting 20:08:24 (3864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 15:36:23 (3704): No heartbeat from core client for 30 sec - exiting 15:36:24 (3704): No heartbeat from core client for 30 sec - exiting 15:36:25 (3704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:36:26 (3704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1880, iMonCtr=1 Model crash detected, will try to restart... 00:27:34 (3812): No heartbeat from core client for 30 sec - exiting 00:27:35 (3812): No heartbeat from core client for 30 sec - exiting 00:27:36 (3812): No heartbeat from core client for 30 sec - exiting 00:27:37 (3812): No heartbeat from core client for 30 sec - exiting 00:27:38 (3812): No heartbeat from core client for 30 sec - exiting 00:27:39 (3812): No heartbeat from core client for 30 sec - exiting 00:27:40 (3812): No heartbeat from core client for 30 sec - exiting 00:27:41 (3812): No heartbeat from core client for 30 sec - exiting 00:27:43 (3812): No heartbeat from core client for 30 sec - exiting 00:27:44 (3812): No heartbeat from core client for 30 sec - exiting 00:27:45 (3812): No heartbeat from core client for 30 sec - exiting 00:27:46 (3812): No heartbeat from core client for 30 sec - exiting 00:27:47 (3812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 00:08:19 (3484): No heartbeat from core client for 30 sec - exiting 00:08:20 (3484): No heartbeat from core client for 30 sec - exiting 00:08:21 (3484): No heartbeat from core client for 30 sec - exiting 00:08:22 (3484): No heartbeat from core client for 30 sec - exiting 00:08:23 (3484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:08:24 (3484): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4960, iMonCtr=1 Model crash detected, will try to restart... 22:30:27 (3356): No heartbeat from core client for 30 sec - exiting 22:30:28 (3356): No heartbeat from core client for 30 sec - exiting 22:30:29 (3356): No heartbeat from core client for 30 sec - exiting 22:30:30 (3356): No heartbeat from core client for 30 sec - exiting 22:30:31 (3356): No heartbeat from core client for 30 sec - exiting 22:30:32 (3356): No heartbeat from core client for 30 sec - exiting 22:30:33 (3356): No heartbeat from core client for 30 sec - exiting 22:30:34 (3356): No heartbeat from core client for 30 sec - exiting 22:30:35 (3356): No heartbeat from core client for 30 sec - exiting 22:30:37 (3356): No heartbeat from core client for 30 sec - exiting 22:30:38 (3356): No heartbeat from core client for 30 sec - exiting 22:30:39 (3356): No heartbeat from core client for 30 sec - exiting 22:30:40 (3356): No heartbeat from core client for 30 sec - exiting 22:30:41 (3356): No heartbeat from core client for 30 sec - exiting 22:30:42 (3356): No heartbeat from core client for 30 sec - exiting 22:30:43 (3356): No heartbeat from core client for 30 sec - exiting 22:30:44 (3356): No heartbeat from core client for 30 sec - exiting 22:30:45 (3356): No heartbeat from core client for 30 sec - exiting 22:30:46 (3356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:30:47 (3356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 00:19:53 (2752): No heartbeat from core client for 30 sec - exiting 00:19:54 (2752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:50:43 (3852): No heartbeat from core client for 30 sec - exiting 22:50:44 (3852): No heartbeat from core client for 30 sec - exiting 22:50:45 (3852): No heartbeat from core client for 30 sec - exiting 22:50:46 (3852): No heartbeat from core client for 30 sec - exiting 22:50:48 (3852): No heartbeat from core client for 30 sec - exiting 22:50:49 (3852): No heartbeat from core client for 30 sec - exiting 22:50:50 (3852): No heartbeat from core client for 30 sec - exiting 22:50:51 (3852): No heartbeat from core client for 30 sec - exiting 22:50:52 (3852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:50:53 (3852): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3084, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 15:37:55 (3944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:10:30 (1464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:55:43 (3692): No heartbeat from core client for 30 sec - exiting 18:55:44 (3692): No heartbeat from core client for 30 sec - exiting 18:55:45 (3692): No heartbeat from core client for 30 sec - exiting 18:55:46 (3692): No heartbeat from core client for 30 sec - exiting 18:55:47 (3692): No heartbeat from core client for 30 sec - exiting 18:55:48 (3692): No heartbeat from core client for 30 sec - exiting 18:55:49 (3692): No heartbeat from core client for 30 sec - exiting 18:55:51 (3692): No heartbeat from core client for 30 sec - exiting 18:55:52 (3692): No heartbeat from core client for 30 sec - exiting 18:55:53 (3692): No heartbeat from core client for 30 sec - exiting 18:55:54 (3692): No heartbeat from core client for 30 sec - exiting 18:55:55 (3692): No heartbeat from core client for 30 sec - exiting 18:55:56 (3692): No heartbeat from core client for 30 sec - exiting 18:55:57 (3692): No heartbeat from core client for 30 sec - exiting 18:55:58 (3692): No heartbeat from core client for 30 sec - exiting 18:55:59 (3692): No heartbeat from core client for 30 sec - exiting 18:56:00 (3692): No heartbeat from core client for 30 sec - exiting 18:56:01 (3692): No heartbeat from core client for 30 sec - exiting 18:56:03 (3692): No heartbeat from core client for 30 sec - exiting 18:56:04 (3692): No heartbeat from core client for 30 sec - exiting 18:56:05 (3692): No heartbeat from core client for 30 sec - exiting 18:56:06 (3692): No heartbeat from core client for 30 sec - exiting 18:56:07 (3692): No heartbeat from core client for 30 sec - exiting 18:56:08 (3692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:48:56 (3860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:38:53 (3324): No heartbeat from core client for 30 sec - exiting 21:38:54 (3324): No heartbeat from core client for 30 sec - exiting 21:38:56 (3324): No heartbeat from core client for 30 sec - exiting 21:38:57 (3324): No heartbeat from core client for 30 sec - exiting 21:38:58 (3324): No heartbeat from core client for 30 sec - exiting 21:38:59 (3324): No heartbeat from core client for 30 sec - exiting 21:39:00 (3324): No heartbeat from core client for 30 sec - exiting 21:39:01 (3324): No heartbeat from core client for 30 sec - exiting 21:39:02 (3324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 02:37:34 (1312): No heartbeat from core client for 30 sec - exiting 02:37:35 (1312): No heartbeat from core client for 30 sec - exiting 02:37:36 (1312): No heartbeat from core client for 30 sec - exiting 02:37:37 (1312): No heartbeat from core client for 30 sec - exiting 02:37:38 (1312): No heartbeat from core client for 30 sec - exiting 02:37:39 (1312): No heartbeat from core client for 30 sec - exiting 02:37:40 (1312): No heartbeat from core client for 30 sec - exiting 02:37:41 (1312): No heartbeat from core client for 30 sec - exiting 02:37:42 (1312): No heartbeat from core client for 30 sec - exiting 02:37:43 (1312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:02:00 (3456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 20:24:48 (3856): No heartbeat from core client for 30 sec - exiting 20:24:49 (3856): No heartbeat from core client for 30 sec - exiting 20:24:50 (3856): No heartbeat from core client for 30 sec - exiting 20:24:51 (3856): No heartbeat from core client for 30 sec - exiting 20:24:52 (3856): No heartbeat from core client for 30 sec - exiting 20:24:54 (3856): No heartbeat from core client for 30 sec - exiting 20:24:55 (3856): No heartbeat from core client for 30 sec - exiting 20:24:56 (3856): No heartbeat from core client for 30 sec - exiting 20:24:57 (3856): No heartbeat from core client for 30 sec - exiting 20:24:58 (3856): No heartbeat from core client for 30 sec - exiting 20:24:59 (3856): No heartbeat from core client for 30 sec - exiting 20:25:00 (3856): No heartbeat from core client for 30 sec - exiting 20:25:01 (3856): No heartbeat from core client for 30 sec - exiting 20:25:02 (3856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:25:03 (3856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 23:42:45 (3924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:53:14 (3764): No heartbeat from core client for 30 sec - exiting 18:53:15 (3764): No heartbeat from core client for 30 sec - exiting 18:53:16 (3764): No heartbeat from core client for 30 sec - exiting 18:53:17 (3764): No heartbeat from core client for 30 sec - exiting 18:53:18 (3764): No heartbeat from core client for 30 sec - exiting 18:53:20 (3764): No heartbeat from core client for 30 sec - exiting 18:53:21 (3764): No heartbeat from core client for 30 sec - exiting 18:53:22 (3764): No heartbeat from core client for 30 sec - exiting 18:53:23 (3764): No heartbeat from core client for 30 sec - exiting 18:53:24 (3764): No heartbeat from core client for 30 sec - exiting 18:53:25 (3764): No heartbeat from core client for 30 sec - exiting 18:53:26 (3764): No heartbeat from core client for 30 sec - exiting 18:53:27 (3764): No heartbeat from core client for 30 sec - exiting 18:53:28 (3764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:44:01 (3828): No heartbeat from core client for 30 sec - exiting 19:44:02 (3828): No heartbeat from core client for 30 sec - exiting 19:44:03 (3828): No heartbeat from core client for 30 sec - exiting 19:44:04 (3828): No heartbeat from core client for 30 sec - exiting 19:44:05 (3828): No heartbeat from core client for 30 sec - exiting 19:44:06 (3828): No heartbeat from core client for 30 sec - exiting 19:44:08 (3828): No heartbeat from core client for 30 sec - exiting 19:44:09 (3828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:51:20 (2768): No heartbeat from core client for 30 sec - exiting 23:51:21 (2768): No heartbeat from core client for 30 sec - exiting 23:51:22 (2768): No heartbeat from core client for 30 sec - exiting 23:51:23 (2768): No heartbeat from core client for 30 sec - exiting 23:51:24 (2768): No heartbeat from core client for 30 sec - exiting 23:51:25 (2768): No heartbeat from core client for 30 sec - exiting 23:51:26 (2768): No heartbeat from core client for 30 sec - exiting 23:51:27 (2768): No heartbeat from core client for 30 sec - exiting 23:51:28 (2768): No heartbeat from core client for 30 sec - exiting 23:51:30 (2768): No heartbeat from core client for 30 sec - exiting 23:51:31 (2768): No heartbeat from core client for 30 sec - exiting 23:51:32 (2768): No heartbeat from core client for 30 sec - exiting 23:51:33 (2768): No heartbeat from core client for 30 sec - exiting 23:51:34 (2768): No heartbeat from core client for 30 sec - exiting 23:51:35 (2768): No heartbeat from core client for 30 sec - exiting 23:51:36 (2768): No heartbeat from core client for 30 sec - exiting 23:51:37 (2768): No heartbeat from core client for 30 sec - exiting 23:51:38 (2768): No heartbeat from core client for 30 sec - exiting 23:51:39 (2768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... C19:32:40 (3704): No heartbeat from core client for 30 sec - exiting 19:32:41 (3704): No heartbeat from core client for 30 sec - exiting 19:32:42 (3704): No heartbeat from core client for 30 sec - exiting 19:32:43 (3704): No heartbeat from core client for 30 sec - exiting 19:32:44 (3704): No heartbeat from core client for 30 sec - exiting 19:32:45 (3704): No heartbeat from core client for 30 sec - exiting 19:32:46 (3704): No heartbeat from core client for 30 sec - exiting 19:32:47 (3704): No heartbeat from core client for 30 sec - exiting 19:32:48 (3704): No heartbeat from core client for 30 sec - exiting 19:32:49 (3704): No heartbeat from core client for 30 sec - exiting 19:32:50 (3704): No heartbeat from core client for 30 sec - exiting 19:32:52 (3704): No heartbeat from core client for 30 sec - exiting 19:32:53 (3704): No heartbeat from core client for 30 sec - exiting 19:32:54 (3704): No heartbeat from core client for 30 sec - exiting 19:32:55 (3704): No heartbeat from core client for 30 sec - exiting 19:32:56 (3704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:51:35 (3976): No heartbeat from core client for 30 sec - exiting 19:51:36 (3976): No heartbeat from core client for 30 sec - exiting 19:51:38 (3976): No heartbeat from core client for 30 sec - exiting 19:51:39 (3976): No heartbeat from core client for 30 sec - exiting 19:51:40 (3976): No heartbeat from core client for 30 sec - exiting 19:51:41 (3976): No heartbeat from core client for 30 sec - exiting 19:51:42 (3976): No heartbeat from core client for 30 sec - exiting 19:51:43 (3976): No heartbeat from core client for 30 sec - exiting 19:51:44 (3976): No heartbeat from core client for 30 sec - exiting 19:51:45 (3976): No heartbeat from core client for 30 sec - exiting 19:51:46 (3976): No heartbeat from core client for 30 sec - exiting 19:51:47 (3976): No heartbeat from core client for 30 sec - exiting 19:51:48 (3976): No heartbeat from core client for 30 sec - exiting 19:51:50 (3976): No heartbeat from core client for 30 sec - exiting 19:51:51 (3976): No heartbeat from core client for 30 sec - exiting 19:51:52 (3976): No heartbeat from core client for 30 sec - exiting 19:51:53 (3976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:51:54 (3976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:35:15 (3172): No heartbeat from core client for 30 sec - exiting 22:35:16 (3172): No heartbeat from core client for 30 sec - exiting 22:35:18 (3172): No heartbeat from core client for 30 sec - exiting 22:35:19 (3172): No heartbeat from core client for 30 sec - exiting 22:35:20 (3172): No heartbeat from core client for 30 sec - exiting 22:35:21 (3172): No heartbeat from core client for 30 sec - exiting 22:35:22 (3172): No heartbeat from core client for 30 sec - exiting 22:35:23 (3172): No heartbeat from core client for 30 sec - exiting 22:35:24 (3172): No heartbeat from core client for 30 sec - exiting 22:35:25 (3172): No heartbeat from core client for 30 sec - exiting 22:35:26 (3172): No heartbeat from core client for 30 sec - exiting 22:35:27 (3172): No heartbeat from core client for 30 sec - exiting 22:35:28 (3172): No heartbeat from core client for 30 sec - exiting 22:35:30 (3172): No heartbeat from core client for 30 sec - exiting 22:35:31 (3172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2972, selfPID=2972, iMonCtr=1 21:15:35 (1156): No heartbeat from core client for 30 sec - exiting 21:15:37 (1156): No heartbeat from core client for 30 sec - exiting 21:15:38 (1156): No heartbeat from core client for 30 sec - exiting BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: C I/O Error feof - Unit 62 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: C I/O Error feof - Unit 62 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/ygv5ko.pjc0c10 Error converting file to netcdf: dataout/ygv5ko.pic0c10 Error converting file to netcdf: dataout/ygv5ko.pfc0c10 Error converting file to netcdf: dataout/ygv5ko.pcc0c10 Error converting file to netcdf: dataout/ygv5ko.pbc0c10 Error converting file to netcdf: dataout/ygv5ko.pac0c10 Error converting file to netcdf: dataout/ygv5ka.phc0c10 Error converting file to netcdf: dataout/ygv5ka.pgc0c10 Error converting file to netcdf: dataout/ygv5ka.pec0c10 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77843A93 read attempt to address 0x40863750 Engaging BOINC Windows Runtime Debugger... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1172, selfPID=1172, iMonCtr=1 Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Sep 2011 04:22:05	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	518,400	832,001	1.6049
22 Sep 2011 07:26:59	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	492,480	790,978	1.6061
20 Sep 2011 03:49:22	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	466,560	751,122	1.6099
17 Sep 2011 15:08:29	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	440,640	710,176	1.6117
16 Sep 2011 18:25:02	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	414,720	669,928	1.6154
15 Sep 2011 03:37:43	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	388,800	629,572	1.6193
11 Sep 2011 23:17:44	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	362,880	588,788	1.6225
11 Sep 2011 09:57:30	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	336,960	547,921	1.6261
06 Sep 2011 00:47:18	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	311,040	508,170	1.6338
03 Sep 2011 13:38:09	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	285,120	467,636	1.6401
31 Aug 2011 08:00:19	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	259,200	428,310	1.6524
28 Aug 2011 19:18:53	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	233,280	386,223	1.6556
24 Aug 2011 06:10:18	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	207,360	345,456	1.6660
24 Aug 2011 02:17:29	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	181,440	306,833	1.6911
12 Aug 2011 06:25:47	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	155,520	264,040	1.6978
07 Aug 2011 09:13:28	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	129,600	220,443	1.7009
31 Jul 2011 20:42:47	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	103,680	176,535	1.7027
25 Jul 2011 22:52:29	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	77,760	131,864	1.6958
25 Jul 2011 21:47:02	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	51,840	88,184	1.7011
25 Jul 2011 16:11:27	1109026	13113301	hadcm3n_ygv5_1900_40_007354715_1	25,920	44,583	1.7200