Task 12737021

Name	hadcm3n_o1zv_1900_40_007197918_0
Workunit	7396198
Created	28 Mar 2011, 14:02:08 UTC
Sent	1 Apr 2011, 9:05:10 UTC
Report deadline	1 Jul 2011, 16:32:21 UTC
Received	30 Apr 2011, 7:37:13 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1006135
Run time	6 days 18 hours 28 min 28 sec
CPU time	5 days 10 hours 21 min 49 sec
Validate state	Invalid
Credit	2,488.32
Device peak FLOPS	2.68 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.6.36</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:18:08 (450): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:43:45 (1529): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:58:53 (17131): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:59:47 (17136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:59:48 (17136): No heartbeat from core client for 30 sec - exiting 18:38:38 (17173): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:24:48 (34460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:24:54 (34460): No heartbeat from core client for 30 sec - exiting 09:24:55 (34460): No heartbeat from core client for 30 sec - exiting 09:25:51 (35546): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:34:22 (35571): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:44:19 (35655): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:44:24 (35655): No heartbeat from core client for 30 sec - exiting 09:45:44 (35733): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:49:14 (55614): No heartbeat from core client for 30 sec - exiting 15:49:15 (55614): No heartbeat from core client for 30 sec - exiting 15:49:16 (55614): No heartbeat from core client for 30 sec - exiting 15:49:17 (55614): No heartbeat from core client for 30 sec - exiting 15:49:18 (55614): No heartbeat from core client for 30 sec - exiting 15:49:19 (55614): No heartbeat from core client for 30 sec - exiting 15:49:20 (55614): No heartbeat from core client for 30 sec - exiting 15:49:21 (55614): No heartbeat from core client for 30 sec - exiting 15:49:22 (55614): No heartbeat from core client for 30 sec - exiting 15:49:23 (55614): No heartbeat from core client for 30 sec - exiting 15:49:24 (55614): No heartbeat from core client for 30 sec - exiting 15:49:25 (55614): No heartbeat from core client for 30 sec - exiting 15:49:26 (55614): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:55:20 (55644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:56:25 (55705): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:58:02 (55714): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:58:03 (55714): No heartbeat from core client for 30 sec - exiting 15:58:04 (55714): No heartbeat from core client for 30 sec - exiting 15:58:05 (55714): No heartbeat from core client for 30 sec - exiting 15:58:06 (55714): No heartbeat from core client for 30 sec - exiting 15:58:07 (55714): No heartbeat from core client for 30 sec - exiting 15:58:08 (55714): No heartbeat from core client for 30 sec - exiting 15:58:09 (55714): No heartbeat from core client for 30 sec - exiting 15:58:10 (55714): No heartbeat from core client for 30 sec - exiting 15:58:11 (55714): No heartbeat from core client for 30 sec - exiting 15:58:12 (55714): No heartbeat from core client for 30 sec - exiting 15:58:13 (55714): No heartbeat from core client for 30 sec - exiting 15:58:14 (55714): No heartbeat from core client for 30 sec - exiting 15:58:15 (55714): No heartbeat from core client for 30 sec - exiting 15:58:16 (55714): No heartbeat from core client for 30 sec - exiting 15:58:17 (55714): No heartbeat from core client for 30 sec - exiting 15:58:18 (55714): No heartbeat from core client for 30 sec - exiting 15:58:19 (55714): No heartbeat from core client for 30 sec - exiting 15:58:20 (55714): No heartbeat from core client for 30 sec - exiting 15:58:21 (55714): No heartbeat from core client for 30 sec - exiting 15:58:22 (55714): No heartbeat from core client for 30 sec - exiting 15:58:23 (55714): No heartbeat from core client for 30 sec - exiting 15:58:24 (55714): No heartbeat from core client for 30 sec - exiting 15:58:25 (55714): No heartbeat from core client for 30 sec - exiting 15:58:26 (55714): No heartbeat from core client for 30 sec - exiting 15:58:27 (55714): No heartbeat from core client for 30 sec - exiting 15:58:28 (55714): No heartbeat from core client for 30 sec - exiting 15:58:29 (55714): No heartbeat from core client for 30 sec - exiting 15:58:30 (55714): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:19:51 (65465): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 05:48:07 (69102): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:57:06 (69117): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:57:10 (69117): No heartbeat from core client for 30 sec - exiting 05:57:11 (69117): No heartbeat from core client for 30 sec - exiting 05:57:13 (69117): No heartbeat from core client for 30 sec - exiting 05:57:14 (69117): No heartbeat from core client for 30 sec - exiting 05:57:15 (69117): No heartbeat from core client for 30 sec - exiting 05:57:16 (69117): No heartbeat from core client for 30 sec - exiting 05:57:17 (69117): No heartbeat from core client for 30 sec - exiting 05:57:18 (69117): No heartbeat from core client for 30 sec - exiting 05:57:19 (69117): No heartbeat from core client for 30 sec - exiting 05:57:20 (69117): No heartbeat from core client for 30 sec - exiting 05:57:21 (69117): No heartbeat from core client for 30 sec - exiting 05:57:22 (69117): No heartbeat from core client for 30 sec - exiting 05:57:23 (69117): No heartbeat from core client for 30 sec - exiting 05:57:24 (69117): No heartbeat from core client for 30 sec - exiting 05:57:25 (69117): No heartbeat from core client for 30 sec - exiting 05:57:26 (69117): No heartbeat from core client for 30 sec - exiting 05:57:57 (69117): No heartbeat from core client for 30 sec - exiting 05:57:58 (69117): No heartbeat from core client for 30 sec - exiting 05:57:59 (69117): No heartbeat from core client for 30 sec - exiting 05:58:00 (69117): No heartbeat from core client for 30 sec - exiting 05:58:01 (69117): No heartbeat from core client for 30 sec - exiting 05:58:02 (69117): No heartbeat from core client for 30 sec - exiting 05:58:03 (69117): No heartbeat from core client for 30 sec - exiting 05:58:04 (69117): No heartbeat from core client for 30 sec - exiting 05:58:05 (69117): No heartbeat from core client for 30 sec - exiting 05:58:06 (69117): No heartbeat from core client for 30 sec - exiting 05:58:07 (69117): No heartbeat from core client for 30 sec - exiting 05:58:08 (69117): No heartbeat from core client for 30 sec - exiting 05:58:09 (69117): No heartbeat from core client for 30 sec - exiting 05:58:10 (69117): No heartbeat from core client for 30 sec - exiting 05:58:11 (69117): No heartbeat from core client for 30 sec - exiting 05:58:12 (69117): No heartbeat from core client for 30 sec - exiting 05:58:13 (69117): No heartbeat from core client for 30 sec - exiting 05:58:14 (69117): No heartbeat from core client for 30 sec - exiting 05:58:15 (69117): No heartbeat from core client for 30 sec - exiting 05:58:16 (69117): No heartbeat from core client for 30 sec - exiting 05:58:17 (69117): No heartbeat from core client for 30 sec - exiting 05:58:18 (69117): No heartbeat from core client for 30 sec - exiting 05:58:19 (69117): No heartbeat from core client for 30 sec - exiting 05:58:20 (69117): No heartbeat from core client for 30 sec - exiting 05:58:21 (69117): No heartbeat from core client for 30 sec - exiting 05:58:22 (69117): No heartbeat from core client for 30 sec - exiting 05:58:23 (69117): No heartbeat from core client for 30 sec - exiting 05:58:24 (69117): No heartbeat from core client for 30 sec - exiting 05:58:25 (69117): No heartbeat from core client for 30 sec - exiting 05:58:26 (69117): No heartbeat from core client for 30 sec - exiting 05:58:27 (69117): No heartbeat from core client for 30 sec - exiting 05:58:28 (69117): No heartbeat from core client for 30 sec - exiting 05:58:29 (69117): No heartbeat from core client for 30 sec - exiting 05:58:30 (69117): No heartbeat from core client for 30 sec - exiting 05:58:31 (69117): No heartbeat from core client for 30 sec - exiting 05:58:32 (69117): No heartbeat from core client for 30 sec - exiting 05:58:33 (69117): No heartbeat from core client for 30 sec - exiting 05:58:34 (69117): No heartbeat from core client for 30 sec - exiting 05:58:35 (69117): No heartbeat from core client for 30 sec - exiting 05:58:36 (69117): No heartbeat from core client for 30 sec - exiting 05:58:37 (69117): No heartbeat from core client for 30 sec - exiting 05:58:38 (69117): No heartbeat from core client for 30 sec - exiting 05:58:39 (69117): No heartbeat from core client for 30 sec - exiting 05:58:40 (69117): No heartbeat from core client for 30 sec - exiting 05:58:41 (69117): No heartbeat from core client for 30 sec - exiting 05:58:42 (69117): No heartbeat from core client for 30 sec - exiting 06:08:48 (69206): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:08:50 (69206): No heartbeat from core client for 30 sec - exiting 06:08:51 (69206): No heartbeat from core client for 30 sec - exiting 06:08:52 (69206): No heartbeat from core client for 30 sec - exiting 06:08:53 (69206): No heartbeat from core client for 30 sec - exiting 06:08:54 (69206): No heartbeat from core client for 30 sec - exiting 06:08:55 (69206): No heartbeat from core client for 30 sec - exiting 06:08:56 (69206): No heartbeat from core client for 30 sec - exiting 06:08:57 (69206): No heartbeat from core client for 30 sec - exiting 06:08:58 (69206): No heartbeat from core client for 30 sec - exiting 06:10:50 (69271): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:11:54 (69294): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:13:00 (69320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:15:59 (69343): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:16:00 (69343): No heartbeat from core client for 30 sec - exiting 06:19:03 (69384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:19:05 (69384): No heartbeat from core client for 30 sec - exiting 06:19:06 (69384): No heartbeat from core client for 30 sec - exiting 06:20:03 (69418): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:21:01 (69442): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:21:57 (69466): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:22:54 (69492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:23:50 (69512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:24:40 (69539): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:29:19 (69560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:37:04 (69612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:48:34 (69675): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:48:40 (69675): No heartbeat from core client for 30 sec - exiting 06:48:41 (69675): No heartbeat from core client for 30 sec - exiting 06:48:42 (69675): No heartbeat from core client for 30 sec - exiting 06:48:43 (69675): No heartbeat from core client for 30 sec - exiting 06:51:01 (69803): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:58:35 (69826): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:08:36 (69900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:18:26 (69972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:18:33 (69972): No heartbeat from core client for 30 sec - exiting 07:28:06 (70055): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:28:24 (70055): No heartbeat from core client for 30 sec - exiting 07:28:25 (70055): No heartbeat from core client for 30 sec - exiting 07:28:26 (70055): No heartbeat from core client for 30 sec - exiting 07:28:27 (70055): No heartbeat from core client for 30 sec - exiting 07:39:29 (70125): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:49:32 (70212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:59:32 (70285): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:16:28 (70368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:16:32 (70368): No heartbeat from core client for 30 sec - exiting 08:24:34 (70482): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:34:37 (70562): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:34:43 (70562): No heartbeat from core client for 30 sec - exiting 08:35:41 (70638): No heartbeat from core client for 30 sec - exiting 08:35:42 (70638): No heartbeat from core client for 30 sec - exiting 08:35:43 (70638): No heartbeat from core client for 30 sec - exiting 08:35:44 (70638): No heartbeat from core client for 30 sec - exiting 08:35:45 (70638): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:48:00 (70655): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:48:04 (70655): No heartbeat from core client for 30 sec - exiting 09:20:10 (70762): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:38:17 (70965): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:38:24 (70965): No heartbeat from core client for 30 sec - exiting 09:38:25 (70965): No heartbeat from core client for 30 sec - exiting 09:44:07 (71093): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:47:08 (71142): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:47:09 (71142): No heartbeat from core client for 30 sec - exiting 09:47:10 (71142): No heartbeat from core client for 30 sec - exiting 09:47:11 (71142): No heartbeat from core client for 30 sec - exiting 09:47:12 (71142): No heartbeat from core client for 30 sec - exiting 09:47:13 (71142): No heartbeat from core client for 30 sec - exiting 09:52:36 (71183): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:57:02 (71229): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:02:14 (71284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:04:35 (71330): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:04:36 (71330): No heartbeat from core client for 30 sec - exiting 10:15:43 (71370): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:15:49 (71370): No heartbeat from core client for 30 sec - exiting 10:15:50 (71370): No heartbeat from core client for 30 sec - exiting 10:29:00 (71455): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:29:03 (71455): No heartbeat from core client for 30 sec - exiting 10:29:04 (71455): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation 10:35:11 (71562): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:44:16 (71617): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:44:20 (71617): No heartbeat from core client for 30 sec - exiting 10:44:21 (71617): No heartbeat from core client for 30 sec - exiting 10:44:22 (71617): No heartbeat from core client for 30 sec - exiting 10:44:23 (71617): No heartbeat from core client for 30 sec - exiting 10:44:24 (71617): No heartbeat from core client for 30 sec - exiting 10:44:25 (71617): No heartbeat from core client for 30 sec - exiting 10:44:26 (71617): No heartbeat from core client for 30 sec - exiting 10:51:24 (71701): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:15:52 (77416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:15:55 (77416): No heartbeat from core client for 30 sec - exiting 15:15:56 (77416): No heartbeat from core client for 30 sec - exiting 15:16:03 (77416): No heartbeat from core client for 30 sec - exiting 15:24:11 (77821): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:24:18 (77821): No heartbeat from core client for 30 sec - exiting 15:24:19 (77821): No heartbeat from core client for 30 sec - exiting 15:24:20 (77821): No heartbeat from core client for 30 sec - exiting 15:24:21 (77821): No heartbeat from core client for 30 sec - exiting 15:24:22 (77821): No heartbeat from core client for 30 sec - exiting 15:35:12 (77906): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:35:18 (77906): No heartbeat from core client for 30 sec - exiting 15:35:19 (77906): No heartbeat from core client for 30 sec - exiting 15:35:20 (77906): No heartbeat from core client for 30 sec - exiting 15:35:21 (77906): No heartbeat from core client for 30 sec - exiting 15:35:22 (77906): No heartbeat from core client for 30 sec - exiting 15:35:23 (77906): No heartbeat from core client for 30 sec - exiting 15:35:24 (77906): No heartbeat from core client for 30 sec - exiting 15:35:25 (77906): No heartbeat from core client for 30 sec - exiting 15:35:26 (77906): No heartbeat from core client for 30 sec - exiting 15:35:27 (77906): No heartbeat from core client for 30 sec - exiting 15:35:28 (77906): No heartbeat from core client for 30 sec - exiting 15:35:29 (77906): No heartbeat from core client for 30 sec - exiting 15:35:30 (77906): No heartbeat from core client for 30 sec - exiting 15:46:24 (77984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:46:27 (77984): No heartbeat from core client for 30 sec - exiting 15:46:28 (77984): No heartbeat from core client for 30 sec - exiting 15:46:29 (77984): No heartbeat from core client for 30 sec - exiting 15:46:30 (77984): No heartbeat from core client for 30 sec - exiting 15:46:31 (77984): No heartbeat from core client for 30 sec - exiting 15:56:19 (78097): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:56:27 (78097): No heartbeat from core client for 30 sec - exiting 16:06:02 (78163): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:06:11 (78163): No heartbeat from core client for 30 sec - exiting 16:06:12 (78163): No heartbeat from core client for 30 sec - exiting 16:06:13 (78163): No heartbeat from core client for 30 sec - exiting 16:06:14 (78163): No heartbeat from core client for 30 sec - exiting 16:06:15 (78163): No heartbeat from core client for 30 sec - exiting 16:06:16 (78163): No heartbeat from core client for 30 sec - exiting 16:06:17 (78163): No heartbeat from core client for 30 sec - exiting 16:06:18 (78163): No heartbeat from core client for 30 sec - exiting 16:06:19 (78163): No heartbeat from core client for 30 sec - exiting 16:06:20 (78163): No heartbeat from core client for 30 sec - exiting 16:06:21 (78163): No heartbeat from core client for 30 sec - exiting 16:06:22 (78163): No heartbeat from core client for 30 sec - exiting 16:06:23 (78163): No heartbeat from core client for 30 sec - exiting 16:06:24 (78163): No heartbeat from core client for 30 sec - exiting 16:06:25 (78163): No heartbeat from core client for 30 sec - exiting 16:06:26 (78163): No heartbeat from core client for 30 sec - exiting 16:06:27 (78163): No heartbeat from core client for 30 sec - exiting 16:07:11 (78163): No heartbeat from core client for 30 sec - exiting 16:07:12 (78163): No heartbeat from core client for 30 sec - exiting 16:07:13 (78163): No heartbeat from core client for 30 sec - exiting 16:07:14 (78163): No heartbeat from core client for 30 sec - exiting 16:07:15 (78163): No heartbeat from core client for 30 sec - exiting 16:07:16 (78163): No heartbeat from core client for 30 sec - exiting 16:07:17 (78163): No heartbeat from core client for 30 sec - exiting 16:07:18 (78163): No heartbeat from core client for 30 sec - exiting 16:07:19 (78163): No heartbeat from core client for 30 sec - exiting 16:32:08 (78282): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:32:12 (78282): No heartbeat from core client for 30 sec - exiting 16:32:13 (78282): No heartbeat from core client for 30 sec - exiting 17:03:15 (78446): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:15:04 (78657): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:25:45 (78757): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:25:50 (78757): No heartbeat from core client for 30 sec - exiting 17:25:51 (78757): No heartbeat from core client for 30 sec - exiting 17:25:52 (78757): No heartbeat from core client for 30 sec - exiting 17:43:03 (78846): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:43:07 (78846): No heartbeat from core client for 30 sec - exiting 17:43:08 (78846): No heartbeat from core client for 30 sec - exiting 17:43:09 (78846): No heartbeat from core client for 30 sec - exiting 17:43:10 (78846): No heartbeat from core client for 30 sec - exiting 17:43:11 (78846): No heartbeat from core client for 30 sec - exiting 17:43:12 (78846): No heartbeat from core client for 30 sec - exiting 17:43:13 (78846): No heartbeat from core client for 30 sec - exiting 17:43:14 (78846): No heartbeat from core client for 30 sec - exiting 17:43:15 (78846): No heartbeat from core client for 30 sec - exiting 17:43:16 (78846): No heartbeat from core client for 30 sec - exiting 17:43:17 (78846): No heartbeat from core client for 30 sec - exiting 17:43:18 (78846): No heartbeat from core client for 30 sec - exiting 17:43:19 (78846): No heartbeat from core client for 30 sec - exiting 17:57:19 (78971): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:12:21 (79079): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:22:26 (79201): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:32:52 (79285): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:32:57 (79285): No heartbeat from core client for 30 sec - exiting 18:32:58 (79285): No heartbeat from core client for 30 sec - exiting 18:32:59 (79285): No heartbeat from core client for 30 sec - exiting 18:33:00 (79285): No heartbeat from core client for 30 sec - exiting 18:33:01 (79285): No heartbeat from core client for 30 sec - exiting 18:47:26 (79370): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:02:26 (79483): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:13:37 (79596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:23:28 (79691): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:23:34 (79691): No heartbeat from core client for 30 sec - exiting 19:37:30 (79772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:50:38 (79883): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:02:46 (79981): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:16:10 (80088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:27:48 (80188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:42:48 (80289): No heartbeat from core client for 30 sec - exiting 20:42:59 (80289): No heartbeat from core client for 30 sec - exiting 20:43:00 (80289): No heartbeat from core client for 30 sec - exiting 20:43:01 (80289): No heartbeat from core client for 30 sec - exiting 20:43:02 (80289): No heartbeat from core client for 30 sec - exiting 20:43:03 (80289): No heartbeat from core client for 30 sec - exiting 20:43:04 (80289): No heartbeat from core client for 30 sec - exiting 20:43:05 (80289): No heartbeat from core client for 30 sec - exiting 20:43:06 (80289): No heartbeat from core client for 30 sec - exiting 20:43:07 (80289): No heartbeat from core client for 30 sec - exiting 20:43:08 (80289): No heartbeat from core client for 30 sec - exiting 20:43:09 (80289): No heartbeat from core client for 30 sec - exiting 20:43:10 (80289): No heartbeat from core client for 30 sec - exiting 20:43:11 (80289): No heartbeat from core client for 30 sec - exiting 20:43:12 (80289): No heartbeat from core client for 30 sec - exiting 20:43:13 (80289): No heartbeat from core client for 30 sec - exiting 20:43:44 (80289): No heartbeat from core client for 30 sec - exiting 20:43:45 (80289): No heartbeat from core client for 30 sec - exiting 20:43:46 (80289): No heartbeat from core client for 30 sec - exiting 20:43:47 (80289): No heartbeat from core client for 30 sec - exiting 20:43:48 (80289): No heartbeat from core client for 30 sec - exiting 20:43:49 (80289): No heartbeat from core client for 30 sec - exiting 20:43:50 (80289): No heartbeat from core client for 30 sec - exiting 20:43:51 (80289): No heartbeat from core client for 30 sec - exiting 20:43:52 (80289): No heartbeat from core client for 30 sec - exiting 20:44:31 (80289): No heartbeat from core client for 30 sec - exiting 20:44:32 (80289): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/o1zvko.pja6c10 is not a valid UM file. Error converting file to netcdf: dataout/o1zvko.pja6c10 Error: Input file: dataout/o1zvko.pia6c10 is not a valid UM file. Error converting file to netcdf: dataout/o1zvko.pia6c10 Error: Input file: dataout/o1zvko.pfa6c10 is not a valid UM file. Error converting file to netcdf: dataout/o1zvko.pfa6c10 Error: Input file: dataout/o1zvka.pha6c10 is not a valid UM file. Error converting file to netcdf: dataout/o1zvka.pha6c10 Error: Input file: dataout/o1zvka.pga6c10 is not a valid UM file. Error converting file to netcdf: dataout/o1zvka.pga6c10 Error: Input file: dataout/o1zvka.pea6c10 is not a valid UM file. Error converting file to netcdf: dataout/o1zvka.pea6c10 Error: Input file: dataout/o1zvka.pda6c10 is not a valid UM file. Error converting file to netcdf: dataout/o1zvka.pda6c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:20:53 (99858): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:35:54 (99928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:35:58 (99928): No heartbeat from core client for 30 sec - exiting 17:35:59 (99928): No heartbeat from core client for 30 sec - exiting 17:45:55 (191): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:05:26 (257): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:21:14 (598): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:21:16 (598): No heartbeat from core client for 30 sec - exiting 18:21:17 (598): No heartbeat from core client for 30 sec - exiting 18:21:18 (598): No heartbeat from core client for 30 sec - exiting 18:21:19 (598): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 22:51:19 (5643): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:53:21 (5809): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:53:25 (5809): No heartbeat from core client for 30 sec - exiting 23:53:26 (5809): No heartbeat from core client for 30 sec - exiting 23:53:27 (5809): No heartbeat from core client for 30 sec - exiting 23:54:14 (6217): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:07:32 (6228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:08:50 (6350): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:11:48 (6357): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:12:53 (6797): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:12:55 (6797): No heartbeat from core client for 30 sec - exiting 01:23:28 (6808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:24:40 (6903): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:35:18 (6913): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:45:46 (7014): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:07:41 (7083): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:26:42 (7247): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:37:46 (7360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:47:48 (7471): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:57:49 (7533): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:31:37 (7634): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:31:41 (7634): No heartbeat from core client for 30 sec - exiting 03:31:42 (7634): No heartbeat from core client for 30 sec - exiting 03:31:43 (7634): No heartbeat from core client for 30 sec - exiting 03:31:44 (7634): No heartbeat from core client for 30 sec - exiting 03:31:45 (7634): No heartbeat from core client for 30 sec - exiting 03:32:37 (9975): No heartbeat from core client for 30 sec - exiting 03:32:38 (9975): No heartbeat from core client for 30 sec - exiting 03:32:39 (9975): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:17:21 (9987): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:38:52 (18859): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:56:29 (19148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:49:56 (40270): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:50:02 (40270): No heartbeat from core client for 30 sec - exiting 09:50:03 (40270): No heartbeat from core client for 30 sec - exiting 09:50:04 (40270): No heartbeat from core client for 30 sec - exiting 09:50:05 (40270): No heartbeat from core client for 30 sec - exiting 09:50:06 (40270): No heartbeat from core client for 30 sec - exiting 09:50:07 (40270): No heartbeat from core client for 30 sec - exiting 09:50:08 (40270): No heartbeat from core client for 30 sec - exiting 09:50:09 (40270): No heartbeat from core client for 30 sec - exiting 09:50:10 (40270): No heartbeat from core client for 30 sec - exiting 09:50:11 (40270): No heartbeat from core client for 30 sec - exiting 09:50:12 (40270): No heartbeat from core client for 30 sec - exiting 09:50:13 (40270): No heartbeat from core client for 30 sec - exiting 09:50:14 (40270): No heartbeat from core client for 30 sec - exiting 09:50:15 (40270): No heartbeat from core client for 30 sec - exiting 09:50:16 (40270): No heartbeat from core client for 30 sec - exiting 09:50:17 (40270): No heartbeat from core client for 30 sec - exiting 09:50:18 (40270): No heartbeat from core client for 30 sec - exiting 09:50:19 (40270): No heartbeat from core client for 30 sec - exiting 09:50:20 (40270): No heartbeat from core client for 30 sec - exiting 09:50:21 (40270): No heartbeat from core client for 30 sec - exiting 09:57:01 (40431): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:57:04 (40431): No heartbeat from core client for 30 sec - exiting 10:22:52 (40499): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:22:55 (40499): No heartbeat from core client for 30 sec - exiting 10:22:56 (40499): No heartbeat from core client for 30 sec - exiting 10:22:57 (40499): No heartbeat from core client for 30 sec - exiting 10:29:51 (40690): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:36:48 (40756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:51:50 (40824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:20:33 (42932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:20:38 (42932): No heartbeat from core client for 30 sec - exiting 15:20:39 (42932): No heartbeat from core client for 30 sec - exiting 15:20:40 (42932): No heartbeat from core client for 30 sec - exiting 15:20:41 (42932): No heartbeat from core client for 30 sec - exiting 15:20:42 (42932): No heartbeat from core client for 30 sec - exiting 15:20:43 (42932): No heartbeat from core client for 30 sec - exiting 15:21:45 (44927): No heartbeat from core client for 30 sec - exiting 15:21:46 (44927): No heartbeat from core client for 30 sec - exiting 15:21:47 (44927): No heartbeat from core client for 30 sec - exiting 15:21:48 (44927): No heartbeat from core client for 30 sec - exiting 15:21:49 (44927): No heartbeat from core client for 30 sec - exiting 15:21:50 (44927): No heartbeat from core client for 30 sec - exiting 15:21:51 (44927): No heartbeat from core client for 30 sec - exiting 15:21:52 (44927): No heartbeat from core client for 30 sec - exiting 15:21:53 (44927): No heartbeat from core client for 30 sec - exiting 15:21:54 (44927): No heartbeat from core client for 30 sec - exiting 15:21:55 (44927): No heartbeat from core client for 30 sec - exiting 15:21:56 (44927): No heartbeat from core client for 30 sec - exiting 15:21:57 (44927): No heartbeat from core client for 30 sec - exiting 15:21:58 (44927): No heartbeat from core client for 30 sec - exiting 15:21:59 (44927): No heartbeat from core client for 30 sec - exiting 15:22:00 (44927): No heartbeat from core client for 30 sec - exiting 15:22:01 (44927): No heartbeat from core client for 30 sec - exiting 15:22:02 (44927): No heartbeat from core client for 30 sec - exiting 15:22:03 (44927): No heartbeat from core client for 30 sec - exiting 15:22:04 (44927): No heartbeat from core client for 30 sec - exiting 15:22:05 (44927): No heartbeat from core client for 30 sec - exiting 15:22:06 (44927): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:36:35 (44936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:36:39 (44936): No heartbeat from core client for 30 sec - exiting 16:09:42 (45075): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:09:46 (45075): No heartbeat from core client for 30 sec - exiting 16:09:47 (45075): No heartbeat from core client for 30 sec - exiting 16:09:48 (45075): No heartbeat from core client for 30 sec - exiting 16:09:49 (45075): No heartbeat from core client for 30 sec - exiting 16:09:50 (45075): No heartbeat from core client for 30 sec - exiting 16:09:51 (45075): No heartbeat from core client for 30 sec - exiting 16:09:52 (45075): No heartbeat from core client for 30 sec - exiting 16:09:53 (45075): No heartbeat from core client for 30 sec - exiting 16:09:54 (45075): No heartbeat from core client for 30 sec - exiting 16:09:55 (45075): No heartbeat from core client for 30 sec - exiting 16:09:56 (45075): No heartbeat from core client for 30 sec - exiting 16:09:57 (45075): No heartbeat from core client for 30 sec - exiting 16:09:58 (45075): No heartbeat from core client for 30 sec - exiting 16:09:59 (45075): No heartbeat from core client for 30 sec - exiting 16:10:00 (45075): No heartbeat from core client for 30 sec - exiting 16:10:01 (45075): No heartbeat from core client for 30 sec - exiting 16:10:02 (45075): No heartbeat from core client for 30 sec - exiting 16:10:03 (45075): No heartbeat from core client for 30 sec - exiting 16:10:04 (45075): No heartbeat from core client for 30 sec - exiting 16:10:05 (45075): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.7 build 10J869 Fri Apr 29 18:01:44 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x97eb1156 ___kill + 10 1 libSystem.B.dylib 0x97f43899 _raise + 26 2 libSystem.B.dylib 0x97f599b8 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x97e501ae _mach_wait_until + 10 1 libSystem.B.dylib 0x97ed7809 _nanosleep + 345 2 libSystem.B.dylib 0x97ed76aa _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x97e7d7fd __pthread_start + 345 6 libSystem.B.dylib 0x97e7d682 _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcab8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x96d00000 - 0x96d0efff /usr/lib/libz.1.dylib 0x97e4f000 - 0x97ff6fff /usr/lib/libSystem.B.dylib 0x98318000 - 0x9831bfff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=48182, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.7 build 10J869 Fri Apr 29 18:01:47 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x97eb1156 ___kill + 10 1 libSystem.B.dylib 0x97f43899 _raise + 26 2 libSystem.B.dylib 0x97f599b8 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x97e501ae _mach_wait_until + 10 1 libSystem.B.dylib 0x97ed7809 _nanosleep + 345 2 libSystem.B.dylib 0x97ed76aa _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x97e7d7fd __pthread_start + 345 6 libSystem.B.dylib 0x97e7d682 _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcab8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x96d00000 - 0x96d0efff /usr/lib/libz.1.dylib 0x97e4f000 - 0x97ff6fff /usr/lib/libSystem.B.dylib 0x98318000 - 0x9831bfff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=48182, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.7 build 10J869 Fri Apr 29 18:01:50 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x97eb1156 ___kill + 10 1 libSystem.B.dylib 0x97f43899 _raise + 26 2 libSystem.B.dylib 0x97f599b8 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x97e501ae _mach_wait_until + 10 1 libSystem.B.dylib 0x97ed7809 _nanosleep + 345 2 libSystem.B.dylib 0x97ed76aa _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x97e7d7fd __pthread_start + 345 6 libSystem.B.dylib 0x97e7d682 _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcab8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x96d00000 - 0x96d0efff /usr/lib/libz.1.dylib 0x97e4f000 - 0x97ff6fff /usr/lib/libSystem.B.dylib 0x98318000 - 0x9831bfff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=48182, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.7 build 10J869 Fri Apr 29 18:01:53 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x97eb1156 ___kill + 10 1 libSystem.B.dylib 0x97f43899 _raise + 26 2 libSystem.B.dylib 0x97f599b8 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x97e501ae _mach_wait_until + 10 1 libSystem.B.dylib 0x97ed7809 _nanosleep + 345 2 libSystem.B.dylib 0x97ed76aa _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x97e7d7fd __pthread_start + 345 6 libSystem.B.dylib 0x97e7d682 _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcab8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x96d00000 - 0x96d0efff /usr/lib/libz.1.dylib 0x97e4f000 - 0x97ff6fff /usr/lib/libSystem.B.dylib 0x98318000 - 0x9831bfff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=48182, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.7 build 10J869 Fri Apr 29 18:01:56 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x97eb1156 ___kill + 10 1 libSystem.B.dylib 0x97f43899 _raise + 26 2 libSystem.B.dylib 0x97f599b8 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x97e501ae _mach_wait_until + 10 1 libSystem.B.dylib 0x97ed7809 _nanosleep + 345 2 libSystem.B.dylib 0x97ed76aa _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x97e7d7fd __pthread_start + 345 6 libSystem.B.dylib 0x97e7d682 _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcab8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x96d00000 - 0x96d0efff /usr/lib/libz.1.dylib 0x97e4f000 - 0x97ff6fff /usr/lib/libSystem.B.dylib 0x98318000 - 0x9831bfff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=48182, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.7 build 10J869 Fri Apr 29 18:02:00 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x97eb1156 ___kill + 10 1 libSystem.B.dylib 0x97f43899 _raise + 26 2 libSystem.B.dylib 0x97f599b8 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x97e501ae _mach_wait_until + 10 1 libSystem.B.dylib 0x97ed7809 _nanosleep + 345 2 libSystem.B.dylib 0x97ed76aa _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x97e7d7fd __pthread_start + 345 6 libSystem.B.dylib 0x97e7d682 _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcab8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x96d00000 - 0x96d0efff /usr/lib/libz.1.dylib 0x97e4f000 - 0x97ff6fff /usr/lib/libSystem.B.dylib 0x98318000 - 0x9831bfff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=48182, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Apr 2011 05:25:34	1006135	12737021	hadcm3n_o1zv_1900_40_007197918_0	207,360	424,920	2.0492
28 Apr 2011 08:24:24	1006135	12737021	hadcm3n_o1zv_1900_40_007197918_0	181,440	370,098	2.0398
27 Apr 2011 00:49:27	1006135	12737021	hadcm3n_o1zv_1900_40_007197918_0	155,520	316,763	2.0368
25 Apr 2011 10:41:14	1006135	12737021	hadcm3n_o1zv_1900_40_007197918_0	129,600	262,941	2.0289
23 Apr 2011 10:54:41	1006135	12737021	hadcm3n_o1zv_1900_40_007197918_0	103,680	210,190	2.0273
12 Apr 2011 21:38:18	1006135	12737021	hadcm3n_o1zv_1900_40_007197918_0	77,760	158,235	2.0349
02 Apr 2011 21:53:44	1006135	12737021	hadcm3n_o1zv_1900_40_007197918_0	51,840	105,204	2.0294
02 Apr 2011 00:56:37	1006135	12737021	hadcm3n_o1zv_1900_40_007197918_0	25,920	51,851	2.0004