Task 15477883

Name	hadcm3n_zhew_1880_40_008250605_3
Workunit	8405729
Created	14 Dec 2012, 11:01:14 UTC
Sent	14 Dec 2012, 11:01:27 UTC
Report deadline	15 Mar 2013, 18:28:38 UTC
Received	5 Mar 2013, 1:28:36 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1213970
Run time	25 days 2 hours 26 min 13 sec
CPU time	22 days 19 hours 19 min 48 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.32 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <stderr_txt> 23:38:18 (56956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:18:22 (4880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:18:31 (4880): No heartbeat from core client for 30 sec - exiting 23:18:32 (4880): No heartbeat from core client for 30 sec - exiting 23:18:33 (4880): No heartbeat from core client for 30 sec - exiting 23:18:34 (4880): No heartbeat from core client for 30 sec - exiting 23:18:35 (4880): No heartbeat from core client for 30 sec - exiting 22:28:14 (3772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:28:17 (3772): No heartbeat from core client for 30 sec - exiting 22:28:18 (3772): No heartbeat from core client for 30 sec - exiting 22:28:19 (3772): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 12:37:04 (4052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:37:05 (4052): No heartbeat from core client for 30 sec - exiting 13:57:59 (35852): No heartbeat from core client for 30 sec - exiting 13:58:13 (35852): No heartbeat from core client for 30 sec - exiting 13:58:14 (35852): No heartbeat from core client for 30 sec - exiting 13:58:15 (35852): No heartbeat from core client for 30 sec - exiting 13:58:16 (35852): No heartbeat from core client for 30 sec - exiting 13:58:17 (35852): No heartbeat from core client for 30 sec - exiting 13:58:18 (35852): No heartbeat from core client for 30 sec - exiting 13:58:19 (35852): No heartbeat from core client for 30 sec - exiting 13:58:20 (35852): No heartbeat from core client for 30 sec - exiting 13:58:21 (35852): No heartbeat from core client for 30 sec - exiting 13:58:22 (35852): No heartbeat from core client for 30 sec - exiting 13:58:23 (35852): No heartbeat from core client for 30 sec - exiting 13:58:24 (35852): No heartbeat from core client for 30 sec - exiting 13:58:25 (35852): No heartbeat from core client for 30 sec - exiting 13:59:13 (35852): No heartbeat from core client for 30 sec - exiting 13:59:14 (35852): No heartbeat from core client for 30 sec - exiting 13:59:15 (35852): No heartbeat from core client for 30 sec - exiting 13:59:16 (35852): No heartbeat from core client for 30 sec - exiting 13:59:17 (35852): No heartbeat from core client for 30 sec - exiting 13:59:18 (35852): No heartbeat from core client for 30 sec - exiting 13:59:19 (35852): No heartbeat from core client for 30 sec - exiting 13:59:20 (35852): No heartbeat from core client for 30 sec - exiting 13:59:21 (35852): No heartbeat from core client for 30 sec - exiting 13:59:22 (35852): No heartbeat from core client for 30 sec - exiting 13:59:23 (35852): No heartbeat from core client for 30 sec - exiting 13:59:24 (35852): No heartbeat from core client for 30 sec - exiting 13:59:25 (35852): No heartbeat from core client for 30 sec - exiting 13:59:26 (35852): No heartbeat from core client for 30 sec - exiting 13:59:27 (35852): No heartbeat from core client for 30 sec - exiting 13:59:28 (35852): No heartbeat from core client for 30 sec - exiting 13:59:29 (35852): No heartbeat from core client for 30 sec - exiting 13:59:30 (35852): No heartbeat from core client for 30 sec - exiting 13:59:31 (35852): No heartbeat from core client for 30 sec - exiting 13:59:32 (35852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zhewko.pj81c10 Error converting file to netcdf: dataout/zhewko.pi81c10 Error converting file to netcdf: dataout/zhewko.pf81c10 Error converting file to netcdf: dataout/zhewka.ph81c10 Error converting file to netcdf: dataout/zhewka.pg81c10 Error converting file to netcdf: dataout/zhewka.pe81c10 Error converting file to netcdf: dataout/zhewka.pd81c10 14:21:10 (41152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:11:05 (3040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:11:27 (3040): No heartbeat from core client for 30 sec - exiting 13:11:29 (3040): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CSuspended CPDN Monitor - Suspend request from BOINC... 11:19:19 (2752): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 11:19:34 (2752): No heartbeat from core client for 30 sec - exiting 11:19:35 (2752): No heartbeat from core client for 30 sec - exiting 11:19:36 (2752): No heartbeat from core client for 30 sec - exiting 11:19:37 (2752): No heartbeat from core client for 30 sec - exiting 11:19:38 (2752): No heartbeat from core client for 30 sec - exiting 11:19:39 (2752): No heartbeat from core client for 30 sec - exiting 12:28:11 (3756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:28:14 (3756): No heartbeat from core client for 30 sec - exiting 14:40:59 (4996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:41:12 (4996): No heartbeat from core client for 30 sec - exiting 14:41:13 (4996): No heartbeat from core client for 30 sec - exiting 14:41:14 (4996): No heartbeat from core client for 30 sec - exiting 14:41:15 (4996): No heartbeat from core client for 30 sec - exiting 14:41:16 (4996): No heartbeat from core client for 30 sec - exiting 14:41:17 (4996): No heartbeat from core client for 30 sec - exiting 14:41:18 (4996): No heartbeat from core client for 30 sec - exiting 14:41:19 (4996): No heartbeat from core client for 30 sec - exiting 14:41:20 (4996): No heartbeat from core client for 30 sec - exiting 14:41:21 (4996): No heartbeat from core client for 30 sec - exiting 14:41:22 (4996): No heartbeat from core client for 30 sec - exiting 14:41:23 (4996): No heartbeat from core client for 30 sec - exiting 14:41:24 (4996): No heartbeat from core client for 30 sec - exiting 14:41:25 (4996): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 23:12:12 (25736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:12:22 (25736): No heartbeat from core client for 30 sec - exiting 23:12:23 (25736): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 18:48:22 (2400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:48:28 (2400): No heartbeat from core client for 30 sec - exiting 18:48:29 (2400): No heartbeat from core client for 30 sec - exiting 18:48:30 (2400): No heartbeat from core client for 30 sec - exiting 18:48:31 (2400): No heartbeat from core client for 30 sec - exiting 18:48:32 (2400): No heartbeat from core client for 30 sec - exiting 18:48:33 (2400): No heartbeat from core client for 30 sec - exiting 18:48:34 (2400): No heartbeat from core client for 30 sec - exiting 18:48:35 (2400): No heartbeat from core client for 30 sec - exiting 18:48:36 (2400): No heartbeat from core client for 30 sec - exiting 18:48:37 (2400): No heartbeat from core client for 30 sec - exiting 18:48:38 (2400): No heartbeat from core client for 30 sec - exiting 18:48:40 (2400): No heartbeat from core client for 30 sec - exiting 18:50:04 (43280): No heartbeat from core client for 30 sec - exiting 18:50:05 (43280): No heartbeat from core client for 30 sec - exiting 18:50:06 (43280): No heartbeat from core client for 30 sec - exiting 18:50:07 (43280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:10:52 (372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:10:56 (372): No heartbeat from core client for 30 sec - exiting 13:10:57 (372): No heartbeat from core client for 30 sec - exiting 13:10:58 (372): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:58:00 (2560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:59:24 (27424): No heartbeat from core client for 30 sec - exiting 16:59:27 (27424): No heartbeat from core client for 30 sec - exiting 16:59:28 (27424): No heartbeat from core client for 30 sec - exiting 16:59:29 (27424): No heartbeat from core client for 30 sec - exiting 16:59:30 (27424): No heartbeat from core client for 30 sec - exiting 16:59:31 (27424): No heartbeat from core client for 30 sec - exiting 16:59:32 (27424): No heartbeat from core client for 30 sec - exiting 16:59:33 (27424): No heartbeat from core client for 30 sec - exiting 16:59:34 (27424): No heartbeat from core client for 30 sec - exiting 16:59:35 (27424): No heartbeat from core client for 30 sec - exiting 16:59:36 (27424): No heartbeat from core client for 30 sec - exiting 16:59:37 (27424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:52:20 (12232): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 14:52:26 (12232): No heartbeat from core client for 30 sec - exiting 14:52:27 (12232): No heartbeat from core client for 30 sec - exiting 14:52:28 (12232): No heartbeat from core client for 30 sec - exiting 14:52:29 (12232): No heartbeat from core client for 30 sec - exiting 14:52:30 (12232): No heartbeat from core client for 30 sec - exiting 14:52:31 (12232): No heartbeat from core client for 30 sec - exiting 12:08:11 (4008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:08:15 (4008): No heartbeat from core client for 30 sec - exiting 12:08:16 (4008): No heartbeat from core client for 30 sec - exiting 12:08:17 (4008): No heartbeat from core client for 30 sec - exiting 12:08:18 (4008): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 16:26:52 (14432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:27:05 (14432): No heartbeat from core client for 30 sec - exiting 16:27:06 (14432): No heartbeat from core client for 30 sec - exiting 16:27:07 (14432): No heartbeat from core client for 30 sec - exiting 16:27:08 (14432): No heartbeat from core client for 30 sec - exiting 16:27:09 (14432): No heartbeat from core client for 30 sec - exiting 16:27:10 (14432): No heartbeat from core client for 30 sec - exiting 16:36:33 (13736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:36:40 (13736): No heartbeat from core client for 30 sec - exiting 16:36:41 (13736): No heartbeat from core client for 30 sec - exiting 16:36:43 (13736): No heartbeat from core client for 30 sec - exiting 16:36:44 (13736): No heartbeat from core client for 30 sec - exiting 16:36:45 (13736): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 16:44:26 (17168): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 16:44:51 (17168): No heartbeat from core client for 30 sec - exiting 16:44:52 (17168): No heartbeat from core client for 30 sec - exiting 16:44:53 (17168): No heartbeat from core client for 30 sec - exiting 16:44:54 (17168): No heartbeat from core client for 30 sec - exiting 16:44:55 (17168): No heartbeat from core client for 30 sec - exiting 16:44:56 (17168): No heartbeat from core client for 30 sec - exiting 18:13:00 (28596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:13:10 (28596): No heartbeat from core client for 30 sec - exiting 18:13:12 (28596): No heartbeat from core client for 30 sec - exiting 18:13:13 (28596): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:24:00 (716): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 12:24:05 (716): No heartbeat from core client for 30 sec - exiting 12:24:07 (716): No heartbeat from core client for 30 sec - exiting 14:04:26 (9496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:24:47 (296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:24:51 (296): No heartbeat from core client for 30 sec - exiting 09:24:52 (296): No heartbeat from core client for 30 sec - exiting 09:24:53 (296): No heartbeat from core client for 30 sec - exiting 09:24:54 (296): No heartbeat from core client for 30 sec - exiting 09:24:55 (296): No heartbeat from core client for 30 sec - exiting 09:24:56 (296): No heartbeat from core client for 30 sec - exiting 09:24:57 (296): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5272, iMonCtr=1 Model crash detected, will try to restart... 12:21:52 (4080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:22:21 (4080): No heartbeat from core client for 30 sec - exiting 12:22:23 (4080): No heartbeat from core client for 30 sec - exiting 12:22:24 (4080): No heartbeat from core client for 30 sec - exiting 12:22:25 (4080): No heartbeat from core client for 30 sec - exiting 12:24:12 (28408): No heartbeat from core client for 30 sec - exiting 12:24:13 (28408): No heartbeat from core client for 30 sec - exiting 12:24:14 (28408): No heartbeat from core client for 30 sec - exiting 12:24:15 (28408): No heartbeat from core client for 30 sec - exiting 12:24:16 (28408): No heartbeat from core client for 30 sec - exiting 12:24:17 (28408): No heartbeat from core client for 30 sec - exiting 12:24:18 (28408): No heartbeat from core client for 30 sec - exiting 12:24:19 (28408): No heartbeat from core client for 30 sec - exiting 12:24:20 (28408): No heartbeat from core client for 30 sec - exiting 12:24:21 (28408): No heartbeat from core client for 30 sec - exiting 12:24:22 (28408): No heartbeat from core client for 30 sec - exiting 12:24:23 (28408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:27:18 (3376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:27:21 (3376): No heartbeat from core client for 30 sec - exiting 14:27:22 (3376): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 10:29:07 (2812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:29:11 (2812): No heartbeat from core client for 30 sec - exiting 10:29:12 (2812): No heartbeat from core client for 30 sec - exiting 10:29:14 (2812): No heartbeat from core client for 30 sec - exiting 10:29:15 (2812): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:42:26 (2984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:42:27 (2984): No heartbeat from core client for 30 sec - exiting 20:42:29 (2984): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold Suspended CPDN Monitor - Suspend request from BOINC... 16:29:35 (3936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:21:17 (936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:21:24 (936): No heartbeat from core client for 30 sec - exiting 09:21:25 (936): No heartbeat from core client for 30 sec - exiting 09:21:26 (936): No heartbeat from core client for 30 sec - exiting 09:21:27 (936): No heartbeat from core client for 30 sec - exiting 09:21:28 (936): No heartbeat from core client for 30 sec - exiting 09:21:29 (936): No heartbeat from core client for 30 sec - exiting 10:18:23 (9376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=20656, iMonCtr=1 Model crash detected, will try to restart... 09:23:37 (3672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:23:41 (3672): No heartbeat from core client for 30 sec - exiting 09:23:42 (3672): No heartbeat from core client for 30 sec - exiting 09:23:43 (3672): No heartbeat from core client for 30 sec - exiting 09:23:44 (3672): No heartbeat from core client for 30 sec - exiting 09:23:45 (3672): No heartbeat from core client for 30 sec - exiting 09:23:46 (3672): No heartbeat from core client for 30 sec - exiting 09:23:47 (3672): No heartbeat from core client for 30 sec - exiting 09:23:48 (3672): No heartbeat from core client for 30 sec - exiting 10:47:14 (15260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:47:18 (15260): No heartbeat from core client for 30 sec - exiting 10:47:19 (15260): No heartbeat from core client for 30 sec - exiting 10:47:20 (15260): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=39680, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3588, iMonCtr=1 Model crash detected, will try to restart... 22:49:26 (3484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:49:29 (3484): No heartbeat from core client for 30 sec - exiting 22:49:30 (3484): No heartbeat from core client for 30 sec - exiting 22:49:31 (3484): No heartbeat from core client for 30 sec - exiting 22:49:32 (3484): No heartbeat from core client for 30 sec - exiting 22:49:33 (3484): No heartbeat from core client for 30 sec - exiting 22:49:34 (3484): No heartbeat from core client for 30 sec - exiting 22:49:35 (3484): No heartbeat from core client for 30 sec - exiting 22:49:36 (3484): No heartbeat from core client for 30 sec - exiting 22:49:37 (3484): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3832, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:10:24 (3640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:10:29 (3640): No heartbeat from core client for 30 sec - exiting 14:10:30 (3640): No heartbeat from core client for 30 sec - exiting 14:10:31 (3640): No heartbeat from core client for 30 sec - exiting 14:10:32 (3640): No heartbeat from core client for 30 sec - exiting 14:10:33 (3640): No heartbeat from core client for 30 sec - exiting 14:10:34 (3640): No heartbeat from core client for 30 sec - exiting 16:29:29 (20472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=51916, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2476, iMonCtr=1 Model crash detected, will try to restart... 16:36:02 (3820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=18428, iMonCtr=1 Model crash detected, will try to restart... 10:57:46 (4064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:57:50 (4064): No heartbeat from core client for 30 sec - exiting 10:57:51 (4064): No heartbeat from core client for 30 sec - exiting 10:57:52 (4064): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=18288, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2300, iMonCtr=1 Model crash detected, will try to restart... 11:13:55 (2744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:13:57 (2744): No heartbeat from core client for 30 sec - exiting 11:13:58 (2744): No heartbeat from core client for 30 sec - exiting 11:13:59 (2744): No heartbeat from core client for 30 sec - exiting 11:14:00 (2744): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2092, iMonCtr=1 Model crash detected, will try to restart... 11:24:04 (1428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2240, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4076, iMonCtr=1 Model crash detected, will try to restart... 13:18:08 (2740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:28:30 (17104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:28:43 (17104): No heartbeat from core client for 30 sec - exiting 14:28:44 (17104): No heartbeat from core client for 30 sec - exiting 14:28:45 (17104): No heartbeat from core client for 30 sec - exiting 14:29:58 (58324): No heartbeat from core client for 30 sec - exiting 14:29:59 (58324): No heartbeat from core client for 30 sec - exiting 14:30:00 (58324): No heartbeat from core client for 30 sec - exiting 14:30:01 (58324): No heartbeat from core client for 30 sec - exiting 14:30:02 (58324): No heartbeat from core client for 30 sec - exiting 14:30:03 (58324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=59096, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3732, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2968, iMonCtr=1 Model crash detected, will try to restart... 21:17:43 (3796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:17:47 (3796): No heartbeat from core client for 30 sec - exiting 21:17:49 (3796): No heartbeat from core client for 30 sec - exiting 21:17:50 (3796): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=41920, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2120, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3728, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=688, iMonCtr=1 Model crash detected, will try to restart... 09:50:43 (2636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:50:45 (2636): No heartbeat from core client for 30 sec - exiting 15:04:06 (3028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:04:11 (3028): No heartbeat from core client for 30 sec - exiting 15:04:14 (3028): No heartbeat from core client for 30 sec - exiting 15:04:15 (3028): No heartbeat from core client for 30 sec - exiting 15:04:16 (3028): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=41356, iMonCtr=1 Model crash detected, will try to restart... 10:48:30 (2228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=380, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4076, iMonCtr=1 Model crash detected, will try to restart... 10:43:24 (2480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:43:25 (2480): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=16576, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3164, iMonCtr=1 Model crash detected, will try to restart... 14:01:02 (3276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:01:07 (3276): No heartbeat from core client for 30 sec - exiting 14:01:08 (3276): No heartbeat from core client for 30 sec - exiting 14:01:09 (3276): No heartbeat from core client for 30 sec - exiting 14:01:10 (3276): No heartbeat from core client for 30 sec - exiting 14:01:11 (3276): No heartbeat from core client for 30 sec - exiting 14:01:12 (3276): No heartbeat from core client for 30 sec - exiting 14:01:13 (3276): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=57052, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=33100, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1256, iMonCtr=1 Model crash detected, will try to restart... 09:51:02 (3636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Mar 2013 00:28:20	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	1,036,800	1,969,704	1.8998
03 Mar 2013 12:50:01	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	1,010,880	1,922,635	1.9019
01 Mar 2013 09:17:36	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	984,960	1,875,346	1.9040
27 Feb 2013 15:05:33	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	959,040	1,828,732	1.9068
26 Feb 2013 13:40:25	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	933,120	1,782,862	1.9106
25 Feb 2013 13:31:22	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	907,200	1,736,925	1.9146
24 Feb 2013 13:15:19	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	881,280	1,691,088	1.9189
23 Feb 2013 15:05:36	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	855,360	1,646,041	1.9244
21 Feb 2013 14:00:26	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	829,440	1,599,759	1.9287
20 Feb 2013 11:18:34	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	803,520	1,552,681	1.9323
19 Feb 2013 00:56:31	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	777,600	1,506,367	1.9372
16 Feb 2013 10:14:16	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	751,680	1,460,607	1.9431
15 Feb 2013 06:46:44	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	725,760	1,415,223	1.9500
14 Feb 2013 04:44:52	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	699,840	1,369,542	1.9569
09 Feb 2013 03:15:30	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	673,920	1,323,411	1.9638
07 Feb 2013 14:10:58	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	648,000	1,275,447	1.9683
06 Feb 2013 07:12:53	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	622,080	1,225,837	1.9705
05 Feb 2013 07:56:42	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	596,160	1,178,996	1.9777
04 Feb 2013 08:22:43	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	570,240	1,131,275	1.9839
03 Feb 2013 07:52:38	1213970	15477883	hadcm3n_zhew_1880_40_008250605_3	544,320	1,083,624	1.9908