Task 16085577

Name	hadcm3n_n3yz_1920_40_008371574_2
Workunit	8522433
Created	25 Nov 2013, 22:22:43 UTC
Sent	25 Nov 2013, 22:33:22 UTC
Report deadline	25 Feb 2014, 6:00:33 UTC
Received	4 Feb 2014, 18:21:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1179495
Run time	41 days 3 hours 20 min 47 sec
CPU time	31 days 20 hours 57 min 13 sec
Validate state	Invalid
Credit	9,642.24
Device peak FLOPS	2.54 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> Das Gerät erkennt den Befehl nicht. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3828, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3864, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3396, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3396, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3396, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3904, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3904, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3904, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4960, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4960, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6420, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3812, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3812, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3812, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3812, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3812, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3812, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3812, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3812, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3812, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3812, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3812, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3812, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3812, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/n3yzko.pjc7c10 Error converting file to netcdf: dataout/n3yzko.pic7c10 Error converting file to netcdf: dataout/n3yzko.pfc7c10 Error converting file to netcdf: dataout/n3yzka.phc7c10 Error converting file to netcdf: dataout/n3yzka.pgc7c10 Error converting file to netcdf: dataout/n3yzka.pec7c10 Error converting file to netcdf: dataout/n3yzka.pdc7c10 CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7052, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3844, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3844, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3844, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3616, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3616, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3912, iMonCtr=1 Model crash detected, will try to restart... Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... 04:22:14 (4428): No heartbeat from core client for 30 sec - exiting 04:22:16 (4428): No heartbeat from core client for 30 sec - exiting 04:22:17 (4428): No heartbeat from core client for 30 sec - exiting 04:22:18 (4428): No heartbeat from core client for 30 sec - exiting 04:22:19 (4428): No heartbeat from core client for 30 sec - exiting 04:22:20 (4428): No heartbeat from core client for 30 sec - exiting 04:22:21 (4428): No heartbeat from core client for 30 sec - exiting 04:22:22 (4428): No heartbeat from core client for 30 sec - exiting 04:22:23 (4428): No heartbeat from core client for 30 sec - exiting 04:22:24 (4428): No heartbeat from core client for 30 sec - exiting 04:22:25 (4428): No heartbeat from core client for 30 sec - exiting 04:22:26 (4428): No heartbeat from core client for 30 sec - exiting 04:22:27 (4428): No heartbeat from core client for 30 sec - exiting 04:22:28 (4428): No heartbeat from core client for 30 sec - exiting 04:22:29 (4428): No heartbeat from core client for 30 sec - exiting 04:22:30 (4428): No heartbeat from core client for 30 sec - exiting 04:22:31 (4428): No heartbeat from core client for 30 sec - exiting 04:22:32 (4428): No heartbeat from core client for 30 sec - exiting 04:22:33 (4428): No heartbeat from core client for 30 sec - exiting 04:22:34 (4428): No heartbeat from core client for 30 sec - exiting 04:22:35 (4428): No heartbeat from core client for 30 sec - exiting 04:22:36 (4428): No heartbeat from core client for 30 sec - exiting 04:22:37 (4428): No heartbeat from core client for 30 sec - exiting 04:22:38 (4428): No heartbeat from core client for 30 sec - exiting 04:22:39 (4428): No heartbeat from core client for 30 sec - exiting 04:22:40 (4428): No heartbeat from core client for 30 sec - exiting 04:22:41 (4428): No heartbeat from core client for 30 sec - exiting 04:22:42 (4428): No heartbeat from core client for 30 sec - exiting 04:22:43 (4428): No heartbeat from core client for 30 sec - exiting 04:22:44 (4428): No heartbeat from core client for 30 sec - exiting 04:22:45 (4428): No heartbeat from core client for 30 sec - exiting 04:22:46 (4428): No heartbeat from core client for 30 sec - exiting 04:22:47 (4428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1420, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:19:10 (7480): No heartbeat from core client for 30 sec - exiting 14:19:11 (7480): No heartbeat from core client for 30 sec - exiting 14:19:12 (7480): No heartbeat from core client for 30 sec - exiting 14:19:13 (7480): No heartbeat from core client for 30 sec - exiting 14:19:14 (7480): No heartbeat from core client for 30 sec - exiting 14:19:15 (7480): No heartbeat from core client for 30 sec - exiting 14:19:16 (7480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:37:19 (504): No heartbeat from core client for 30 sec - exiting 20:37:20 (504): No heartbeat from core client for 30 sec - exiting 20:37:21 (504): No heartbeat from core client for 30 sec - exiting 20:37:22 (504): No heartbeat from core client for 30 sec - exiting 20:37:23 (504): No heartbeat from core client for 30 sec - exiting 20:37:24 (504): No heartbeat from core client for 30 sec - exiting 20:37:25 (504): No heartbeat from core client for 30 sec - exiting 20:37:26 (504): No heartbeat from core client for 30 sec - exiting 20:37:27 (504): No heartbeat from core client for 30 sec - exiting 20:37:28 (504): No heartbeat from core client for 30 sec - exiting 20:37:29 (504): No heartbeat from core client for 30 sec - exiting 20:37:30 (504): No heartbeat from core client for 30 sec - exiting 20:37:31 (504): No heartbeat from core client for 30 sec - exiting 20:37:32 (504): No heartbeat from core client for 30 sec - exiting 20:37:33 (504): No heartbeat from core client for 30 sec - exiting 20:37:34 (504): No heartbeat from core client for 30 sec - exiting 20:37:35 (504): No heartbeat from core client for 30 sec - exiting 20:37:36 (504): No heartbeat from core client for 30 sec - exiting 20:37:37 (504): No heartbeat from core client for 30 sec - exiting 20:37:38 (504): No heartbeat from core client for 30 sec - exiting 20:37:39 (504): No heartbeat from core client for 30 sec - exiting 20:37:40 (504): No heartbeat from core client for 30 sec - exiting 20:37:41 (504): No heartbeat from core client for 30 sec - exiting 20:37:42 (504): No heartbeat from core client for 30 sec - exiting 20:37:43 (504): No heartbeat from core client for 30 sec - exiting 20:37:44 (504): No heartbeat from core client for 30 sec - exiting 20:37:45 (504): No heartbeat from core client for 30 sec - exiting 20:37:46 (504): No heartbeat from core client for 30 sec - exiting 20:37:47 (504): No heartbeat from core client for 30 sec - exiting 20:37:48 (504): No heartbeat from core client for 30 sec - exiting 20:37:49 (504): No heartbeat from core client for 30 sec - exiting 20:37:50 (504): No heartbeat from core client for 30 sec - exiting 20:37:51 (504): No heartbeat from core client for 30 sec - exiting 20:37:52 (504): No heartbeat from core client for 30 sec - exiting 20:37:53 (504): No heartbeat from core client for 30 sec - exiting 20:37:54 (504): No heartbeat from core client for 30 sec - exiting 20:37:55 (504): No heartbeat from core client for 30 sec - exiting 20:37:56 (504): No heartbeat from core client for 30 sec - exiting 20:37:57 (504): No heartbeat from core client for 30 sec - exiting 20:37:58 (504): No heartbeat from core client for 30 sec - exiting 20:37:59 (504): No heartbeat from core client for 30 sec - exiting 20:38:00 (504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4000, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4000, iMonCtr=1 Model crash detected, will try to restart... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4076, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:03:43 (1760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:30:19 (3788): No heartbeat from core client for 30 sec - exiting 17:30:20 (3788): No heartbeat from core client for 30 sec - exiting 17:30:21 (3788): No heartbeat from core client for 30 sec - exiting 17:30:22 (3788): No heartbeat from core client for 30 sec - exiting 17:30:23 (3788): No heartbeat from core client for 30 sec - exiting 17:30:24 (3788): No heartbeat from core client for 30 sec - exiting 17:30:25 (3788): No heartbeat from core client for 30 sec - exiting 17:30:26 (3788): No heartbeat from core client for 30 sec - exiting 17:30:27 (3788): No heartbeat from core client for 30 sec - exiting 17:30:28 (3788): No heartbeat from core client for 30 sec - exiting 17:30:29 (3788): No heartbeat from core client for 30 sec - exiting 17:30:30 (3788): No heartbeat from core client for 30 sec - exiting 17:30:31 (3788): No heartbeat from core client for 30 sec - exiting 17:30:32 (3788): No heartbeat from core client for 30 sec - exiting 17:30:33 (3788): No heartbeat from core client for 30 sec - exiting 17:30:34 (3788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied 17:41:01 (3792): No heartbeat from core client for 30 sec - exiting 17:41:03 (3792): No heartbeat from core client for 30 sec - exiting 17:41:04 (3792): No heartbeat from core client for 30 sec - exiting 17:41:05 (3792): No heartbeat from core client for 30 sec - exiting 17:41:06 (3792): No heartbeat from core client for 30 sec - exiting 17:41:07 (3792): No heartbeat from core client for 30 sec - exiting 17:41:08 (3792): No heartbeat from core client for 30 sec - exiting 17:41:09 (3792): No heartbeat from core client for 30 sec - exiting 17:41:10 (3792): No heartbeat from core client for 30 sec - exiting 17:41:11 (3792): No heartbeat from core client for 30 sec - exiting 17:41:12 (3792): No heartbeat from core client for 30 sec - exiting 17:41:13 (3792): No heartbeat from core client for 30 sec - exiting 17:41:14 (3792): No heartbeat from core client for 30 sec - exiting 17:41:15 (3792): No heartbeat from core client for 30 sec - exiting 17:41:16 (3792): No heartbeat from core client for 30 sec - exiting 17:41:17 (3792): No heartbeat from core client for 30 sec - exiting 17:41:18 (3792): No heartbeat from core client for 30 sec - exiting 17:41:19 (3792): No heartbeat from core client for 30 sec - exiting 17:41:20 (3792): No heartbeat from core client for 30 sec - exiting 17:41:21 (3792): No heartbeat from core client for 30 sec - exiting 17:41:22 (3792): No heartbeat from core client for 30 sec - exiting 17:41:23 (3792): No heartbeat from core client for 30 sec - exiting 17:41:24 (3792): No heartbeat from core client for 30 sec - exiting 17:41:25 (3792): No heartbeat from core client for 30 sec - exiting 17:41:26 (3792): No heartbeat from core client for 30 sec - exiting 17:41:27 (3792): No heartbeat from core client for 30 sec - exiting 17:41:28 (3792): No heartbeat from core client for 30 sec - exiting 17:41:29 (3792): No heartbeat from core client for 30 sec - exiting 17:41:30 (3792): No heartbeat from core client for 30 sec - exiting 17:41:31 (3792): No heartbeat from core client for 30 sec - exiting 17:41:32 (3792): No heartbeat from core client for 30 sec - exiting 17:41:33 (3792): No heartbeat from core client for 30 sec - exiting 17:41:34 (3792): No heartbeat from core client for 30 sec - exiting 17:41:35 (3792): No heartbeat from core client for 30 sec - exiting 17:41:36 (3792): No heartbeat from core client for 30 sec - exiting 17:41:37 (3792): No heartbeat from core client for 30 sec - exiting 17:41:38 (3792): No heartbeat from core client for 30 sec - exiting 17:41:39 (3792): No heartbeat from core client for 30 sec - exiting 17:41:40 (3792): No heartbeat from core client for 30 sec - exiting 17:41:41 (3792): No heartbeat from core client for 30 sec - exiting 17:41:42 (3792): No heartbeat from core client for 30 sec - exiting 17:41:43 (3792): No heartbeat from core client for 30 sec - exiting 17:41:44 (3792): No heartbeat from core client for 30 sec - exiting 17:41:45 (3792): No heartbeat from core client for 30 sec - exiting 17:41:46 (3792): No heartbeat from core client for 30 sec - exiting 17:41:47 (3792): No heartbeat from core client for 30 sec - exiting 17:41:48 (3792): No heartbeat from core client for 30 sec - exiting 17:41:49 (3792): No heartbeat from core client for 30 sec - exiting 17:41:50 (3792): No heartbeat from core client for 30 sec - exiting 17:41:51 (3792): No heartbeat from core client for 30 sec - exiting 17:41:52 (3792): No heartbeat from core client for 30 sec - exiting 17:41:53 (3792): No heartbeat from core client for 30 sec - exiting 17:41:54 (3792): No heartbeat from core client for 30 sec - exiting 17:41:55 (3792): No heartbeat from core client for 30 sec - exiting 17:41:56 (3792): No heartbeat from core client for 30 sec - exiting 17:41:57 (3792): No heartbeat from core client for 30 sec - exiting 17:41:58 (3792): No heartbeat from core client for 30 sec - exiting 17:41:59 (3792): No heartbeat from core client for 30 sec - exiting 17:42:00 (3792): No heartbeat from core client for 30 sec - exiting 17:42:01 (3792): No heartbeat from core client for 30 sec - exiting 17:42:02 (3792): No heartbeat from core client for 30 sec - exiting 17:42:03 (3792): No heartbeat from core client for 30 sec - exiting 17:42:04 (3792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied 03:09:04 (3788): No heartbeat from core client for 30 sec - exiting 03:09:06 (3788): No heartbeat from core client for 30 sec - exiting 03:09:07 (3788): No heartbeat from core client for 30 sec - exiting 03:09:08 (3788): No heartbeat from core client for 30 sec - exiting 03:09:09 (3788): No heartbeat from core client for 30 sec - exiting 03:09:10 (3788): No heartbeat from core client for 30 sec - exiting 03:09:11 (3788): No heartbeat from core client for 30 sec - exiting 03:09:12 (3788): No heartbeat from core client for 30 sec - exiting 03:09:13 (3788): No heartbeat from core client for 30 sec - exiting 03:09:14 (3788): No heartbeat from core client for 30 sec - exiting 03:09:15 (3788): No heartbeat from core client for 30 sec - exiting 03:09:16 (3788): No heartbeat from core client for 30 sec - exiting 03:09:17 (3788): No heartbeat from core client for 30 sec - exiting 03:09:18 (3788): No heartbeat from core client for 30 sec - exiting 03:09:19 (3788): No heartbeat from core client for 30 sec - exiting 03:09:20 (3788): No heartbeat from core client for 30 sec - exiting 03:09:21 (3788): No heartbeat from core client for 30 sec - exiting 03:09:22 (3788): No heartbeat from core client for 30 sec - exiting 03:09:23 (3788): No heartbeat from core client for 30 sec - exiting 03:09:24 (3788): No heartbeat from core client for 30 sec - exiting 03:09:25 (3788): No heartbeat from core client for 30 sec - exiting 03:09:26 (3788): No heartbeat from core client for 30 sec - exiting 03:09:27 (3788): No heartbeat from core client for 30 sec - exiting 03:09:28 (3788): No heartbeat from core client for 30 sec - exiting 03:09:29 (3788): No heartbeat from core client for 30 sec - exiting 03:09:30 (3788): No heartbeat from core client for 30 sec - exiting 03:09:31 (3788): No heartbeat from core client for 30 sec - exiting 03:09:32 (3788): No heartbeat from core client for 30 sec - exiting 03:09:33 (3788): No heartbeat from core client for 30 sec - exiting 03:09:34 (3788): No heartbeat from core client for 30 sec - exiting 03:09:35 (3788): No heartbeat from core client for 30 sec - exiting 03:09:36 (3788): No heartbeat from core client for 30 sec - exiting 03:09:37 (3788): No heartbeat from core client for 30 sec - exiting 03:09:38 (3788): No heartbeat from core client for 30 sec - exiting 03:09:39 (3788): No heartbeat from core client for 30 sec - exiting 03:09:40 (3788): No heartbeat from core client for 30 sec - exiting 03:09:41 (3788): No heartbeat from core client for 30 sec - exiting 03:09:42 (3788): No heartbeat from core client for 30 sec - exiting 03:09:43 (3788): No heartbeat from core client for 30 sec - exiting 03:09:44 (3788): No heartbeat from core client for 30 sec - exiting 03:09:45 (3788): No heartbeat from core client for 30 sec - exiting 03:09:46 (3788): No heartbeat from core client for 30 sec - exiting 03:09:47 (3788): No heartbeat from core client for 30 sec - exiting 03:09:48 (3788): No heartbeat from core client for 30 sec - exiting 03:09:49 (3788): No heartbeat from core client for 30 sec - exiting 03:09:50 (3788): No heartbeat from core client for 30 sec - exiting 03:09:51 (3788): No heartbeat from core client for 30 sec - exiting 03:09:52 (3788): No heartbeat from core client for 30 sec - exiting 03:09:53 (3788): No heartbeat from core client for 30 sec - exiting 03:09:54 (3788): No heartbeat from core client for 30 sec - exiting 03:09:55 (3788): No heartbeat from core client for 30 sec - exiting 03:09:56 (3788): No heartbeat from core client for 30 sec - exiting 03:09:57 (3788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:09:58 (3788): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 13:24:26 (3268): No heartbeat from core client for 30 sec - exiting 13:24:27 (3268): No heartbeat from core client for 30 sec - exiting 13:24:28 (3268): No heartbeat from core client for 30 sec - exiting 13:24:29 (3268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:47:50 (5036): No heartbeat from core client for 30 sec - exiting 19:47:51 (5036): No heartbeat from core client for 30 sec - exiting 19:47:52 (5036): No heartbeat from core client for 30 sec - exiting 19:47:53 (5036): No heartbeat from core client for 30 sec - exiting 19:47:54 (5036): No heartbeat from core client for 30 sec - exiting 19:47:55 (5036): No heartbeat from core client for 30 sec - exiting 19:47:56 (5036): No heartbeat from core client for 30 sec - exiting 19:47:57 (5036): No heartbeat from core client for 30 sec - exiting 19:47:58 (5036): No heartbeat from core client for 30 sec - exiting 19:47:59 (5036): No heartbeat from core client for 30 sec - exiting 19:48:00 (5036): No heartbeat from core client for 30 sec - exiting 19:48:01 (5036): No heartbeat from core client for 30 sec - exiting 19:48:02 (5036): No heartbeat from core client for 30 sec - exiting 19:48:03 (5036): No heartbeat from core client for 30 sec - exiting 19:48:04 (5036): No heartbeat from core client for 30 sec - exiting 19:48:05 (5036): No heartbeat from core client for 30 sec - exiting 19:48:06 (5036): No heartbeat from core client for 30 sec - exiting 19:48:07 (5036): No heartbeat from core client for 30 sec - exiting 19:48:08 (5036): No heartbeat from core client for 30 sec - exiting 19:48:09 (5036): No heartbeat from core client for 30 sec - exiting 19:48:10 (5036): No heartbeat from core client for 30 sec - exiting 19:48:11 (5036): No heartbeat from core client for 30 sec - exiting 19:48:12 (5036): No heartbeat from core client for 30 sec - exiting 19:48:13 (5036): No heartbeat from core client for 30 sec - exiting 19:48:14 (5036): No heartbeat from core client for 30 sec - exiting 19:48:15 (5036): No heartbeat from core client for 30 sec - exiting 19:48:16 (5036): No heartbeat from core client for 30 sec - exiting 19:48:17 (5036): No heartbeat from core client for 30 sec - exiting 19:48:18 (5036): No heartbeat from core client for 30 sec - exiting 19:48:19 (5036): No heartbeat from core client for 30 sec - exiting 19:48:20 (5036): No heartbeat from core client for 30 sec - exiting 19:48:21 (5036): No heartbeat from core client for 30 sec - exiting 19:48:22 (5036): No heartbeat from core client for 30 sec - exiting 19:48:23 (5036): No heartbeat from core client for 30 sec - exiting 19:48:24 (5036): No heartbeat from core client for 30 sec - exiting 19:48:25 (5036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied 18:22:18 (3984): No heartbeat from core client for 30 sec - exiting 18:22:20 (3984): No heartbeat from core client for 30 sec - exiting 18:22:21 (3984): No heartbeat from core client for 30 sec - exiting 18:22:22 (3984): No heartbeat from core client for 30 sec - exiting 18:22:23 (3984): No heartbeat from core client for 30 sec - exiting 18:22:24 (3984): No heartbeat from core client for 30 sec - exiting 18:22:25 (3984): No heartbeat from core client for 30 sec - exiting 18:22:26 (3984): No heartbeat from core client for 30 sec - exiting 18:22:27 (3984): No heartbeat from core client for 30 sec - exiting 18:22:28 (3984): No heartbeat from core client for 30 sec - exiting 18:22:29 (3984): No heartbeat from core client for 30 sec - exiting 18:22:30 (3984): No heartbeat from core client for 30 sec - exiting 18:22:31 (3984): No heartbeat from core client for 30 sec - exiting 18:22:32 (3984): No heartbeat from core client for 30 sec - exiting 18:22:33 (3984): No heartbeat from core client for 30 sec - exiting 18:22:34 (3984): No heartbeat from core client for 30 sec - exiting 18:22:35 (3984): No heartbeat from core client for 30 sec - exiting 18:22:36 (3984): No heartbeat from core client for 30 sec - exiting 18:22:37 (3984): No heartbeat from core client for 30 sec - exiting 18:22:38 (3984): No heartbeat from core client for 30 sec - exiting 18:22:39 (3984): No heartbeat from core client for 30 sec - exiting 18:22:40 (3984): No heartbeat from core client for 30 sec - exiting 18:22:41 (3984): No heartbeat from core client for 30 sec - exiting 18:22:42 (3984): No heartbeat from core client for 30 sec - exiting 18:22:43 (3984): No heartbeat from core client for 30 sec - exiting 18:22:44 (3984): No heartbeat from core client for 30 sec - exiting 18:22:45 (3984): No heartbeat from core client for 30 sec - exiting 18:22:46 (3984): No heartbeat from core client for 30 sec - exiting 18:22:47 (3984): No heartbeat from core client for 30 sec - exiting 18:22:48 (3984): No heartbeat from core client for 30 sec - exiting 18:22:49 (3984): No heartbeat from core client for 30 sec - exiting 18:22:50 (3984): No heartbeat from core client for 30 sec - exiting 18:22:51 (3984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:22:52 (3984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 22:11:05 (5188): No heartbeat from core client for 30 sec - exiting 22:11:07 (5188): No heartbeat from core client for 30 sec - exiting 22:11:08 (5188): No heartbeat from core client for 30 sec - exiting 22:11:09 (5188): No heartbeat from core client for 30 sec - exiting 22:11:10 (5188): No heartbeat from core client for 30 sec - exiting 22:11:11 (5188): No heartbeat from core client for 30 sec - exiting 22:11:12 (5188): No heartbeat from core client for 30 sec - exiting 22:11:13 (5188): No heartbeat from core client for 30 sec - exiting 22:11:14 (5188): No heartbeat from core client for 30 sec - exiting 22:11:15 (5188): No heartbeat from core client for 30 sec - exiting 22:11:16 (5188): No heartbeat from core client for 30 sec - exiting 22:11:17 (5188): No heartbeat from core client for 30 sec - exiting 22:11:18 (5188): No heartbeat from core client for 30 sec - exiting 22:11:19 (5188): No heartbeat from core client for 30 sec - exiting 22:11:20 (5188): No heartbeat from core client for 30 sec - exiting 22:11:21 (5188): No heartbeat from core client for 30 sec - exiting 22:11:22 (5188): No heartbeat from core client for 30 sec - exiting 22:11:23 (5188): No heartbeat from core client for 30 sec - exiting 22:11:24 (5188): No heartbeat from core client for 30 sec - exiting 22:11:25 (5188): No heartbeat from core client for 30 sec - exiting 22:11:26 (5188): No heartbeat from core client for 30 sec - exiting 22:11:27 (5188): No heartbeat from core client for 30 sec - exiting 22:11:28 (5188): No heartbeat from core client for 30 sec - exiting 22:11:29 (5188): No heartbeat from core client for 30 sec - exiting 22:11:30 (5188): No heartbeat from core client for 30 sec - exiting 22:11:31 (5188): No heartbeat from core client for 30 sec - exiting 22:11:32 (5188): No heartbeat from core client for 30 sec - exiting 22:11:33 (5188): No heartbeat from core client for 30 sec - exiting 22:11:34 (5188): No heartbeat from core client for 30 sec - exiting 22:11:35 (5188): No heartbeat from core client for 30 sec - exiting 22:11:36 (5188): No heartbeat from core client for 30 sec - exiting 22:11:37 (5188): No heartbeat from core client for 30 sec - exiting 22:11:38 (5188): No heartbeat from core client for 30 sec - exiting 22:11:39 (5188): No heartbeat from core client for 30 sec - exiting 22:11:40 (5188): No heartbeat from core client for 30 sec - exiting 22:11:41 (5188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:11:42 (5188): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 17:40:43 (3928): No heartbeat from core client for 30 sec - exiting 17:40:44 (3928): No heartbeat from core client for 30 sec - exiting 17:40:45 (3928): No heartbeat from core client for 30 sec - exiting 17:40:46 (3928): No heartbeat from core client for 30 sec - exiting 17:40:47 (3928): No heartbeat from core client for 30 sec - exiting 17:40:48 (3928): No heartbeat from core client for 30 sec - exiting 17:40:49 (3928): No heartbeat from core client for 30 sec - exiting 17:40:50 (3928): No heartbeat from core client for 30 sec - exiting 17:40:51 (3928): No heartbeat from core client for 30 sec - exiting 17:40:52 (3928): No heartbeat from core client for 30 sec - exiting 17:40:53 (3928): No heartbeat from core client for 30 sec - exiting 17:40:54 (3928): No heartbeat from core client for 30 sec - exiting 17:40:55 (3928): No heartbeat from core client for 30 sec - exiting 17:40:56 (3928): No heartbeat from core client for 30 sec - exiting 17:40:57 (3928): No heartbeat from core client for 30 sec - exiting 17:40:58 (3928): No heartbeat from core client for 30 sec - exiting 17:40:59 (3928): No heartbeat from core client for 30 sec - exiting 17:41:00 (3928): No heartbeat from core client for 30 sec - exiting 17:41:01 (3928): No heartbeat from core client for 30 sec - exiting 17:41:02 (3928): No heartbeat from core client for 30 sec - exiting 17:41:03 (3928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:41:04 (3928): No heartbeat from core client for 30 sec - exiting 17:41:05 (3928): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... 02:15:25 (3756): No heartbeat from core client for 30 sec - exiting 02:15:26 (3756): No heartbeat from core client for 30 sec - exiting 02:15:27 (3756): No heartbeat from core client for 30 sec - exiting 02:15:28 (3756): No heartbeat from core client for 30 sec - exiting 02:15:29 (3756): No heartbeat from core client for 30 sec - exiting 02:15:30 (3756): No heartbeat from core client for 30 sec - exiting 02:15:31 (3756): No heartbeat from core client for 30 sec - exiting 02:15:32 (3756): No heartbeat from core client for 30 sec - exiting 02:15:33 (3756): No heartbeat from core client for 30 sec - exiting 02:15:34 (3756): No heartbeat from core client for 30 sec - exiting 02:15:35 (3756): No heartbeat from core client for 30 sec - exiting 02:15:36 (3756): No heartbeat from core client for 30 sec - exiting 02:15:37 (3756): No heartbeat from core client for 30 sec - exiting 02:15:38 (3756): No heartbeat from core client for 30 sec - exiting 02:15:39 (3756): No heartbeat from core client for 30 sec - exiting 02:15:40 (3756): No heartbeat from core client for 30 sec - exiting 02:15:41 (3756): No heartbeat from core client for 30 sec - exiting 02:15:42 (3756): No heartbeat from core client for 30 sec - exiting 02:15:43 (3756): No heartbeat from core client for 30 sec - exiting 02:15:44 (3756): No heartbeat from core client for 30 sec - exiting 02:15:45 (3756): No heartbeat from core client for 30 sec - exiting 02:15:46 (3756): No heartbeat from core client for 30 sec - exiting 02:15:47 (3756): No heartbeat from core client for 30 sec - exiting 02:15:48 (3756): No heartbeat from core client for 30 sec - exiting 02:15:49 (3756): No heartbeat from core client for 30 sec - exiting 02:15:50 (3756): No heartbeat from core client for 30 sec - exiting 02:15:51 (3756): No heartbeat from core client for 30 sec - exiting 02:15:52 (3756): No heartbeat from core client for 30 sec - exiting 02:15:53 (3756): No heartbeat from core client for 30 sec - exiting 02:15:54 (3756): No heartbeat from core client for 30 sec - exiting 02:15:55 (3756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:27:48 (1336): No heartbeat from core client for 30 sec - exiting 18:27:49 (1336): No heartbeat from core client for 30 sec - exiting 18:27:50 (1336): No heartbeat from core client for 30 sec - exiting 18:27:51 (1336): No heartbeat from core client for 30 sec - exiting 18:27:52 (1336): No heartbeat from core client for 30 sec - exiting 18:27:53 (1336): No heartbeat from core client for 30 sec - exiting 18:27:54 (1336): No heartbeat from core client for 30 sec - exiting 18:27:55 (1336): No heartbeat from core client for 30 sec - exiting 18:27:56 (1336): No heartbeat from core client for 30 sec - exiting 18:27:57 (1336): No heartbeat from core client for 30 sec - exiting 18:27:58 (1336): No heartbeat from core client for 30 sec - exiting 18:27:59 (1336): No heartbeat from core client for 30 sec - exiting 18:28:00 (1336): No heartbeat from core client for 30 sec - exiting 18:28:01 (1336): No heartbeat from core client for 30 sec - exiting 18:28:02 (1336): No heartbeat from core client for 30 sec - exiting 18:28:03 (1336): No heartbeat from core client for 30 sec - exiting 18:28:04 (1336): No heartbeat from core client for 30 sec - exiting 18:28:05 (1336): No heartbeat from core client for 30 sec - exiting 18:28:06 (1336): No heartbeat from core client for 30 sec - exiting 18:28:07 (1336): No heartbeat from core client for 30 sec - exiting 18:28:08 (1336): No heartbeat from core client for 30 sec - exiting 18:28:09 (1336): No heartbeat from core client for 30 sec - exiting 18:28:10 (1336): No heartbeat from core client for 30 sec - exiting 18:28:11 (1336): No heartbeat from core client for 30 sec - exiting 18:28:12 (1336): No heartbeat from core client for 30 sec - exiting 18:28:13 (1336): No heartbeat from core client for 30 sec - exiting 18:28:14 (1336): No heartbeat from core client for 30 sec - exiting 18:28:15 (1336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:28:16 (1336): No heartbeat from core client for 30 sec - exiting 18:28:17 (1336): No heartbeat from core client for 30 sec - exiting 18:28:18 (1336): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 14:37:10 (6376): No heartbeat from core client for 30 sec - exiting 14:37:11 (6376): No heartbeat from core client for 30 sec - exiting 14:37:12 (6376): No heartbeat from core client for 30 sec - exiting 14:37:13 (6376): No heartbeat from core client for 30 sec - exiting 14:37:14 (6376): No heartbeat from core client for 30 sec - exiting 14:37:15 (6376): No heartbeat from core client for 30 sec - exiting 14:37:16 (6376): No heartbeat from core client for 30 sec - exiting 14:37:17 (6376): No heartbeat from core client for 30 sec - exiting 14:37:18 (6376): No heartbeat from core client for 30 sec - exiting 14:37:19 (6376): No heartbeat from core client for 30 sec - exiting 14:37:20 (6376): No heartbeat from core client for 30 sec - exiting 14:37:21 (6376): No heartbeat from core client for 30 sec - exiting 14:37:22 (6376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4068, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3544, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3544, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3384, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3384, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3408, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3408, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5000, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3668, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3668, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3668, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3668, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3668, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3532, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3532, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3556, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3556, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3556, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3556, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3688, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3688, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3688, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6908, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3540, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3540, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:00:43 (5824): No heartbeat from core client for 30 sec - exiting 16:00:45 (5824): No heartbeat from core client for 30 sec - exiting 16:00:46 (5824): No heartbeat from core client for 30 sec - exiting 16:00:47 (5824): No heartbeat from core client for 30 sec - exiting 16:00:48 (5824): No heartbeat from core client for 30 sec - exiting 16:00:49 (5824): No heartbeat from core client for 30 sec - exiting 16:00:50 (5824): No heartbeat from core client for 30 sec - exiting 16:00:51 (5824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3404, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3404, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3404, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3404, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3404, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3404, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Feb 2014 21:01:52	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	803,520	2,730,405	3.3981
01 Feb 2014 01:45:32	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	777,600	2,637,317	3.3916
30 Jan 2014 16:42:22	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	751,680	2,546,652	3.3879
26 Jan 2014 14:43:13	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	725,760	2,453,159	3.3801
25 Jan 2014 01:55:55	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	699,840	2,359,167	3.3710
22 Jan 2014 19:37:29	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	673,920	2,267,756	3.3650
19 Jan 2014 21:02:44	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	648,000	2,178,038	3.3612
17 Jan 2014 23:19:52	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	622,080	2,089,073	3.3582
15 Jan 2014 05:33:04	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	596,160	1,995,583	3.3474
12 Jan 2014 18:59:29	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	570,240	1,912,411	3.3537
11 Jan 2014 03:15:19	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	544,320	1,839,809	3.3800
08 Jan 2014 20:33:31	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	518,400	1,750,630	3.3770
06 Jan 2014 07:58:28	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	492,480	1,656,079	3.3627
04 Jan 2014 17:45:53	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	466,560	1,562,740	3.3495
02 Jan 2014 23:03:23	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	440,640	1,471,094	3.3385
01 Jan 2014 10:04:39	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	414,720	1,385,423	3.3406
30 Dec 2013 06:46:28	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	388,800	1,295,077	3.3310
28 Dec 2013 13:56:51	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	362,880	1,206,500	3.3248
26 Dec 2013 20:59:00	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	336,960	1,118,292	3.3188
24 Dec 2013 08:13:31	1179495	16085577	hadcm3n_n3yz_1920_40_008371574_2	311,040	1,034,298	3.3253