climateprediction.net home page
Task 14917181

Task 14917181

Name hadam3p_pnw_bctl_1977_1_008039095_2
Workunit 8194209
Created 15 Jul 2012, 11:43:43 UTC
Sent 15 Jul 2012, 11:43:55 UTC
Report deadline 27 Jun 2013, 17:03:55 UTC
Received 31 Jul 2012, 19:06:51 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1220942
Run time 6 days 8 hours 35 min 2 sec
CPU time 5 days 5 hours 5 min 47 sec
Validate state Workunit error - check skipped
Credit 3,006.03
Device peak FLOPS 2.25 GFLOPS
Application version UK Met Office HadAM3P-HadRM3P Pacific North West v6.09
windows_intelx86
Stderr 
<core_client_version>7.0.25</core_client_version>
<![CDATA[
<stderr_txt>
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3288, iMonCtr=2
Model crash detected, will try to restart...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6216, iMonCtr=2
CPDN Monitor - Quit request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4828, selfPID=4828, iMonCtr=2
20:21:30 (1808): No heartbeat from core client for 30 sec - exiting
20:21:31 (1808): No heartbeat from core client for 30 sec - exiting
20:21:32 (1808): No heartbeat from core client for 30 sec - exiting
20:21:33 (1808): No heartbeat from core client for 30 sec - exiting
20:21:34 (1808): No heartbeat from core client for 30 sec - exiting
20:21:35 (1808): No heartbeat from core client for 30 sec - exiting
20:21:36 (1808): No heartbeat from core client for 30 sec - exiting
20:21:38 (1808): No heartbeat from core client for 30 sec - exiting
20:21:39 (1808): No heartbeat from core client for 30 sec - exiting
20:21:40 (1808): No heartbeat from core client for 30 sec - exiting
20:21:41 (1808): No heartbeat from core client for 30 sec - exiting
20:21:42 (1808): No heartbeat from core client for 30 sec - exiting
20:21:43 (1808): No heartbeat from core client for 30 sec - exiting
20:21:44 (1808): No heartbeat from core client for 30 sec - exiting
20:21:45 (1808): No heartbeat from core client for 30 sec - exiting
20:21:46 (1808): No heartbeat from core client for 30 sec - exiting
20:21:47 (1808): No heartbeat from core client for 30 sec - exiting
20:21:49 (1808): No heartbeat from core client for 30 sec - exiting
20:21:50 (1808): No heartbeat from core client for 30 sec - exiting
20:21:51 (1808): No heartbeat from core client for 30 sec - exiting
20:21:52 (1808): No heartbeat from core client for 30 sec - exiting
20:21:53 (1808): No heartbeat from core client for 30 sec - exiting
20:21:54 (1808): No heartbeat from core client for 30 sec - exiting
20:21:55 (1808): No heartbeat from core client for 30 sec - exiting
20:21:56 (1808): No heartbeat from core client for 30 sec - exiting
20:21:57 (1808): No heartbeat from core client for 30 sec - exiting
20:21:58 (1808): No heartbeat from core client for 30 sec - exiting
20:21:59 (1808): No heartbeat from core client for 30 sec - exiting
20:22:01 (1808): No heartbeat from core client for 30 sec - exiting
20:22:02 (1808): No heartbeat from core client for 30 sec - exiting
20:22:03 (1808): No heartbeat from core client for 30 sec - exiting
20:22:04 (1808): No heartbeat from core client for 30 sec - exiting
20:22:05 (1808): No heartbeat from core client for 30 sec - exiting
20:22:06 (1808): No heartbeat from core client for 30 sec - exiting
20:22:07 (1808): No heartbeat from core client for 30 sec - exiting
20:22:08 (1808): No heartbeat from core client for 30 sec - exiting
20:22:09 (1808): No heartbeat from core client for 30 sec - exiting
20:22:10 (1808): No heartbeat from core client for 30 sec - exiting
20:22:11 (1808): No heartbeat from core client for 30 sec - exiting
20:22:12 (1808): No heartbeat from core client for 30 sec - exiting
20:22:13 (1808): No heartbeat from core client for 30 sec - exiting
20:22:14 (1808): No heartbeat from core client for 30 sec - exiting
20:22:15 (1808): No heartbeat from core client for 30 sec - exiting
20:22:16 (1808): No heartbeat from core client for 30 sec - exiting
20:22:17 (1808): No heartbeat from core client for 30 sec - exiting
20:22:18 (1808): No heartbeat from core client for 30 sec - exiting
20:22:19 (1808): No heartbeat from core client for 30 sec - exiting
20:22:20 (1808): No heartbeat from core client for 30 sec - exiting
20:22:21 (1808): No heartbeat from core client for 30 sec - exiting
20:22:22 (1808): No heartbeat from core client for 30 sec - exiting
20:22:23 (1808): No heartbeat from core client for 30 sec - exiting
20:22:24 (1808): No heartbeat from core client for 30 sec - exiting
20:22:25 (1808): No heartbeat from core client for 30 sec - exiting
20:22:26 (1808): No heartbeat from core client for 30 sec - exiting
20:23:02 (1808): No heartbeat from core client for 30 sec - exiting
20:23:03 (1808): No heartbeat from core client for 30 sec - exiting
20:23:04 (1808): No heartbeat from core client for 30 sec - exiting
20:23:05 (1808): No heartbeat from core client for 30 sec - exiting
20:23:06 (1808): No heartbeat from core client for 30 sec - exiting
20:23:07 (1808): No heartbeat from core client for 30 sec - exiting
20:23:08 (1808): No heartbeat from core client for 30 sec - exiting
20:23:09 (1808): No heartbeat from core client for 30 sec - exiting
20:23:10 (1808): No heartbeat from core client for 30 sec - exiting
20:23:11 (1808): No heartbeat from core client for 30 sec - exiting
20:23:12 (1808): No heartbeat from core client for 30 sec - exiting
20:23:13 (1808): No heartbeat from core client for 30 sec - exiting
20:23:14 (1808): No heartbeat from core client for 30 sec - exiting
20:23:15 (1808): No heartbeat from core client for 30 sec - exiting
20:23:16 (1808): No heartbeat from core client for 30 sec - exiting
20:23:17 (1808): No heartbeat from core client for 30 sec - exiting
20:23:18 (1808): No heartbeat from core client for 30 sec - exiting
20:23:19 (1808): No heartbeat from core client for 30 sec - exiting
20:23:20 (1808): No heartbeat from core client for 30 sec - exiting
20:23:21 (1808): No heartbeat from core client for 30 sec - exiting
20:23:22 (1808): No heartbeat from core client for 30 sec - exiting
20:23:23 (1808): No heartbeat from core client for 30 sec - exiting
20:23:24 (1808): No heartbeat from core client for 30 sec - exiting
20:23:25 (1808): No heartbeat from core client for 30 sec - exiting
20:23:26 (1808): No heartbeat from core client for 30 sec - exiting
20:23:27 (1808): No heartbeat from core client for 30 sec - exiting
20:23:28 (1808): No heartbeat from core client for 30 sec - exiting
20:23:29 (1808): No heartbeat from core client for 30 sec - exiting
20:23:30 (1808): No heartbeat from core client for 30 sec - exiting
20:23:31 (1808): No heartbeat from core client for 30 sec - exiting
20:23:32 (1808): No heartbeat from core client for 30 sec - exiting
20:23:33 (1808): No heartbeat from core client for 30 sec - exiting
20:23:34 (1808): No heartbeat from core client for 30 sec - exiting
20:23:35 (1808): No heartbeat from core client for 30 sec - exiting
20:23:36 (1808): No heartbeat from core client for 30 sec - exiting
20:23:37 (1808): No heartbeat from core client for 30 sec - exiting
20:23:38 (1808): No heartbeat from core client for 30 sec - exiting
20:23:39 (1808): No heartbeat from core client for 30 sec - exiting
20:23:40 (1808): No heartbeat from core client for 30 sec - exiting
20:23:41 (1808): No heartbeat from core client for 30 sec - exiting
20:23:42 (1808): No heartbeat from core client for 30 sec - exiting
20:23:43 (1808): No heartbeat from core client for 30 sec - exiting
20:23:44 (1808): No heartbeat from core client for 30 sec - exiting
20:23:45 (1808): No heartbeat from core client for 30 sec - exiting
20:23:46 (1808): No heartbeat from core client for 30 sec - exiting
20:23:47 (1808): No heartbeat from core client for 30 sec - exiting
20:23:48 (1808): No heartbeat from core client for 30 sec - exiting
20:23:49 (1808): No heartbeat from core client for 30 sec - exiting
20:23:50 (1808): No heartbeat from core client for 30 sec - exiting
20:23:51 (1808): No heartbeat from core client for 30 sec - exiting
20:23:52 (1808): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
GClntroller:: oPDN process is not running,i exiting, bRetVal = 1, checkPID=0, selfPID=4420, iMonCtr=21
Model cnCtr=2r
ash detected, will try to restart...
08:53:42 (5704): No heartbeat from core client for 30 sec - exiting
08:53:44 (5704): No heartbeat from core client for 30 sec - exiting
08:53:45 (5704): No heartbeat from core client for 30 sec - exiting
08:53:46 (5704): No heartbeat from core client for 30 sec - exiting
08:53:47 (5704): No heartbeat from core client for 30 sec - exiting
08:53:48 (5704): No heartbeat from core client for 30 sec - exiting
08:53:49 (5704): No heartbeat from core client for 30 sec - exiting
08:53:50 (5704): No heartbeat from core client for 30 sec - exiting
08:53:51 (5704): No heartbeat from core client for 30 sec - exiting
08:53:52 (5704): No heartbeat from core client for 30 sec - exiting
08:53:53 (5704): No heartbeat from core client for 30 sec - exiting
08:53:54 (5704): No heartbeat from core client for 30 sec - exiting
08:53:55 (5704): No heartbeat from core client for 30 sec - exiting
08:53:56 (5704): No heartbeat from core client for 30 sec - exiting
08:53:57 (5704): No heartbeat from core client for 30 sec - exiting
08:53:58 (5704): No heartbeat from core client for 30 sec - exiting
08:54:00 (5704): No heartbeat from core client for 30 sec - exiting
08:54:01 (5704): No heartbeat from core client for 30 sec - exiting
08:54:02 (5704): No heartbeat from core client for 30 sec - exiting
08:54:03 (5704): No heartbeat from core client for 30 sec - exiting
08:54:04 (5704): No heartbeat from core client for 30 sec - exiting
08:54:05 (5704): No heartbeat from core client for 30 sec - exiting
08:54:06 (5704): No heartbeat from core client for 30 sec - exiting
08:54:07 (5704): No heartbeat from core client for 30 sec - exiting
08:54:08 (5704): No heartbeat from core client for 30 sec - exiting
08:54:09 (5704): No heartbeat from core client for 30 sec - exiting
08:54:10 (5704): No heartbeat from core client for 30 sec - exiting
08:54:12 (5704): No heartbeat from core client for 30 sec - exiting
08:54:13 (5704): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5956, selfPID=5956, iMonCtr=2
CPDN Monitor - Quit request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4816, selfPID=4816, iMonCtr=2
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
10:53:26 (5300): No heartbeat from core client for 30 sec - exiting
10:53:28 (5300): No heartbeat from core client for 30 sec - exiting
10:53:29 (5300): No heartbeat from core client for 30 sec - exiting
10:53:30 (5300): No heartbeat from core client for 30 sec - exiting
10:53:31 (5300): No heartbeat from core client for 30 sec - exiting
10:53:32 (5300): No heartbeat from core client for 30 sec - exiting
10:53:33 (5300): No heartbeat from core client for 30 sec - exiting
10:53:34 (5300): No heartbeat from core client for 30 sec - exiting
10:53:35 (5300): No heartbeat from core client for 30 sec - exiting
10:53:36 (5300): No heartbeat from core client for 30 sec - exiting
10:53:37 (5300): No heartbeat from core client for 30 sec - exiting
10:53:38 (5300): No heartbeat from core client for 30 sec - exiting
10:53:39 (5300): No heartbeat from core client for 30 sec - exiting
10:53:40 (5300): No heartbeat from core client for 30 sec - exiting
10:53:41 (5300): No heartbeat from core client for 30 sec - exiting
10:53:42 (5300): No heartbeat from core client for 30 sec - exiting
10:53:43 (5300): No heartbeat from core client for 30 sec - exiting
10:53:44 (5300): No heartbeat from core client for 30 sec - exiting
10:53:45 (5300): No heartbeat from core client for 30 sec - exiting
10:53:46 (5300): No heartbeat from core client for 30 sec - exiting
10:53:47 (5300): No heartbeat from core client for 30 sec - exiting
10:53:48 (5300): No heartbeat from core client for 30 sec - exiting
10:53:49 (5300): No heartbeat from core client for 30 sec - exiting
10:53:50 (5300): No heartbeat from core client for 30 sec - exiting
10:53:51 (5300): No heartbeat from core client for 30 sec - exiting
10:53:52 (5300): No heartbeat from core client for 30 sec - exiting
10:53:53 (5300): No heartbeat from core client for 30 sec - exiting
10:53:54 (5300): No heartbeat from core client for 30 sec - exiting
10:53:55 (5300): No heartbeat from core client for 30 sec - exiting
10:53:56 (5300): No heartbeat from core client for 30 sec - exiting
10:53:57 (5300): No heartbeat from core client for 30 sec - exiting
10:53:58 (5300): No heartbeat from core client for 30 sec - exiting
10:53:59 (5300): No heartbeat from core client for 30 sec - exiting
10:54:01 (5300): No heartbeat from core client for 30 sec - exiting
10:54:02 (5300): No heartbeat from core client for 30 sec - exiting
10:54:03 (5300): No heartbeat from core client for 30 sec - exiting
10:54:04 (5300): No heartbeat from core client for 30 sec - exiting
10:54:05 (5300): No heartbeat from core client for 30 sec - exiting
10:54:06 (5300): No heartbeat from core client for 30 sec - exiting
10:54:07 (5300): No heartbeat from core client for 30 sec - exiting
10:54:08 (5300): No heartbeat from core client for 30 sec - exiting
10:54:09 (5300): No heartbeat from core client for 30 sec - exiting
10:54:10 (5300): No heartbeat from core client for 30 sec - exiting
10:54:11 (5300): No heartbeat from core client for 30 sec - exiting
10:54:13 (5300): No heartbeat from core client for 30 sec - exiting
10:54:14 (5300): No heartbeat from core client for 30 sec - exiting
10:54:15 (5300): No heartbeat from core client for 30 sec - exiting
10:54:16 (5300): No heartbeat from core client for 30 sec - exiting
10:54:17 (5300): No heartbeat from core client for 30 sec - exiting
10:54:18 (5300): No heartbeat from core client for 30 sec - exiting
10:54:19 (5300): No heartbeat from core client for 30 sec - exiting
10:54:20 (5300): No heartbeat from core client for 30 sec - exiting
10:54:21 (5300): No heartbeat from core client for 30 sec - exiting
10:54:22 (5300): No heartbeat from core client for 30 sec - exiting
10:54:23 (5300): No heartbeat from core client for 30 sec - exiting
10:54:24 (5300): No heartbeat from core client for 30 sec - exiting
10:54:25 (5300): No heartbeat from core client for 30 sec - exiting
10:54:26 (5300): No heartbeat from core client for 30 sec - exiting
10:54:27 (5300): No heartbeat from core client for 30 sec - exiting
10:54:29 (5300): No heartbeat from core client for 30 sec - exiting
10:54:30 (5300): No heartbeat from core client for 30 sec - exiting
10:54:31 (5300): No heartbeat from core client for 30 sec - exiting
10:54:32 (5300): No heartbeat from core client for 30 sec - exiting
10:54:33 (5300): No heartbeat from core client for 30 sec - exiting
10:54:34 (5300): No heartbeat from core client for 30 sec - exiting
10:54:35 (5300): No heartbeat from core client for 30 sec - exiting
10:54:36 (5300): No heartbeat from core client for 30 sec - exiting
10:54:37 (5300): No heartbeat from core client for 30 sec - exiting
10:54:38 (5300): No heartbeat from core client for 30 sec - exiting
10:54:39 (5300): No heartbeat from core client for 30 sec - exiting
10:54:40 (5300): No heartbeat from core client for 30 sec - exiting
10:54:41 (5300): No heartbeat from core client for 30 sec - exiting
10:54:42 (5300): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6056, selfPID=6056, iMonCtr=2
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
07:35:03 (4668): No heartbeat from core client for 30 sec - exiting
07:35:05 (4668): No heartbeat from core client for 30 sec - exiting
07:35:06 (4668): No heartbeat from core client for 30 sec - exiting
07:35:07 (4668): No heartbeat from core client for 30 sec - exiting
07:35:08 (4668): No heartbeat from core client for 30 sec - exiting
07:35:09 (4668): No heartbeat from core client for 30 sec - exiting
07:35:10 (4668): No heartbeat from core client for 30 sec - exiting
07:35:11 (4668): No heartbeat from core client for 30 sec - exiting
07:35:12 (4668): No heartbeat from core client for 30 sec - exiting
07:35:13 (4668): No heartbeat from core client for 30 sec - exiting
07:35:14 (4668): No heartbeat from core client for 30 sec - exiting
07:35:15 (4668): No heartbeat from core client for 30 sec - exiting
07:35:17 (4668): No heartbeat from core client for 30 sec - exiting
07:35:18 (4668): No heartbeat from core client for 30 sec - exiting
07:35:19 (4668): No heartbeat from core client for 30 sec - exiting
07:35:20 (4668): No heartbeat from core client for 30 sec - exiting
07:35:21 (4668): No heartbeat from core client for 30 sec - exiting
07:35:22 (4668): No heartbeat from core client for 30 sec - exiting
07:35:23 (4668): No heartbeat from core client for 30 sec - exiting
07:35:24 (4668): No heartbeat from core client for 30 sec - exiting
07:35:25 (4668): No heartbeat from core client for 30 sec - exiting
07:35:26 (4668): No heartbeat from core client for 30 sec - exiting
07:35:27 (4668): No heartbeat from core client for 30 sec - exiting
07:35:29 (4668): No heartbeat from core client for 30 sec - exiting
07:35:30 (4668): No heartbeat from core client for 30 sec - exiting
07:35:31 (4668): No heartbeat from core client for 30 sec - exiting
07:35:32 (4668): No heartbeat from core client for 30 sec - exiting
07:35:33 (4668): No heartbeat from core client for 30 sec - exiting
07:35:34 (4668): No heartbeat from core client for 30 sec - exiting
07:35:35 (4668): No heartbeat from core client for 30 sec - exiting
07:35:36 (4668): No heartbeat from core client for 30 sec - exiting
07:35:37 (4668): No heartbeat from core client for 30 sec - exiting
07:35:38 (4668): No heartbeat from core client for 30 sec - exiting
07:35:39 (4668): No heartbeat from core client for 30 sec - exiting
07:35:40 (4668): No heartbeat from core client for 30 sec - exiting
07:35:41 (4668): No heartbeat from core client for 30 sec - exiting
07:35:42 (4668): No heartbeat from core client for 30 sec - exiting
07:35:43 (4668): No heartbeat from core client for 30 sec - exiting
07:35:44 (4668): No heartbeat from core client for 30 sec - exiting
07:35:45 (4668): No heartbeat from core client for 30 sec - exiting
07:35:46 (4668): No heartbeat from core client for 30 sec - exiting
07:35:47 (4668): No heartbeat from core client for 30 sec - exiting
07:35:48 (4668): No heartbeat from core client for 30 sec - exiting
07:35:49 (4668): No heartbeat from core client for 30 sec - exiting
07:35:50 (4668): No heartbeat from core client for 30 sec - exiting
07:35:51 (4668): No heartbeat from core client for 30 sec - exiting
07:35:52 (4668): No heartbeat from core client for 30 sec - exiting
07:35:53 (4668): No heartbeat from core client for 30 sec - exiting
07:35:54 (4668): No heartbeat from core client for 30 sec - exiting
07:35:55 (4668): No heartbeat from core client for 30 sec - exiting
07:35:56 (4668): No heartbeat from core client for 30 sec - exiting
07:35:57 (4668): No heartbeat from core client for 30 sec - exiting
07:35:58 (4668): No heartbeat from core client for 30 sec - exiting
07:35:59 (4668): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7596, selfPID=6300, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
08:14:25 (5116): No heartbeat from core client for 30 sec - exiting
08:14:27 (5116): No heartbeat from core client for 30 sec - exiting
08:14:28 (5116): No heartbeat from core client for 30 sec - exiting
08:14:29 (5116): No heartbeat from core client for 30 sec - exiting
08:14:30 (5116): No heartbeat from core client for 30 sec - exiting
08:14:31 (5116): No heartbeat from core client for 30 sec - exiting
08:14:32 (5116): No heartbeat from core client for 30 sec - exiting
08:14:33 (5116): No heartbeat from core client for 30 sec - exiting
08:14:34 (5116): No heartbeat from core client for 30 sec - exiting
08:14:35 (5116): No heartbeat from core client for 30 sec - exiting
08:14:36 (5116): No heartbeat from core client for 30 sec - exiting
08:14:37 (5116): No heartbeat from core client for 30 sec - exiting
08:14:38 (5116): No heartbeat from core client for 30 sec - exiting
08:14:39 (5116): No heartbeat from core client for 30 sec - exiting
08:14:40 (5116): No heartbeat from core client for 30 sec - exiting
08:14:41 (5116): No heartbeat from core client for 30 sec - exiting
08:14:42 (5116): No heartbeat from core client for 30 sec - exiting
08:14:43 (5116): No heartbeat from core client for 30 sec - exiting
08:14:44 (5116): No heartbeat from core client for 30 sec - exiting
08:14:45 (5116): No heartbeat from core client for 30 sec - exiting
08:14:46 (5116): No heartbeat from core client for 30 sec - exiting
08:14:47 (5116): No heartbeat from core client for 30 sec - exiting
08:14:48 (5116): No heartbeat from core client for 30 sec - exiting
08:14:49 (5116): No heartbeat from core client for 30 sec - exiting
08:14:50 (5116): No heartbeat from core client for 30 sec - exiting
08:14:51 (5116): No heartbeat from core client for 30 sec - exiting
08:14:52 (5116): No heartbeat from core client for 30 sec - exiting
08:14:53 (5116): No heartbeat from core client for 30 sec - exiting
08:14:54 (5116): No heartbeat from core client for 30 sec - exiting
08:14:55 (5116): No heartbeat from core client for 30 sec - exiting
08:14:56 (5116): No heartbeat from core client for 30 sec - exiting
08:14:57 (5116): No heartbeat from core client for 30 sec - exiting
08:14:58 (5116): No heartbeat from core client for 30 sec - exiting
08:14:59 (5116): No heartbeat from core client for 30 sec - exiting
08:15:00 (5116): No heartbeat from core client for 30 sec - exiting
08:15:01 (5116): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
14:25:42 (7104): No heartbeat from core client for 30 sec - exiting
14:25:43 (7104): No heartbeat from core client for 30 sec - exiting
14:25:44 (7104): No heartbeat from core client for 30 sec - exiting
14:25:45 (7104): No heartbeat from core client for 30 sec - exiting
14:25:46 (7104): No heartbeat from core client for 30 sec - exiting
14:25:48 (7104): No heartbeat from core client for 30 sec - exiting
14:25:49 (7104): No heartbeat from core client for 30 sec - exiting
14:25:50 (7104): No heartbeat from core client for 30 sec - exiting
14:25:51 (7104): No heartbeat from core client for 30 sec - exiting
14:25:52 (7104): No heartbeat from core client for 30 sec - exiting
14:25:53 (7104): No heartbeat from core client for 30 sec - exiting
14:25:54 (7104): No heartbeat from core client for 30 sec - exiting
14:25:55 (7104): No heartbeat from core client for 30 sec - exiting
14:25:56 (7104): No heartbeat from core client for 30 sec - exiting
14:25:57 (7104): No heartbeat from core client for 30 sec - exiting
14:25:58 (7104): No heartbeat from core client for 30 sec - exiting
14:26:00 (7104): No heartbeat from core client for 30 sec - exiting
14:26:01 (7104): No heartbeat from core client for 30 sec - exiting
14:26:02 (7104): No heartbeat from core client for 30 sec - exiting
14:26:03 (7104): No heartbeat from core client for 30 sec - exiting
14:26:04 (7104): No heartbeat from core client for 30 sec - exiting
14:26:05 (7104): No heartbeat from core client for 30 sec - exiting
14:26:06 (7104): No heartbeat from core client for 30 sec - exiting
14:26:07 (7104): No heartbeat from core client for 30 sec - exiting
14:26:08 (7104): No heartbeat from core client for 30 sec - exiting
14:26:09 (7104): No heartbeat from core client for 30 sec - exiting
14:26:11 (7104): No heartbeat from core client for 30 sec - exiting
14:26:12 (7104): No heartbeat from core client for 30 sec - exiting
14:26:13 (7104): No heartbeat from core client for 30 sec - exiting
14:26:14 (7104): No heartbeat from core client for 30 sec - exiting
14:26:15 (7104): No heartbeat from core client for 30 sec - exiting
14:26:16 (7104): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8828, iMonCtr=2
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2592, iMonCtr=2
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7436, selfPID=6584, iMonCtr=1
Model crash detected, will try to restart...
08:18:32 (6004): No heartbeat from core client for 30 sec - exiting
08:18:33 (6004): No heartbeat from core client for 30 sec - exiting
08:18:34 (6004): No heartbeat from core client for 30 sec - exiting
08:18:35 (6004): No heartbeat from core client for 30 sec - exiting
08:18:36 (6004): No heartbeat from core client for 30 sec - exiting
08:18:37 (6004): No heartbeat from core client for 30 sec - exiting
08:18:38 (6004): No heartbeat from core client for 30 sec - exiting
08:18:39 (6004): No heartbeat from core client for 30 sec - exiting
08:18:41 (6004): No heartbeat from core client for 30 sec - exiting
08:18:42 (6004): No heartbeat from core client for 30 sec - exiting
08:18:43 (6004): No heartbeat from core client for 30 sec - exiting
08:18:44 (6004): No heartbeat from core client for 30 sec - exiting
08:18:45 (6004): No heartbeat from core client for 30 sec - exiting
08:18:46 (6004): No heartbeat from core client for 30 sec - exiting
08:18:47 (6004): No heartbeat from core client for 30 sec - exiting
08:18:48 (6004): No heartbeat from core client for 30 sec - exiting
08:18:49 (6004): No heartbeat from core client for 30 sec - exiting
08:18:50 (6004): No heartbeat from core client for 30 sec - exiting
08:18:51 (6004): No heartbeat from core client for 30 sec - exiting
08:18:53 (6004): No heartbeat from core client for 30 sec - exiting
08:18:54 (6004): No heartbeat from core client for 30 sec - exiting
08:18:55 (6004): No heartbeat from core client for 30 sec - exiting
08:18:56 (6004): No heartbeat from core client for 30 sec - exiting
08:18:57 (6004): No heartbeat from core client for 30 sec - exiting
08:18:58 (6004): No heartbeat from core client for 30 sec - exiting
08:18:59 (6004): No heartbeat from core client for 30 sec - exiting
08:19:00 (6004): No heartbeat from core client for 30 sec - exiting
08:19:01 (6004): No heartbeat from core client for 30 sec - exiting
08:19:02 (6004): No heartbeat from core client for 30 sec - exiting
08:19:03 (6004): No heartbeat from core client for 30 sec - exiting
08:19:05 (6004): No heartbeat from core client for 30 sec - exiting
08:19:06 (6004): No heartbeat from core client for 30 sec - exiting
08:19:07 (6004): No heartbeat from core client for 30 sec - exiting
08:19:08 (6004): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1252, selfPID=3264, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
13:27:35 (6284): No heartbeat from core client for 30 sec - exiting
13:27:36 (6284): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7248, iMonCtr=2
Model crash detected, will try to restart...
22:07:27 (9488): No heartbeat from core client for 30 sec - exiting
22:07:28 (9488): No heartbeat from core client for 30 sec - exiting
22:07:30 (9488): No heartbeat from core client for 30 sec - exiting
22:07:31 (9488): No heartbeat from core client for 30 sec - exiting
22:07:32 (9488): No heartbeat from core client for 30 sec - exiting
22:07:33 (9488): No heartbeat from core client for 30 sec - exiting
22:07:34 (9488): No heartbeat from core client for 30 sec - exiting
22:07:35 (9488): No heartbeat from core client for 30 sec - exiting
22:07:36 (9488): No heartbeat from core client for 30 sec - exiting
22:07:37 (9488): No heartbeat from core client for 30 sec - exiting
22:07:38 (9488): No heartbeat from core client for 30 sec - exiting
22:07:39 (9488): No heartbeat from core client for 30 sec - exiting
22:07:41 (9488): No heartbeat from core client for 30 sec - exiting
22:07:42 (9488): No heartbeat from core client for 30 sec - exiting
22:07:43 (9488): No heartbeat from core client for 30 sec - exiting
22:07:44 (9488): No heartbeat from core client for 30 sec - exiting
22:07:45 (9488): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9784, selfPID=8260, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5360, iMonCtr=2
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6024, selfPID=5508, iMonCtr=1
Model crash detected, will try to restart...
Leaving CPDN_Main::Monitor...
Regional yearly means requires 12 input files got 9
07:35:25 (5904): No heartbeat from core client for 30 sec - exiting
07:35:26 (5904): No heartbeat from core client for 30 sec - exiting
07:35:27 (5904): No heartbeat from core client for 30 sec - exiting
07:35:28 (5904): No heartbeat from core client for 30 sec - exiting
07:35:30 (5904): No heartbeat from core client for 30 sec - exiting
07:35:31 (5904): No heartbeat from core client for 30 sec - exiting
07:35:32 (5904): No heartbeat from core client for 30 sec - exiting
07:35:33 (5904): No heartbeat from core client for 30 sec - exiting
07:35:34 (5904): No heartbeat from core client for 30 sec - exiting
07:35:35 (5904): No heartbeat from core client for 30 sec - exiting
07:35:36 (5904): No heartbeat from core client for 30 sec - exiting
07:35:37 (5904): No heartbeat from core client for 30 sec - exiting
07:35:38 (5904): No heartbeat from core client for 30 sec - exiting
07:35:39 (5904): No heartbeat from core client for 30 sec - exiting
07:35:40 (5904): No heartbeat from core client for 30 sec - exiting
07:35:41 (5904): No heartbeat from core client for 30 sec - exiting
07:35:42 (5904): No heartbeat from core client for 30 sec - exiting
07:35:43 (5904): No heartbeat from core client for 30 sec - exiting
07:35:44 (5904): No heartbeat from core client for 30 sec - exiting
07:35:45 (5904): No heartbeat from core client for 30 sec - exiting
07:35:46 (5904): No heartbeat from core client for 30 sec - exiting
07:35:47 (5904): No heartbeat from core client for 30 sec - exiting
07:35:48 (5904): No heartbeat from core client for 30 sec - exiting
07:35:49 (5904): No heartbeat from core client for 30 sec - exiting
07:35:50 (5904): No heartbeat from core client for 30 sec - exiting
07:35:51 (5904): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
08:08:44 (5760): No heartbeat from core client for 30 sec - exiting
08:08:46 (5760): No heartbeat from core client for 30 sec - exiting
08:08:47 (5760): No heartbeat from core client for 30 sec - exiting
08:08:48 (5760): No heartbeat from core client for 30 sec - exiting
08:08:49 (5760): No heartbeat from core client for 30 sec - exiting
08:08:50 (5760): No heartbeat from core client for 30 sec - exiting
08:08:51 (5760): No heartbeat from core client for 30 sec - exiting
08:08:52 (5760): No heartbeat from core client for 30 sec - exiting
08:08:53 (5760): No heartbeat from core client for 30 sec - exiting
08:08:54 (5760): No heartbeat from core client for 30 sec - exiting
08:08:55 (5760): No heartbeat from core client for 30 sec - exiting
08:08:56 (5760): No heartbeat from core client for 30 sec - exiting
08:08:58 (5760): No heartbeat from core client for 30 sec - exiting
08:08:59 (5760): No heartbeat from core client for 30 sec - exiting
08:09:00 (5760): No heartbeat from core client for 30 sec - exiting
08:09:01 (5760): No heartbeat from core client for 30 sec - exiting
08:09:02 (5760): No heartbeat from core client for 30 sec - exiting
08:09:04 (5760): No heartbeat from core client for 30 sec - exiting
08:09:05 (5760): No heartbeat from core client for 30 sec - exiting
08:09:06 (5760): No heartbeat from core client for 30 sec - exiting
08:09:07 (5760): No heartbeat from core client for 30 sec - exiting
08:09:08 (5760): No heartbeat from core client for 30 sec - exiting
08:09:09 (5760): No heartbeat from core client for 30 sec - exiting
08:09:10 (5760): No heartbeat from core client for 30 sec - exiting
08:09:11 (5760): No heartbeat from core client for 30 sec - exiting
08:09:12 (5760): No heartbeat from core client for 30 sec - exiting
08:09:13 (5760): No heartbeat from core client for 30 sec - exiting
08:09:14 (5760): No heartbeat from core client for 30 sec - exiting
08:09:16 (5760): No heartbeat from core client for 30 sec - exiting
08:09:17 (5760): No heartbeat from core client for 30 sec - exiting
08:09:18 (5760): No heartbeat from core client for 30 sec - exiting
08:09:19 (5760): No heartbeat from core client for 30 sec - exiting
08:09:20 (5760): No heartbeat from core client for 30 sec - exiting
08:09:21 (5760): No heartbeat from core client for 30 sec - exiting
08:09:22 (5760): No heartbeat from core client for 30 sec - exiting
08:09:23 (5760): No heartbeat from core client for 30 sec - exiting
08:09:24 (5760): No heartbeat from core client for 30 sec - exiting
08:09:25 (5760): No heartbeat from core client for 30 sec - exiting
08:09:26 (5760): No heartbeat from core client for 30 sec - exiting
08:09:27 (5760): No heartbeat from core client for 30 sec - exiting
08:09:28 (5760): No heartbeat from core client for 30 sec - exiting
08:09:29 (5760): No heartbeat from core client for 30 sec - exiting
08:09:30 (5760): No heartbeat from core client for 30 sec - exiting
08:09:31 (5760): No heartbeat from core client for 30 sec - exiting
08:09:32 (5760): No heartbeat from core client for 30 sec - exiting
08:09:33 (5760): No heartbeat from core client for 30 sec - exiting
08:09:34 (5760): No heartbeat from core client for 30 sec - exiting
08:09:35 (5760): No heartbeat from core client for 30 sec - exiting
08:09:36 (5760): No heartbeat from core client for 30 sec - exiting
08:09:37 (5760): No heartbeat from core client for 30 sec - exiting
08:09:38 (5760): No heartbeat from core client for 30 sec - exiting
08:09:39 (5760): No heartbeat from core client for 30 sec - exiting
08:09:40 (5760): No heartbeat from core client for 30 sec - exiting
08:09:41 (5760): No heartbeat from core client for 30 sec - exiting
08:09:42 (5760): No heartbeat from core client for 30 sec - exiting
08:09:43 (5760): No heartbeat from core client for 30 sec - exiting
08:09:44 (5760): No heartbeat from core client for 30 sec - exiting
08:09:45 (5760): No heartbeat from core client for 30 sec - exiting
08:09:46 (5760): No heartbeat from core client for 30 sec - exiting
08:09:47 (5760): No heartbeat from core client for 30 sec - exiting
08:09:48 (5760): No heartbeat from core client for 30 sec - exiting
08:09:49 (5760): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
16:10:26 (7052): No heartbeat from core client for 30 sec - exiting
16:10:27 (7052): No heartbeat from core client for 30 sec - exiting
16:10:28 (7052): No heartbeat from core client for 30 sec - exiting
16:10:29 (7052): No heartbeat from core client for 30 sec - exiting
16:10:30 (7052): No heartbeat from core client for 30 sec - exiting
16:10:31 (7052): No heartbeat from core client for 30 sec - exiting
16:10:32 (7052): No heartbeat from core client for 30 sec - exiting
16:10:33 (7052): No heartbeat from core client for 30 sec - exiting
16:10:34 (7052): No heartbeat from core client for 30 sec - exiting
16:10:35 (7052): No heartbeat from core client for 30 sec - exiting
16:10:37 (7052): No heartbeat from core client for 30 sec - exiting
16:10:38 (7052): No heartbeat from core client for 30 sec - exiting
16:10:39 (7052): No heartbeat from core client for 30 sec - exiting
16:10:40 (7052): No heartbeat from core client for 30 sec - exiting
16:10:41 (7052): No heartbeat from core client for 30 sec - exiting
16:10:42 (7052): No heartbeat from core client for 30 sec - exiting
16:10:43 (7052): No heartbeat from core client for 30 sec - exiting
16:10:44 (7052): No heartbeat from core client for 30 sec - exiting
16:10:45 (7052): No heartbeat from core client for 30 sec - exiting
16:10:46 (7052): No heartbeat from core client for 30 sec - exiting
16:10:48 (7052): No heartbeat from core client for 30 sec - exiting
16:10:49 (7052): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
07:36:43 (5724): No heartbeat from core client for 30 sec - exiting
07:36:44 (5724): No heartbeat from core client for 30 sec - exiting
07:36:46 (5724): No heartbeat from core client for 30 sec - exiting
07:36:47 (5724): No heartbeat from core client for 30 sec - exiting
07:36:48 (5724): No heartbeat from core client for 30 sec - exiting
07:36:49 (5724): No heartbeat from core client for 30 sec - exiting
07:36:50 (5724): No heartbeat from core client for 30 sec - exiting
07:36:51 (5724): No heartbeat from core client for 30 sec - exiting
07:36:52 (5724): No heartbeat from core client for 30 sec - exiting
07:36:53 (5724): No heartbeat from core client for 30 sec - exiting
07:36:54 (5724): No heartbeat from core client for 30 sec - exiting
07:36:55 (5724): No heartbeat from core client for 30 sec - exiting
07:36:56 (5724): No heartbeat from core client for 30 sec - exiting
07:36:58 (5724): No heartbeat from core client for 30 sec - exiting
07:36:59 (5724): No heartbeat from core client for 30 sec - exiting
07:37:00 (5724): No heartbeat from core client for 30 sec - exiting
07:37:01 (5724): No heartbeat from core client for 30 sec - exiting
07:37:02 (5724): No heartbeat from core client for 30 sec - exiting
07:37:03 (5724): No heartbeat from core client for 30 sec - exiting
07:37:04 (5724): No heartbeat from core client for 30 sec - exiting
07:37:05 (5724): No heartbeat from core client for 30 sec - exiting
07:37:06 (5724): No heartbeat from core client for 30 sec - exiting
07:37:07 (5724): No heartbeat from core client for 30 sec - exiting
07:37:08 (5724): No heartbeat from core client for 30 sec - exiting
07:37:09 (5724): No heartbeat from core client for 30 sec - exiting
07:37:10 (5724): No heartbeat from core client for 30 sec - exiting
07:37:11 (5724): No heartbeat from core client for 30 sec - exiting
07:37:12 (5724): No heartbeat from core client for 30 sec - exiting
07:37:13 (5724): No heartbeat from core client for 30 sec - exiting
07:37:14 (5724): No heartbeat from core client for 30 sec - exiting
07:37:15 (5724): No heartbeat from core client for 30 sec - exiting
07:37:16 (5724): No heartbeat from core client for 30 sec - exiting
07:37:17 (5724): No heartbeat from core client for 30 sec - exiting
07:37:18 (5724): No heartbeat from core client for 30 sec - exiting
07:37:19 (5724): No heartbeat from core client for 30 sec - exiting
07:37:20 (5724): No heartbeat from core client for 30 sec - exiting
07:37:21 (5724): No heartbeat from core client for 30 sec - exiting
07:37:22 (5724): No heartbeat from core client for 30 sec - exiting
07:37:23 (5724): No heartbeat from core client for 30 sec - exiting
07:37:24 (5724): No heartbeat from core client for 30 sec - exiting
07:37:25 (5724): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
12:29:00 (2424): No heartbeat from core client for 30 sec - exiting
12:29:01 (2424): No heartbeat from core client for 30 sec - exiting
12:29:02 (2424): No heartbeat from core client for 30 sec - exiting
12:29:03 (2424): No heartbeat from core client for 30 sec - exiting
12:29:04 (2424): No heartbeat from core client for 30 sec - exiting
12:29:05 (2424): No heartbeat from core client for 30 sec - exiting
12:29:06 (2424): No heartbeat from core client for 30 sec - exiting
12:29:07 (2424): No heartbeat from core client for 30 sec - exiting
12:29:08 (2424): No heartbeat from core client for 30 sec - exiting
12:29:09 (2424): No heartbeat from core client for 30 sec - exiting
12:29:10 (2424): No heartbeat from core client for 30 sec - exiting
12:29:11 (2424): No heartbeat from core client for 30 sec - exiting
12:29:12 (2424): No heartbeat from core client for 30 sec - exiting
12:29:13 (2424): No heartbeat from core client for 30 sec - exiting
12:29:14 (2424): No heartbeat from core client for 30 sec - exiting
12:29:15 (2424): No heartbeat from core client for 30 sec - exiting
12:29:16 (2424): No heartbeat from core client for 30 sec - exiting
12:29:17 (2424): No heartbeat from core client for 30 sec - exiting
12:29:18 (2424): No heartbeat from core client for 30 sec - exiting
12:29:19 (2424): No heartbeat from core client for 30 sec - exiting
12:29:20 (2424): No heartbeat from core client for 30 sec - exiting
12:29:21 (2424): No heartbeat from core client for 30 sec - exiting
12:29:22 (2424): No heartbeat from core client for 30 sec - exiting
12:29:24 (2424): No heartbeat from core client for 30 sec - exiting
12:29:25 (2424): No heartbeat from core client for 30 sec - exiting
12:29:26 (2424): No heartbeat from core client for 30 sec - exiting
12:29:27 (2424): No heartbeat from core client for 30 sec - exiting
12:29:28 (2424): No heartbeat from core client for 30 sec - exiting
12:29:29 (2424): No heartbeat from core client for 30 sec - exiting
12:29:30 (2424): No heartbeat from core client for 30 sec - exiting
12:29:31 (2424): No heartbeat from core client for 30 sec - exiting
12:29:32 (2424): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
20:14:31 (2072): No heartbeat from core client for 30 sec - exiting
20:14:33 (2072): No heartbeat from core client for 30 sec - exiting
20:14:34 (2072): No heartbeat from core client for 30 sec - exiting
20:14:35 (2072): No heartbeat from core client for 30 sec - exiting
20:14:36 (2072): No heartbeat from core client for 30 sec - exiting
20:14:37 (2072): No heartbeat from core client for 30 sec - exiting
20:14:38 (2072): No heartbeat from core client for 30 sec - exiting
20:14:39 (2072): No heartbeat from core client for 30 sec - exiting
20:14:40 (2072): No heartbeat from core client for 30 sec - exiting
20:14:41 (2072): No heartbeat from core client for 30 sec - exiting
20:14:42 (2072): No heartbeat from core client for 30 sec - exiting
20:14:43 (2072): No heartbeat from core client for 30 sec - exiting
20:14:45 (2072): No heartbeat from core client for 30 sec - exiting
20:14:46 (2072): No heartbeat from core client for 30 sec - exiting
20:14:47 (2072): No heartbeat from core client for 30 sec - exiting
20:14:48 (2072): No heartbeat from core client for 30 sec - exiting
20:14:49 (2072): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Leaving CPDN_Main::Monitor...
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
31 Jul 2012 19:09:31 1220942 14917181 hadam3p_pnw_bctl_1977_1_008039095_2 138,343 449,553 3.2496
31 Jul 2012 18:12:27 1220942 14917181 hadam3p_pnw_bctl_1977_1_008039095_2 138,336 449,135 3.2467
30 Jul 2012 17:19:05 1220942 14917181 hadam3p_pnw_bctl_1977_1_008039095_2 126,816 412,648 3.2539
29 Jul 2012 12:50:11 1220942 14917181 hadam3p_pnw_bctl_1977_1_008039095_2 115,296 377,039 3.2702
27 Jul 2012 18:08:25 1220942 14917181 hadam3p_pnw_bctl_1977_1_008039095_2 103,776 338,868 3.2654
26 Jul 2012 19:51:46 1220942 14917181 hadam3p_pnw_bctl_1977_1_008039095_2 92,256 301,539 3.2685
25 Jul 2012 18:24:39 1220942 14917181 hadam3p_pnw_bctl_1977_1_008039095_2 80,736 264,085 3.2710
25 Jul 2012 08:22:08 1220942 14917181 hadam3p_pnw_bctl_1977_1_008039095_2 69,216 227,567 3.2878
24 Jul 2012 08:23:54 1220942 14917181 hadam3p_pnw_bctl_1977_1_008039095_2 57,696 189,221 3.2796
22 Jul 2012 16:59:47 1220942 14917181 hadam3p_pnw_bctl_1977_1_008039095_2 46,176 152,481 3.3022
20 Jul 2012 20:13:06 1220942 14917181 hadam3p_pnw_bctl_1977_1_008039095_2 34,656 113,427 3.2729
19 Jul 2012 17:38:59 1220942 14917181 hadam3p_pnw_bctl_1977_1_008039095_2 23,136 76,399 3.3022
18 Jul 2012 11:34:41 1220942 14917181 hadam3p_pnw_bctl_1977_1_008039095_2 11,621 39,343 3.3855
18 Jul 2012 09:34:16 1220942 14917181 hadam3p_pnw_bctl_1977_1_008039095_2 11,616 38,891 3.3481


©2024 climateprediction.net