climateprediction.net home page
Task 15237132

Task 15237132

Name hadam3p_pnw_6vug_2008_1_008185551_0
Workunit 8340675
Created 5 Sep 2012, 1:37:21 UTC
Sent 5 Sep 2012, 16:31:17 UTC
Report deadline 18 Aug 2013, 21:51:17 UTC
Received 24 Sep 2012, 10:29:34 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1143449
Run time 3 days 0 hours 5 min 57 sec
CPU time 2 days 19 hours 5 min 55 sec
Validate state Invalid
Credit 1,503.98
Device peak FLOPS 2.97 GFLOPS
Application version UK Met Office HadAM3P-HadRM3P Pacific North West v6.09
windows_intelx86
Stderr
<core_client_version>7.0.28</core_client_version>
<![CDATA[
<stderr_txt>
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5616, selfPID=4004, iMonCtr=1
Model crash detected, will try to restart...
12:25:25 (3584): No heartbeat from core client for 30 sec - exiting
12:25:26 (3584): No heartbeat from core client for 30 sec - exiting
12:25:27 (3584): No heartbeat from core client for 30 sec - exiting
12:25:28 (3584): No heartbeat from core client for 30 sec - exiting
12:25:29 (3584): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
12:09:06 (6304): No heartbeat from core client for 30 sec - exiting
12:09:07 (6304): No heartbeat from core client for 30 sec - exiting
12:09:08 (6304): No heartbeat from core client for 30 sec - exiting
12:09:09 (6304): No heartbeat from core client for 30 sec - exiting
12:09:10 (6304): No heartbeat from core client for 30 sec - exiting
12:09:11 (6304): No heartbeat from core client for 30 sec - exiting
12:09:12 (6304): No heartbeat from core client for 30 sec - exiting
12:09:13 (6304): No heartbeat from core client for 30 sec - exiting
12:09:14 (6304): No heartbeat from core client for 30 sec - exiting
12:09:15 (6304): No heartbeat from core client for 30 sec - exiting
12:09:16 (6304): No heartbeat from core client for 30 sec - exiting
12:09:17 (6304): No heartbeat from core client for 30 sec - exiting
12:09:18 (6304): No heartbeat from core client for 30 sec - exiting
12:09:19 (6304): No heartbeat from core client for 30 sec - exiting
12:09:20 (6304): No heartbeat from core client for 30 sec - exiting
12:09:21 (6304): No heartbeat from core client for 30 sec - exiting
12:09:22 (6304): No heartbeat from core client for 30 sec - exiting
12:09:23 (6304): No heartbeat from core client for 30 sec - exiting
12:09:24 (6304): No heartbeat from core client for 30 sec - exiting
12:09:25 (6304): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6880, iMonCtr=2
Model crash detected, will try to restart...
15:17:35 (5876): No heartbeat from core client for 30 sec - exiting
15:17:36 (5876): No heartbeat from core client for 30 sec - exiting
15:17:37 (5876): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7392, selfPID=7288, iMonCtr=1
Model crash detected, will try to restart...
11:38:16 (5800): No heartbeat from core client for 30 sec - exiting
11:38:17 (5800): No heartbeat from core client for 30 sec - exiting
11:38:18 (5800): No heartbeat from core client for 30 sec - exiting
11:38:19 (5800): No heartbeat from core client for 30 sec - exiting
11:38:20 (5800): No heartbeat from core client for 30 sec - exiting
11:38:21 (5800): No heartbeat from core client for 30 sec - exiting
11:38:22 (5800): No heartbeat from core client for 30 sec - exiting
11:38:23 (5800): No heartbeat from core client for 30 sec - exiting
11:38:24 (5800): No heartbeat from core client for 30 sec - exiting
11:38:25 (5800): No heartbeat from core client for 30 sec - exiting
11:38:26 (5800): No heartbeat from core client for 30 sec - exiting
11:38:27 (5800): No heartbeat from core client for 30 sec - exiting
11:38:28 (5800): No heartbeat from core client for 30 sec - exiting
11:38:29 (5800): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7816, iMonCtr=2
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7748, iMonCtr=2
Model crash detected, will try to restart...
Leaving CPDN_Main::Monitor...
Regional yearly means requires 12 input files got 2
20:22:29 (1412): No heartbeat from core client for 30 sec - exiting
20:22:30 (1412): No heartbeat from core client for 30 sec - exiting
20:22:31 (1412): No heartbeat from core client for 30 sec - exiting
20:22:32 (1412): No heartbeat from core client for 30 sec - exiting
20:22:33 (1412): No heartbeat from core client for 30 sec - exiting
20:22:34 (1412): No heartbeat from core client for 30 sec - exiting
20:22:35 (1412): No heartbeat from core client for 30 sec - exiting
20:22:36 (1412): No heartbeat from core client for 30 sec - exiting
20:22:37 (1412): No heartbeat from core client for 30 sec - exiting
20:22:38 (1412): No heartbeat from core client for 30 sec - exiting
20:22:39 (1412): No heartbeat from core client for 30 sec - exiting
20:22:40 (1412): No heartbeat from core client for 30 sec - exiting
20:22:41 (1412): No heartbeat from core client for 30 sec - exiting
20:22:42 (1412): No heartbeat from core client for 30 sec - exiting
20:22:43 (1412): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:05:09 (5724): No heartbeat from core client for 30 sec - exiting
17:05:10 (5724): No heartbeat from core client for 30 sec - exiting
17:05:11 (5724): No heartbeat from core client for 30 sec - exiting
17:05:12 (5724): No heartbeat from core client for 30 sec - exiting
17:05:13 (5724): No heartbeat from core client for 30 sec - exiting
17:05:14 (5724): No heartbeat from core client for 30 sec - exiting
17:05:15 (5724): No heartbeat from core client for 30 sec - exiting
17:05:16 (5724): No heartbeat from core client for 30 sec - exiting
17:05:17 (5724): No heartbeat from core client for 30 sec - exiting
17:05:18 (5724): No heartbeat from core client for 30 sec - exiting
17:05:19 (5724): No heartbeat from core client for 30 sec - exiting
17:05:20 (5724): No heartbeat from core client for 30 sec - exiting
17:05:21 (5724): No heartbeat from core client for 30 sec - exiting
17:05:22 (5724): No heartbeat from core client for 30 sec - exiting
17:05:23 (5724): No heartbeat from core client for 30 sec - exiting
17:05:24 (5724): No heartbeat from core client for 30 sec - exiting
17:05:25 (5724): No heartbeat from core client for 30 sec - exiting
17:05:26 (5724): No heartbeat from core client for 30 sec - exiting
17:05:27 (5724): No heartbeat from core client for 30 sec - exiting
17:05:28 (5724): No heartbeat from core client for 30 sec - exiting
17:05:29 (5724): No heartbeat from core client for 30 sec - exiting
17:05:30 (5724): No heartbeat from core client for 30 sec - exiting
17:05:31 (5724): No heartbeat from core client for 30 sec - exiting
17:05:32 (5724): No heartbeat from core client for 30 sec - exiting
17:05:33 (5724): No heartbeat from core client for 30 sec - exiting
17:05:34 (5724): No heartbeat from core client for 30 sec - exiting
17:05:35 (5724): No heartbeat from core client for 30 sec - exiting
17:05:36 (5724): No heartbeat from core client for 30 sec - exiting
17:05:37 (5724): No heartbeat from core client for 30 sec - exiting
17:05:38 (5724): No heartbeat from core client for 30 sec - exiting
17:05:39 (5724): No heartbeat from core client for 30 sec - exiting
17:05:40 (5724): No heartbeat from core client for 30 sec - exiting
17:05:41 (5724): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3264, iMonCtr=2
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4208, iMonCtr=2
Model crash detected, will try to restart...
Leaving CPDN_Main::Monitor...
Regional yearly means requires 12 input files got 2
16:45:07 (5312): No heartbeat from core client for 30 sec - exiting
16:45:08 (5312): No heartbeat from core client for 30 sec - exiting
16:45:09 (5312): No heartbeat from core client for 30 sec - exiting
16:45:10 (5312): No heartbeat from core client for 30 sec - exiting
16:45:11 (5312): No heartbeat from core client for 30 sec - exiting
16:45:12 (5312): No heartbeat from core client for 30 sec - exiting
16:45:13 (5312): No heartbeat from core client for 30 sec - exiting
16:45:14 (5312): No heartbeat from core client for 30 sec - exiting
16:45:15 (5312): No heartbeat from core client for 30 sec - exiting
16:45:16 (5312): No heartbeat from core client for 30 sec - exiting
16:45:17 (5312): No heartbeat from core client for 30 sec - exiting
16:45:18 (5312): No heartbeat from core client for 30 sec - exiting
16:45:19 (5312): No heartbeat from core client for 30 sec - exiting
16:45:20 (5312): No heartbeat from core client for 30 sec - exiting
16:45:21 (5312): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3788, selfPID=7324, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4460, selfPID=2700, iMonCtr=1
Model crash detected, will try to restart...
Leaving CPDN_Main::Monitor...
Regional yearly means requires 12 input files got 4
20:42:37 (5812): No heartbeat from core client for 30 sec - exiting
20:42:38 (5812): No heartbeat from core client for 30 sec - exiting
20:42:39 (5812): No heartbeat from core client for 30 sec - exiting
20:42:40 (5812): No heartbeat from core client for 30 sec - exiting
20:42:41 (5812): No heartbeat from core client for 30 sec - exiting
20:42:42 (5812): No heartbeat from core client for 30 sec - exiting
20:42:43 (5812): No heartbeat from core client for 30 sec - exiting
20:42:44 (5812): No heartbeat from core client for 30 sec - exiting
20:42:45 (5812): No heartbeat from core client for 30 sec - exiting
20:42:46 (5812): No heartbeat from core client for 30 sec - exiting
20:42:47 (5812): No heartbeat from core client for 30 sec - exiting
20:42:48 (5812): No heartbeat from core client for 30 sec - exiting
20:42:49 (5812): No heartbeat from core client for 30 sec - exiting
20:42:50 (5812): No heartbeat from core client for 30 sec - exiting
20:42:51 (5812): No heartbeat from core client for 30 sec - exiting
20:42:52 (5812): No heartbeat from core client for 30 sec - exiting
20:42:53 (5812): No heartbeat from core client for 30 sec - exiting
20:42:54 (5812): No heartbeat from core client for 30 sec - exiting
20:42:55 (5812): No heartbeat from core client for 30 sec - exiting
20:42:56 (5812): No heartbeat from core client for 30 sec - exiting
20:42:57 (5812): No heartbeat from core client for 30 sec - exiting
20:42:58 (5812): No heartbeat from core client for 30 sec - exiting
20:42:59 (5812): No heartbeat from core client for 30 sec - exiting
20:43:00 (5812): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5196, iMonCtr=2
Model crash detected, will try to restart...
Leaving CPDN_Main::Monitor...
Regional yearly means requires 12 input files got 4
22:01:53 (4796): No heartbeat from core client for 30 sec - exiting
22:01:55 (4796): No heartbeat from core client for 30 sec - exiting
22:01:56 (4796): No heartbeat from core client for 30 sec - exiting
22:01:57 (4796): No heartbeat from core client for 30 sec - exiting
22:01:58 (4796): No heartbeat from core client for 30 sec - exiting
22:01:59 (4796): No heartbeat from core client for 30 sec - exiting
22:02:00 (4796): No heartbeat from core client for 30 sec - exiting
22:02:01 (4796): No heartbeat from core client for 30 sec - exiting
22:02:02 (4796): No heartbeat from core client for 30 sec - exiting
22:02:03 (4796): No heartbeat from core client for 30 sec - exiting
22:02:05 (4796): No heartbeat from core client for 30 sec - exiting
22:02:06 (4796): No heartbeat from core client for 30 sec - exiting
22:02:07 (4796): No heartbeat from core client for 30 sec - exiting
22:02:08 (4796): No heartbeat from core client for 30 sec - exiting
22:02:09 (4796): No heartbeat from core client for 30 sec - exiting
22:02:10 (4796): No heartbeat from core client for 30 sec - exiting
22:02:11 (4796): No heartbeat from core client for 30 sec - exiting
22:02:12 (4796): No heartbeat from core client for 30 sec - exiting
22:02:13 (4796): No heartbeat from core client for 30 sec - exiting
22:02:14 (4796): No heartbeat from core client for 30 sec - exiting
22:02:15 (4796): No heartbeat from core client for 30 sec - exiting
22:02:16 (4796): No heartbeat from core client for 30 sec - exiting
22:02:17 (4796): No heartbeat from core client for 30 sec - exiting
22:02:18 (4796): No heartbeat from core client for 30 sec - exiting
22:02:19 (4796): No heartbeat from core client for 30 sec - exiting
22:02:20 (4796): No heartbeat from core client for 30 sec - exiting
22:02:21 (4796): No heartbeat from core client for 30 sec - exiting
22:02:22 (4796): No heartbeat from core client for 30 sec - exiting
22:02:23 (4796): No heartbeat from core client for 30 sec - exiting
22:02:24 (4796): No heartbeat from core client for 30 sec - exiting
22:02:25 (4796): No heartbeat from core client for 30 sec - exiting
22:02:26 (4796): No heartbeat from core client for 30 sec - exiting
22:02:27 (4796): No heartbeat from core client for 30 sec - exiting
22:02:28 (4796): No heartbeat from core client for 30 sec - exiting
22:02:29 (4796): No heartbeat from core client for 30 sec - exiting
22:02:30 (4796): No heartbeat from core client for 30 sec - exiting
22:02:31 (4796): No heartbeat from core client for 30 sec - exiting
22:02:32 (4796): No heartbeat from core client for 30 sec - exiting
22:02:33 (4796): No heartbeat from core client for 30 sec - exiting
22:02:34 (4796): No heartbeat from core client for 30 sec - exiting
22:02:35 (4796): No heartbeat from core client for 30 sec - exiting
22:02:36 (4796): No heartbeat from core client for 30 sec - exiting
22:02:37 (4796): No heartbeat from core client for 30 sec - exiting
22:02:38 (4796): No heartbeat from core client for 30 sec - exiting
22:02:39 (4796): No heartbeat from core client for 30 sec - exiting
22:02:40 (4796): No heartbeat from core client for 30 sec - exiting
22:02:41 (4796): No heartbeat from core client for 30 sec - exiting
22:02:42 (4796): No heartbeat from core client for 30 sec - exiting
22:02:43 (4796): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6616, iMonCtr=2
21:07:07 (5272): No heartbeat from core client for 30 sec - exiting
21:07:08 (5272): No heartbeat from core client for 30 sec - exiting
21:07:09 (5272): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7536, selfPID=7480, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4976, selfPID=4128, iMonCtr=1
Model crash detected, will try to restart...
Leaving CPDN_Main::Monitor...
Regional yearly means requires 12 input files got 5
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7676, selfPID=7072, iMonCtr=1
Model crash detected, will try to restart...
Leaving CPDN_Main::Monitor...
Regional yearly means requires 12 input files got 6
forrtl: severe (24): end-of-file during read, unit 9, file D:\ProgramData\BOINC\projects\climateprediction.net\hadam3p_pnw_6vug_2008_1_008185551\tmp\xaakg.namelists

Image              PC        Routine            Line        Source             
hadrm3p_pnw_um_6.  00C4C52A  Unknown               Unknown  Unknown
hadrm3p_pnw_um_6.  00BF4460  Unknown               Unknown  Unknown
hadrm3p_pnw_um_6.  00BF362A  Unknown               Unknown  Unknown
hadrm3p_pnw_um_6.  00BD2469  Unknown               Unknown  Unknown
hadrm3p_pnw_um_6.  00AD66EB  Unknown               Unknown  Unknown
hadrm3p_pnw_um_6.  00B72AE2  Unknown               Unknown  Unknown
hadrm3p_pnw_um_6.  00B735AF  Unknown               Unknown  Unknown
hadrm3p_pnw_um_6.  00919860  Unknown               Unknown  Unknown
hadrm3p_pnw_um_6.  00C30893  Unknown               Unknown  Unknown
kernel32.dll       7789ED6C  Unknown               Unknown  Unknown
ntdll.dll          77CC377B  Unknown               Unknown  Unknown
ntdll.dll          77CC374E  Unknown               Unknown  Unknown
forrtl: severe (24): end-of-file during read, unit 9, file D:\ProgramData\BOINC\projects\climateprediction.net\hadam3p_pnw_6vug_2008_1_008185551\tmp\xaakm.namelists

Image              PC        Routine            Line        Source             
hadam3p_pnw_um_6.  00D6A39A  Unknown               Unknown  Unknown
hadam3p_pnw_um_6.  00D12CD0  Unknown               Unknown  Unknown
hadam3p_pnw_um_6.  00D11E9A  Unknown               Unknown  Unknown
hadam3p_pnw_um_6.  00CF2819  Unknown               Unknown  Unknown
hadam3p_pnw_um_6.  00BF2287  Unknown               Unknown  Unknown
hadam3p_pnw_um_6.  00C8E7B2  Unknown               Unknown  Unknown
hadam3p_pnw_um_6.  00C8F2DA  Unknown               Unknown  Unknown
hadam3p_pnw_um_6.  00A09BD2  Unknown               Unknown  Unknown
hadam3p_pnw_um_6.  00D4E638  Unknown               Unknown  Unknown
kernel32.dll       7789ED6C  Unknown               Unknown  Unknown
ntdll.dll          77CC377B  Unknown               Unknown  Unknown
ntdll.dll          77CC374E  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7908, selfPID=740, iMonCtr=1
Model crash detected, will try to restart...
Leaving CPDN_Main::Monitor...
Regional yearly means requires 12 input files got 6
Called boinc_finish

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>hadam3p_pnw_6vug_2008_1_008185551_0_7.zip</file_name>
  <error_code>-161</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>hadam3p_pnw_6vug_2008_1_008185551_0_8.zip</file_name>
  <error_code>-161</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>hadam3p_pnw_6vug_2008_1_008185551_0_9.zip</file_name>
  <error_code>-161</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>hadam3p_pnw_6vug_2008_1_008185551_0_10.zip</file_name>
  <error_code>-161</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>hadam3p_pnw_6vug_2008_1_008185551_0_11.zip</file_name>
  <error_code>-161</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>hadam3p_pnw_6vug_2008_1_008185551_0_12.zip</file_name>
  <error_code>-161</error_code>
</file_xfer_error>

</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
23 Sep 2012 13:52:17 1143449 15237132 hadam3p_pnw_6vug_2008_1_008185551_0 69,216 213,741 3.0880
21 Sep 2012 12:53:45 1143449 15237132 hadam3p_pnw_6vug_2008_1_008185551_0 57,696 178,199 3.0886
16 Sep 2012 21:42:17 1143449 15237132 hadam3p_pnw_6vug_2008_1_008185551_0 46,176 142,915 3.0950
14 Sep 2012 16:42:38 1143449 15237132 hadam3p_pnw_6vug_2008_1_008185551_0 34,656 107,601 3.1048
09 Sep 2012 19:26:17 1143449 15237132 hadam3p_pnw_6vug_2008_1_008185551_0 23,136 71,901 3.1078
07 Sep 2012 15:46:20 1143449 15237132 hadam3p_pnw_6vug_2008_1_008185551_0 11,616 36,104 3.1081


©2024 climateprediction.net