climateprediction.net home page
Task 12933147

Task 12933147

Name hadcm3n_o3rs_1940_40_007266496_2
Workunit 7464736
Created 4 Jun 2011, 10:27:32 UTC
Sent 4 Jun 2011, 10:27:34 UTC
Report deadline 3 Sep 2011, 17:54:45 UTC
Received 18 Aug 2011, 0:32:22 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 957259
Run time 58 days 18 hours 34 min 2 sec
CPU time 55 days 21 hours 57 min 36 sec
Validate state Invalid
Credit 12,441.60
Device peak FLOPS 1.26 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr
<core_client_version>6.10.18</core_client_version>
<![CDATA[
<message>
 - exit code 193 (0xc1)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
17:53:18 (3160): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:53:19 (3160): No heartbeat from core client for 30 sec - exiting
17:53:20 (3160): No heartbeat from core client for 30 sec - exiting
17:53:21 (3160): No heartbeat from core client for 30 sec - exiting
17:53:22 (3160): No heartbeat from core client for 30 sec - exiting
17:53:24 (3160): No heartbeat from core client for 30 sec - exiting
17:53:25 (3160): No heartbeat from core client for 30 sec - exiting
17:53:26 (3160): No heartbeat from core client for 30 sec - exiting
17:53:27 (3160): No heartbeat from core client for 30 sec - exiting
17:53:28 (3160): No heartbeat from core client for 30 sec - exiting
17:53:29 (3160): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
00:30:41 (4196): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:29:36 (5824): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:29:37 (5824): No heartbeat from core client for 30 sec - exiting
10:28:33 (1032): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
01:22:49 (4116): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:21:49 (1052): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:20:43 (3740): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:19:36 (5328): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:18:35 (4012): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:17:34 (1916): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:16:33 (4404): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
01:15:31 (864): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:14:31 (3728): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:13:26 (4148): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
01:40:50 (312): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:39:41 (4736): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:38:38 (6028): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:37:30 (4800): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:36:24 (5464): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:35:19 (1648): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:34:08 (6080): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:33:07 (2068): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:31:58 (5292): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
00:24:40 (3028): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:23:41 (4856): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:22:40 (3136): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
01:21:25 (4480): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:20:17 (5548): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
22:15:14 (4256): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:14:06 (5224): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:12:59 (3724): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:11:52 (6140): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:10:46 (3108): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:09:39 (2056): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:08:31 (5804): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:07:26 (5280): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:06:18 (4120): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:05:13 (5924): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:04:09 (2452): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:03:05 (4472): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:01:58 (3336): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:00:51 (4508): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:59:45 (5168): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:58:37 (5068): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
00:57:31 (5828): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:56:24 (4428): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:55:18 (5288): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:54:18 (2856): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:53:10 (2212): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:52:03 (3480): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:50:56 (4064): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:49:50 (2936): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:48:42 (5804): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:47:35 (1256): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:46:35 (2192): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:45:32 (364): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x7C920CCE read attempt to address 0x00000000

Engaging BOINC Windows Runtime Debugger...

02:44:31 (3820): No heartbeat from core client for 30 sec - exiting
02:44:32 (3820): No heartbeat from core client for 30 sec - exiting
02:44:33 (3820): No heartbeat from core client for 30 sec - exiting
02:44:35 (3820): No heartbeat from core client for 30 sec - exiting
02:44:36 (3820): No heartbeat from core client for 30 sec - exiting
02:44:37 (3820): No heartbeat from core client for 30 sec - exiting
02:44:38 (3820): No heartbeat from core client for 30 sec - exiting
02:44:39 (3820): No heartbeat from core client for 30 sec - exiting
02:44:40 (3820): No heartbeat from core client for 30 sec - exiting
02:44:41 (3820): No heartbeat from core client for 30 sec - exiting
02:44:42 (3820): No heartbeat from core client for 30 sec - exiting
02:44:43 (3820): No heartbeat from core client for 30 sec - exiting
02:44:44 (3820): No heartbeat from core client for 30 sec - exiting
02:44:45 (3820): No heartbeat from core client for 30 sec - exiting
02:44:47 (3820): No heartbeat from core client for 30 sec - exiting
02:44:48 (3820): No heartbeat from core client for 30 sec - exiting
02:44:49 (3820): No heartbeat from core client for 30 sec - exiting
02:44:50 (3820): No heartbeat from core client for 30 sec - exiting
02:44:51 (3820): No heartbeat from core client for 30 sec - exiting
02:44:52 (3820): No heartbeat from core client for 30 sec - exiting
02:44:53 (3820): No heartbeat from core client for 30 sec - exiting
02:44:54 (3820): No heartbeat from core client for 30 sec - exiting
02:44:55 (3820): No heartbeat from core client for 30 sec - exiting
02:44:56 (3820): No heartbeat from core client for 30 sec - exiting
02:44:58 (3820): No heartbeat from core client for 30 sec - exiting
02:44:59 (3820): No heartbeat from core client for 30 sec - exiting
02:45:00 (3820): No heartbeat from core client for 30 sec - exiting
09:43:25 (3820): No heartbeat from core client for 30 sec - exiting
09:43:26 (3820): No heartbeat from core client for 30 sec - exiting
09:43:27 (3820): No heartbeat from core client for 30 sec - exiting
09:43:28 (3820): No heartbeat from core client for 30 sec - exiting
09:43:29 (3820): No heartbeat from core client for 30 sec - exiting
09:43:30 (3820): No heartbeat from core client for 30 sec - exiting
09:43:32 (3820): No heartbeat from core client for 30 sec - exiting
09:43:33 (3820): No heartbeat from core client for 30 sec - exiting
09:43:34 (3820): No heartbeat from core client for 30 sec - exiting
09:43:35 (3820): No heartbeat from core client for 30 sec - exiting
09:43:36 (3820): No heartbeat from core client for 30 sec - exiting
09:43:37 (3820): No heartbeat from core client for 30 sec - exiting
09:43:38 (3820): No heartbeat from core client for 30 sec - exiting
09:43:39 (3820): No heartbeat from core client for 30 sec - exiting
09:43:40 (3820): No heartbeat from core client for 30 sec - exiting
09:43:41 (3820): No heartbeat from core client for 30 sec - exiting
09:43:42 (3820): No heartbeat from core client for 30 sec - exiting
09:43:44 (3820): No heartbeat from core client for 30 sec - exiting
09:43:45 (3820): No heartbeat from core client for 30 sec - exiting
09:43:46 (3820): No heartbeat from core client for 30 sec - exiting
09:43:47 (3820): No heartbeat from core client for 30 sec - exiting
09:43:48 (3820): No heartbeat from core client for 30 sec - exiting
09:43:49 (3820): No heartbeat from core client for 30 sec - exiting
09:43:50 (3820): No heartbeat from core client for 30 sec - exiting
09:43:51 (3820): No heartbeat from core client for 30 sec - exiting
09:43:52 (3820): No heartbeat from core client for 30 sec - exiting
09:43:53 (3820): No heartbeat from core client for 30 sec - exiting
09:43:55 (3820): No heartbeat from core client for 30 sec - exiting
09:43:56 (3820): No heartbeat from core client for 30 sec - exiting
09:43:57 (3820): No heartbeat from core client for 30 sec - exiting
09:43:58 (3820): No heartbeat from core client for 30 sec - exiting
09:43:59 (3820): No heartbeat from core client for 30 sec - exiting
16:42:21 (3820): No heartbeat from core client for 30 sec - exiting
16:42:22 (3820): No heartbeat from core client for 30 sec - exiting
16:42:23 (3820): No heartbeat from core client for 30 sec - exiting
16:42:24 (3820): No heartbeat from core client for 30 sec - exiting
16:42:25 (3820): No heartbeat from core client for 30 sec - exiting
16:42:26 (3820): No heartbeat from core client for 30 sec - exiting
16:42:27 (3820): No heartbeat from core client for 30 sec - exiting
16:42:28 (3820): No heartbeat from core client for 30 sec - exiting
16:42:29 (3820): No heartbeat from core client for 30 sec - exiting
16:42:30 (3820): No heartbeat from core client for 30 sec - exiting
16:42:31 (3820): No heartbeat from core client for 30 sec - exiting
16:42:33 (3820): No heartbeat from core client for 30 sec - exiting
16:42:34 (3820): No heartbeat from core client for 30 sec - exiting
16:42:35 (3820): No heartbeat from core client for 30 sec - exiting
16:42:36 (3820): No heartbeat from core client for 30 sec - exiting
16:42:37 (3820): No heartbeat from core client for 30 sec - exiting
16:42:38 (3820): No heartbeat from core client for 30 sec - exiting
16:42:39 (3820): No heartbeat from core client for 30 sec - exiting
16:42:40 (3820): No heartbeat from core client for 30 sec - exiting
16:42:41 (3820): No heartbeat from core client for 30 sec - exiting
16:42:42 (3820): No heartbeat from core client for 30 sec - exiting
16:42:44 (3820): No heartbeat from core client for 30 sec - exiting
16:42:45 (3820): No heartbeat from core client for 30 sec - exiting
16:42:46 (3820): No heartbeat from core client for 30 sec - exiting
16:42:47 (3820): No heartbeat from core client for 30 sec - exiting
16:42:48 (3820): No heartbeat from core client for 30 sec - exiting
16:42:49 (3820): No heartbeat from core client for 30 sec - exiting
16:42:50 (3820): No heartbeat from core client for 30 sec - exiting
16:42:51 (3820): No heartbeat from core client for 30 sec - exiting
16:42:52 (3820): No heartbeat from core client for 30 sec - exiting
23:41:13 (3820): No heartbeat from core client for 30 sec - exiting
23:41:14 (3820): No heartbeat from core client for 30 sec - exiting
23:41:15 (3820): No heartbeat from core client for 30 sec - exiting
23:41:16 (3820): No heartbeat from core client for 30 sec - exiting
23:41:17 (3820): No heartbeat from core client for 30 sec - exiting
23:41:19 (3820): No heartbeat from core client for 30 sec - exiting
23:41:20 (3820): No heartbeat from core client for 30 sec - exiting
23:41:21 (3820): No heartbeat from core client for 30 sec - exiting
23:41:22 (3820): No heartbeat from core client for 30 sec - exiting
23:41:23 (3820): No heartbeat from core client for 30 sec - exiting
23:41:24 (3820): No heartbeat from core client for 30 sec - exiting
23:41:25 (3820): No heartbeat from core client for 30 sec - exiting
23:41:26 (3820): No heartbeat from core client for 30 sec - exiting
23:41:27 (3820): No heartbeat from core client for 30 sec - exiting
23:41:28 (3820): No heartbeat from core client for 30 sec - exiting
23:41:29 (3820): No heartbeat from core client for 30 sec - exiting
23:41:31 (3820): No heartbeat from core client for 30 sec - exiting
23:41:32 (3820): No heartbeat from core client for 30 sec - exiting
23:41:33 (3820): No heartbeat from core client for 30 sec - exiting
23:41:34 (3820): No heartbeat from core client for 30 sec - exiting
23:41:35 (3820): No heartbeat from core client for 30 sec - exiting
23:41:36 (3820): No heartbeat from core client for 30 sec - exiting
23:41:37 (3820): No heartbeat from core client for 30 sec - exiting
23:41:38 (3820): No heartbeat from core client for 30 sec - exiting
23:41:39 (3820): No heartbeat from core client for 30 sec - exiting
23:41:40 (3820): No heartbeat from core client for 30 sec - exiting
23:41:41 (3820): No heartbeat from core client for 30 sec - exiting
23:41:43 (3820): No heartbeat from core client for 30 sec - exiting
23:41:44 (3820): No heartbeat from core client for 30 sec - exiting
23:41:45 (3820): No heartbeat from core client for 30 sec - exiting
23:41:46 (3820): No heartbeat from core client for 30 sec - exiting
23:41:47 (3820): No heartbeat from core client for 30 sec - exiting
06:40:08 (3820): No heartbeat from core client for 30 sec - exiting
06:40:09 (3820): No heartbeat from core client for 30 sec - exiting
06:40:10 (3820): No heartbeat from core client for 30 sec - exiting
06:40:11 (3820): No heartbeat from core client for 30 sec - exiting
06:40:13 (3820): No heartbeat from core client for 30 sec - exiting
06:40:14 (3820): No heartbeat from core client for 30 sec - exiting
06:40:15 (3820): No heartbeat from core client for 30 sec - exiting
06:40:16 (3820): No heartbeat from core client for 30 sec - exiting
06:40:17 (3820): No heartbeat from core client for 30 sec - exiting
06:40:18 (3820): No heartbeat from core client for 30 sec - exiting
06:40:19 (3820): No heartbeat from core client for 30 sec - exiting
06:40:20 (3820): No heartbeat from core client for 30 sec - exiting
06:40:21 (3820): No heartbeat from core client for 30 sec - exiting
06:40:22 (3820): No heartbeat from core client for 30 sec - exiting
06:40:23 (3820): No heartbeat from core client for 30 sec - exiting
06:40:25 (3820): No heartbeat from core client for 30 sec - exiting
06:40:26 (3820): No heartbeat from core client for 30 sec - exiting
06:40:27 (3820): No heartbeat from core client for 30 sec - exiting
06:40:28 (3820): No heartbeat from core client for 30 sec - exiting
06:40:29 (3820): No heartbeat from core client for 30 sec - exiting
06:40:30 (3820): No heartbeat from core client for 30 sec - exiting
06:40:31 (3820): No heartbeat from core client for 30 sec - exiting
06:40:32 (3820): No heartbeat from core client for 30 sec - exiting
06:40:33 (3820): No heartbeat from core client for 30 sec - exiting
06:40:35 (3820): No heartbeat from core client for 30 sec - exiting
06:40:36 (3820): No heartbeat from core client for 30 sec - exiting
06:40:37 (3820): No heartbeat from core client for 30 sec - exiting
06:40:38 (3820): No heartbeat from core client for 30 sec - exiting
06:40:39 (3820): No heartbeat from core client for 30 sec - exiting
06:40:40 (3820): No heartbeat from core client for 30 sec - exiting
06:40:41 (3820): No heartbeat from core client for 30 sec - exiting
06:40:42 (3820): No heartbeat from core client for 30 sec - exiting


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x7C921295 read attempt to address 0xFFFFFFF8

Engaging BOINC Windows Runtime Debugger...



Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x7C921295 read attempt to address 0xFFFFFFF8

Engaging BOINC Windows Runtime Debugger...



Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x7C921295 read attempt to address 0xFFFFFFF8

Engaging BOINC Windows Runtime Debugger...

02:22:07 (2116): No heartbeat from core client for 30 sec - exiting
02:22:08 (2116): No heartbeat from core client for 30 sec - exiting
02:22:09 (2116): No heartbeat from core client for 30 sec - exiting
02:22:10 (2116): No heartbeat from core client for 30 sec - exiting
02:22:11 (2116): No heartbeat from core client for 30 sec - exiting
02:22:13 (2116): No heartbeat from core client for 30 sec - exiting
02:22:14 (2116): No heartbeat from core client for 30 sec - exiting
02:22:15 (2116): No heartbeat from core client for 30 sec - exiting
02:22:16 (2116): No heartbeat from core client for 30 sec - exiting
02:22:17 (2116): No heartbeat from core client for 30 sec - exiting
02:22:18 (2116): No heartbeat from core client for 30 sec - exiting
02:22:19 (2116): No heartbeat from core client for 30 sec - exiting
02:22:20 (2116): No heartbeat from core client for 30 sec - exiting
02:22:21 (2116): No heartbeat from core client for 30 sec - exiting
02:22:22 (2116): No heartbeat from core client for 30 sec - exiting
02:22:24 (2116): No heartbeat from core client for 30 sec - exiting
02:22:25 (2116): No heartbeat from core client for 30 sec - exiting
02:22:26 (2116): No heartbeat from core client for 30 sec - exiting
02:22:27 (2116): No heartbeat from core client for 30 sec - exiting
02:22:28 (2116): No heartbeat from core client for 30 sec - exiting
02:22:29 (2116): No heartbeat from core client for 30 sec - exiting
02:22:30 (2116): No heartbeat from core client for 30 sec - exiting
02:22:31 (2116): No heartbeat from core client for 30 sec - exiting
02:22:32 (2116): No heartbeat from core client for 30 sec - exiting
02:22:33 (2116): No heartbeat from core client for 30 sec - exiting
02:22:34 (2116): No heartbeat from core client for 30 sec - exiting
02:22:36 (2116): No heartbeat from core client for 30 sec - exiting
02:22:37 (2116): No heartbeat from core client for 30 sec - exiting
02:22:38 (2116): No heartbeat from core client for 30 sec - exiting
02:22:39 (2116): No heartbeat from core client for 30 sec - exiting
08:21:03 (2116): No heartbeat from core client for 30 sec - exiting
08:21:04 (2116): No heartbeat from core client for 30 sec - exiting
08:21:05 (2116): No heartbeat from core client for 30 sec - exiting
08:21:06 (2116): No heartbeat from core client for 30 sec - exiting
08:21:08 (2116): No heartbeat from core client for 30 sec - exiting
08:21:09 (2116): No heartbeat from core client for 30 sec - exiting
08:21:10 (2116): No heartbeat from core client for 30 sec - exiting
08:21:11 (2116): No heartbeat from core client for 30 sec - exiting
08:21:12 (2116): No heartbeat from core client for 30 sec - exiting
08:21:13 (2116): No heartbeat from core client for 30 sec - exiting
08:21:14 (2116): No heartbeat from core client for 30 sec - exiting
08:21:15 (2116): No heartbeat from core client for 30 sec - exiting
08:21:16 (2116): No heartbeat from core client for 30 sec - exiting
08:21:17 (2116): No heartbeat from core client for 30 sec - exiting
08:21:19 (2116): No heartbeat from core client for 30 sec - exiting
08:21:20 (2116): No heartbeat from core client for 30 sec - exiting
08:21:21 (2116): No heartbeat from core client for 30 sec - exiting
08:21:22 (2116): No heartbeat from core client for 30 sec - exiting
08:21:23 (2116): No heartbeat from core client for 30 sec - exiting
08:21:24 (2116): No heartbeat from core client for 30 sec - exiting
08:21:25 (2116): No heartbeat from core client for 30 sec - exiting
08:21:26 (2116): No heartbeat from core client for 30 sec - exiting
08:21:27 (2116): No heartbeat from core client for 30 sec - exiting
08:21:28 (2116): No heartbeat from core client for 30 sec - exiting
08:21:29 (2116): No heartbeat from core client for 30 sec - exiting
08:21:31 (2116): No heartbeat from core client for 30 sec - exiting
08:21:32 (2116): No heartbeat from core client for 30 sec - exiting
08:21:33 (2116): No heartbeat from core client for 30 sec - exiting
08:21:34 (2116): No heartbeat from core client for 30 sec - exiting
14:20:00 (2116): No heartbeat from core client for 30 sec - exiting
14:20:01 (2116): No heartbeat from core client for 30 sec - exiting
14:20:02 (2116): No heartbeat from core client for 30 sec - exiting
14:20:03 (2116): No heartbeat from core client for 30 sec - exiting
14:20:04 (2116): No heartbeat from core client for 30 sec - exiting
14:20:05 (2116): No heartbeat from core client for 30 sec - exiting
14:20:07 (2116): No heartbeat from core client for 30 sec - exiting
14:20:08 (2116): No heartbeat from core client for 30 sec - exiting
14:20:09 (2116): No heartbeat from core client for 30 sec - exiting
14:20:10 (2116): No heartbeat from core client for 30 sec - exiting
14:20:11 (2116): No heartbeat from core client for 30 sec - exiting
14:20:12 (2116): No heartbeat from core client for 30 sec - exiting
14:20:13 (2116): No heartbeat from core client for 30 sec - exiting
14:20:14 (2116): No heartbeat from core client for 30 sec - exiting
14:20:15 (2116): No heartbeat from core client for 30 sec - exiting
14:20:16 (2116): No heartbeat from core client for 30 sec - exiting
14:20:18 (2116): No heartbeat from core client for 30 sec - exiting
14:20:19 (2116): No heartbeat from core client for 30 sec - exiting
14:20:20 (2116): No heartbeat from core client for 30 sec - exiting
14:20:21 (2116): No heartbeat from core client for 30 sec - exiting
14:20:22 (2116): No heartbeat from core client for 30 sec - exiting
14:20:23 (2116): No heartbeat from core client for 30 sec - exiting
14:20:24 (2116): No heartbeat from core client for 30 sec - exiting
14:20:25 (2116): No heartbeat from core client for 30 sec - exiting
14:20:26 (2116): No heartbeat from core client for 30 sec - exiting
14:20:27 (2116): No heartbeat from core client for 30 sec - exiting
14:20:28 (2116): No heartbeat from core client for 30 sec - exiting
14:20:30 (2116): No heartbeat from core client for 30 sec - exiting
14:20:31 (2116): No heartbeat from core client for 30 sec - exiting
14:20:32 (2116): No heartbeat from core client for 30 sec - exiting


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x7C921295 read attempt to address 0xFFFFFFF8

Engaging BOINC Windows Runtime Debugger...



Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x7C921295 read attempt to address 0xFFFFFFF8

Engaging BOINC Windows Runtime Debugger...

02:07:40 (3280): No heartbeat from core client for 30 sec - exiting
02:07:41 (3280): No heartbeat from core client for 30 sec - exiting
02:07:42 (3280): No heartbeat from core client for 30 sec - exiting
02:07:43 (3280): No heartbeat from core client for 30 sec - exiting
02:07:44 (3280): No heartbeat from core client for 30 sec - exiting
02:07:46 (3280): No heartbeat from core client for 30 sec - exiting
02:07:47 (3280): No heartbeat from core client for 30 sec - exiting
02:07:48 (3280): No heartbeat from core client for 30 sec - exiting
02:07:49 (3280): No heartbeat from core client for 30 sec - exiting
02:07:50 (3280): No heartbeat from core client for 30 sec - exiting
02:07:51 (3280): No heartbeat from core client for 30 sec - exiting
02:07:52 (3280): No heartbeat from core client for 30 sec - exiting
02:07:53 (3280): No heartbeat from core client for 30 sec - exiting
02:07:54 (3280): No heartbeat from core client for 30 sec - exiting
02:07:55 (3280): No heartbeat from core client for 30 sec - exiting
02:07:57 (3280): No heartbeat from core client for 30 sec - exiting
02:07:58 (3280): No heartbeat from core client for 30 sec - exiting
02:07:59 (3280): No heartbeat from core client for 30 sec - exiting
02:08:00 (3280): No heartbeat from core client for 30 sec - exiting
02:08:01 (3280): No heartbeat from core client for 30 sec - exiting
02:08:02 (3280): No heartbeat from core client for 30 sec - exiting
02:08:03 (3280): No heartbeat from core client for 30 sec - exiting
02:08:04 (3280): No heartbeat from core client for 30 sec - exiting
02:08:05 (3280): No heartbeat from core client for 30 sec - exiting
02:08:06 (3280): No heartbeat from core client for 30 sec - exiting
02:08:07 (3280): No heartbeat from core client for 30 sec - exiting
02:08:09 (3280): No heartbeat from core client for 30 sec - exiting
02:08:10 (3280): No heartbeat from core client for 30 sec - exiting
02:08:11 (3280): No heartbeat from core client for 30 sec - exiting
02:08:12 (3280): No heartbeat from core client for 30 sec - exiting
02:08:13 (3280): No heartbeat from core client for 30 sec - exiting
02:08:14 (3280): No heartbeat from core client for 30 sec - exiting
02:08:15 (3280): No heartbeat from core client for 30 sec - exiting
02:08:16 (3280): No heartbeat from core client for 30 sec - exiting
07:06:39 (3280): No heartbeat from core client for 30 sec - exiting
07:06:40 (3280): No heartbeat from core client for 30 sec - exiting
07:06:41 (3280): No heartbeat from core client for 30 sec - exiting
07:06:42 (3280): No heartbeat from core client for 30 sec - exiting
07:06:43 (3280): No heartbeat from core client for 30 sec - exiting
07:06:44 (3280): No heartbeat from core client for 30 sec - exiting
07:06:45 (3280): No heartbeat from core client for 30 sec - exiting
07:06:47 (3280): No heartbeat from core client for 30 sec - exiting
07:06:48 (3280): No heartbeat from core client for 30 sec - exiting
07:06:49 (3280): No heartbeat from core client for 30 sec - exiting
07:06:50 (3280): No heartbeat from core client for 30 sec - exiting
07:06:51 (3280): No heartbeat from core client for 30 sec - exiting
07:06:52 (3280): No heartbeat from core client for 30 sec - exiting
07:06:53 (3280): No heartbeat from core client for 30 sec - exiting
07:06:54 (3280): No heartbeat from core client for 30 sec - exiting
07:06:55 (3280): No heartbeat from core client for 30 sec - exiting
07:06:56 (3280): No heartbeat from core client for 30 sec - exiting
07:06:57 (3280): No heartbeat from core client for 30 sec - exiting
07:06:59 (3280): No heartbeat from core client for 30 sec - exiting
07:07:00 (3280): No heartbeat from core client for 30 sec - exiting
07:07:01 (3280): No heartbeat from core client for 30 sec - exiting
07:07:02 (3280): No heartbeat from core client for 30 sec - exiting
07:07:03 (3280): No heartbeat from core client for 30 sec - exiting
07:07:04 (3280): No heartbeat from core client for 30 sec - exiting
07:07:05 (3280): No heartbeat from core client for 30 sec - exiting
07:07:06 (3280): No heartbeat from core client for 30 sec - exiting
12:05:39 (3280): No heartbeat from core client for 30 sec - exiting
12:05:40 (3280): No heartbeat from core client for 30 sec - exiting
12:05:41 (3280): No heartbeat from core client for 30 sec - exiting
12:05:43 (3280): No heartbeat from core client for 30 sec - exiting
12:05:44 (3280): No heartbeat from core client for 30 sec - exiting
12:05:45 (3280): No heartbeat from core client for 30 sec - exiting
12:05:46 (3280): No heartbeat from core client for 30 sec - exiting
12:05:47 (3280): No heartbeat from core client for 30 sec - exiting
12:05:48 (3280): No heartbeat from core client for 30 sec - exiting
12:05:49 (3280): No heartbeat from core client for 30 sec - exiting
12:05:50 (3280): No heartbeat from core client for 30 sec - exiting
12:05:51 (3280): No heartbeat from core client for 30 sec - exiting
12:05:52 (3280): No heartbeat from core client for 30 sec - exiting
12:05:53 (3280): No heartbeat from core client for 30 sec - exiting
12:05:55 (3280): No heartbeat from core client for 30 sec - exiting
12:05:56 (3280): No heartbeat from core client for 30 sec - exiting
12:05:57 (3280): No heartbeat from core client for 30 sec - exiting
12:05:58 (3280): No heartbeat from core client for 30 sec - exiting
12:05:59 (3280): No heartbeat from core client for 30 sec - exiting
12:06:00 (3280): No heartbeat from core client for 30 sec - exiting
12:06:01 (3280): No heartbeat from core client for 30 sec - exiting
12:06:02 (3280): No heartbeat from core client for 30 sec - exiting
12:06:03 (3280): No heartbeat from core client for 30 sec - exiting
12:06:04 (3280): No heartbeat from core client for 30 sec - exiting
12:06:05 (3280): No heartbeat from core client for 30 sec - exiting
12:06:07 (3280): No heartbeat from core client for 30 sec - exiting
17:04:39 (3280): No heartbeat from core client for 30 sec - exiting
17:04:40 (3280): No heartbeat from core client for 30 sec - exiting
17:04:41 (3280): No heartbeat from core client for 30 sec - exiting
17:04:42 (3280): No heartbeat from core client for 30 sec - exiting
17:04:43 (3280): No heartbeat from core client for 30 sec - exiting
17:04:45 (3280): No heartbeat from core client for 30 sec - exiting
17:04:46 (3280): No heartbeat from core client for 30 sec - exiting
17:04:47 (3280): No heartbeat from core client for 30 sec - exiting
17:04:48 (3280): No heartbeat from core client for 30 sec - exiting
17:04:49 (3280): No heartbeat from core client for 30 sec - exiting
17:04:50 (3280): No heartbeat from core client for 30 sec - exiting
17:04:51 (3280): No heartbeat from core client for 30 sec - exiting
17:04:52 (3280): No heartbeat from core client for 30 sec - exiting
17:04:53 (3280): No heartbeat from core client for 30 sec - exiting
17:04:54 (3280): No heartbeat from core client for 30 sec - exiting
17:04:55 (3280): No heartbeat from core client for 30 sec - exiting
17:04:57 (3280): No heartbeat from core client for 30 sec - exiting
17:04:58 (3280): No heartbeat from core client for 30 sec - exiting
17:04:59 (3280): No heartbeat from core client for 30 sec - exiting
17:05:00 (3280): No heartbeat from core client for 30 sec - exiting
17:05:01 (3280): No heartbeat from core client for 30 sec - exiting
17:05:02 (3280): No heartbeat from core client for 30 sec - exiting
17:05:03 (3280): No heartbeat from core client for 30 sec - exiting
17:05:04 (3280): No heartbeat from core client for 30 sec - exiting
17:05:05 (3280): No heartbeat from core client for 30 sec - exiting
17:05:06 (3280): No heartbeat from core client for 30 sec - exiting
17:05:07 (3280): No heartbeat from core client for 30 sec - exiting
23:03:29 (3280): No heartbeat from core client for 30 sec - exiting
23:03:30 (3280): No heartbeat from core client for 30 sec - exiting
23:03:31 (3280): No heartbeat from core client for 30 sec - exiting
23:03:32 (3280): No heartbeat from core client for 30 sec - exiting
23:03:33 (3280): No heartbeat from core client for 30 sec - exiting
23:03:34 (3280): No heartbeat from core client for 30 sec - exiting
23:03:35 (3280): No heartbeat from core client for 30 sec - exiting
23:03:36 (3280): No heartbeat from core client for 30 sec - exiting
23:03:37 (3280): No heartbeat from core client for 30 sec - exiting
23:03:39 (3280): No heartbeat from core client for 30 sec - exiting
23:03:40 (3280): No heartbeat from core client for 30 sec - exiting
23:03:41 (3280): No heartbeat from core client for 30 sec - exiting
23:03:42 (3280): No heartbeat from core client for 30 sec - exiting
23:03:43 (3280): No heartbeat from core client for 30 sec - exiting
23:03:44 (3280): No heartbeat from core client for 30 sec - exiting
23:03:45 (3280): No heartbeat from core client for 30 sec - exiting
23:03:46 (3280): No heartbeat from core client for 30 sec - exiting
23:03:47 (3280): No heartbeat from core client for 30 sec - exiting
23:03:48 (3280): No heartbeat from core client for 30 sec - exiting
23:03:50 (3280): No heartbeat from core client for 30 sec - exiting
23:03:51 (3280): No heartbeat from core client for 30 sec - exiting
23:03:52 (3280): No heartbeat from core client for 30 sec - exiting
23:03:53 (3280): No heartbeat from core client for 30 sec - exiting
23:03:54 (3280): No heartbeat from core client for 30 sec - exiting
23:03:55 (3280): No heartbeat from core client for 30 sec - exiting
23:03:56 (3280): No heartbeat from core client for 30 sec - exiting
23:03:57 (3280): No heartbeat from core client for 30 sec - exiting
23:03:58 (3280): No heartbeat from core client for 30 sec - exiting
23:03:59 (3280): No heartbeat from core client for 30 sec - exiting
23:04:00 (3280): No heartbeat from core client for 30 sec - exiting
23:04:02 (3280): No heartbeat from core client for 30 sec - exiting
23:04:03 (3280): No heartbeat from core client for 30 sec - exiting
23:04:04 (3280): No heartbeat from core client for 30 sec - exiting
23:04:05 (3280): No heartbeat from core client for 30 sec - exiting
23:04:06 (3280): No heartbeat from core client for 30 sec - exiting
23:04:07 (3280): No heartbeat from core client for 30 sec - exiting
05:02:19 (3280): No heartbeat from core client for 30 sec - exiting
05:02:20 (3280): No heartbeat from core client for 30 sec - exiting
05:02:21 (3280): No heartbeat from core client for 30 sec - exiting
05:02:22 (3280): No heartbeat from core client for 30 sec - exiting
05:02:23 (3280): No heartbeat from core client for 30 sec - exiting
05:02:24 (3280): No heartbeat from core client for 30 sec - exiting
05:02:25 (3280): No heartbeat from core client for 30 sec - exiting
05:02:27 (3280): No heartbeat from core client for 30 sec - exiting
05:02:28 (3280): No heartbeat from core client for 30 sec - exiting
05:02:29 (3280): No heartbeat from core client for 30 sec - exiting
05:02:30 (3280): No heartbeat from core client for 30 sec - exiting
05:02:31 (3280): No heartbeat from core client for 30 sec - exiting
05:02:32 (3280): No heartbeat from core client for 30 sec - exiting
05:02:33 (3280): No heartbeat from core client for 30 sec - exiting
05:02:34 (3280): No heartbeat from core client for 30 sec - exiting
05:02:35 (3280): No heartbeat from core client for 30 sec - exiting
05:02:36 (3280): No heartbeat from core client for 30 sec - exiting
05:02:37 (3280): No heartbeat from core client for 30 sec - exiting
05:02:39 (3280): No heartbeat from core client for 30 sec - exiting
05:02:40 (3280): No heartbeat from core client for 30 sec - exiting
05:02:41 (3280): No heartbeat from core client for 30 sec - exiting
05:02:42 (3280): No heartbeat from core client for 30 sec - exiting
05:02:43 (3280): No heartbeat from core client for 30 sec - exiting
05:02:44 (3280): No heartbeat from core client for 30 sec - exiting
05:02:45 (3280): No heartbeat from core client for 30 sec - exiting
05:02:46 (3280): No heartbeat from core client for 30 sec - exiting
05:02:47 (3280): No heartbeat from core client for 30 sec - exiting
05:02:48 (3280): No heartbeat from core client for 30 sec - exiting
05:02:49 (3280): No heartbeat from core client for 30 sec - exiting
05:02:51 (3280): No heartbeat from core client for 30 sec - exiting
05:02:52 (3280): No heartbeat from core client for 30 sec - exiting
05:02:53 (3280): No heartbeat from core client for 30 sec - exiting
05:02:54 (3280): No heartbeat from core client for 30 sec - exiting
05:02:55 (3280): No heartbeat from core client for 30 sec - exiting
05:02:56 (3280): No heartbeat from core client for 30 sec - exiting
10:01:19 (3280): No heartbeat from core client for 30 sec - exiting
10:01:20 (3280): No heartbeat from core client for 30 sec - exiting
10:01:21 (3280): No heartbeat from core client for 30 sec - exiting
10:01:22 (3280): No heartbeat from core client for 30 sec - exiting
10:01:24 (3280): No heartbeat from core client for 30 sec - exiting
10:01:25 (3280): No heartbeat from core client for 30 sec - exiting
10:01:26 (3280): No heartbeat from core client for 30 sec - exiting
10:01:27 (3280): No heartbeat from core client for 30 sec - exiting
10:01:28 (3280): No heartbeat from core client for 30 sec - exiting
10:01:29 (3280): No heartbeat from core client for 30 sec - exiting
10:01:30 (3280): No heartbeat from core client for 30 sec - exiting
10:01:31 (3280): No heartbeat from core client for 30 sec - exiting
10:01:32 (3280): No heartbeat from core client for 30 sec - exiting
10:01:33 (3280): No heartbeat from core client for 30 sec - exiting
10:01:35 (3280): No heartbeat from core client for 30 sec - exiting
10:01:36 (3280): No heartbeat from core client for 30 sec - exiting
10:01:37 (3280): No heartbeat from core client for 30 sec - exiting
10:01:38 (3280): No heartbeat from core client for 30 sec - exiting
10:01:39 (3280): No heartbeat from core client for 30 sec - exiting
10:01:40 (3280): No heartbeat from core client for 30 sec - exiting
10:01:41 (3280): No heartbeat from core client for 30 sec - exiting
10:01:42 (3280): No heartbeat from core client for 30 sec - exiting
10:01:43 (3280): No heartbeat from core client for 30 sec - exiting
10:01:44 (3280): No heartbeat from core client for 30 sec - exiting
10:01:45 (3280): No heartbeat from core client for 30 sec - exiting
10:01:47 (3280): No heartbeat from core client for 30 sec - exiting
16:00:09 (3280): No heartbeat from core client for 30 sec - exiting
16:00:11 (3280): No heartbeat from core client for 30 sec - exiting
16:00:12 (3280): No heartbeat from core client for 30 sec - exiting
16:00:13 (3280): No heartbeat from core client for 30 sec - exiting
16:00:14 (3280): No heartbeat from core client for 30 sec - exiting
16:00:15 (3280): No heartbeat from core client for 30 sec - exiting
16:00:16 (3280): No heartbeat from core client for 30 sec - exiting
16:00:17 (3280): No heartbeat from core client for 30 sec - exiting
16:00:18 (3280): No heartbeat from core client for 30 sec - exiting
16:00:19 (3280): No heartbeat from core client for 30 sec - exiting
16:00:20 (3280): No heartbeat from core client for 30 sec - exiting
16:00:22 (3280): No heartbeat from core client for 30 sec - exiting
16:00:23 (3280): No heartbeat from core client for 30 sec - exiting
16:00:24 (3280): No heartbeat from core client for 30 sec - exiting
16:00:25 (3280): No heartbeat from core client for 30 sec - exiting
16:00:26 (3280): No heartbeat from core client for 30 sec - exiting
16:00:27 (3280): No heartbeat from core client for 30 sec - exiting
16:00:28 (3280): No heartbeat from core client for 30 sec - exiting
16:00:29 (3280): No heartbeat from core client for 30 sec - exiting
16:00:30 (3280): No heartbeat from core client for 30 sec - exiting
16:00:31 (3280): No heartbeat from core client for 30 sec - exiting
16:00:32 (3280): No heartbeat from core client for 30 sec - exiting
16:00:34 (3280): No heartbeat from core client for 30 sec - exiting
16:00:35 (3280): No heartbeat from core client for 30 sec - exiting
16:00:36 (3280): No heartbeat from core client for 30 sec - exiting
16:00:37 (3280): No heartbeat from core client for 30 sec - exiting
16:00:38 (3280): No heartbeat from core client for 30 sec - exiting
16:00:39 (3280): No heartbeat from core client for 30 sec - exiting
16:00:40 (3280): No heartbeat from core client for 30 sec - exiting
16:00:41 (3280): No heartbeat from core client for 30 sec - exiting
16:00:42 (3280): No heartbeat from core client for 30 sec - exiting
16:00:43 (3280): No heartbeat from core client for 30 sec - exiting
16:00:45 (3280): No heartbeat from core client for 30 sec - exiting
16:00:46 (3280): No heartbeat from core client for 30 sec - exiting
Signal 11 received, exiting...
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
07 Aug 2011 06:55:09 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 1,036,800 4,829,680 4.6583
05 Aug 2011 22:24:17 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 1,010,880 4,716,203 4.6654
04 Aug 2011 13:36:54 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 984,960 4,603,043 4.6733
03 Aug 2011 05:37:47 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 959,040 4,490,500 4.6823
01 Aug 2011 20:24:57 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 933,120 4,374,329 4.6879
31 Jul 2011 09:57:11 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 907,200 4,252,932 4.6880
29 Jul 2011 15:34:03 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 881,280 4,131,895 4.6885
28 Jul 2011 16:17:00 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 855,360 4,012,414 4.6909
26 Jul 2011 11:30:10 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 829,440 3,890,624 4.6907
25 Jul 2011 22:20:37 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 803,520 3,769,810 4.6916
25 Jul 2011 20:37:20 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 777,600 3,648,028 4.6914
25 Jul 2011 19:27:23 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 751,680 3,529,214 4.6951
25 Jul 2011 18:14:59 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 725,760 3,406,935 4.6943
25 Jul 2011 17:16:43 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 699,840 3,285,479 4.6946
25 Jul 2011 15:46:38 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 673,920 3,163,581 4.6943
25 Jul 2011 13:30:45 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 648,000 3,040,135 4.6916
25 Jul 2011 13:30:45 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 622,080 2,918,373 4.6913
25 Jul 2011 13:30:45 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 596,160 2,797,268 4.6921
10 Jul 2011 15:44:31 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 570,240 2,676,231 4.6932
09 Jul 2011 04:02:55 957259 12933147 hadcm3n_o3rs_1940_40_007266496_2 544,320 2,555,524 4.6949


©2024 cpdn.org