Task 13355958

Name	hadcm3n_t2tl_1940_40_007445989_0
Workunit	7643492
Created	9 Sep 2011, 14:26:25 UTC
Sent	9 Sep 2011, 14:28:51 UTC
Report deadline	9 Dec 2011, 21:56:02 UTC
Received	13 Dec 2011, 7:53:59 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1099968
Run time	19 days 17 hours 58 min 52 sec
CPU time	17 days 1 hours 11 min 4 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.84 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:56:01 (1696): No heartbeat from core client for 30 sec - exiting 07:56:03 (1696): No heartbeat from core client for 30 sec - exiting 07:56:04 (1696): No heartbeat from core client for 30 sec - exiting 07:56:05 (1696): No heartbeat from core client for 30 sec - exiting 07:56:06 (1696): No heartbeat from core client for 30 sec - exiting 07:56:07 (1696): No heartbeat from core client for 30 sec - exiting 07:56:08 (1696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:41:09 (1722): No heartbeat from core client for 30 sec - exiting 08:41:10 (1722): No heartbeat from core client for 30 sec - exiting 08:41:11 (1722): No heartbeat from core client for 30 sec - exiting 08:41:12 (1722): No heartbeat from core client for 30 sec - exiting 08:41:13 (1722): No heartbeat from core client for 30 sec - exiting 08:41:14 (1722): No heartbeat from core client for 30 sec - exiting 08:41:15 (1722): No heartbeat from core client for 30 sec - exiting 08:41:16 (1722): No heartbeat from core client for 30 sec - exiting 08:41:17 (1722): No heartbeat from core client for 30 sec - exiting 08:41:18 (1722): No heartbeat from core client for 30 sec - exiting 08:41:19 (1722): No heartbeat from core client for 30 sec - exiting 08:41:20 (1722): No heartbeat from core client for 30 sec - exiting 08:41:21 (1722): No heartbeat from core client for 30 sec - exiting 08:41:23 (1722): No heartbeat from core client for 30 sec - exiting 08:41:24 (1722): No heartbeat from core client for 30 sec - exiting 08:41:25 (1722): No heartbeat from core client for 30 sec - exiting 08:41:26 (1722): No heartbeat from core client for 30 sec - exiting 08:41:27 (1722): No heartbeat from core client for 30 sec - exiting 08:41:28 (1722): No heartbeat from core client for 30 sec - exiting 08:41:29 (1722): No heartbeat from core client for 30 sec - exiting 08:41:30 (1722): No heartbeat from core client for 30 sec - exiting 08:41:31 (1722): No heartbeat from core client for 30 sec - exiting 08:41:32 (1722): No heartbeat from core client for 30 sec - exiting 08:41:33 (1722): No heartbeat from core client for 30 sec - exiting 08:41:34 (1722): No heartbeat from core client for 30 sec - exiting 08:41:35 (1722): No heartbeat from core client for 30 sec - exiting 08:41:36 (1722): No heartbeat from core client for 30 sec - exiting 08:41:37 (1722): No heartbeat from core client for 30 sec - exiting 08:41:38 (1722): No heartbeat from core client for 30 sec - exiting 08:41:39 (1722): No heartbeat from core client for 30 sec - exiting 08:41:40 (1722): No heartbeat from core client for 30 sec - exiting 08:41:41 (1722): No heartbeat from core client for 30 sec - exiting 08:41:42 (1722): No heartbeat from core client for 30 sec - exiting 08:41:43 (1722): No heartbeat from core client for 30 sec - exiting 08:41:44 (1722): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:47:01 (2095): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(1701,0xacef82c0) malloc: * error for object 0x1220a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1701,0xacef82c0) malloc: * error for object 0x1220a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1701,0xacef82c0) malloc: * error for object 0x9e8804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1701,0xacef82c0) malloc: * error for object 0x9e8800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1701,0xacef82c0) malloc: * error for object 0x9e4a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1701,0xacef82c0) malloc: * error for object 0x9e4a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1701,0xacef82c0) malloc: * error for object 0x9e4a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1701,0xacef82c0) malloc: * error for object 0x9e4a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1701,0xacef82c0) malloc: * error for object 0x9e4a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1701,0xacef82c0) malloc: * error for object 0x9e4a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1701,0xacef82c0) malloc: * error for object 0x9e4a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1701,0xacef82c0) malloc: * error for object 0x9e4a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.2 build 11C74 hadcm3n_um_6.07_i686-apple-darwin(1803,0xc0005000) malloc: * error for object 0x3000000: pointer being freed was not allocated * set a breakpoint in malloc_error_break to debug Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish hadcm3n_6.07_i686-apple-darwin(1759,0xacef82c0) malloc: * error for object 0x1432004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1759,0xacef82c0) malloc: * error for object 0x1432000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1759,0xacef82c0) malloc: * error for object 0x1432004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1759,0xacef82c0) malloc: * error for object 0x1432000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1759,0xacef82c0) malloc: * error for object 0x1432004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1759,0xacef82c0) malloc: * error for object 0x1432000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1759,0xacef82c0) malloc: * error for object 0x1432004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1759,0xacef82c0) malloc: * error for object 0x1432000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1759,0xacef82c0) malloc: * error for object 0x1432004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1759,0xacef82c0) malloc: * error for object 0x1432000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1759,0xacef82c0) malloc: * error for object 0x1432004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1759,0xacef82c0) malloc: * error for object 0x1432000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(3067,0xacef82c0) malloc: * error for object 0xbf5004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3067,0xacef82c0) malloc: * error for object 0xbf5000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3067,0xacef82c0) malloc: * error for object 0x142ae04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3067,0xacef82c0) malloc: * error for object 0x142ae00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3067,0xacef82c0) malloc: * error for object 0x142ae00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(1820,0xacef82c0) malloc: * error for object 0xb7ac04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1820,0xacef82c0) malloc: * error for object 0xb7ac00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1820,0xacef82c0) malloc: * error for object 0xb7ac00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(1728,0xacef82c0) malloc: * error for object 0x1146804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1728,0xacef82c0) malloc: * error for object 0x1146800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1728,0xacef82c0) malloc: * error for object 0x1146804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1728,0xacef82c0) malloc: * error for object 0x1146800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1728,0xacef82c0) malloc: * error for object 0x1146804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1728,0xacef82c0) malloc: * error for object 0x1146800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1728,0xacef82c0) malloc: * error for object 0x911e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1728,0xacef82c0) malloc: * error for object 0x911e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1728,0xacef82c0) malloc: * error for object 0x911e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1728,0xacef82c0) malloc: * error for object 0x911e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1728,0xacef82c0) malloc: * error for object 0x911e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1728,0xacef82c0) malloc: * error for object 0x911e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Dec 2011 10:48:08	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	1,036,800	1,472,491	1.4202
06 Dec 2011 14:06:30	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	1,010,880	1,436,541	1.4211
05 Dec 2011 12:06:50	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	984,960	1,400,909	1.4223
02 Dec 2011 08:31:56	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	959,040	1,365,382	1.4237
29 Nov 2011 16:43:17	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	933,120	1,327,590	1.4227
25 Nov 2011 09:01:02	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	907,200	1,288,556	1.4204
23 Nov 2011 13:53:24	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	881,280	1,249,838	1.4182
22 Nov 2011 11:24:25	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	855,360	1,210,847	1.4156
21 Nov 2011 08:28:22	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	829,440	1,171,915	1.4129
17 Nov 2011 17:10:34	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	803,520	1,132,941	1.4100
16 Nov 2011 14:51:58	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	777,600	1,096,830	1.4105
15 Nov 2011 18:23:30	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	751,680	1,060,889	1.4114
15 Nov 2011 18:23:29	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	725,760	1,024,963	1.4123
15 Nov 2011 18:23:29	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	699,840	988,981	1.4132
15 Nov 2011 18:23:29	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	673,920	953,553	1.4149
31 Oct 2011 17:11:20	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	648,000	918,045	1.4167
31 Oct 2011 16:24:09	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	622,080	882,183	1.4181
19 Oct 2011 09:12:37	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	596,160	845,986	1.4191
17 Oct 2011 17:12:32	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	570,240	809,618	1.4198
13 Oct 2011 08:29:22	1099968	13355958	hadcm3n_t2tl_1940_40_007445989_0	544,320	772,900	1.4199