Task 15279891

Name	hadcm3n_zk0h_1880_40_008200583_1
Workunit	8355707
Created	13 Sep 2012, 8:03:46 UTC
Sent	14 Sep 2012, 1:22:37 UTC
Report deadline	14 Dec 2012, 8:49:48 UTC
Received	5 Nov 2012, 23:41:28 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1170228
Run time	28 days 0 hours 20 min 44 sec
CPU time	24 days 1 hours 6 min
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.20 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11336, selfPID=11336, iMonCtr=1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/zk0hko.pj82c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hko.pj82c10 Error: Input file: dataout/zk0hko.pi82c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hko.pi82c10 Error: Input file: dataout/zk0hko.pf82c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hko.pf82c10 Error: Input file: dataout/zk0hka.ph82c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hka.ph82c10 Error: Input file: dataout/zk0hka.pg82c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hka.pg82c10 Error: Input file: dataout/zk0hka.pe82c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hka.pe82c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/zk0hko.pj83c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hko.pj83c10 Error: Input file: dataout/zk0hko.pi83c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hko.pi83c10 Error: Input file: dataout/zk0hko.pf83c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hko.pf83c10 Error: Input file: dataout/zk0hka.ph83c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hka.ph83c10 Error: Input file: dataout/zk0hka.pg83c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hka.pg83c10 Error: Input file: dataout/zk0hka.pe83c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hka.pe83c10 Error: Input file: dataout/zk0hka.pd83c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hka.pd83c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/zk0hko.pj84c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hko.pj84c10 Error: Input file: dataout/zk0hko.pi84c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hko.pi84c10 Error: Input file: dataout/zk0hko.pf84c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hko.pf84c10 Error: Input file: dataout/zk0hka.ph84c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hka.ph84c10 Error: Input file: dataout/zk0hka.pg84c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hka.pg84c10 Error: Input file: dataout/zk0hka.pe84c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hka.pe84c10 Error: Input file: dataout/zk0hka.pd84c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hka.pd84c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/zk0hko.pj85c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hko.pj85c10 Error: Input file: dataout/zk0hko.pi85c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hko.pi85c10 Error: Input file: dataout/zk0hko.pf85c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hko.pf85c10 Error: Input file: dataout/zk0hka.ph85c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hka.ph85c10 Error: Input file: dataout/zk0hka.pg85c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hka.pg85c10 Error: Input file: dataout/zk0hka.pe85c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hka.pe85c10 Error: Input file: dataout/zk0hka.pd85c10 is not a valid UM file. Error converting file to netcdf: dataout/zk0hka.pd85c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:50:31 (86761): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:23:20 (3317): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:08:26 (11426): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:47:06 (902): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x82ba04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x82ba00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x847004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x847000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x104bc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x82ba04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x82d204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x82d200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x1016804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x1016800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x1016804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x1016800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x1016804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x1016800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x1016804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x1016800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x802204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(4944,0xa020a540) malloc: * error for object 0x802200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Nov 2012 23:44:23	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	1,036,800	2,076,736	2.0030
05 Nov 2012 03:02:20	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	1,010,880	2,024,193	2.0024
04 Nov 2012 06:25:18	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	984,960	1,971,571	2.0017
03 Nov 2012 12:00:14	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	959,040	1,919,050	2.0010
02 Nov 2012 17:36:32	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	933,120	1,866,538	2.0003
01 Nov 2012 23:18:25	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	907,200	1,814,036	1.9996
01 Nov 2012 04:40:56	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	881,280	1,761,487	1.9988
31 Oct 2012 10:23:59	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	855,360	1,708,929	1.9979
30 Oct 2012 15:12:07	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	829,440	1,656,449	1.9971
29 Oct 2012 19:33:13	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	803,520	1,604,011	1.9962
28 Oct 2012 23:55:40	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	777,600	1,551,362	1.9951
28 Oct 2012 05:53:37	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	751,680	1,498,763	1.9939
27 Oct 2012 11:59:46	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	725,760	1,446,261	1.9928
26 Oct 2012 18:04:32	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	699,840	1,393,682	1.9914
25 Oct 2012 23:37:05	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	673,920	1,341,123	1.9900
24 Oct 2012 00:35:10	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	648,000	1,288,750	1.9888
23 Oct 2012 08:28:10	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	622,080	1,236,378	1.9875
22 Oct 2012 12:33:15	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	596,160	1,183,924	1.9859
21 Oct 2012 09:40:05	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	570,240	1,131,411	1.9841
20 Oct 2012 16:52:54	1170228	15279891	hadcm3n_zk0h_1880_40_008200583_1	544,320	1,078,865	1.9820