Task 15630931

Name	hadcm3n_zes0_1960_40_008317285_0
Workunit	8468420
Created	24 Feb 2013, 0:19:10 UTC
Sent	24 Feb 2013, 0:19:32 UTC
Report deadline	26 May 2013, 7:46:43 UTC
Received	28 Mar 2013, 6:39:12 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1176125
Run time	20 days 3 hours 47 min 9 sec
CPU time	18 days 18 hours 58 min 12 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.24 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/zes0ko.pji9c10 is not a valid UM file. Error converting file to netcdf: dataout/zes0ko.pji9c10 Error: Input file: dataout/zes0ko.pii9c10 is not a valid UM file. Error converting file to netcdf: dataout/zes0ko.pii9c10 Error: Input file: dataout/zes0ko.pfi9c10 is not a valid UM file. Error converting file to netcdf: dataout/zes0ko.pfi9c10 Error: Input file: dataout/zes0ka.phi9c10 is not a valid UM file. Error converting file to netcdf: dataout/zes0ka.phi9c10 Error: Input file: dataout/zes0ka.pgi9c10 is not a valid UM file. Error converting file to netcdf: dataout/zes0ka.pgi9c10 Error: Input file: dataout/zes0ka.pei9c10 is not a valid UM file. Error converting file to netcdf: dataout/zes0ka.pei9c10 Error: Input file: dataout/zes0ka.pdi9c10 is not a valid UM file. Error converting file to netcdf: dataout/zes0ka.pdi9c10 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x48ad404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x48ad400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x40d8204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x30b4604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x30cfc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x30cfc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x30cfc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x30cfc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x8ad604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x8c8c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x8c8c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x20b2604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x40d8204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x109fe04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x109fe00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x109fe04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x109fe00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61490,0xac29b2c0) malloc: * error for object 0x109fe04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(64691,0xac29b2c0) malloc: * error for object 0x203a804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Thu Mar 28 04:48:15 2013 Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fc98 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/4/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x90476000 - 0x904a5fff /usr/lib/system/libsystem_info.dylib 0x90812000 - 0x90812fff /usr/lib/system/libdnsinfo.dylib 0x918fe000 - 0x91902fff /usr/lib/system/libcache.dylib 0x92e3c000 - 0x92e3efff /usr/lib/system/libdyld.dylib 0x92f41000 - 0x9300cfff /usr/lib/system/libsystem_c.dylib 0x93a46000 - 0x93a49fff /usr/lib/system/libmathCommon.A.dylib 0x93ea0000 - 0x93eaefff /usr/lib/libz.1.dylib 0x94514000 - 0x9451dfff /usr/lib/libc++abi.dylib 0x945f8000 - 0x9465afff /usr/lib/libstdc++.6.dylib 0x94d56000 - 0x94d6cfff /usr/lib/system/libxpc.dylib 0x956ee000 - 0x9571cfff /usr/lib/libSystem.B.dylib 0x95f42000 - 0x95f4afff /usr/lib/system/libcopyfile.dylib 0x9600e000 - 0x96015fff /usr/lib/system/libsystem_dnssd.dylib 0x965e2000 - 0x965e6fff /usr/lib/system/libsystem_network.dylib 0x9708f000 - 0x97094fff /usr/lib/system/libmacho.dylib 0x9716f000 - 0x9718dfff /usr/lib/system/libsystem_kernel.dylib 0x98773000 - 0x9877afff /usr/lib/system/libsystem_notify.dylib 0x98782000 - 0x98783fff /usr/lib/system/libsystem_blocks.dylib 0x987b0000 - 0x987befff /usr/lib/system/libdispatch.dylib 0x998f6000 - 0x998fefff /usr/lib/system/libunwind.dylib 0x998ff000 - 0x99900fff /usr/lib/system/libunc.dylib 0x99ea7000 - 0x99eaffff /usr/lib/system/liblaunch.dylib 0x99ef5000 - 0x99ef8fff /usr/lib/system/libcompiler_rt.dylib 0x9bc0e000 - 0x9bc0ffff /usr/lib/system/libquarantine.dylib 0x9c200000 - 0x9c201fff /usr/lib/system/libremovefile.dylib 0x9c22b000 - 0x9c22bfff /usr/lib/system/libkeymgr.dylib 0x9cb58000 - 0x9cb9bfff /usr/lib/system/libcommonCrypto.dylib 0x9cf31000 - 0x9cf32fff /usr/lib/system/libsystem_sandbox.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Mar 2013 22:44:56	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	777,600	1,612,694	2.0739
26 Mar 2013 11:27:07	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	751,680	1,559,601	2.0748
25 Mar 2013 06:51:47	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	725,760	1,508,671	2.0787
24 Mar 2013 02:35:03	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	699,840	1,456,678	2.0814
22 Mar 2013 20:40:35	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	673,920	1,402,558	2.0812
21 Mar 2013 20:28:32	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	648,000	1,350,392	2.0839
20 Mar 2013 02:15:27	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	622,080	1,301,798	2.0927
19 Mar 2013 06:08:26	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	596,160	1,252,705	2.1013
18 Mar 2013 15:35:31	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	570,240	1,203,312	2.1102
18 Mar 2013 00:26:29	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	544,320	1,151,923	2.1163
17 Mar 2013 09:59:16	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	518,400	1,103,037	2.1278
16 Mar 2013 19:48:51	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	492,480	1,054,593	2.1414
16 Mar 2013 05:47:14	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	466,560	1,007,050	2.1585
15 Mar 2013 10:28:58	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	440,640	959,049	2.1765
14 Mar 2013 15:05:32	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	414,720	908,047	2.1895
13 Mar 2013 19:36:56	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	388,800	855,696	2.2009
08 Mar 2013 21:33:56	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	362,880	806,957	2.2238
07 Mar 2013 20:04:46	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	336,960	755,341	2.2416
06 Mar 2013 07:45:35	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	311,040	701,279	2.2546
05 Mar 2013 11:21:33	1176125	15630931	hadcm3n_zes0_1960_40_008317285_0	285,120	648,068	2.2730