Task 15453081

Name	hadcm3n_zn4p_1880_40_008251009_0
Workunit	8406133
Created	22 Nov 2012, 9:27:31 UTC
Sent	22 Nov 2012, 9:28:17 UTC
Report deadline	21 Feb 2013, 16:55:28 UTC
Received	14 Dec 2012, 15:40:28 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	789649
Run time	15 days 15 hours 47 min 8 sec
CPU time	13 days 18 hours 5 min 38 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	3.47 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(26099,0xac2232c0) malloc: * error for object 0x12e5404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26099,0xac2232c0) malloc: * error for object 0xacbe04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26099,0xac2232c0) malloc: * error for object 0xae7404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26099,0xac2232c0) malloc: * error for object 0xae7400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26099,0xac2232c0) malloc: * error for object 0xae7404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26099,0xac2232c0) malloc: * error for object 0xae7400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26099,0xac2232c0) malloc: * error for object 0x2af2204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26099,0xac2232c0) malloc: * error for object 0x12e5404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26099,0xac2232c0) malloc: * error for object 0xacbe04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26099,0xac2232c0) malloc: * error for object 0x22d7a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26099,0xac2232c0) malloc: * error for object 0x2ab3604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26099,0xac2232c0) malloc: * error for object 0x2ab3600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26099,0xac2232c0) malloc: * error for object 0x2ab3604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26099,0xac2232c0) malloc: * error for object 0x2ab3600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26099,0xac2232c0) malloc: * error for object 0x2ab3604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(26813,0xac2232c0) malloc: * error for object 0x2a62204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26813,0xac2232c0) malloc: * error for object 0x228a204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26813,0xac2232c0) malloc: * error for object 0x228a200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26813,0xac2232c0) malloc: * error for object 0x1a7fe00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(26908,0xac2232c0) malloc: * error for object 0x2069804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26908,0xac2232c0) malloc: * error for object 0x2069800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26908,0xac2232c0) malloc: * error for object 0x87d404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26908,0xac2232c0) malloc: * error for object 0x87d400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26908,0xac2232c0) malloc: * error for object 0x87d404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26908,0xac2232c0) malloc: * error for object 0x87d400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26908,0xac2232c0) malloc: * error for object 0x205ba04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26908,0xac2232c0) malloc: * error for object 0x205ba00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26908,0xac2232c0) malloc: * error for object 0x205ba04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26908,0xac2232c0) malloc: * error for object 0x205ba00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26908,0xac2232c0) malloc: * error for object 0x2068a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26908,0xac2232c0) malloc: * error for object 0x2077004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26908,0xac2232c0) malloc: * error for object 0x2085604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(26908,0xac2232c0) malloc: * error for object 0x2085600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... hadcm3n_6.07_i686-apple-darwin(50500,0xac2232c0) malloc: * error for object 0x22d9604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(50500,0xac2232c0) malloc: * error for object 0x22d9600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(50500,0xac2232c0) malloc: * error for object 0x22f4c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(50500,0xac2232c0) malloc: * error for object 0x22f4c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(50500,0xac2232c0) malloc: * error for object 0x2adfa04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63b Thu Dec 13 10:40:14 2012 Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fa18 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/2/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x903ef000 - 0x903f1fff /usr/lib/system/libdyld.dylib 0x91291000 - 0x91291fff /usr/lib/system/libdnsinfo.dylib 0x9203a000 - 0x9203bfff /usr/lib/system/libsystem_blocks.dylib 0x920ab000 - 0x92176fff /usr/lib/system/libsystem_c.dylib 0x92194000 - 0x92197fff /usr/lib/system/libcompiler_rt.dylib 0x927ef000 - 0x927f3fff /usr/lib/system/libsystem_network.dylib 0x929f6000 - 0x929fefff /usr/lib/system/libunwind.dylib 0x929ff000 - 0x92a08fff /usr/lib/libc++abi.dylib 0x92a3c000 - 0x92a44fff /usr/lib/system/libcopyfile.dylib 0x93ec7000 - 0x93eccfff /usr/lib/system/libmacho.dylib 0x93ecd000 - 0x93ee3fff /usr/lib/system/libxpc.dylib 0x942b3000 - 0x942c1fff /usr/lib/libz.1.dylib 0x9479c000 - 0x9479dfff /usr/lib/system/libunc.dylib 0x9480a000 - 0x9486cfff /usr/lib/libstdc++.6.dylib 0x94a8a000 - 0x94ab9fff /usr/lib/system/libsystem_info.dylib 0x94cf1000 - 0x94cf1fff /usr/lib/system/libkeymgr.dylib 0x94ebb000 - 0x94ec3fff /usr/lib/system/liblaunch.dylib 0x95d30000 - 0x95d31fff /usr/lib/system/libremovefile.dylib 0x98462000 - 0x98466fff /usr/lib/system/libcache.dylib 0x98e31000 - 0x98e32fff /usr/lib/system/libsystem_sandbox.dylib 0x98e33000 - 0x98e41fff /usr/lib/system/libdispatch.dylib 0x993de000 - 0x993e5fff /usr/lib/system/libsystem_dnssd.dylib 0x996af000 - 0x996b2fff /usr/lib/system/libmathCommon.A.dylib 0x9a46f000 - 0x9a49dfff /usr/lib/libSystem.B.dylib 0x9a4df000 - 0x9a522fff /usr/lib/system/libcommonCrypto.dylib 0x9ad3b000 - 0x9ad42fff /usr/lib/system/libsystem_notify.dylib 0x9c69b000 - 0x9c69cfff /usr/lib/system/libquarantine.dylib 0x9ceb3000 - 0x9ced1fff /usr/lib/system/libsystem_kernel.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Dec 2012 15:44:04	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	1,036,800	1,188,335	1.1462
14 Dec 2012 15:44:04	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	1,010,880	1,159,010	1.1465
14 Dec 2012 15:44:04	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	984,960	1,129,003	1.1462
14 Dec 2012 15:44:04	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	959,040	1,099,534	1.1465
14 Dec 2012 15:44:04	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	933,120	1,070,066	1.1468
08 Dec 2012 04:05:20	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	907,200	1,039,584	1.1459
07 Dec 2012 06:46:38	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	881,280	1,009,203	1.1452
06 Dec 2012 14:10:06	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	855,360	979,104	1.1447
06 Dec 2012 04:12:33	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	829,440	949,604	1.1449
05 Dec 2012 10:54:51	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	803,520	920,081	1.1451
04 Dec 2012 21:51:52	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	777,600	890,497	1.1452
04 Dec 2012 10:57:42	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	751,680	861,019	1.1455
04 Dec 2012 01:30:35	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	725,760	831,383	1.1455
03 Dec 2012 10:34:42	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	699,840	801,661	1.1455
03 Dec 2012 01:32:59	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	673,920	771,930	1.1454
02 Dec 2012 16:35:07	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	648,000	742,372	1.1456
02 Dec 2012 07:38:25	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	622,080	712,974	1.1461
01 Dec 2012 22:41:32	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	596,160	683,474	1.1465
01 Dec 2012 13:47:25	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	570,240	654,159	1.1472
01 Dec 2012 04:55:26	789649	15453081	hadcm3n_zn4p_1880_40_008251009_0	544,320	624,951	1.1481