Task 15925443

Name	hadcm3n_4jt4_1940_40_008311144_4
Workunit	8462279
Created	18 Aug 2013, 18:21:20 UTC
Sent	18 Aug 2013, 18:21:56 UTC
Report deadline	18 Nov 2013, 1:49:07 UTC
Received	18 Oct 2013, 12:06:41 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1221991
Run time	8 days 2 hours 45 min 7 sec
CPU time	7 days 7 hours 13 min 10 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.75 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:25:39 (3167): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:17:59 (7228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... 01:17:26 (11397): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... 04:06:15 (14330): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... 04:32:53 (18729): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:00:50 (21250): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... 09:44:56 (26479): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid 11:16:05 (29998): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:47:04 (32647): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:59:16 (37991): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... 18:12:03 (43186): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:18:38 (46201): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:12:08 (50524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:24:25 (54605): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:50:59 (57777): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... 07:12:02 (64266): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... hadcm3n_6.07_i686-apple-darwin(3022,0xac369a28) malloc: * error for object 0x4077804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3022,0xac369a28) malloc: * error for object 0x4077800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3022,0xac369a28) malloc: * error for object 0x4077804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3022,0xac369a28) malloc: * error for object 0x4077800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3022,0xac369a28) malloc: * error for object 0x4077804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3022,0xac369a28) malloc: * error for object 0x4077800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3022,0xac369a28) malloc: * error for object 0x4077804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3022,0xac369a28) malloc: * error for object 0x4077800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3022,0xac369a28) malloc: * error for object 0x4077804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3022,0xac369a28) malloc: * error for object 0x4077800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3022,0xac369a28) malloc: * error for object 0x407c004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3022,0xac369a28) malloc: * error for object 0x408a604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3022,0xac369a28) malloc: * error for object 0x4098c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3022,0xac369a28) malloc: * error for object 0x4098c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Signal 1 received, exiting... Called boinc_finish dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:21:59 (1499): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... 22:52:21 (1048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/0/../../switcher/switcher) is setuid or setgid hadcm3n_6.07_i686-apple-darwin(3965,0xacd9ba28) malloc: * error for object 0x1065c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3965,0xacd9ba28) malloc: * error for object 0x1065c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3965,0xacd9ba28) malloc: * error for object 0x303dc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3965,0xacd9ba28) malloc: * error for object 0x303dc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(3965,0xacd9ba28) malloc: * error for object 0x303dc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation dyld: DYLD_ environment variables being ignored because main executable (/bin/ps) is setuid or setgid dyld: DYLD_ environment variables being ignored because main executable (/bin/ps) is setuid or setgid Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.8.5 build 12F37 Fri Oct 18 07:05:59 2013 dyld: DYLD_ environment variables being ignored because main executable (/bin/ps) is setuid or setgid dyld: DYLD_ environment variables being ignored because main executable (/bin/ps) is setuid or setgid Thread 0 Crashed: Thread 1: dyld: DYLD_ environment variables being ignored because main executable (/usr/bin/atos) has __RESTRICT/__restrict section atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fa88 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/0/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x9023a000 - 0x9023efff /usr/lib/system/libcache.dylib 0x902d8000 - 0x902dffff /usr/lib/system/libunwind.dylib 0x9035e000 - 0x903c2fff /usr/lib/libstdc++.6.dylib 0x903c3000 - 0x903e0fff /usr/lib/system/libxpc.dylib 0x9046f000 - 0x90472fff /usr/lib/system/libcompiler_rt.dylib 0x91583000 - 0x9158bfff /usr/lib/system/libcopyfile.dylib 0x91af0000 - 0x91afefff /usr/lib/system/libsystem_network.dylib 0x92fed000 - 0x92fedfff /usr/lib/system/libsystem_blocks.dylib 0x94410000 - 0x94411fff /usr/lib/system/libremovefile.dylib 0x94412000 - 0x94414fff /usr/lib/system/libdyld.dylib 0x94417000 - 0x94421fff /usr/lib/system/libsystem_notify.dylib 0x945de000 - 0x945ecfff /usr/lib/libz.1.dylib 0x94b95000 - 0x94be1fff /usr/lib/system/libcorecrypto.dylib 0x95fde000 - 0x95fdffff /usr/lib/system/libsystem_sandbox.dylib 0x967d9000 - 0x967f3fff /usr/lib/system/libsystem_kernel.dylib 0x9681a000 - 0x9682cfff /usr/lib/system/libdispatch.dylib 0x9686a000 - 0x9686afff /usr/lib/libSystem.B.dylib 0x9686b000 - 0x96898fff /usr/lib/system/libsystem_m.dylib 0x96d3d000 - 0x96d3efff /usr/lib/system/libquarantine.dylib 0x96d3f000 - 0x96d3ffff /usr/lib/system/libkeymgr.dylib 0x980ef000 - 0x980f3fff /usr/lib/system/libmacho.dylib 0x98e1e000 - 0x98e25fff /usr/lib/system/libsystem_dnssd.dylib 0x98e26000 - 0x98e27fff /usr/lib/system/libdnsinfo.dylib 0x98fab000 - 0x98fcdfff /usr/lib/libc++abi.dylib 0x99525000 - 0x9952cfff /usr/lib/system/liblaunch.dylib 0x99916000 - 0x99921fff /usr/lib/system/libcommonCrypto.dylib 0x99df4000 - 0x99df5fff /usr/lib/system/libunc.dylib 0x99e52000 - 0x99e7efff /usr/lib/system/libsystem_info.dylib 0x9a23e000 - 0x9a2fbfff /usr/lib/system/libsystem_c.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Oct 2013 11:10:57	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	518,400	630,783	1.2168
11 Oct 2013 16:19:51	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	492,480	597,609	1.2135
11 Oct 2013 06:03:24	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	466,560	563,871	1.2086
04 Oct 2013 04:29:09	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	440,640	530,604	1.2042
30 Sep 2013 04:43:21	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	414,720	497,513	1.1996
29 Sep 2013 15:20:02	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	388,800	464,512	1.1947
29 Sep 2013 04:57:15	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	362,880	433,522	1.1947
28 Sep 2013 19:34:30	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	336,960	403,881	1.1986
21 Sep 2013 10:25:55	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	311,040	374,814	1.2050
21 Sep 2013 01:48:12	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	285,120	344,593	1.2086
20 Sep 2013 17:04:54	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	259,200	314,146	1.2120
20 Sep 2013 08:19:09	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	233,280	283,523	1.2154
09 Sep 2013 07:52:50	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	207,360	252,856	1.2194
05 Sep 2013 08:38:02	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	181,440	222,896	1.2285
26 Aug 2013 10:24:47	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	155,520	191,747	1.2329
26 Aug 2013 00:06:38	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	129,600	160,897	1.2415
25 Aug 2013 12:18:45	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	103,680	129,896	1.2529
25 Aug 2013 01:41:45	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	77,760	99,154	1.2751
19 Aug 2013 17:41:42	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	51,840	67,160	1.2955
19 Aug 2013 07:14:25	1221991	15925443	hadcm3n_4jt4_1940_40_008311144_4	25,920	34,215	1.3200