Task 15554074

Name	hadcm3n_n1ln_1880_40_008284371_2
Workunit	8435506
Created	21 Jan 2013, 20:15:43 UTC
Sent	21 Jan 2013, 20:16:02 UTC
Report deadline	23 Apr 2013, 3:43:13 UTC
Received	26 Mar 2013, 13:48:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1252582
Run time	11 days 11 hours 5 min 57 sec
CPU time	11 days 10 hours 8 min 35 sec
Validate state	Invalid
Credit	6,531.84
Device peak FLOPS	2.97 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:49:24 (6508): No heartbeat from core client for 30 sec - exiting 17:49:25 (6508): No heartbeat from core client for 30 sec - exiting 17:49:26 (6508): No heartbeat from core client for 30 sec - exiting 17:49:28 (6508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:14:49 (4572): No heartbeat from core client for 30 sec - exiting 19:14:50 (4572): No heartbeat from core client for 30 sec - exiting 19:14:51 (4572): No heartbeat from core client for 30 sec - exiting 19:14:52 (4572): No heartbeat from core client for 30 sec - exiting 19:14:53 (4572): No heartbeat from core client for 30 sec - exiting 19:14:54 (4572): No heartbeat from core client for 30 sec - exiting 19:14:56 (4572): No heartbeat from core client for 30 sec - exiting 19:14:57 (4572): No heartbeat from core client for 30 sec - exiting 19:14:58 (4572): No heartbeat from core client for 30 sec - exiting 19:14:59 (4572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:32:02 (5960): No heartbeat from core client for 30 sec - exiting 19:32:03 (5960): No heartbeat from core client for 30 sec - exiting 19:32:04 (5960): No heartbeat from core client for 30 sec - exiting 19:32:05 (5960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:16:49 (4172): No heartbeat from core client for 30 sec - exiting 19:16:51 (4172): No heartbeat from core client for 30 sec - exiting 19:16:52 (4172): No heartbeat from core client for 30 sec - exiting 19:16:53 (4172): No heartbeat from core client for 30 sec - exiting 19:16:54 (4172): No heartbeat from core client for 30 sec - exiting 19:16:55 (4172): No heartbeat from core client for 30 sec - exiting 19:16:56 (4172): No heartbeat from core client for 30 sec - exiting 19:16:57 (4172): No heartbeat from core client for 30 sec - exiting 19:16:58 (4172): No heartbeat from core client for 30 sec - exiting 19:16:59 (4172): No heartbeat from core client for 30 sec - exiting 19:17:01 (4172): No heartbeat from core client for 30 sec - exiting 19:17:02 (4172): No heartbeat from core client for 30 sec - exiting 19:17:03 (4172): No heartbeat from core client for 30 sec - exiting 19:17:04 (4172): No heartbeat from core client for 30 sec - exiting 19:17:05 (4172): No heartbeat from core client for 30 sec - exiting 19:17:06 (4172): No heartbeat from core client for 30 sec - exiting 19:17:07 (4172): No heartbeat from core client for 30 sec - exiting 19:17:08 (4172): No heartbeat from core client for 30 sec - exiting 19:17:09 (4172): No heartbeat from core client for 30 sec - exiting 19:17:10 (4172): No heartbeat from core client for 30 sec - exiting 19:17:11 (4172): No heartbeat from core client for 30 sec - exiting 19:17:13 (4172): No heartbeat from core client for 30 sec - exiting 19:17:14 (4172): No heartbeat from core client for 30 sec - exiting 19:17:15 (4172): No heartbeat from core client for 30 sec - exiting 19:17:16 (4172): No heartbeat from core client for 30 sec - exiting 19:17:17 (4172): No heartbeat from core client for 30 sec - exiting 19:17:18 (4172): No heartbeat from core client for 30 sec - exiting 19:17:19 (4172): No heartbeat from core client for 30 sec - exiting 19:17:20 (4172): No heartbeat from core client for 30 sec - exiting 19:17:21 (4172): No heartbeat from core client for 30 sec - exiting 19:17:22 (4172): No heartbeat from core client for 30 sec - exiting 19:17:23 (4172): No heartbeat from core client for 30 sec - exiting 19:17:25 (4172): No heartbeat from core client for 30 sec - exiting 19:17:26 (4172): No heartbeat from core client for 30 sec - exiting 19:17:27 (4172): No heartbeat from core client for 30 sec - exiting 19:17:28 (4172): No heartbeat from core client for 30 sec - exiting 19:17:29 (4172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:59:25 (5552): No heartbeat from core client for 30 sec - exiting 19:59:27 (5552): No heartbeat from core client for 30 sec - exiting 19:59:28 (5552): No heartbeat from core client for 30 sec - exiting 19:59:29 (5552): No heartbeat from core client for 30 sec - exiting 19:59:30 (5552): No heartbeat from core client for 30 sec - exiting 19:59:31 (5552): No heartbeat from core client for 30 sec - exiting 19:59:32 (5552): No heartbeat from core client for 30 sec - exiting 19:59:34 (5552): No heartbeat from core client for 30 sec - exiting 19:59:35 (5552): No heartbeat from core client for 30 sec - exiting 19:59:36 (5552): No heartbeat from core client for 30 sec - exiting 19:59:37 (5552): No heartbeat from core client for 30 sec - exiting 19:59:38 (5552): No heartbeat from core client for 30 sec - exiting 19:59:39 (5552): No heartbeat from core client for 30 sec - exiting 19:59:40 (5552): No heartbeat from core client for 30 sec - exiting 19:59:41 (5552): No heartbeat from core client for 30 sec - exiting 19:59:42 (5552): No heartbeat from core client for 30 sec - exiting 19:59:43 (5552): No heartbeat from core client for 30 sec - exiting 19:59:44 (5552): No heartbeat from core client for 30 sec - exiting 19:59:46 (5552): No heartbeat from core client for 30 sec - exiting 19:59:47 (5552): No heartbeat from core client for 30 sec - exiting 19:59:48 (5552): No heartbeat from core client for 30 sec - exiting 19:59:49 (5552): No heartbeat from core client for 30 sec - exiting 19:59:50 (5552): No heartbeat from core client for 30 sec - exiting 19:59:51 (5552): No heartbeat from core client for 30 sec - exiting 19:59:52 (5552): No heartbeat from core client for 30 sec - exiting 19:59:53 (5552): No heartbeat from core client for 30 sec - exiting 19:59:54 (5552): No heartbeat from core client for 30 sec - exiting 19:59:55 (5552): No heartbeat from core client for 30 sec - exiting 19:59:56 (5552): No heartbeat from core client for 30 sec - exiting 19:59:58 (5552): No heartbeat from core client for 30 sec - exiting 19:59:59 (5552): No heartbeat from core client for 30 sec - exiting 20:00:00 (5552): No heartbeat from core client for 30 sec - exiting 20:00:01 (5552): No heartbeat from core client for 30 sec - exiting 20:00:02 (5552): No heartbeat from core client for 30 sec - exiting 20:00:03 (5552): No heartbeat from core client for 30 sec - exiting 20:00:04 (5552): No heartbeat from core client for 30 sec - exiting 20:00:05 (5552): No heartbeat from core client for 30 sec - exiting 20:00:06 (5552): No heartbeat from core client for 30 sec - exiting 20:00:07 (5552): No heartbeat from core client for 30 sec - exiting 20:00:08 (5552): No heartbeat from core client for 30 sec - exiting 20:00:10 (5552): No heartbeat from core client for 30 sec - exiting 20:00:11 (5552): No heartbeat from core client for 30 sec - exiting 20:00:12 (5552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4700, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4700, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4700, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4700, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4700, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4700, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:41:53 (476): No heartbeat from core client for 30 sec - exiting 21:41:54 (476): No heartbeat from core client for 30 sec - exiting 21:41:56 (476): No heartbeat from core client for 30 sec - exiting 21:41:57 (476): No heartbeat from core client for 30 sec - exiting 21:41:58 (476): No heartbeat from core client for 30 sec - exiting 21:41:59 (476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:42:00 (476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/n1lnko.pj85c10 Error converting file to netcdf: dataout/n1lnko.pi85c10 Error converting file to netcdf: dataout/n1lnko.pf85c10 Error converting file to netcdf: dataout/n1lnka.ph85c10 Error converting file to netcdf: dataout/n1lnka.pg85c10 Error converting file to netcdf: dataout/n1lnka.pe85c10 Error converting file to netcdf: dataout/n1lnka.pd85c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:14:57 (3752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:03:27 (3752): No heartbeat from core client for 30 sec - exiting 10:03:29 (3752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:04:54 (4892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:23:09 (2768): No heartbeat from core client for 30 sec - exiting 19:23:10 (2768): No heartbeat from core client for 30 sec - exiting 19:23:11 (2768): No heartbeat from core client for 30 sec - exiting 19:23:12 (2768): No heartbeat from core client for 30 sec - exiting 19:23:13 (2768): No heartbeat from core client for 30 sec - exiting 19:23:14 (2768): No heartbeat from core client for 30 sec - exiting 19:23:15 (2768): No heartbeat from core client for 30 sec - exiting 19:23:17 (2768): No heartbeat from core client for 30 sec - exiting 19:23:18 (2768): No heartbeat from core client for 30 sec - exiting 19:23:19 (2768): No heartbeat from core client for 30 sec - exiting 19:23:20 (2768): No heartbeat from core client for 30 sec - exiting 19:23:21 (2768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x774F3AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x004703A7 write attempt to address 0x00B1D188 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x774F3AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Feb 2013 03:47:54	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	544,320	491,935	0.9038
12 Feb 2013 14:56:04	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	518,400	467,562	0.9019
11 Feb 2013 05:40:30	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	492,480	443,392	0.9003
10 Feb 2013 06:56:31	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	466,560	419,780	0.8997
09 Feb 2013 16:30:54	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	440,640	395,369	0.8973
07 Feb 2013 05:51:56	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	414,720	371,997	0.8970
06 Feb 2013 04:47:19	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	388,800	347,900	0.8948
04 Feb 2013 07:12:24	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	362,880	324,665	0.8947
29 Jan 2013 04:58:56	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	336,960	301,714	0.8954
27 Jan 2013 13:46:57	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	311,040	278,769	0.8962
25 Jan 2013 18:38:11	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	285,120	255,729	0.8969
25 Jan 2013 13:15:13	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	259,200	232,783	0.8981
24 Jan 2013 20:52:02	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	233,280	210,017	0.9003
24 Jan 2013 12:30:07	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	207,360	187,126	0.9024
24 Jan 2013 06:20:43	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	181,440	164,869	0.9087
23 Jan 2013 21:38:21	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	155,520	141,636	0.9107
23 Jan 2013 08:13:07	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	129,600	118,828	0.9169
23 Jan 2013 00:21:38	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	103,680	95,265	0.9188
22 Jan 2013 16:05:50	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	77,760	70,907	0.9119
22 Jan 2013 09:35:39	1252582	15554074	hadcm3n_n1ln_1880_40_008284371_2	51,840	47,618	0.9186