Task 17250780

Name	hadcm3n_saq0_1940_40_009109152_0
Workunit	9239488
Created	22 Oct 2014, 14:13:02 UTC
Sent	27 Oct 2014, 1:45:18 UTC
Report deadline	26 Jan 2015, 9:12:29 UTC
Received	15 Nov 2014, 20:26:18 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1330529
Run time	13 days 15 hours 39 min 14 sec
CPU time	13 days 2 hours 36 min 35 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.26 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:37:37 (7576): No heartbeat from core client for 30 sec - exiting 08:37:38 (7576): No heartbeat from core client for 30 sec - exiting 08:37:39 (7576): No heartbeat from core client for 30 sec - exiting 08:37:40 (7576): No heartbeat from core client for 30 sec - exiting 08:37:41 (7576): No heartbeat from core client for 30 sec - exiting 08:37:42 (7576): No heartbeat from core client for 30 sec - exiting 08:37:43 (7576): No heartbeat from core client for 30 sec - exiting 08:37:44 (7576): No heartbeat from core client for 30 sec - exiting 08:37:45 (7576): No heartbeat from core client for 30 sec - exiting 08:38:19 (7576): No heartbeat from core client for 30 sec - exiting 08:38:20 (7576): No heartbeat from core client for 30 sec - exiting 08:38:21 (7576): No heartbeat from core client for 30 sec - exiting 08:38:22 (7576): No heartbeat from core client for 30 sec - exiting 08:38:23 (7576): No heartbeat from core client for 30 sec - exiting 08:38:25 (7576): No heartbeat from core client for 30 sec - exiting 08:38:26 (7576): No heartbeat from core client for 30 sec - exiting 08:38:27 (7576): No heartbeat from core client for 30 sec - exiting 08:38:28 (7576): No heartbeat from core client for 30 sec - exiting 08:38:29 (7576): No heartbeat from core client for 30 sec - exiting 08:38:30 (7576): No heartbeat from core client for 30 sec - exiting 08:38:31 (7576): No heartbeat from core client for 30 sec - exiting 08:38:32 (7576): No heartbeat from core client for 30 sec - exiting 08:38:33 (7576): No heartbeat from core client for 30 sec - exiting 08:38:34 (7576): No heartbeat from core client for 30 sec - exiting 08:38:35 (7576): No heartbeat from core client for 30 sec - exiting 08:38:37 (7576): No heartbeat from core client for 30 sec - exiting 08:38:38 (7576): No heartbeat from core client for 30 sec - exiting 08:38:39 (7576): No heartbeat from core client for 30 sec - exiting 08:38:40 (7576): No heartbeat from core client for 30 sec - exiting 08:38:41 (7576): No heartbeat from core client for 30 sec - exiting 08:38:42 (7576): No heartbeat from core client for 30 sec - exiting 08:38:43 (7576): No heartbeat from core client for 30 sec - exiting 08:38:44 (7576): No heartbeat from core client for 30 sec - exiting 08:38:45 (7576): No heartbeat from core client for 30 sec - exiting 08:38:47 (7576): No heartbeat from core client for 30 sec - exiting 08:38:48 (7576): No heartbeat from core client for 30 sec - exiting 08:38:49 (7576): No heartbeat from core client for 30 sec - exiting 08:38:50 (7576): No heartbeat from core client for 30 sec - exiting 08:38:51 (7576): No heartbeat from core client for 30 sec - exiting 08:38:52 (7576): No heartbeat from core client for 30 sec - exiting 08:38:53 (7576): No heartbeat from core client for 30 sec - exiting 08:38:54 (7576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8816, iMonCtr=1 Model crash detected, will try to restart... 19:19:00 (9272): No heartbeat from core client for 30 sec - exiting 19:19:01 (9272): No heartbeat from core client for 30 sec - exiting 19:19:02 (9272): No heartbeat from core client for 30 sec - exiting 19:19:03 (9272): No heartbeat from core client for 30 sec - exiting 19:19:05 (9272): No heartbeat from core client for 30 sec - exiting 19:19:06 (9272): No heartbeat from core client for 30 sec - exiting 19:19:07 (9272): No heartbeat from core client for 30 sec - exiting 19:19:08 (9272): No heartbeat from core client for 30 sec - exiting 19:19:09 (9272): No heartbeat from core client for 30 sec - exiting 19:19:10 (9272): No heartbeat from core client for 30 sec - exiting 19:19:11 (9272): No heartbeat from core client for 30 sec - exiting 19:19:12 (9272): No heartbeat from core client for 30 sec - exiting 19:19:13 (9272): No heartbeat from core client for 30 sec - exiting 19:19:14 (9272): No heartbeat from core client for 30 sec - exiting 19:19:15 (9272): No heartbeat from core client for 30 sec - exiting 19:19:17 (9272): No heartbeat from core client for 30 sec - exiting 19:19:18 (9272): No heartbeat from core client for 30 sec - exiting 19:19:19 (9272): No heartbeat from core client for 30 sec - exiting 19:19:20 (9272): No heartbeat from core client for 30 sec - exiting 19:19:21 (9272): No heartbeat from core client for 30 sec - exiting 19:19:22 (9272): No heartbeat from core client for 30 sec - exiting 19:19:23 (9272): No heartbeat from core client for 30 sec - exiting 19:19:24 (9272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:17:33 (6784): No heartbeat from core client for 30 sec - exiting 08:17:34 (6784): No heartbeat from core client for 30 sec - exiting 08:17:35 (6784): No heartbeat from core client for 30 sec - exiting 08:17:36 (6784): No heartbeat from core client for 30 sec - exiting 08:17:37 (6784): No heartbeat from core client for 30 sec - exiting 08:17:38 (6784): No heartbeat from core client for 30 sec - exiting 08:17:39 (6784): No heartbeat from core client for 30 sec - exiting 08:17:40 (6784): No heartbeat from core client for 30 sec - exiting 08:17:42 (6784): No heartbeat from core client for 30 sec - exiting 08:17:43 (6784): No heartbeat from core client for 30 sec - exiting 08:17:44 (6784): No heartbeat from core client for 30 sec - exiting 08:17:45 (6784): No heartbeat from core client for 30 sec - exiting 08:17:46 (6784): No heartbeat from core client for 30 sec - exiting 08:17:47 (6784): No heartbeat from core client for 30 sec - exiting 08:17:48 (6784): No heartbeat from core client for 30 sec - exiting 08:17:49 (6784): No heartbeat from core client for 30 sec - exiting 08:17:50 (6784): No heartbeat from core client for 30 sec - exiting 08:17:51 (6784): No heartbeat from core client for 30 sec - exiting 08:17:52 (6784): No heartbeat from core client for 30 sec - exiting 08:17:53 (6784): No heartbeat from core client for 30 sec - exiting 08:17:54 (6784): No heartbeat from core client for 30 sec - exiting 08:17:55 (6784): No heartbeat from core client for 30 sec - exiting 08:17:57 (6784): No heartbeat from core client for 30 sec - exiting 08:17:58 (6784): No heartbeat from core client for 30 sec - exiting 08:17:59 (6784): No heartbeat from core client for 30 sec - exiting 08:18:00 (6784): No heartbeat from core client for 30 sec - exiting 08:18:01 (6784): No heartbeat from core client for 30 sec - exiting 08:18:02 (6784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:30:58 (9772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:30:59 (9772): No heartbeat from core client for 30 sec - exiting 01:31:00 (9772): No heartbeat from core client for 30 sec - exiting 01:31:01 (9772): No heartbeat from core client for 30 sec - exiting 01:31:02 (9772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:15:22 (7396): No heartbeat from core client for 30 sec - exiting 17:15:23 (7396): No heartbeat from core client for 30 sec - exiting 17:15:24 (7396): No heartbeat from core client for 30 sec - exiting 17:15:25 (7396): No heartbeat from core client for 30 sec - exiting 17:15:26 (7396): No heartbeat from core client for 30 sec - exiting 17:15:27 (7396): No heartbeat from core client for 30 sec - exiting 17:15:28 (7396): No heartbeat from core client for 30 sec - exiting 17:15:29 (7396): No heartbeat from core client for 30 sec - exiting 17:15:30 (7396): No heartbeat from core client for 30 sec - exiting 17:15:31 (7396): No heartbeat from core client for 30 sec - exiting 17:15:32 (7396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:15:41 (9832): Can't acquire lockfile (32) - waiting 35s 19:14:47 (9832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:54:25 (8360): No heartbeat from core client for 30 sec - exiting 21:54:26 (8360): No heartbeat from core client for 30 sec - exiting 21:54:27 (8360): No heartbeat from core client for 30 sec - exiting 21:54:29 (8360): No heartbeat from core client for 30 sec - exiting 21:54:30 (8360): No heartbeat from core client for 30 sec - exiting 21:54:31 (8360): No heartbeat from core client for 30 sec - exiting 21:54:32 (8360): No heartbeat from core client for 30 sec - exiting 21:54:33 (8360): No heartbeat from core client for 30 sec - exiting 21:54:34 (8360): No heartbeat from core client for 30 sec - exiting 21:54:35 (8360): No heartbeat from core client for 30 sec - exiting 21:54:36 (8360): No heartbeat from core client for 30 sec - exiting 21:54:37 (8360): No heartbeat from core client for 30 sec - exiting 21:54:38 (8360): No heartbeat from core client for 30 sec - exiting 21:54:39 (8360): No heartbeat from core client for 30 sec - exiting 21:54:41 (8360): No heartbeat from core client for 30 sec - exiting 21:54:42 (8360): No heartbeat from core client for 30 sec - exiting 21:54:43 (8360): No heartbeat from core client for 30 sec - exiting 21:54:44 (8360): No heartbeat from core client for 30 sec - exiting 21:54:45 (8360): No heartbeat from core client for 30 sec - exiting 21:54:46 (8360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:54:47 (8360): No heartbeat from core client for 30 sec - exiting 21:54:48 (8360): No heartbeat from core client for 30 sec - exiting 21:54:49 (8360): No heartbeat from core client for 30 sec - exiting 21:54:50 (8360): No heartbeat from core client for 30 sec - exiting 21:54:51 (8360): No heartbeat from core client for 30 sec - exiting 21:54:53 (8360): No heartbeat from core client for 30 sec - exiting 21:55:44 (8636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:01:21 (9212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:38:28 (7772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:50:42 (6864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:31:28 (5796): No heartbeat from core client for 30 sec - exiting 17:31:29 (5796): No heartbeat from core client for 30 sec - exiting 17:31:30 (5796): No heartbeat from core client for 30 sec - exiting 17:31:31 (5796): No heartbeat from core client for 30 sec - exiting 17:31:32 (5796): No heartbeat from core client for 30 sec - exiting 17:31:33 (5796): No heartbeat from core client for 30 sec - exiting 17:31:34 (5796): No heartbeat from core client for 30 sec - exiting 17:31:35 (5796): No heartbeat from core client for 30 sec - exiting 17:31:36 (5796): No heartbeat from core client for 30 sec - exiting 17:31:37 (5796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/saq0ko.pjg6c10 Error converting file to netcdf: dataout/saq0ko.pig6c10 Error converting file to netcdf: dataout/saq0ko.pfg6c10 Error converting file to netcdf: dataout/saq0ka.phg6c10 Error converting file to netcdf: dataout/saq0ka.pgg6c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=43500, iMonCtr=1 Model crash detected, will try to restart... 07:25:42 (9532): No heartbeat from core client for 30 sec - exiting 07:25:43 (9532): No heartbeat from core client for 30 sec - exiting 07:25:44 (9532): No heartbeat from core client for 30 sec - exiting 07:25:45 (9532): No heartbeat from core client for 30 sec - exiting 07:25:46 (9532): No heartbeat from core client for 30 sec - exiting 07:25:48 (9532): No heartbeat from core client for 30 sec - exiting 07:25:49 (9532): No heartbeat from core client for 30 sec - exiting 07:25:50 (9532): No heartbeat from core client for 30 sec - exiting 07:25:51 (9532): No heartbeat from core client for 30 sec - exiting 07:25:52 (9532): No heartbeat from core client for 30 sec - exiting 07:25:53 (9532): No heartbeat from core client for 30 sec - exiting 07:25:54 (9532): No heartbeat from core client for 30 sec - exiting 07:25:55 (9532): No heartbeat from core client for 30 sec - exiting 07:25:56 (9532): No heartbeat from core client for 30 sec - exiting 07:25:57 (9532): No heartbeat from core client for 30 sec - exiting 07:25:58 (9532): No heartbeat from core client for 30 sec - exiting 07:26:00 (9532): No heartbeat from core client for 30 sec - exiting 07:26:01 (9532): No heartbeat from core client for 30 sec - exiting 07:26:02 (9532): No heartbeat from core client for 30 sec - exiting 07:26:03 (9532): No heartbeat from core client for 30 sec - exiting 07:26:04 (9532): No heartbeat from core client for 30 sec - exiting 07:26:05 (9532): No heartbeat from core client for 30 sec - exiting 07:26:06 (9532): No heartbeat from core client for 30 sec - exiting 07:26:07 (9532): No heartbeat from core client for 30 sec - exiting 07:26:08 (9532): No heartbeat from core client for 30 sec - exiting 07:26:09 (9532): No heartbeat from core client for 30 sec - exiting 07:26:10 (9532): No heartbeat from core client for 30 sec - exiting 07:26:12 (9532): No heartbeat from core client for 30 sec - exiting 07:26:13 (9532): No heartbeat from core client for 30 sec - exiting 07:26:14 (9532): No heartbeat from core client for 30 sec - exiting 07:26:15 (9532): No heartbeat from core client for 30 sec - exiting 07:26:16 (9532): No heartbeat from core client for 30 sec - exiting 07:26:17 (9532): No heartbeat from core client for 30 sec - exiting 07:26:18 (9532): No heartbeat from core client for 30 sec - exiting 07:26:19 (9532): No heartbeat from core client for 30 sec - exiting 07:26:20 (9532): No heartbeat from core client for 30 sec - exiting 07:26:21 (9532): No heartbeat from core client for 30 sec - exiting 07:26:23 (9532): No heartbeat from core client for 30 sec - exiting 07:26:24 (9532): No heartbeat from core client for 30 sec - exiting 07:26:25 (9532): No heartbeat from core client for 30 sec - exiting 07:26:26 (9532): No heartbeat from core client for 30 sec - exiting 07:26:27 (9532): No heartbeat from core client for 30 sec - exiting 07:26:28 (9532): No heartbeat from core client for 30 sec - exiting 07:26:29 (9532): No heartbeat from core client for 30 sec - exiting 07:26:30 (9532): No heartbeat from core client for 30 sec - exiting 07:26:31 (9532): No heartbeat from core client for 30 sec - exiting 07:26:33 (9532): No heartbeat from core client for 30 sec - exiting 07:26:34 (9532): No heartbeat from core client for 30 sec - exiting 07:26:35 (9532): No heartbeat from core client for 30 sec - exiting 07:26:36 (9532): No heartbeat from core client for 30 sec - exiting 07:26:37 (9532): No heartbeat from core client for 30 sec - exiting 07:26:38 (9532): No heartbeat from core client for 30 sec - exiting 07:26:39 (9532): No heartbeat from core client for 30 sec - exiting 07:26:40 (9532): No heartbeat from core client for 30 sec - exiting 07:26:41 (9532): No heartbeat from core client for 30 sec - exiting 07:26:42 (9532): No heartbeat from core client for 30 sec - exiting 07:26:43 (9532): No heartbeat from core client for 30 sec - exiting 07:26:45 (9532): No heartbeat from core client for 30 sec - exiting 07:26:46 (9532): No heartbeat from core client for 30 sec - exiting 07:26:47 (9532): No heartbeat from core client for 30 sec - exiting 07:26:48 (9532): No heartbeat from core client for 30 sec - exiting 07:26:49 (9532): No heartbeat from core client for 30 sec - exiting 07:26:50 (9532): No heartbeat from core client for 30 sec - exiting 07:26:51 (9532): No heartbeat from core client for 30 sec - exiting 07:26:52 (9532): No heartbeat from core client for 30 sec - exiting 07:26:53 (9532): No heartbeat from core client for 30 sec - exiting 07:26:54 (9532): No heartbeat from core client for 30 sec - exiting 07:26:55 (9532): No heartbeat from core client for 30 sec - exiting 07:26:57 (9532): No heartbeat from core client for 30 sec - exiting 07:26:58 (9532): No heartbeat from core client for 30 sec - exiting 07:26:59 (9532): No heartbeat from core client for 30 sec - exiting 07:27:00 (9532): No heartbeat from core client for 30 sec - exiting 07:27:01 (9532): No heartbeat from core client for 30 sec - exiting 07:27:02 (9532): No heartbeat from core client for 30 sec - exiting 07:27:03 (9532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:29:56 (9276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 09:14:34 (9156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:32:17 (9324): No heartbeat from core client for 30 sec - exiting 17:32:18 (9324): No heartbeat from core client for 30 sec - exiting 17:32:19 (9324): No heartbeat from core client for 30 sec - exiting 17:32:20 (9324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:31:06 (9124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:31:07 (9124): No heartbeat from core client for 30 sec - exiting 01:31:08 (9124): No heartbeat from core client for 30 sec - exiting 01:31:09 (9124): No heartbeat from core client for 30 sec - exiting 01:31:10 (9124): No heartbeat from core client for 30 sec - exiting 01:31:11 (9124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 13:46:45 (7472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:46:46 (7472): No heartbeat from core client for 30 sec - exiting 13:46:47 (7472): No heartbeat from core client for 30 sec - exiting 13:46:48 (7472): No heartbeat from core client for 30 sec - exiting 13:46:49 (7472): No heartbeat from core client for 30 sec - exiting 13:46:50 (7472): No heartbeat from core client for 30 sec - exiting 13:46:51 (7472): No heartbeat from core client for 30 sec - exiting 13:46:52 (7472): No heartbeat from core client for 30 sec - exiting 13:46:53 (7472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 20:01:13 (8624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:01:14 (8624): No heartbeat from core client for 30 sec - exiting 09:01:14 (9780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:01:15 (9780): No heartbeat from core client for 30 sec - exiting 09:01:16 (9780): No heartbeat from core client for 30 sec - exiting 09:01:17 (9780): No heartbeat from core client for 30 sec - exiting 09:01:18 (9780): No heartbeat from core client for 30 sec - exiting 09:01:19 (9780): No heartbeat from core client for 30 sec - exiting 09:01:20 (9780): No heartbeat from core client for 30 sec - exiting 09:01:21 (9780): No heartbeat from core client for 30 sec - exiting 09:01:22 (9780): No heartbeat from core client for 30 sec - exiting Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Nov 2014 19:26:09	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	1,036,800	1,132,349	1.0922
15 Nov 2014 14:56:41	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	1,010,880	1,116,013	1.1040
15 Nov 2014 10:22:07	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	984,960	1,099,655	1.1164
15 Nov 2014 05:26:57	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	959,040	1,082,093	1.1283
14 Nov 2014 23:55:13	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	933,120	1,065,552	1.1419
14 Nov 2014 19:10:13	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	907,200	1,048,787	1.1561
14 Nov 2014 09:53:46	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	881,280	1,019,220	1.1565
14 Nov 2014 07:23:37	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	855,360	994,408	1.1626
13 Nov 2014 20:15:05	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	829,440	973,285	1.1734
13 Nov 2014 15:56:08	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	803,520	927,055	1.1537
11 Nov 2014 23:38:17	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	777,600	875,669	1.1261
11 Nov 2014 11:13:07	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	751,680	837,328	1.1139
10 Nov 2014 04:48:14	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	725,760	785,592	1.0824
09 Nov 2014 07:49:34	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	699,840	739,315	1.0564
08 Nov 2014 03:06:05	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	673,920	682,277	1.0124
07 Nov 2014 16:13:49	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	648,000	654,788	1.0105
07 Nov 2014 08:57:23	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	622,080	628,986	1.0111
06 Nov 2014 21:59:32	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	596,160	606,851	1.0179
06 Nov 2014 15:19:45	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	570,240	586,038	1.0277
06 Nov 2014 07:41:14	1330529	17250780	hadcm3n_saq0_1940_40_009109152_0	544,320	558,856	1.0267