Task 16697460

Name	hadcm3n_876s_1980_40_008515819_3
Workunit	8663331
Created	1 Jul 2014, 3:52:38 UTC
Sent	1 Jul 2014, 3:52:43 UTC
Report deadline	30 Sep 2014, 11:19:54 UTC
Received	23 Jul 2014, 15:16:18 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1284999
Run time	16 days 7 hours 53 min 55 sec
CPU time	15 days 4 hours 20 min 51 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.98 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:14:13 (4060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:18:30 (3748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3084, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 19:43:12 (7636): Can't acquire lockfile (32) - waiting 35s 19:43:20 (2060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2252, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:41:57 (6932): No heartbeat from core client for 30 sec - exiting 19:41:58 (6932): No heartbeat from core client for 30 sec - exiting 19:41:59 (6932): No heartbeat from core client for 30 sec - exiting 19:42:00 (6932): No heartbeat from core client for 30 sec - exiting 19:42:01 (6932): No heartbeat from core client for 30 sec - exiting 19:42:02 (6932): No heartbeat from core client for 30 sec - exiting 19:42:03 (6932): No heartbeat from core client for 30 sec - exiting 19:42:05 (6932): No heartbeat from core client for 30 sec - exiting 19:42:06 (6932): No heartbeat from core client for 30 sec - exiting 19:42:07 (6932): No heartbeat from core client for 30 sec - exiting 19:42:08 (6932): No heartbeat from core client for 30 sec - exiting 19:42:09 (6932): No heartbeat from core client for 30 sec - exiting 19:42:10 (6932): No heartbeat from core client for 30 sec - exiting 19:42:11 (6932): No heartbeat from core client for 30 sec - exiting 19:42:12 (6932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:51:22 (1912): No heartbeat from core client for 30 sec - exiting 11:51:24 (1912): No heartbeat from core client for 30 sec - exiting 11:51:25 (1912): No heartbeat from core client for 30 sec - exiting 11:51:26 (1912): No heartbeat from core client for 30 sec - exiting 11:51:27 (1912): No heartbeat from core client for 30 sec - exiting 11:51:28 (1912): No heartbeat from core client for 30 sec - exiting 11:51:29 (1912): No heartbeat from core client for 30 sec - exiting 11:51:30 (1912): No heartbeat from core client for 30 sec - exiting 11:51:31 (1912): No heartbeat from core client for 30 sec - exiting 11:51:32 (1912): No heartbeat from core client for 30 sec - exiting 11:51:33 (1912): No heartbeat from core client for 30 sec - exiting 11:51:34 (1912): No heartbeat from core client for 30 sec - exiting 11:51:35 (1912): No heartbeat from core client for 30 sec - exiting 11:51:36 (1912): No heartbeat from core client for 30 sec - exiting 11:51:37 (1912): No heartbeat from core client for 30 sec - exiting 11:51:38 (1912): No heartbeat from core client for 30 sec - exiting 11:51:39 (1912): No heartbeat from core client for 30 sec - exiting 11:51:40 (1912): No heartbeat from core client for 30 sec - exiting 11:51:41 (1912): No heartbeat from core client for 30 sec - exiting 11:51:42 (1912): No heartbeat from core client for 30 sec - exiting 11:51:43 (1912): No heartbeat from core client for 30 sec - exiting 11:51:44 (1912): No heartbeat from core client for 30 sec - exiting 11:51:45 (1912): No heartbeat from core client for 30 sec - exiting 11:51:46 (1912): No heartbeat from core client for 30 sec - exiting 11:51:47 (1912): No heartbeat from core client for 30 sec - exiting 11:51:48 (1912): No heartbeat from core client for 30 sec - exiting 11:51:49 (1912): No heartbeat from core client for 30 sec - exiting 11:51:50 (1912): No heartbeat from core client for 30 sec - exiting 11:51:51 (1912): No heartbeat from core client for 30 sec - exiting 11:51:52 (1912): No heartbeat from core client for 30 sec - exiting 11:51:53 (1912): No heartbeat from core client for 30 sec - exiting 11:51:54 (1912): No heartbeat from core client for 30 sec - exiting 11:51:55 (1912): No heartbeat from core client for 30 sec - exiting 11:51:56 (1912): No heartbeat from core client for 30 sec - exiting 11:51:57 (1912): No heartbeat from core client for 30 sec - exiting 11:51:58 (1912): No heartbeat from core client for 30 sec - exiting 11:51:59 (1912): No heartbeat from core client for 30 sec - exiting 11:52:00 (1912): No heartbeat from core client for 30 sec - exiting 11:52:01 (1912): No heartbeat from core client for 30 sec - exiting 11:52:02 (1912): No heartbeat from core client for 30 sec - exiting 11:52:03 (1912): No heartbeat from core client for 30 sec - exiting 11:52:04 (1912): No heartbeat from core client for 30 sec - exiting 11:52:05 (1912): No heartbeat from core client for 30 sec - exiting 11:52:06 (1912): No heartbeat from core client for 30 sec - exiting 11:52:07 (1912): No heartbeat from core client for 30 sec - exiting 11:52:08 (1912): No heartbeat from core client for 30 sec - exiting 11:52:09 (1912): No heartbeat from core client for 30 sec - exiting 11:52:10 (1912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Jul 2014 14:22:12	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	777,600	1,311,645	1.6868
23 Jul 2014 00:33:23	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	751,680	1,268,355	1.6874
22 Jul 2014 13:34:27	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	725,760	1,224,406	1.6871
22 Jul 2014 13:34:27	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	699,840	1,180,745	1.6872
21 Jul 2014 01:50:57	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	673,920	1,137,348	1.6877
20 Jul 2014 11:07:11	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	648,000	1,093,177	1.6870
19 Jul 2014 19:56:07	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	622,080	1,048,979	1.6862
19 Jul 2014 05:49:26	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	596,160	1,005,301	1.6863
18 Jul 2014 12:14:39	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	570,240	961,685	1.6865
17 Jul 2014 21:08:39	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	544,320	918,131	1.6867
17 Jul 2014 05:35:00	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	518,400	874,758	1.6874
16 Jul 2014 12:29:24	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	492,480	830,920	1.6872
15 Jul 2014 22:27:54	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	466,560	787,172	1.6872
15 Jul 2014 08:18:48	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	440,640	743,413	1.6871
14 Jul 2014 14:52:13	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	414,720	699,808	1.6874
14 Jul 2014 01:18:36	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	388,800	656,034	1.6873
13 Jul 2014 08:21:26	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	362,880	612,235	1.6872
12 Jul 2014 18:35:44	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	336,960	568,365	1.6867
12 Jul 2014 02:15:00	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	311,040	524,488	1.6862
11 Jul 2014 09:19:31	1284999	16697460	hadcm3n_876s_1980_40_008515819_3	285,120	480,303	1.6846