Task 15731798

Name	hadcm3n_n2vc_1880_40_008285212_1
Workunit	8436347
Created	18 Apr 2013, 2:20:48 UTC
Sent	18 Apr 2013, 2:24:15 UTC
Report deadline	18 Jul 2013, 9:51:26 UTC
Received	19 Sep 2013, 3:26:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1 (0xFFFFFFFF) Unknown error code
Computer ID	1155777
Run time	10 days 0 hours 53 min 50 sec
CPU time	9 days 13 hours 58 min
Validate state	Invalid
Credit	6,842.88
Device peak FLOPS	2.64 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code -1 (0xffffffff) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 10:37:23 (3488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:37:36 (3488): No heartbeat from core client for 30 sec - exiting 10:37:37 (3488): No heartbeat from core client for 30 sec - exiting 10:37:38 (3488): No heartbeat from core client for 30 sec - exiting 10:37:39 (3488): No heartbeat from core client for 30 sec - exiting 10:37:40 (3488): No heartbeat from core client for 30 sec - exiting 10:37:41 (3488): No heartbeat from core client for 30 sec - exiting 10:37:42 (3488): No heartbeat from core client for 30 sec - exiting 10:37:43 (3488): No heartbeat from core client for 30 sec - exiting 10:37:44 (3488): No heartbeat from core client for 30 sec - exiting 10:37:45 (3488): No heartbeat from core client for 30 sec - exiting 10:40:13 (904): No heartbeat from core client for 30 sec - exiting 10:40:14 (904): No heartbeat from core client for 30 sec - exiting 10:40:15 (904): No heartbeat from core client for 30 sec - exiting 10:40:16 (904): No heartbeat from core client for 30 sec - exiting 10:40:17 (904): No heartbeat from core client for 30 sec - exiting 10:40:18 (904): No heartbeat from core client for 30 sec - exiting 10:40:19 (904): No heartbeat from core client for 30 sec - exiting 10:40:20 (904): No heartbeat from core client for 30 sec - exiting 10:40:21 (904): No heartbeat from core client for 30 sec - exiting 10:40:22 (904): No heartbeat from core client for 30 sec - exiting 10:40:23 (904): No heartbeat from core client for 30 sec - exiting 10:40:24 (904): No heartbeat from core client for 30 sec - exiting 10:40:25 (904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/n2vcko.pj82c10 Error converting file to netcdf: dataout/n2vcko.pi82c10 Error converting file to netcdf: dataout/n2vcko.pf82c10 Error converting file to netcdf: dataout/n2vcka.ph82c10 Error converting file to netcdf: dataout/n2vcka.pg82c10 Error converting file to netcdf: dataout/n2vcka.pe82c10 Error converting file to netcdf: dataout/n2vcka.pd82c10 14:07:22 (4864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/n2vcko.pj82c10 Error converting file to netcdf: dataout/n2vcko.pi82c10 Error converting file to netcdf: dataout/n2vcko.pf82c10 Error converting file to netcdf: dataout/n2vcka.ph82c10 Error converting file to netcdf: dataout/n2vcka.pg82c10 Error converting file to netcdf: dataout/n2vcka.pe82c10 Error converting file to netcdf: dataout/n2vcka.pd82c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6192, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:30:17 (8032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:31:37 (8032): No heartbeat from core client for 30 sec - exiting 09:31:41 (8032): No heartbeat from core client for 30 sec - exiting 09:31:43 (8032): No heartbeat from core client for 30 sec - exiting 09:31:44 (8032): No heartbeat from core client for 30 sec - exiting 09:31:46 (8032): No heartbeat from core client for 30 sec - exiting 09:31:47 (8032): No heartbeat from core client for 30 sec - exiting 09:33:27 (7492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:13:30 (8516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:13:42 (8516): No heartbeat from core client for 30 sec - exiting 00:13:43 (8516): No heartbeat from core client for 30 sec - exiting 00:13:44 (8516): No heartbeat from core client for 30 sec - exiting 00:13:45 (8516): No heartbeat from core client for 30 sec - exiting 00:13:46 (8516): No heartbeat from core client for 30 sec - exiting 00:13:47 (8516): No heartbeat from core client for 30 sec - exiting 00:13:48 (8516): No heartbeat from core client for 30 sec - exiting 00:13:49 (8516): No heartbeat from core client for 30 sec - exiting 00:13:50 (8516): No heartbeat from core client for 30 sec - exiting 00:13:52 (8516): No heartbeat from core client for 30 sec - exiting 00:13:53 (8516): No heartbeat from core client for 30 sec - exiting 00:13:54 (8516): No heartbeat from core client for 30 sec - exiting 00:13:55 (8516): No heartbeat from core client for 30 sec - exiting 00:16:46 (1976): No heartbeat from core client for 30 sec - exiting 00:16:47 (1976): No heartbeat from core client for 30 sec - exiting 00:16:48 (1976): No heartbeat from core client for 30 sec - exiting 00:16:49 (1976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:06:59 (2832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:07:04 (2832): No heartbeat from core client for 30 sec - exiting 00:12:45 (7264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:58:17 (968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:58:21 (968): No heartbeat from core client for 30 sec - exiting 03:58:22 (968): No heartbeat from core client for 30 sec - exiting 03:58:23 (968): No heartbeat from core client for 30 sec - exiting 04:01:40 (6012): No heartbeat from core client for 30 sec - exiting 04:01:41 (6012): No heartbeat from core client for 30 sec - exiting 04:01:42 (6012): No heartbeat from core client for 30 sec - exiting 04:01:43 (6012): No heartbeat from core client for 30 sec - exiting 04:01:44 (6012): No heartbeat from core client for 30 sec - exiting 04:01:45 (6012): No heartbeat from core client for 30 sec - exiting 04:01:46 (6012): No heartbeat from core client for 30 sec - exiting 04:01:47 (6012): No heartbeat from core client for 30 sec - exiting 04:01:48 (6012): No heartbeat from core client for 30 sec - exiting 04:01:49 (6012): No heartbeat from core client for 30 sec - exiting 04:01:50 (6012): No heartbeat from core client for 30 sec - exiting 04:01:51 (6012): No heartbeat from core client for 30 sec - exiting 04:01:52 (6012): No heartbeat from core client for 30 sec - exiting 04:01:53 (6012): No heartbeat from core client for 30 sec - exiting 04:01:54 (6012): No heartbeat from core client for 30 sec - exiting 04:01:55 (6012): No heartbeat from core client for 30 sec - exiting 04:01:56 (6012): No heartbeat from core client for 30 sec - exiting 04:01:57 (6012): No heartbeat from core client for 30 sec - exiting 04:01:58 (6012): No heartbeat from core client for 30 sec - exiting 04:01:59 (6012): No heartbeat from core client for 30 sec - exiting 04:02:00 (6012): No heartbeat from core client for 30 sec - exiting 04:02:01 (6012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:37:35 (7164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:37:39 (7164): No heartbeat from core client for 30 sec - exiting 04:37:40 (7164): No heartbeat from core client for 30 sec - exiting 04:37:42 (7164): No heartbeat from core client for 30 sec - exiting 04:37:43 (7164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:05:49 (4076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:05:57 (4076): No heartbeat from core client for 30 sec - exiting 19:13:43 (5544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:32:46 (7660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:33:08 (7660): No heartbeat from core client for 30 sec - exiting 20:33:13 (7660): No heartbeat from core client for 30 sec - exiting 20:33:14 (7660): No heartbeat from core client for 30 sec - exiting 20:33:17 (7660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 21:19:36 (6952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:20:23 (6952): No heartbeat from core client for 30 sec - exiting 21:23:42 (5180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1084, iMonCtr=1 Model crash detected, will try to restart... 21:59:58 (1084): No heartbeat from core client for 30 sec - exiting 21:59:59 (1084): No heartbeat from core client for 30 sec - exiting 22:00:00 (1084): No heartbeat from core client for 30 sec - exiting 22:00:01 (1084): No heartbeat from core client for 30 sec - exiting 22:48:05 (7404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:48:09 (7404): No heartbeat from core client for 30 sec - exiting 22:48:10 (7404): No heartbeat from core client for 30 sec - exiting 22:48:11 (7404): No heartbeat from core client for 30 sec - exiting 22:49:48 (2668): No heartbeat from core client for 30 sec - exiting 22:49:49 (2668): No heartbeat from core client for 30 sec - exiting 22:49:50 (2668): No heartbeat from core client for 30 sec - exiting 22:49:51 (2668): No heartbeat from core client for 30 sec - exiting 22:49:52 (2668): No heartbeat from core client for 30 sec - exiting 22:49:53 (2668): No heartbeat from core client for 30 sec - exiting 22:49:54 (2668): No heartbeat from core client for 30 sec - exiting 22:49:55 (2668): No heartbeat from core client for 30 sec - exiting 22:49:56 (2668): No heartbeat from core client for 30 sec - exiting 22:49:57 (2668): No heartbeat from core client for 30 sec - exiting 22:49:58 (2668): No heartbeat from core client for 30 sec - exiting 22:49:59 (2668): No heartbeat from core client for 30 sec - exiting 22:50:00 (2668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 May 2013 20:44:06	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	570,240	819,510	1.4371
02 May 2013 18:51:52	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	544,320	783,718	1.4398
02 May 2013 07:45:30	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	518,400	745,570	1.4382
01 May 2013 20:15:59	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	492,480	707,711	1.4370
01 May 2013 09:24:05	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	466,560	670,543	1.4372
30 Apr 2013 21:40:25	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	440,640	633,015	1.4366
30 Apr 2013 10:56:09	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	414,720	595,291	1.4354
29 Apr 2013 09:32:58	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	388,800	557,324	1.4334
28 Apr 2013 18:55:56	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	362,880	520,915	1.4355
28 Apr 2013 08:10:56	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	336,960	485,014	1.4394
27 Apr 2013 19:09:08	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	311,040	448,291	1.4413
27 Apr 2013 07:07:27	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	285,120	410,060	1.4382
26 Apr 2013 19:52:30	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	259,200	372,488	1.4371
26 Apr 2013 09:08:05	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	233,280	334,837	1.4353
24 Apr 2013 17:51:42	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	207,360	296,936	1.4320
24 Apr 2013 06:59:30	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	181,440	259,548	1.4305
23 Apr 2013 20:09:21	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	155,520	222,230	1.4289
23 Apr 2013 05:58:11	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	129,600	184,228	1.4215
22 Apr 2013 16:41:21	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	103,680	147,004	1.4179
22 Apr 2013 06:18:00	1155777	15731798	hadcm3n_n2vc_1880_40_008285212_1	77,760	110,861	1.4257