Task 17432856

Name	hadcm3n_xcog_1940_40_009152726_2
Workunit	9283062
Created	19 Nov 2014, 5:01:43 UTC
Sent	19 Nov 2014, 5:02:00 UTC
Report deadline	18 Feb 2015, 12:29:11 UTC
Received	27 Nov 2014, 2:57:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1296791
Run time	3 days 20 hours 38 min 51 sec
CPU time	3 days 18 hours 29 min 58 sec
Validate state	Invalid
Credit	5,909.76
Device peak FLOPS	4.97 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.4.27</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 03:25:31 (6348): No heartbeat from core client for 30 sec - exiting 03:25:32 (6348): No heartbeat from core client for 30 sec - exiting 03:25:33 (6348): No heartbeat from core client for 30 sec - exiting 03:25:34 (6348): No heartbeat from core client for 30 sec - exiting 03:25:35 (6348): No heartbeat from core client for 30 sec - exiting 03:25:36 (6348): No heartbeat from core client for 30 sec - exiting 03:25:37 (6348): No heartbeat from core client for 30 sec - exiting 03:25:38 (6348): No heartbeat from core client for 30 sec - exiting 03:25:39 (6348): No heartbeat from core client for 30 sec - exiting 03:25:40 (6348): No heartbeat from core client for 30 sec - exiting 03:25:41 (6348): No heartbeat from core client for 30 sec - exiting 03:25:42 (6348): No heartbeat from core client for 30 sec - exiting 03:25:43 (6348): No heartbeat from core client for 30 sec - exiting 03:25:44 (6348): No heartbeat from core client for 30 sec - exiting 03:25:45 (6348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:51:06 (6648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 05:50:52 (7136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:19:16 (5652): No heartbeat from core client for 30 sec - exiting 07:19:17 (5652): No heartbeat from core client for 30 sec - exiting 07:19:18 (5652): No heartbeat from core client for 30 sec - exiting 07:19:19 (5652): No heartbeat from core client for 30 sec - exiting 07:19:20 (5652): No heartbeat from core client for 30 sec - exiting 07:19:21 (5652): No heartbeat from core client for 30 sec - exiting 07:19:22 (5652): No heartbeat from core client for 30 sec - exiting 07:19:23 (5652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:20:47 (4488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:37:19 (5256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:28:15 (6312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:44:09 (3956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:36:20 (9000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:29:19 (6672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:09:26 (7464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:15:27 (8916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:34:28 (5984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/xcogko.pjf8c10 Error converting file to netcdf: dataout/xcogko.pif8c10 Error converting file to netcdf: dataout/xcogko.pff8c10 Error converting file to netcdf: dataout/xcogka.phf8c10 Error converting file to netcdf: dataout/xcogka.pgf8c10 Error converting file to netcdf: dataout/xcogka.pef8c10 Error converting file to netcdf: dataout/xcogka.pdf8c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:16:07 (7192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:37:23 (1536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:38:03 (7648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:39:41 (7556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:40:23 (908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:45:46 (1460): No heartbeat from core client for 30 sec - exiting 10:45:47 (1460): No heartbeat from core client for 30 sec - exiting 10:45:48 (1460): No heartbeat from core client for 30 sec - exiting 10:45:49 (1460): No heartbeat from core client for 30 sec - exiting 10:45:50 (1460): No heartbeat from core client for 30 sec - exiting 10:45:51 (1460): No heartbeat from core client for 30 sec - exiting 10:45:52 (1460): No heartbeat from core client for 30 sec - exiting 10:45:53 (1460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:54:22 (7708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:02:27 (1476): No heartbeat from core client for 30 sec - exiting 11:02:31 (1476): No heartbeat from core client for 30 sec - exiting 11:02:32 (1476): No heartbeat from core client for 30 sec - exiting 11:02:33 (1476): No heartbeat from core client for 30 sec - exiting 11:02:34 (1476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:04:56 (5292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:48:16 (2456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:58:21 (640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:00:34 (6896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:04:25 (7172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:28:21 (3028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:44:01 (6236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1328, iMonCtr=1 Model crash detected, will try to restart... 21:32:42 (1328): No heartbeat from core client for 30 sec - exiting 21:32:43 (1328): No heartbeat from core client for 30 sec - exiting 21:32:44 (1328): No heartbeat from core client for 30 sec - exiting 21:32:45 (1328): No heartbeat from core client for 30 sec - exiting forrtl: Access is denied. 21:32:46 (1328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:32:47 (1328): No heartbeat from core client for 30 sec - exiting 21:32:48 (1328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:07:05 (6028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:07:11 (3272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:34:10 (3724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:03:27 (5168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:30:49 (6232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:46:55 (5844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:48:09 (7040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:48:01 (3688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:45:26 (4640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:37:27 (7040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:38:09 (6404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:13:13 (4140): No heartbeat from core client for 30 sec - exiting 10:13:14 (4140): No heartbeat from core client for 30 sec - exiting 10:13:15 (4140): No heartbeat from core client for 30 sec - exiting 10:13:16 (4140): No heartbeat from core client for 30 sec - exiting 10:13:17 (4140): No heartbeat from core client for 30 sec - exiting 10:13:18 (4140): No heartbeat from core client for 30 sec - exiting 10:13:19 (4140): No heartbeat from core client for 30 sec - exiting 10:13:20 (4140): No heartbeat from core client for 30 sec - exiting 10:13:21 (4140): No heartbeat from core client for 30 sec - exiting 10:13:22 (4140): No heartbeat from core client for 30 sec - exiting 10:13:23 (4140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:24:24 (4832): No heartbeat from core client for 30 sec - exiting 10:24:35 (4832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:27:02 (4756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:37:15 (5692): No heartbeat from core client for 30 sec - exiting 10:37:16 (5692): No heartbeat from core client for 30 sec - exiting 10:37:17 (5692): No heartbeat from core client for 30 sec - exiting 10:37:18 (5692): No heartbeat from core client for 30 sec - exiting 10:37:19 (5692): No heartbeat from core client for 30 sec - exiting 10:37:20 (5692): No heartbeat from core client for 30 sec - exiting 10:37:21 (5692): No heartbeat from core client for 30 sec - exiting 10:37:22 (5692): No heartbeat from core client for 30 sec - exiting 10:37:23 (5692): No heartbeat from core client for 30 sec - exiting 10:37:24 (5692): No heartbeat from core client for 30 sec - exiting 10:37:25 (5692): No heartbeat from core client for 30 sec - exiting 10:37:26 (5692): No heartbeat from core client for 30 sec - exiting 10:37:27 (5692): No heartbeat from core client for 30 sec - exiting 10:37:28 (5692): No heartbeat from core client for 30 sec - exiting 10:37:29 (5692): No heartbeat from core client for 30 sec - exiting 10:37:30 (5692): No heartbeat from core client for 30 sec - exiting 10:37:31 (5692): No heartbeat from core client for 30 sec - exiting 10:37:32 (5692): No heartbeat from core client for 30 sec - exiting 10:37:33 (5692): No heartbeat from core client for 30 sec - exiting 10:37:34 (5692): No heartbeat from core client for 30 sec - exiting 10:37:35 (5692): No heartbeat from core client for 30 sec - exiting 10:37:36 (5692): No heartbeat from core client for 30 sec - exiting 10:37:37 (5692): No heartbeat from core client for 30 sec - exiting 10:37:38 (5692): No heartbeat from core client for 30 sec - exiting 10:37:39 (5692): No heartbeat from core client for 30 sec - exiting 10:37:40 (5692): No heartbeat from core client for 30 sec - exiting 10:37:41 (5692): No heartbeat from core client for 30 sec - exiting 10:37:42 (5692): No heartbeat from core client for 30 sec - exiting 10:37:43 (5692): No heartbeat from core client for 30 sec - exiting 10:37:44 (5692): No heartbeat from core client for 30 sec - exiting 10:37:45 (5692): No heartbeat from core client for 30 sec - exiting 10:37:46 (5692): No heartbeat from core client for 30 sec - exiting 10:37:47 (5692): No heartbeat from core client for 30 sec - exiting 10:37:48 (5692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:37:49 (5692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 10:44:26 (5860): No heartbeat from core client for 30 sec - exiting 10:44:27 (5860): No heartbeat from core client for 30 sec - exiting 10:44:28 (5860): No heartbeat from core client for 30 sec - exiting 10:44:29 (5860): No heartbeat from core client for 30 sec - exiting 10:44:30 (5860): No heartbeat from core client for 30 sec - exiting 10:44:31 (5860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:13:22 (264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:38:50 (5144): No heartbeat from core client for 30 sec - exiting 17:38:51 (5144): No heartbeat from core client for 30 sec - exiting 17:38:52 (5144): No heartbeat from core client for 30 sec - exiting 17:38:53 (5144): No heartbeat from core client for 30 sec - exiting 17:38:54 (5144): No heartbeat from core client for 30 sec - exiting 17:38:55 (5144): No heartbeat from core client for 30 sec - exiting 17:38:56 (5144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:58:19 (7532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7116, iMonCtr=1 Model crash detected, will try to restart... 19:25:59 (7116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11616, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11616, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11616, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11616, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
24 Nov 2014 00:06:15	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	492,480	324,903	0.6597
23 Nov 2014 10:38:41	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	466,560	305,626	0.6551
23 Nov 2014 02:38:17	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	440,640	285,844	0.6487
22 Nov 2014 10:00:58	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	414,720	267,125	0.6441
22 Nov 2014 05:09:30	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	388,800	249,823	0.6425
21 Nov 2014 22:45:47	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	362,880	232,354	0.6403
21 Nov 2014 17:54:26	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	336,960	214,892	0.6377
21 Nov 2014 12:52:04	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	311,040	197,081	0.6336
21 Nov 2014 08:00:18	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	285,120	179,516	0.6296
21 Nov 2014 03:03:58	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	259,200	162,092	0.6254
20 Nov 2014 21:41:41	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	233,280	144,750	0.6205
20 Nov 2014 17:11:34	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	207,360	128,538	0.6199
20 Nov 2014 12:24:36	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	181,440	111,500	0.6145
20 Nov 2014 08:39:42	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	155,520	98,001	0.6302
20 Nov 2014 04:54:23	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	129,600	84,463	0.6517
20 Nov 2014 00:44:30	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	103,680	69,607	0.6714
19 Nov 2014 21:04:19	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	77,760	53,771	0.6915
19 Nov 2014 15:37:23	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	51,840	37,055	0.7148
19 Nov 2014 11:08:26	1296791	17432856	hadcm3n_xcog_1940_40_009152726_2	25,920	19,251	0.7427