Task 16061126

Name	hadcm3n_of82_1900_40_008474853_1
Workunit	8625692
Created	8 Oct 2013, 5:49:06 UTC
Sent	8 Oct 2013, 6:25:15 UTC
Report deadline	7 Jan 2014, 13:52:26 UTC
Received	2 Oct 2014, 21:03:06 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	1 (0x00000001) Unknown error code
Computer ID	1274843
Run time	6 days 17 hours 33 min 27 sec
CPU time	5 days 9 hours 47 min 11 sec
Validate state	Invalid
Credit	2,799.36
Device peak FLOPS	2.52 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> Incorrect function. (0x1) - exit code 1 (0x1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2252, iMonCtr=1 Model crash detected, will try to restart... 22:30:47 (2252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:38:09 (4648): No heartbeat from core client for 30 sec - exiting 06:38:10 (4648): No heartbeat from core client for 30 sec - exiting 06:38:11 (4648): No heartbeat from core client for 30 sec - exiting 06:38:12 (4648): No heartbeat from core client for 30 sec - exiting 06:38:13 (4648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5068, selfPID=5068, iMonCtr=1 08:52:44 (4588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3752, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3752, iMonCtr=1 Model crash detected, will try to restart... 11:10:09 (3752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:47:04 (3588): No heartbeat from core client for 30 sec - exiting 17:47:05 (3588): No heartbeat from core client for 30 sec - exiting 17:47:06 (3588): No heartbeat from core client for 30 sec - exiting 17:47:07 (3588): No heartbeat from core client for 30 sec - exiting 17:47:09 (3588): No heartbeat from core client for 30 sec - exiting 17:47:10 (3588): No heartbeat from core client for 30 sec - exiting 17:47:11 (3588): No heartbeat from core client for 30 sec - exiting 17:47:12 (3588): No heartbeat from core client for 30 sec - exiting 17:47:13 (3588): No heartbeat from core client for 30 sec - exiting 17:47:14 (3588): No heartbeat from core client for 30 sec - exiting 17:47:15 (3588): No heartbeat from core client for 30 sec - exiting 17:47:16 (3588): No heartbeat from core client for 30 sec - exiting 17:47:17 (3588): No heartbeat from core client for 30 sec - exiting 17:47:18 (3588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3668, iMonCtr=1 Model crash detected, will try to restart... 17:48:22 (3668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:16:55 (4564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4792, selfPID=4792, iMonCtr=1 05:16:33 (3164): No heartbeat from core client for 30 sec - exiting 05:16:34 (3164): No heartbeat from core client for 30 sec - exiting 05:16:35 (3164): No heartbeat from core client for 30 sec - exiting 05:16:37 (3164): No heartbeat from core client for 30 sec - exiting 05:16:38 (3164): No heartbeat from core client for 30 sec - exiting 05:16:39 (3164): No heartbeat from core client for 30 sec - exiting 05:16:40 (3164): No heartbeat from core client for 30 sec - exiting 05:16:41 (3164): No heartbeat from core client for 30 sec - exiting 05:16:42 (3164): No heartbeat from core client for 30 sec - exiting 05:16:43 (3164): No heartbeat from core client for 30 sec - exiting 05:16:44 (3164): No heartbeat from core client for 30 sec - exiting 05:16:45 (3164): No heartbeat from core client for 30 sec - exiting 05:16:46 (3164): No heartbeat from core client for 30 sec - exiting 05:16:47 (3164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4704, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4704, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4780, selfPID=4780, iMonCtr=1 07:57:05 (4564): No heartbeat from core client for 30 sec - exiting 07:57:06 (4564): No heartbeat from core client for 30 sec - exiting 07:57:07 (4564): No heartbeat from core client for 30 sec - exiting 07:57:09 (4564): No heartbeat from core client for 30 sec - exiting 07:57:10 (4564): No heartbeat from core client for 30 sec - exiting 07:57:11 (4564): No heartbeat from core client for 30 sec - exiting 07:57:12 (4564): No heartbeat from core client for 30 sec - exiting 07:57:13 (4564): No heartbeat from core client for 30 sec - exiting 07:57:14 (4564): No heartbeat from core client for 30 sec - exiting 07:57:15 (4564): No heartbeat from core client for 30 sec - exiting 07:57:16 (4564): No heartbeat from core client for 30 sec - exiting 07:57:17 (4564): No heartbeat from core client for 30 sec - exiting 07:57:18 (4564): No heartbeat from core client for 30 sec - exiting 07:57:20 (4564): No heartbeat from core client for 30 sec - exiting 07:57:21 (4564): No heartbeat from core client for 30 sec - exiting 07:57:22 (4564): No heartbeat from core client for 30 sec - exiting 07:57:23 (4564): No heartbeat from core client for 30 sec - exiting 07:57:24 (4564): No heartbeat from core client for 30 sec - exiting 07:57:25 (4564): No heartbeat from core client for 30 sec - exiting 07:57:26 (4564): No heartbeat from core client for 30 sec - exiting 07:57:27 (4564): No heartbeat from core client for 30 sec - exiting 07:57:28 (4564): No heartbeat from core client for 30 sec - exiting 07:57:29 (4564): No heartbeat from core client for 30 sec - exiting 07:57:30 (4564): No heartbeat from core client for 30 sec - exiting 07:57:32 (4564): No heartbeat from core client for 30 sec - exiting 07:57:33 (4564): No heartbeat from core client for 30 sec - exiting 07:57:34 (4564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:59:15 (4652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1488, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1488, iMonCtr=1 Model crash detected, will try to restart... 10:30:08 (4304): No heartbeat from core client for 30 sec - exiting 10:30:10 (4304): No heartbeat from core client for 30 sec - exiting 10:30:11 (4304): No heartbeat from core client for 30 sec - exiting 10:30:12 (4304): No heartbeat from core client for 30 sec - exiting 10:30:13 (4304): No heartbeat from core client for 30 sec - exiting 10:30:14 (4304): No heartbeat from core client for 30 sec - exiting 10:30:15 (4304): No heartbeat from core client for 30 sec - exiting 10:30:16 (4304): No heartbeat from core client for 30 sec - exiting 10:30:18 (4304): No heartbeat from core client for 30 sec - exiting 10:30:19 (4304): No heartbeat from core client for 30 sec - exiting 10:30:20 (4304): No heartbeat from core client for 30 sec - exiting 10:30:21 (4304): No heartbeat from core client for 30 sec - exiting 10:30:22 (4304): No heartbeat from core client for 30 sec - exiting 10:30:23 (4304): No heartbeat from core client for 30 sec - exiting 10:30:24 (4304): No heartbeat from core client for 30 sec - exiting 10:30:25 (4304): No heartbeat from core client for 30 sec - exiting 10:30:26 (4304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4324, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4324, iMonCtr=1 Model crash detected, will try to restart... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6040, selfPID=6040, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8084, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8084, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8084, iMonCtr=1 Model crash detected, will try to restart... 17:14:55 (8084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:00:56 (4132): No heartbeat from core client for 30 sec - exiting 04:00:57 (4132): No heartbeat from core client for 30 sec - exiting 04:00:58 (4132): No heartbeat from core client for 30 sec - exiting 04:00:59 (4132): No heartbeat from core client for 30 sec - exiting 04:01:01 (4132): No heartbeat from core client for 30 sec - exiting 04:01:02 (4132): No heartbeat from core client for 30 sec - exiting 04:01:03 (4132): No heartbeat from core client for 30 sec - exiting 04:01:04 (4132): No heartbeat from core client for 30 sec - exiting 04:01:05 (4132): No heartbeat from core client for 30 sec - exiting 04:01:06 (4132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:01:07 (4132): No heartbeat from core client for 30 sec - exiting 04:01:08 (4132): No heartbeat from core client for 30 sec - exiting 04:01:09 (4132): No heartbeat from core client for 30 sec - exiting 04:01:10 (4132): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5084, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5084, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5084, iMonCtr=1 Model crash detected, will try to restart... 08:31:27 (5084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:07:06 (4984): No heartbeat from core client for 30 sec - exiting 21:07:07 (4984): No heartbeat from core client for 30 sec - exiting 21:07:08 (4984): No heartbeat from core client for 30 sec - exiting 21:07:10 (4984): No heartbeat from core client for 30 sec - exiting 21:07:11 (4984): No heartbeat from core client for 30 sec - exiting 21:07:12 (4984): No heartbeat from core client for 30 sec - exiting 21:07:13 (4984): No heartbeat from core client for 30 sec - exiting 21:07:14 (4984): No heartbeat from core client for 30 sec - exiting 21:07:15 (4984): No heartbeat from core client for 30 sec - exiting 21:07:16 (4984): No heartbeat from core client for 30 sec - exiting 21:07:18 (4984): No heartbeat from core client for 30 sec - exiting 21:07:19 (4984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:05:39 (4344): No heartbeat from core client for 30 sec - exiting 10:05:40 (4344): No heartbeat from core client for 30 sec - exiting 10:05:41 (4344): No heartbeat from core client for 30 sec - exiting 10:05:42 (4344): No heartbeat from core client for 30 sec - exiting 10:05:43 (4344): No heartbeat from core client for 30 sec - exiting 10:05:44 (4344): No heartbeat from core client for 30 sec - exiting 10:05:45 (4344): No heartbeat from core client for 30 sec - exiting 10:05:46 (4344): No heartbeat from core client for 30 sec - exiting 10:05:47 (4344): No heartbeat from core client for 30 sec - exiting 10:05:48 (4344): No heartbeat from core client for 30 sec - exiting 10:05:49 (4344): No heartbeat from core client for 30 sec - exiting 10:05:51 (4344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5096, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5096, iMonCtr=1 Model crash detected, will try to restart... 10:11:32 (5096): No heartbeat from core client for 30 sec - exiting 10:11:33 (5096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:42:19 (4208): No heartbeat from core client for 30 sec - exiting 07:42:20 (4208): No heartbeat from core client for 30 sec - exiting 07:42:21 (4208): No heartbeat from core client for 30 sec - exiting 07:42:22 (4208): No heartbeat from core client for 30 sec - exiting 07:42:23 (4208): No heartbeat from core client for 30 sec - exiting 07:42:24 (4208): No heartbeat from core client for 30 sec - exiting 07:42:25 (4208): No heartbeat from core client for 30 sec - exiting 07:42:27 (4208): No heartbeat from core client for 30 sec - exiting 07:42:28 (4208): No heartbeat from core client for 30 sec - exiting 07:42:29 (4208): No heartbeat from core client for 30 sec - exiting 07:42:30 (4208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=720, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=720, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=720, iMonCtr=1 Model crash detected, will try to restart... 19:19:26 (720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:36:22 (4064): Can't set up shared mem: -1. Will run in standalone mode. 06:35:31 (2224): No heartbeat from core client for 30 sec - exiting 06:35:32 (2224): No heartbeat from core client for 30 sec - exiting 06:35:33 (2224): No heartbeat from core client for 30 sec - exiting 06:35:34 (2224): No heartbeat from core client for 30 sec - exiting 06:35:35 (2224): No heartbeat from core client for 30 sec - exiting 06:35:36 (2224): No heartbeat from core client for 30 sec - exiting 06:35:37 (2224): No heartbeat from core client for 30 sec - exiting 06:35:38 (2224): No heartbeat from core client for 30 sec - exiting 06:35:40 (2224): No heartbeat from core client for 30 sec - exiting 06:35:41 (2224): No heartbeat from core client for 30 sec - exiting 06:35:42 (2224): No heartbeat from core client for 30 sec - exiting 06:35:43 (2224): No heartbeat from core client for 30 sec - exiting 06:35:44 (2224): No heartbeat from core client for 30 sec - exiting 06:35:46 (2224): No heartbeat from core client for 30 sec - exiting 06:35:47 (2224): No heartbeat from core client for 30 sec - exiting 06:35:48 (2224): No heartbeat from core client for 30 sec - exiting 06:35:49 (2224): No heartbeat from core client for 30 sec - exiting 06:35:50 (2224): No heartbeat from core client for 30 sec - exiting 06:35:51 (2224): No heartbeat from core client for 30 sec - exiting 06:35:52 (2224): No heartbeat from core client for 30 sec - exiting 06:35:54 (2224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:37:22 (4660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:38:08 (4536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5804, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5804, iMonCtr=1 Model crash detected, will try to restart... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2208, selfPID=2208, iMonCtr=1 No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2600, selfPID=2600, iMonCtr=1 18:29:11 (236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:30:46 (1688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=444, iMonCtr=1 Model crash detected, will try to restart... 18:45:33 (444): No heartbeat from core client for 30 sec - exiting 18:45:34 (444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:59:22 (2632): No heartbeat from core client for 30 sec - exiting 23:59:23 (2632): No heartbeat from core client for 30 sec - exiting 23:59:25 (2632): No heartbeat from core client for 30 sec - exiting 23:59:26 (2632): No heartbeat from core client for 30 sec - exiting 23:59:27 (2632): No heartbeat from core client for 30 sec - exiting 23:59:28 (2632): No heartbeat from core client for 30 sec - exiting 23:59:29 (2632): No heartbeat from core client for 30 sec - exiting 23:59:30 (2632): No heartbeat from core client for 30 sec - exiting 23:59:31 (2632): No heartbeat from core client for 30 sec - exiting 23:59:32 (2632): No heartbeat from core client for 30 sec - exiting 23:59:33 (2632): No heartbeat from core client for 30 sec - exiting 23:59:34 (2632): No heartbeat from core client for 30 sec - exiting 23:59:35 (2632): No heartbeat from core client for 30 sec - exiting 23:59:37 (2632): No heartbeat from core client for 30 sec - exiting 23:59:38 (2632): No heartbeat from core client for 30 sec - exiting 23:59:39 (2632): No heartbeat from core client for 30 sec - exiting 23:59:40 (2632): No heartbeat from core client for 30 sec - exiting 23:59:41 (2632): No heartbeat from core client for 30 sec - exiting 23:59:42 (2632): No heartbeat from core client for 30 sec - exiting 23:59:43 (2632): No heartbeat from core client for 30 sec - exiting 23:59:44 (2632): No heartbeat from core client for 30 sec - exiting 23:59:45 (2632): No heartbeat from core client for 30 sec - exiting 23:59:46 (2632): No heartbeat from core client for 30 sec - exiting 23:59:47 (2632): No heartbeat from core client for 30 sec - exiting 23:59:49 (2632): No heartbeat from core client for 30 sec - exiting 23:59:50 (2632): No heartbeat from core client for 30 sec - exiting 23:59:51 (2632): No heartbeat from core client for 30 sec - exiting 23:59:52 (2632): No heartbeat from core client for 30 sec - exiting 23:59:53 (2632): No heartbeat from core client for 30 sec - exiting 23:59:54 (2632): No heartbeat from core client for 30 sec - exiting 23:59:55 (2632): No heartbeat from core client for 30 sec - exiting 23:59:56 (2632): No heartbeat from core client for 30 sec - exiting 23:59:57 (2632): No heartbeat from core client for 30 sec - exiting 23:59:58 (2632): No heartbeat from core client for 30 sec - exiting 23:59:59 (2632): No heartbeat from core client for 30 sec - exiting 00:00:01 (2632): No heartbeat from core client for 30 sec - exiting 00:00:02 (2632): No heartbeat from core client for 30 sec - exiting 00:00:03 (2632): No heartbeat from core client for 30 sec - exiting 00:00:04 (2632): No heartbeat from core client for 30 sec - exiting 00:00:05 (2632): No heartbeat from core client for 30 sec - exiting 00:00:06 (2632): No heartbeat from core client for 30 sec - exiting 00:00:07 (2632): No heartbeat from core client for 30 sec - exiting 00:00:08 (2632): No heartbeat from core client for 30 sec - exiting 00:00:09 (2632): No heartbeat from core client for 30 sec - exiting 00:00:10 (2632): No heartbeat from core client for 30 sec - exiting 00:00:11 (2632): No heartbeat from core client for 30 sec - exiting 00:00:13 (2632): No heartbeat from core client for 30 sec - exiting 00:00:14 (2632): No heartbeat from core client for 30 sec - exiting 00:00:15 (2632): No heartbeat from core client for 30 sec - exiting 00:00:16 (2632): No heartbeat from core client for 30 sec - exiting 00:00:17 (2632): No heartbeat from core client for 30 sec - exiting 00:00:18 (2632): No heartbeat from core client for 30 sec - exiting 00:00:19 (2632): No heartbeat from core client for 30 sec - exiting 00:00:20 (2632): No heartbeat from core client for 30 sec - exiting 00:00:21 (2632): No heartbeat from core client for 30 sec - exiting 00:00:22 (2632): No heartbeat from core client for 30 sec - exiting 00:00:23 (2632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:00:59 (1756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Sep 2014 07:06:27	1274843	16061126	hadcm3n_of82_1900_40_008474853_1	233,280	465,629	1.9960
21 Jun 2014 14:21:25	1274843	16061126	hadcm3n_of82_1900_40_008474853_1	207,360	414,193	1.9975
17 Mar 2014 21:22:20	1274843	16061126	hadcm3n_of82_1900_40_008474853_1	181,440	360,149	1.9849
16 Mar 2014 10:32:16	1274843	16061126	hadcm3n_of82_1900_40_008474853_1	155,520	303,235	1.9498
11 Mar 2014 22:40:00	1274843	16061126	hadcm3n_of82_1900_40_008474853_1	129,600	250,990	1.9367
09 Mar 2014 04:44:37	1274843	16061126	hadcm3n_of82_1900_40_008474853_1	103,680	199,216	1.9215
05 Mar 2014 18:17:11	1274843	16061126	hadcm3n_of82_1900_40_008474853_1	77,760	150,427	1.9345
05 Mar 2014 02:51:41	1274843	16061126	hadcm3n_of82_1900_40_008474853_1	51,840	100,174	1.9324
21 Feb 2014 06:32:02	1274843	16061126	hadcm3n_of82_1900_40_008474853_1	25,920	52,556	2.0276