 |
|
| Name | hadcm3n_398l_2020_40_008394373_2 |
| Workunit | 8545232 |
| Created | 10 Jul 2013, 11:41:41 UTC |
| Sent | 10 Jul 2013, 12:18:34 UTC |
| Report deadline | 9 Oct 2013, 19:45:45 UTC |
| Received | 29 Aug 2013, 6:10:47 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 22 (0x00000016) Unknown error code |
| Computer ID | 1260695 |
| Run time | 18 days 22 hours 35 min 24 sec |
| CPU time | 17 days 9 hours 5 min 15 sec |
| Validate state | Invalid |
| Credit | 11,819.52 |
| Device peak FLOPS | 2.99 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86 |
| Stderr | <core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:05:05 (6668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:19:41 (7856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:20:40 (9204): No heartbeat from core client for 30 sec - exiting 10:20:41 (9204): No heartbeat from core client for 30 sec - exiting 10:20:42 (9204): No heartbeat from core client for 30 sec - exiting 10:20:43 (9204): No heartbeat from core client for 30 sec - exiting 10:20:44 (9204): No heartbeat from core client for 30 sec - exiting 10:20:45 (9204): No heartbeat from core client for 30 sec - exiting 10:20:46 (9204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:22:14 (7480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:24:41 (9884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:25:30 (7952): No heartbeat from core client for 30 sec - exiting 10:25:31 (7952): No heartbeat from core client for 30 sec - exiting 10:25:32 (7952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:26:07 (9500): No heartbeat from core client for 30 sec - exiting 10:26:08 (9500): No heartbeat from core client for 30 sec - exiting 10:26:09 (9500): No heartbeat from core client for 30 sec - exiting 10:26:10 (9500): No heartbeat from core client for 30 sec - exiting 10:26:11 (9500): No heartbeat from core client for 30 sec - exiting 10:26:12 (9500): No heartbeat from core client for 30 sec - exiting 10:26:13 (9500): No heartbeat from core client for 30 sec - exiting 10:26:14 (9500): No heartbeat from core client for 30 sec - exiting 10:26:15 (9500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:26:16 (9500): No heartbeat from core client for 30 sec - exiting 10:26:48 (8872): No heartbeat from core client for 30 sec - exiting 10:26:49 (8872): No heartbeat from core client for 30 sec - exiting 10:26:50 (8872): No heartbeat from core client for 30 sec - exiting 10:26:52 (8872): No heartbeat from core client for 30 sec - exiting 10:26:53 (8872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:28:16 (9316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:28:17 (9316): No heartbeat from core client for 30 sec - exiting 10:28:53 (4220): No heartbeat from core client for 30 sec - exiting 10:29:00 (4220): No heartbeat from core client for 30 sec - exiting 10:29:01 (4220): No heartbeat from core client for 30 sec - exiting 10:29:02 (4220): No heartbeat from core client for 30 sec - exiting 10:29:03 (4220): No heartbeat from core client for 30 sec - exiting 10:29:04 (4220): No heartbeat from core client for 30 sec - exiting 10:29:05 (4220): No heartbeat from core client for 30 sec - exiting 10:29:06 (4220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:30:10 (9436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:30:48 (2076): No heartbeat from core client for 30 sec - exiting 10:30:49 (2076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:31:26 (6656): No heartbeat from core client for 30 sec - exiting 10:31:27 (6656): No heartbeat from core client for 30 sec - exiting 10:31:28 (6656): No heartbeat from core client for 30 sec - exiting 10:31:29 (6656): No heartbeat from core client for 30 sec - exiting 10:31:30 (6656): No heartbeat from core client for 30 sec - exiting 10:31:31 (6656): No heartbeat from core client for 30 sec - exiting 10:31:32 (6656): No heartbeat from core client for 30 sec - exiting 10:31:33 (6656): No heartbeat from core client for 30 sec - exiting 10:31:34 (6656): No heartbeat from core client for 30 sec - exiting 10:31:35 (6656): No heartbeat from core client for 30 sec - exiting 10:31:36 (6656): No heartbeat from core client for 30 sec - exiting 10:31:37 (6656): No heartbeat from core client for 30 sec - exiting 10:31:38 (6656): No heartbeat from core client for 30 sec - exiting 10:31:39 (6656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:31:40 (6656): No heartbeat from core client for 30 sec - exiting 10:33:24 (9040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:34:23 (7792): No heartbeat from core client for 30 sec - exiting 10:34:24 (7792): No heartbeat from core client for 30 sec - exiting 10:34:25 (7792): No heartbeat from core client for 30 sec - exiting 10:34:26 (7792): No heartbeat from core client for 30 sec - exiting 10:34:27 (7792): No heartbeat from core client for 30 sec - exiting 10:34:28 (7792): No heartbeat from core client for 30 sec - exiting 10:34:29 (7792): No heartbeat from core client for 30 sec - exiting 10:34:30 (7792): No heartbeat from core client for 30 sec - exiting 10:34:31 (7792): No heartbeat from core client for 30 sec - exiting 10:34:32 (7792): No heartbeat from core client for 30 sec - exiting 10:34:33 (7792): No heartbeat from core client for 30 sec - exiting 10:34:34 (7792): No heartbeat from core client for 30 sec - exiting 10:34:36 (7792): No heartbeat from core client for 30 sec - exiting 10:34:37 (7792): No heartbeat from core client for 30 sec - exiting 10:34:38 (7792): No heartbeat from core client for 30 sec - exiting 10:34:39 (7792): No heartbeat from core client for 30 sec - exiting 10:34:40 (7792): No heartbeat from core client for 30 sec - exiting 10:34:41 (7792): No heartbeat from core client for 30 sec - exiting 10:34:42 (7792): No heartbeat from core client for 30 sec - exiting 10:34:43 (7792): No heartbeat from core client for 30 sec - exiting 10:34:44 (7792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:33:49 (9040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:34:28 (10036): No heartbeat from core client for 30 sec - exiting 16:34:29 (10036): No heartbeat from core client for 30 sec - exiting 16:34:30 (10036): No heartbeat from core client for 30 sec - exiting 16:34:31 (10036): No heartbeat from core client for 30 sec - exiting 16:34:32 (10036): No heartbeat from core client for 30 sec - exiting 16:34:33 (10036): No heartbeat from core client for 30 sec - exiting 16:34:34 (10036): No heartbeat from core client for 30 sec - exiting 16:34:35 (10036): No heartbeat from core client for 30 sec - exiting 16:34:36 (10036): No heartbeat from core client for 30 sec - exiting 16:34:37 (10036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:34:38 (10036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2076, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/398lko.pjo6c10 Error converting file to netcdf: dataout/398lko.pio6c10 Error converting file to netcdf: dataout/398lko.pfo6c10 Error converting file to netcdf: dataout/398lka.pho6c10 Error converting file to netcdf: dataout/398lka.pgo6c10 Error converting file to netcdf: dataout/398lka.peo6c10 Error converting file to netcdf: dataout/398lka.pdo6c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:04:16 (3936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:04:44 (10080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:05:22 (8604): No heartbeat from core client for 30 sec - exiting 09:05:23 (8604): No heartbeat from core client for 30 sec - exiting 09:05:24 (8604): No heartbeat from core client for 30 sec - exiting 09:05:25 (8604): No heartbeat from core client for 30 sec - exiting 09:05:26 (8604): No heartbeat from core client for 30 sec - exiting 09:05:27 (8604): No heartbeat from core client for 30 sec - exiting 09:05:28 (8604): No heartbeat from core client for 30 sec - exiting 09:05:29 (8604): No heartbeat from core client for 30 sec - exiting 09:05:30 (8604): No heartbeat from core client for 30 sec - exiting 09:05:31 (8604): No heartbeat from core client for 30 sec - exiting 09:05:32 (8604): No heartbeat from core client for 30 sec - exiting 09:05:33 (8604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:07:09 (2976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:49:29 (9792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:19:39 (3936): No heartbeat from core client for 30 sec - exiting 17:19:40 (3936): No heartbeat from core client for 30 sec - exiting 17:19:41 (3936): No heartbeat from core client for 30 sec - exiting 17:19:42 (3936): No heartbeat from core client for 30 sec - exiting 17:19:43 (3936): No heartbeat from core client for 30 sec - exiting 17:19:44 (3936): No heartbeat from core client for 30 sec - exiting 17:19:46 (3936): No heartbeat from core client for 30 sec - exiting 17:19:47 (3936): No heartbeat from core client for 30 sec - exiting 17:19:48 (3936): No heartbeat from core client for 30 sec - exiting 17:19:49 (3936): No heartbeat from core client for 30 sec - exiting 17:19:50 (3936): No heartbeat from core client for 30 sec - exiting 17:19:51 (3936): No heartbeat from core client for 30 sec - exiting 17:19:52 (3936): No heartbeat from core client for 30 sec - exiting 17:19:53 (3936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 18:34:30 (6964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 28 Aug 2013 06:22:26 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 984,960 | 1,462,569 | 1.4849 |
| 27 Aug 2013 18:03:46 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 959,040 | 1,424,532 | 1.4854 |
| 24 Aug 2013 21:36:00 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 933,120 | 1,387,707 | 1.4872 |
| 23 Aug 2013 23:35:30 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 907,200 | 1,349,044 | 1.4870 |
| 23 Aug 2013 10:38:16 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 881,280 | 1,309,996 | 1.4865 |
| 23 Aug 2013 10:00:49 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 855,360 | 1,271,604 | 1.4866 |
| 21 Aug 2013 19:04:00 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 829,440 | 1,233,046 | 1.4866 |
| 21 Aug 2013 07:51:33 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 803,520 | 1,194,403 | 1.4865 |
| 20 Aug 2013 01:56:12 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 777,600 | 1,155,865 | 1.4865 |
| 19 Aug 2013 14:15:48 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 751,680 | 1,117,485 | 1.4866 |
| 18 Aug 2013 00:07:42 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 725,760 | 1,078,777 | 1.4864 |
| 17 Aug 2013 09:39:46 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 699,840 | 1,039,749 | 1.4857 |
| 14 Aug 2013 16:57:52 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 673,920 | 1,001,267 | 1.4857 |
| 14 Aug 2013 15:29:09 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 648,000 | 964,027 | 1.4877 |
| 14 Aug 2013 15:29:09 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 622,080 | 926,119 | 1.4887 |
| 14 Aug 2013 15:29:09 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 596,160 | 886,682 | 1.4873 |
| 14 Aug 2013 15:29:09 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 570,240 | 847,541 | 1.4863 |
| 14 Aug 2013 15:29:09 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 544,320 | 808,770 | 1.4858 |
| 14 Aug 2013 15:29:09 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 518,400 | 768,907 | 1.4832 |
| 14 Aug 2013 15:29:09 | 1260695 | 15890554 | hadcm3n_398l_2020_40_008394373_2 | 492,480 | 729,650 | 1.4816 |
©2024 climateprediction.net