|
Name | hadcm3n_3lhf_1940_40_008260171_4 |
Workunit | 8415295 |
Created | 25 Jul 2013, 19:33:25 UTC |
Sent | 25 Jul 2013, 19:33:30 UTC |
Report deadline | 25 Oct 2013, 3:00:41 UTC |
Received | 2 Sep 2013, 12:13:23 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 193 (0x000000C1) EXIT_SIGNAL |
Computer ID | 1105487 |
Run time | 20 days 19 hours 14 min 59 sec |
CPU time | 19 days 16 hours 58 min 45 sec |
Validate state | Invalid |
Credit | 12,441.60 |
Device peak FLOPS | 3.03 GFLOPS |
Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86 |
Stderr | <core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:48:42 (2108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:48:43 (2108): No heartbeat from core client for 30 sec - exiting 20:48:44 (2108): No heartbeat from core client for 30 sec - exiting 20:48:45 (2108): No heartbeat from core client for 30 sec - exiting 20:48:47 (2108): No heartbeat from core client for 30 sec - exiting 20:48:48 (2108): No heartbeat from core client for 30 sec - exiting 20:48:49 (2108): No heartbeat from core client for 30 sec - exiting 20:48:50 (2108): No heartbeat from core client for 30 sec - exiting 20:48:51 (2108): No heartbeat from core client for 30 sec - exiting 20:48:52 (2108): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:28:30 (5380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 21:26:29 (5980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 22:41:15 (5628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 13:42:06 (4776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:42:44 (6488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:13:05 (3272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 15:57:30 (5024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:26:50 (5304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:27:35 (4288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 10:53:38 (5996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:06:06 (2928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:06:47 (2756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 09:36:44 (4192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:37:19 (1064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:37:20 (1064): No heartbeat from core client for 30 sec - exiting 09:37:21 (1064): No heartbeat from core client for 30 sec - exiting 12:35:04 (2992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:35:05 (2992): No heartbeat from core client for 30 sec - exiting 12:35:06 (2992): No heartbeat from core client for 30 sec - exiting 12:35:07 (2992): No heartbeat from core client for 30 sec - exiting 12:35:08 (2992): No heartbeat from core client for 30 sec - exiting 12:35:09 (2992): No heartbeat from core client for 30 sec - exiting 12:35:10 (2992): No heartbeat from core client for 30 sec - exiting 12:35:11 (2992): No heartbeat from core client for 30 sec - exiting 12:35:13 (2992): No heartbeat from core client for 30 sec - exiting 12:35:14 (2992): No heartbeat from core client for 30 sec - exiting 12:35:15 (2992): No heartbeat from core client for 30 sec - exiting 23:07:12 (3148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 12:14:46 (4716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:29:40 (6500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:33:04 (1616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:33:05 (1616): No heartbeat from core client for 30 sec - exiting 18:33:06 (1616): No heartbeat from core client for 30 sec - exiting 18:33:07 (1616): No heartbeat from core client for 30 sec - exiting 18:33:08 (1616): No heartbeat from core client for 30 sec - exiting 18:33:09 (1616): No heartbeat from core client for 30 sec - exiting 18:33:10 (1616): No heartbeat from core client for 30 sec - exiting 18:33:12 (1616): No heartbeat from core client for 30 sec - exiting 18:33:13 (1616): No heartbeat from core client for 30 sec - exiting 23:24:49 (3656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 13:45:30 (5084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:09:07 (1460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:10:18 (4580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:10:58 (7088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 22:06:24 (1740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 13:54:24 (5328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:55:06 (2632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:31:54 (6340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:31:55 (6340): No heartbeat from core client for 30 sec - exiting 22:36:16 (2024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4384, iMonCtr=1 Model crash detected, will try to restart... 09:47:17 (5740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 13:52:01 (5020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:10:27 (5812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:10:28 (5812): No heartbeat from core client for 30 sec - exiting 14:10:29 (5812): No heartbeat from core client for 30 sec - exiting 14:10:30 (5812): No heartbeat from core client for 30 sec - exiting 14:10:31 (5812): No heartbeat from core client for 30 sec - exiting 14:10:32 (5812): No heartbeat from core client for 30 sec - exiting 14:11:24 (7080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:40:58 (6720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 15:35:26 (5564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 14:02:04 (6076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:43:44 (1072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 16:13:52 (6044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:45:05 (5032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 21:57:33 (5392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:19:14 (5484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:20:00 (3408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:51:59 (1700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:52:34 (5748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 16:45:20 (5576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:24:54 (4696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 13:26:08 (6000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:55:18 (4440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:55:56 (1720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 18:14:01 (5500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:24:40 (6564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:06:24 (5504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:09:49 (4752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:53:16 (1904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 21:35:10 (5556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 forrtl: There is not enough space on the disk. 09:56:15 (5400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:23:04 (6356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:23:05 (6356): No heartbeat from core client for 30 sec - exiting 11:23:06 (6356): No heartbeat from core client for 30 sec - exiting 11:23:08 (6356): No heartbeat from core client for 30 sec - exiting 11:23:09 (6356): No heartbeat from core client for 30 sec - exiting 11:23:10 (6356): No heartbeat from core client for 30 sec - exiting 15:25:20 (1732): No heartbeat from core client for 30 sec - exiting 15:25:22 (1732): No heartbeat from core client for 30 sec - exiting 15:25:23 (1732): No heartbeat from core client for 30 sec - exiting 15:25:24 (1732): No heartbeat from core client for 30 sec - exiting 15:25:25 (1732): No heartbeat from core client for 30 sec - exiting 15:25:26 (1732): No heartbeat from core client for 30 sec - exiting 15:25:27 (1732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7732FF6B write attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77947383 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... 11:15:23 (5452): No heartbeat from core client for 30 sec - exiting 11:15:24 (5452): No heartbeat from core client for 30 sec - exiting 15:13:11 (5452): No heartbeat from core client for 30 sec - exiting 15:13:12 (5452): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
31 Aug 2013 20:45:16 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 1,036,800 | 1,702,083 | 1.6417 |
31 Aug 2013 08:29:38 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 1,010,880 | 1,666,693 | 1.6488 |
30 Aug 2013 12:52:11 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 984,960 | 1,651,210 | 1.6764 |
29 Aug 2013 12:31:35 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 959,040 | 1,608,304 | 1.6770 |
28 Aug 2013 12:09:22 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 933,120 | 1,565,166 | 1.6773 |
27 Aug 2013 11:47:11 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 907,200 | 1,522,348 | 1.6781 |
26 Aug 2013 11:45:01 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 881,280 | 1,479,419 | 1.6787 |
25 Aug 2013 14:29:16 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 855,360 | 1,436,403 | 1.6793 |
24 Aug 2013 16:19:53 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 829,440 | 1,392,588 | 1.6789 |
23 Aug 2013 20:49:27 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 803,520 | 1,348,952 | 1.6788 |
23 Aug 2013 10:11:32 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 777,600 | 1,304,901 | 1.6781 |
21 Aug 2013 22:55:26 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 751,680 | 1,261,966 | 1.6789 |
21 Aug 2013 10:52:17 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 725,760 | 1,219,757 | 1.6807 |
20 Aug 2013 12:39:12 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 699,840 | 1,177,501 | 1.6825 |
19 Aug 2013 15:36:04 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 673,920 | 1,135,137 | 1.6844 |
18 Aug 2013 17:21:28 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 648,000 | 1,094,819 | 1.6895 |
17 Aug 2013 21:12:12 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 622,080 | 1,053,602 | 1.6937 |
16 Aug 2013 20:27:03 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 596,160 | 1,011,318 | 1.6964 |
15 Aug 2013 21:05:17 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 570,240 | 970,735 | 1.7023 |
14 Aug 2013 20:45:09 | 1105487 | 15908107 | hadcm3n_3lhf_1940_40_008260171_4 | 544,320 | 931,161 | 1.7107 |
©2024 climateprediction.net