climateprediction.net home page
Task 15437700

Task 15437700

Name hadcm3n_o648_2060_40_008245042_1
Workunit 8400166
Created 17 Nov 2012, 0:04:10 UTC
Sent 17 Nov 2012, 0:04:28 UTC
Report deadline 16 Feb 2013, 7:31:39 UTC
Received 14 Dec 2012, 15:51:23 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1157390
Run time 26 days 3 hours 13 min 20 sec
CPU time 26 days 2 hours 49 min
Validate state Invalid
Credit 9,331.20
Device peak FLOPS 1.38 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
03:41:06 (12130): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:47:51 (18043): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
17:47:55 (18043): No heartbeat from core client for 30 sec - exiting
17:47:56 (18043): No heartbeat from core client for 30 sec - exiting
17:47:57 (18043): No heartbeat from core client for 30 sec - exiting
17:48:00 (18043): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
06:34:09 (25914): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:34:10 (25914): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
01:07:44 (2241): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:53:55 (3034): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:01:01 (3429): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:04:16 (3664): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:07:31 (3728): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:41:18 (3786): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
01:30:55 (3843): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
15:09:55 (2243): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
03:46:08 (2784): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
03:46:11 (2784): No heartbeat from core client for 30 sec - exiting
03:46:12 (2784): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
15:52:21 (3479): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:52:23 (3479): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
23:48:08 (4031): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
10:44:13 (4477): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:44:15 (4477): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
19:45:07 (4977): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
06:31:02 (5423): No heartbeat from core client for 30 sec - exiting
06:31:04 (5423): No heartbeat from core client for 30 sec - exiting
06:31:05 (5423): No heartbeat from core client for 30 sec - exiting
06:31:06 (5423): No heartbeat from core client for 30 sec - exiting
06:31:07 (5423): No heartbeat from core client for 30 sec - exiting
06:31:08 (5423): No heartbeat from core client for 30 sec - exiting
06:31:09 (5423): No heartbeat from core client for 30 sec - exiting
06:31:10 (5423): No heartbeat from core client for 30 sec - exiting
06:31:12 (5423): No heartbeat from core client for 30 sec - exiting
06:31:13 (5423): No heartbeat from core client for 30 sec - exiting
06:31:14 (5423): No heartbeat from core client for 30 sec - exiting
06:31:15 (5423): No heartbeat from core client for 30 sec - exiting
06:31:17 (5423): No heartbeat from core client for 30 sec - exiting
06:31:18 (5423): No heartbeat from core client for 30 sec - exiting
06:31:19 (5423): No heartbeat from core client for 30 sec - exiting
06:31:20 (5423): No heartbeat from core client for 30 sec - exiting
06:31:21 (5423): No heartbeat from core client for 30 sec - exiting
06:31:22 (5423): No heartbeat from core client for 30 sec - exiting
06:31:23 (5423): No heartbeat from core client for 30 sec - exiting
06:31:24 (5423): No heartbeat from core client for 30 sec - exiting
06:31:25 (5423): No heartbeat from core client for 30 sec - exiting
06:31:26 (5423): No heartbeat from core client for 30 sec - exiting
06:31:27 (5423): No heartbeat from core client for 30 sec - exiting
06:31:28 (5423): No heartbeat from core client for 30 sec - exiting
06:31:29 (5423): No heartbeat from core client for 30 sec - exiting
06:31:30 (5423): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o648_2060_40_008245042/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o648_2060_40_008245042/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o648_2060_40_008245042/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o648_2060_40_008245042/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o648_2060_40_008245042/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o648_2060_40_008245042/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o648_2060_40_008245042/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o648_2060_40_008245042/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o648_2060_40_008245042/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o648_2060_40_008245042/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o648_2060_40_008245042/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o648_2060_40_008245042/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
14 Dec 2012 12:35:30 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 777,600 2,256,609 2.9020
13 Dec 2012 20:30:55 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 751,680 2,182,575 2.9036
13 Dec 2012 20:30:54 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 725,760 2,107,893 2.9044
13 Dec 2012 20:30:55 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 699,840 2,033,963 2.9063
13 Dec 2012 20:30:55 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 673,920 1,959,492 2.9076
13 Dec 2012 20:30:55 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 648,000 1,884,203 2.9077
08 Dec 2012 00:14:32 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 622,080 1,807,886 2.9062
07 Dec 2012 02:51:00 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 596,160 1,731,411 2.9043
06 Dec 2012 05:37:47 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 570,240 1,655,113 2.9025
05 Dec 2012 08:39:24 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 544,320 1,579,061 2.9010
04 Dec 2012 11:42:51 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 518,400 1,502,745 2.8988
03 Dec 2012 14:30:39 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 492,480 1,426,679 2.8969
02 Dec 2012 17:25:24 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 466,560 1,350,564 2.8947
01 Dec 2012 20:19:00 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 440,640 1,274,218 2.8917
30 Nov 2012 22:03:49 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 414,720 1,197,909 2.8885
30 Nov 2012 00:55:00 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 388,800 1,121,762 2.8852
29 Nov 2012 03:41:31 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 362,880 1,045,317 2.8806
28 Nov 2012 06:26:13 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 336,960 969,089 2.8760
27 Nov 2012 09:13:43 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 311,040 893,209 2.8717
26 Nov 2012 11:56:15 1157390 15437700 hadcm3n_o648_2060_40_008245042_1 285,120 817,022 2.8655


©2024 climateprediction.net