climateprediction.net home page
Task 15766995

Task 15766995

Name hadcm3n_4lht_1940_40_008304259_1
Workunit 8455394
Created 9 May 2013, 9:19:39 UTC
Sent 9 May 2013, 9:19:54 UTC
Report deadline 8 Aug 2013, 16:47:05 UTC
Received 16 Jun 2013, 6:26:09 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1194898
Run time 31 days 10 hours 6 min 12 sec
CPU time 29 days 5 hours 42 min 41 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 1.01 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.10.17</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
07:50:26 (12478): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:54:30 (15992): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:55:50 (15996): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:57:10 (16000): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:58:30 (16004): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:59:50 (16008): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:01:11 (16012): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:02:31 (16016): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:03:51 (16032): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:05:11 (16036): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:07:20 (16040): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:08:41 (16044): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:14:32 (16048): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:15:53 (16052): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:26:08 (16056): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:30:55 (16064): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:37:20 (16068): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:38:41 (16072): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
09:21:44 (16076): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:24:06 (16168): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:26:27 (16172): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:28:48 (16176): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:31:09 (16180): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:33:30 (16184): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:35:51 (16188): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:38:12 (16192): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:40:33 (16196): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:42:54 (16200): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:45:15 (16204): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
18:58:20 (16208): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
12:59:00 (18203): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:55:11 (19534): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:57:33 (19542): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:59:54 (19546): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
10:01:24 (19550): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:02:45 (19872): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:04:05 (19876): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
21:43:00 (1123): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_4lht_1940_40_008304259/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_4lht_1940_40_008304259/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_4lht_1940_40_008304259/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_4lht_1940_40_008304259/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_4lht_1940_40_008304259/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_4lht_1940_40_008304259/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_4lht_1940_40_008304259/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_4lht_1940_40_008304259/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_4lht_1940_40_008304259/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_4lht_1940_40_008304259/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_4lht_1940_40_008304259/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_4lht_1940_40_008304259/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Jun 2013 19:42:57 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 518,400 2,526,208 4.8731
14 Jun 2013 05:46:16 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 492,480 2,396,976 4.8672
12 Jun 2013 15:42:19 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 466,560 2,267,418 4.8599
11 Jun 2013 01:35:13 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 440,640 2,138,094 4.8522
09 Jun 2013 07:44:17 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 414,720 2,009,142 4.8446
07 Jun 2013 18:44:00 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 388,800 1,883,244 4.8437
06 Jun 2013 03:44:03 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 362,880 1,757,176 4.8423
04 Jun 2013 12:41:36 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 336,960 1,630,981 4.8403
02 Jun 2013 23:30:57 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 311,040 1,504,754 4.8378
01 Jun 2013 10:29:54 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 285,120 1,378,608 4.8352
30 May 2013 21:20:43 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 259,200 1,252,575 4.8325
29 May 2013 08:07:45 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 233,280 1,126,337 4.8283
27 May 2013 18:56:06 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 207,360 1,000,452 4.8247
26 May 2013 05:34:12 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 181,440 874,849 4.8217
24 May 2013 15:50:22 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 155,520 749,195 4.8174
18 May 2013 19:46:14 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 129,600 623,548 4.8113
16 May 2013 23:11:25 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 103,680 498,299 4.8061
15 May 2013 03:18:39 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 77,760 373,509 4.8034
13 May 2013 04:51:24 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 51,840 249,008 4.8034
11 May 2013 04:21:04 1194898 15766995 hadcm3n_4lht_1940_40_008304259_1 25,920 124,584 4.8065


©2024 cpdn.org