climateprediction.net home page
Task 13203182

Task 13203182

Name hadcm3n_yebr_1900_40_007351425_2
Workunit 7548855
Created 5 Aug 2011, 16:57:57 UTC
Sent 5 Aug 2011, 16:58:02 UTC
Report deadline 5 Nov 2011, 0:25:13 UTC
Received 31 Aug 2011, 14:40:55 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1145105
Run time 24 days 13 hours 44 min 31 sec
CPU time 24 days 13 hours 44 min 31 sec
Validate state Invalid
Credit 12,441.60
Device peak FLOPS 2.75 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.2.14</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
17:37:24 (31395): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:57:06 (31872): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:43:50 (32506): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:45:32 (32535): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:47:13 (32565): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:48:54 (32587): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:50:35 (32609): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:52:16 (32631): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:53:58 (32653): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:55:38 (32675): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:57:19 (32697): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
08:59:00 (32719): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:00:42 (32741): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:02:23 (32767): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:04:04 (321): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:05:45 (343): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:07:26 (365): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:09:08 (387): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:10:49 (409): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:12:30 (431): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:14:11 (454): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:17:15 (476): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:18:56 (503): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:20:36 (525): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:22:17 (547): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:25:29 (569): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:44:45 (591): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:45:52 (602): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:46:33 (612): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:53:23 (618): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:01:37 (628): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:03:18 (663): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:04:59 (674): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:06:40 (684): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:08:22 (694): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:10:03 (704): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:11:44 (714): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:13:27 (725): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:15:08 (735): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:17:46 (745): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:19:29 (759): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:21:10 (769): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:23:51 (779): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:25:33 (794): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:27:13 (808): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:28:54 (814): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:30:34 (824): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:32:15 (834): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:33:56 (846): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:35:36 (856): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:37:18 (866): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:38:58 (876): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:40:39 (886): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:35:47 (896): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
23:47:30 (921): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
07:13:34 (2847): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:15:20 (5505): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:18:03 (5812): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yebr_1900_40_007351425/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yebr_1900_40_007351425/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yebr_1900_40_007351425/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yebr_1900_40_007351425/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yebr_1900_40_007351425/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yebr_1900_40_007351425/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yebr_1900_40_007351425/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yebr_1900_40_007351425/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yebr_1900_40_007351425/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yebr_1900_40_007351425/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yebr_1900_40_007351425/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yebr_1900_40_007351425/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
31 Aug 2011 14:43:11 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 1,036,800 2,123,117 2.0478
31 Aug 2011 14:43:11 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 1,010,880 2,068,115 2.0459
31 Aug 2011 14:43:11 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 984,960 2,013,116 2.0439
28 Aug 2011 11:26:23 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 959,040 1,958,273 2.0419
27 Aug 2011 20:14:06 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 933,120 1,903,513 2.0399
27 Aug 2011 04:58:13 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 907,200 1,848,723 2.0378
26 Aug 2011 13:43:56 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 881,280 1,793,895 2.0356
25 Aug 2011 22:28:26 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 855,360 1,739,052 2.0331
25 Aug 2011 07:13:57 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 829,440 1,684,359 2.0307
24 Aug 2011 16:58:55 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 803,520 1,632,842 2.0321
24 Aug 2011 04:47:40 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 777,600 1,586,205 2.0399
23 Aug 2011 13:06:13 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 751,680 1,532,895 2.0393
22 Aug 2011 22:18:20 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 725,760 1,479,630 2.0387
22 Aug 2011 08:03:45 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 699,840 1,426,419 2.0382
21 Aug 2011 16:44:00 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 673,920 1,373,223 2.0377
21 Aug 2011 01:55:20 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 648,000 1,319,966 2.0370
20 Aug 2011 11:07:36 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 622,080 1,266,658 2.0362
19 Aug 2011 20:59:19 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 596,160 1,213,294 2.0352
19 Aug 2011 06:06:06 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 570,240 1,160,807 2.0356
18 Aug 2011 14:56:03 1145105 13203182 hadcm3n_yebr_1900_40_007351425_2 544,320 1,106,970 2.0337


©2024 cpdn.org