climateprediction.net home page
Task 16185968

Task 16185968

Name hadcm3n_obxz_1900_40_008470602_1
Workunit 8621441
Created 1 Jan 2014, 0:39:25 UTC
Sent 1 Jan 2014, 0:39:32 UTC
Report deadline 2 Apr 2014, 8:06:43 UTC
Received 7 Jan 2014, 14:32:29 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1292656
Run time 6 days 7 hours 41 min 22 sec
CPU time 4 days 9 hours 38 min 22 sec
Validate state Invalid
Credit 4,976.64
Device peak FLOPS 3.94 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>7.0.27</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
07:58:44 (27223): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:02:28 (27223): No heartbeat from core client for 30 sec - exiting
08:02:29 (27223): No heartbeat from core client for 30 sec - exiting
08:02:30 (27223): No heartbeat from core client for 30 sec - exiting
08:02:31 (27223): No heartbeat from core client for 30 sec - exiting
08:02:32 (27223): No heartbeat from core client for 30 sec - exiting
08:02:33 (27223): No heartbeat from core client for 30 sec - exiting
08:02:34 (27223): No heartbeat from core client for 30 sec - exiting
08:02:35 (27223): No heartbeat from core client for 30 sec - exiting
08:02:36 (27223): No heartbeat from core client for 30 sec - exiting
08:02:37 (27223): No heartbeat from core client for 30 sec - exiting
08:02:38 (27223): No heartbeat from core client for 30 sec - exiting
08:02:39 (27223): No heartbeat from core client for 30 sec - exiting
08:02:40 (27223): No heartbeat from core client for 30 sec - exiting
08:02:41 (27223): No heartbeat from core client for 30 sec - exiting
08:02:42 (27223): No heartbeat from core client for 30 sec - exiting
08:02:43 (27223): No heartbeat from core client for 30 sec - exiting
08:02:44 (27223): No heartbeat from core client for 30 sec - exiting
08:02:45 (27223): No heartbeat from core client for 30 sec - exiting
08:02:46 (27223): No heartbeat from core client for 30 sec - exiting
08:02:47 (27223): No heartbeat from core client for 30 sec - exiting
08:02:48 (27223): No heartbeat from core client for 30 sec - exiting
08:02:49 (27223): No heartbeat from core client for 30 sec - exiting
08:02:50 (27223): No heartbeat from core client for 30 sec - exiting
08:02:51 (27223): No heartbeat from core client for 30 sec - exiting
08:02:52 (27223): No heartbeat from core client for 30 sec - exiting
08:02:53 (27223): No heartbeat from core client for 30 sec - exiting
08:02:54 (27223): No heartbeat from core client for 30 sec - exiting
08:02:55 (27223): No heartbeat from core client for 30 sec - exiting
08:02:56 (27223): No heartbeat from core client for 30 sec - exiting
08:02:57 (27223): No heartbeat from core client for 30 sec - exiting
08:02:58 (27223): No heartbeat from core client for 30 sec - exiting
08:02:59 (27223): No heartbeat from core client for 30 sec - exiting
08:03:00 (27223): No heartbeat from core client for 30 sec - exiting
08:03:01 (27223): No heartbeat from core client for 30 sec - exiting
08:03:02 (27223): No heartbeat from core client for 30 sec - exiting
08:03:03 (27223): No heartbeat from core client for 30 sec - exiting
08:03:04 (27223): No heartbeat from core client for 30 sec - exiting
08:03:05 (27223): No heartbeat from core client for 30 sec - exiting
08:03:06 (27223): No heartbeat from core client for 30 sec - exiting
08:03:07 (27223): No heartbeat from core client for 30 sec - exiting
08:03:08 (27223): No heartbeat from core client for 30 sec - exiting
08:03:09 (27223): No heartbeat from core client for 30 sec - exiting
08:03:10 (27223): No heartbeat from core client for 30 sec - exiting
08:03:11 (27223): No heartbeat from core client for 30 sec - exiting
08:03:42 (27223): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
09:44:45 (4531): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:44:49 (4531): No heartbeat from core client for 30 sec - exiting
09:44:50 (4531): No heartbeat from core client for 30 sec - exiting
09:44:51 (4531): No heartbeat from core client for 30 sec - exiting
09:44:52 (4531): No heartbeat from core client for 30 sec - exiting
09:44:53 (4531): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
Suspended CPDN Monitor - Suspend request from BOINC...
10:24:04 (7050): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:24:06 (7050): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:12:23 (8093): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:30:19 (9125): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:37:48 (9522): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
15:42:48 (15427): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:12:09 (16936): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:14:49 (32133): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
07:58:11 (32213): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
10:36:32 (18629): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:36:33 (18629): No heartbeat from core client for 30 sec - exiting
10:36:34 (18629): No heartbeat from core client for 30 sec - exiting
10:36:35 (18629): No heartbeat from core client for 30 sec - exiting
10:36:36 (18629): No heartbeat from core client for 30 sec - exiting

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:08:03 (22818): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Suspended CPDN Monitor - Suspend request from BOINC...

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
07 Jan 2014 08:37:55 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 414,720 379,411 0.9149
06 Jan 2014 22:30:09 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 388,800 355,409 0.9141
06 Jan 2014 12:41:10 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 362,880 331,598 0.9138
06 Jan 2014 02:57:04 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 336,960 307,783 0.9134
05 Jan 2014 17:08:10 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 311,040 283,975 0.9130
05 Jan 2014 07:19:40 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 285,120 260,171 0.9125
04 Jan 2014 21:17:13 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 259,200 236,382 0.9120
04 Jan 2014 11:27:45 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 233,280 212,416 0.9106
04 Jan 2014 01:32:57 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 207,360 188,489 0.9090
03 Jan 2014 15:38:33 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 181,440 164,640 0.9074
03 Jan 2014 05:39:56 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 155,520 140,383 0.9027
02 Jan 2014 21:02:49 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 129,600 116,773 0.9010
02 Jan 2014 12:21:33 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 103,680 93,168 0.8986
02 Jan 2014 01:23:28 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 77,760 69,165 0.8895
01 Jan 2014 16:49:50 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 51,840 45,937 0.8861
01 Jan 2014 09:24:27 1292656 16185968 hadcm3n_obxz_1900_40_008470602_1 25,920 22,967 0.8861


©2024 climateprediction.net