climateprediction.net home page
Task 15529516

Task 15529516

Name hadcm3n_398h_1940_40_008268095_1
Workunit 8423219
Created 12 Jan 2013, 9:13:25 UTC
Sent 12 Jan 2013, 9:13:28 UTC
Report deadline 13 Apr 2013, 16:40:39 UTC
Received 29 Sep 2013, 20:01:20 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1187881
Run time 51 days 15 hours 20 min 31 sec
CPU time 49 days 22 hours 41 min 14 sec
Validate state Invalid
Credit 5,909.76
Device peak FLOPS 0.62 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>7.0.27</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 62 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/398hko.pjf0c10 is not a valid UM file.
Error converting file to netcdf: dataout/398hko.pjf0c10
Error: Input file: dataout/398hko.pif0c10 is not a valid UM file.
Error converting file to netcdf: dataout/398hko.pif0c10
Error: Input file: dataout/398hko.pff0c10 is not a valid UM file.
Error converting file to netcdf: dataout/398hko.pff0c10
Error: Input file: dataout/398hko.pcf0c10 is not a valid UM file.
Error converting file to netcdf: dataout/398hko.pcf0c10
Error: Input file: dataout/398hko.pbf0c10 is not a valid UM file.
Error converting file to netcdf: dataout/398hko.pbf0c10
Error: Input file: dataout/398hko.paf0c10 is not a valid UM file.
Error converting file to netcdf: dataout/398hko.paf0c10
Error: Input file: dataout/398hka.phf0c10 is not a valid UM file.
Error converting file to netcdf: dataout/398hka.phf0c10
Error: Input file: dataout/398hka.pgf0c10 is not a valid UM file.
Error converting file to netcdf: dataout/398hka.pgf0c10
Error: Input file: dataout/398hka.pef0c10 is not a valid UM file.
Error converting file to netcdf: dataout/398hka.pef0c10
Error: Input file: dataout/398hka.pdf0c10 is not a valid UM file.
Error converting file to netcdf: dataout/398hka.pdf0c10
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
00:42:09 (5498): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:21:14 (2591): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITHEAD: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                     CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

SETPOS: Seek Failed: No space left on device
SETPOS: Unit 22 to Word Address 3715072 Failed with Error Code -1

Model crashed: SETPOS: Unit 22 to Word Address 3715072 Failed with Error Code -1

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

SETPOS: Seek Failed: No space left on device
SETPOS: Unit 22 to Word Address 202Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=14822, iMonCtr=1
Model crash detected, will try to restart...
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_398h_1940_40_008268095/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0848EB7D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0848D975  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0845F3CF  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F90D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F257  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08450E2C  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0824EBA6  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  082712DC  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08177076  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  081795D4  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  081908AF  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08391957  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0838F8B7  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0839BDF8  Unknown               Unknown  Unknown
libc.so.6          F7550935  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=14822, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
25 Sep 2013 17:51:09 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 492,480 4,202,450 8.5332
15 Sep 2013 16:31:21 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 466,560 3,992,247 8.5568
22 Aug 2013 15:31:39 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 440,640 3,758,776 8.5303
14 Aug 2013 18:58:32 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 414,720 3,524,047 8.4974
14 Aug 2013 18:00:07 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 388,800 3,293,533 8.4710
14 Aug 2013 18:00:08 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 362,880 3,060,580 8.4341
23 Jul 2013 22:12:51 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 336,960 2,831,455 8.4029
07 Jul 2013 16:37:56 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 311,040 2,605,368 8.3763
11 Jun 2013 16:21:55 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 285,120 2,401,500 8.4228
02 Jun 2013 15:00:58 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 259,200 2,195,997 8.4722
18 May 2013 21:37:52 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 233,280 1,993,539 8.5457
12 May 2013 23:03:55 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 207,360 1,769,487 8.5334
02 May 2013 11:31:37 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 181,440 1,545,543 8.5182
21 Mar 2013 14:50:27 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 155,520 1,323,020 8.5071
10 Mar 2013 11:01:56 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 129,600 1,106,681 8.5392
22 Feb 2013 21:29:07 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 103,680 890,304 8.5870
12 Feb 2013 00:47:41 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 77,760 674,309 8.6717
09 Feb 2013 02:20:16 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 51,840 458,090 8.8366
30 Jan 2013 11:33:32 1187881 15529516 hadcm3n_398h_1940_40_008268095_1 25,920 240,938 9.2954


©2024 cpdn.org