Task 12821124

Name	hadcm3n_p2bt_1900_40_007220601_0
Workunit	7418841
Created	26 Apr 2011, 15:20:43 UTC
Sent	2 May 2011, 17:40:49 UTC
Report deadline	2 Aug 2011, 1:08:00 UTC
Received	17 May 2011, 8:50:58 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	25 (0x00000019) Unknown error code
Computer ID	1133771
Run time	7 days 18 hours 27 min 48 sec
CPU time	7 days 0 hours 20 min 58 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.36 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <message> process exited with code 25 (0x19, -231) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(10434,0xa05fa540) malloc: * error for object 0x802000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10435, selfPID=10435, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/p2btko.pjb2c10 is not a valid UM file. Error converting file to netcdf: dataout/p2btko.pjb2c10 Error: Input file: dataout/p2btko.pib2c10 is not a valid UM file. Error converting file to netcdf: dataout/p2btko.pib2c10 Error: Input file: dataout/p2btko.pfb2c10 is not a valid UM file. Error converting file to netcdf: dataout/p2btko.pfb2c10 Error: Input file: dataout/p2btka.phb2c10 is not a valid UM file. Error converting file to netcdf: dataout/p2btka.phb2c10 Error: Input file: dataout/p2btka.pgb2c10 is not a valid UM file. Error converting file to netcdf: dataout/p2btka.pgb2c10 Error: Input file: dataout/p2btka.peb2c10 is not a valid UM file. Error converting file to netcdf: dataout/p2btka.peb2c10 Error: Input file: dataout/p2btka.pdb2c10 is not a valid UM file. Error converting file to netcdf: dataout/p2btka.pdb2c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... OPEN: File Creation Failed: No space left on device OPEN: Unable to Open File dataout/p2btka.dac1460 for Read/Write Model crashed: DUMPCTL : Fail to open output dump - may already exist tmp/pipe_dummy 2048 forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_p2bt_1900_40_007220601/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0042E622 Unknown Unknown Unknown hadcm3n_um_6.07_i 0042D0EB Unknown Unknown Unknown hadcm3n_um_6.07_i 003EF4DD Unknown Unknown Unknown hadcm3n_um_6.07_i 003A66F5 Unknown Unknown Unknown hadcm3n_um_6.07_i 003A5E57 Unknown Unknown Unknown hadcm3n_um_6.07_i 003E1953 Unknown Unknown Unknown hadcm3n_um_6.07_i 000D9389 Unknown Unknown Unknown hadcm3n_um_6.07_i 0015826A Unknown Unknown Unknown hadcm3n_um_6.07_i 00163AB7 Unknown Unknown Unknown hadcm3n_um_6.07_i 0038CE7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0038A1ED Unknown Unknown Unknown hadcm3n_um_6.07_i 000048E1 Unknown Unknown Unknown hadcm3n_um_6.07_i 000027FB Unknown Unknown Unknown hadcm3n_um_6.07_i 00002729 Unknown Unknown Unknown Unknown 00000003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28422, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_p2bt_1900_40_007220601/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0042E622 Unknown Unknown Unknown hadcm3n_um_6.07_i 0042D0EB Unknown Unknown Unknown hadcm3n_um_6.07_i 003EF4DD Unknown Unknown Unknown hadcm3n_um_6.07_i 003A66F5 Unknown Unknown Unknown hadcm3n_um_6.07_i 003A5E57 Unknown Unknown Unknown hadcm3n_um_6.07_i 003E1E32 Unknown Unknown Unknown hadcm3n_um_6.07_i 003DF4CC Unknown Unknown Unknown hadcm3n_um_6.07_i 003651C5 Unknown Unknown Unknown hadcm3n_um_6.07_i 002DE95F Unknown Unknown Unknown hadcm3n_um_6.07_i 00389F90 Unknown Unknown Unknown hadcm3n_um_6.07_i 000048E1 Unknown Unknown Unknown hadcm3n_um_6.07_i 000027FB Unknown Unknown UnknCalled boinc_finish SIGSEGV: segmentation violation hadcm3n_p2bt_1900_40_007220601/datain/masks/cpdn_omsk_global_700.nc: write error (disk full?). Continue? (y/n/^C) warning: hadcm3n_p2bt_1900_40_007220601/datain/masks/cpdn_omsk_global_700.nc is probably truncated checkdir: cannot create extraction directory: hadcm3n_p2bt_1900_40_007220601/jobs Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 May 2011 04:34:55	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	518,400	596,000	1.1497
15 May 2011 16:58:31	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	492,480	566,559	1.1504
15 May 2011 05:39:17	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	466,560	537,066	1.1511
14 May 2011 20:49:33	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	440,640	507,740	1.1523
14 May 2011 11:52:25	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	414,720	478,250	1.1532
14 May 2011 03:05:37	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	388,800	448,866	1.1545
13 May 2011 19:19:14	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	362,880	419,316	1.1555
10 May 2011 18:23:57	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	336,960	389,587	1.1562
10 May 2011 03:42:53	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	311,040	359,240	1.1550
09 May 2011 11:23:24	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	285,120	329,589	1.1560
09 May 2011 02:11:50	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	259,200	299,766	1.1565
08 May 2011 17:18:08	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	233,280	270,256	1.1585
08 May 2011 08:03:20	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	207,360	240,172	1.1582
07 May 2011 20:50:08	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	181,440	210,461	1.1599
07 May 2011 10:42:23	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	155,520	179,785	1.1560
07 May 2011 00:56:43	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	129,600	149,797	1.1558
05 May 2011 21:27:39	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	103,680	118,616	1.1441
05 May 2011 09:34:03	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	77,760	89,029	1.1449
04 May 2011 23:05:00	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	51,840	59,428	1.1464
04 May 2011 10:43:54	1133771	12821124	hadcm3n_p2bt_1900_40_007220601_0	25,920	29,644	1.1437