Task 13012013

Name	hadcm3n_t0ed_1940_40_007310760_0
Workunit	7508190
Created	27 Jun 2011, 18:03:18 UTC
Sent	27 Jun 2011, 18:03:29 UTC
Report deadline	27 Sep 2011, 1:30:40 UTC
Received	16 Jul 2011, 3:54:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1140953
Run time	12 days 7 hours 9 min 48 sec
CPU time	9 days 23 hours 47 min 46 sec
Validate state	Invalid
Credit	6,531.84
Device peak FLOPS	2.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(774,0xa042e540) malloc: * error for object 0x1002604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(774,0xa042e540) malloc: * error for object 0x1002600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(774,0xa042e540) malloc: * error for object 0x1002604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(774,0xa042e540) malloc: * error for object 0x1002600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(774,0xa042e540) malloc: * error for object 0x1002604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(774,0xa042e540) malloc: * error for object 0x1002600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(774,0xa042e540) malloc: * error for object 0x1002604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=782, selfPID=782, iMonCtr=1 hadcm3n_6.07_i686-apple-darwin(22621,0xa042e540) malloc: * error for object 0x801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22621,0xa042e540) malloc: * error for object 0x801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22621,0xa042e540) malloc: * error for object 0x1011e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22621,0xa042e540) malloc: * error for object 0x1011e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22621,0xa042e540) malloc: * error for object 0x1011e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22621,0xa042e540) malloc: * error for object 0x1011e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22621,0xa042e540) malloc: * error for object 0x1011e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22621,0xa042e540) malloc: * error for object 0x1011e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22621,0xa042e540) malloc: * error for object 0x1011e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22621,0xa042e540) malloc: * error for object 0x1011e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22621,0xa042e540) malloc: * error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(22621,0xa042e540) malloc: * error for object 0x1000e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 134905) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4174, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 134905) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4174, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 134905) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4174, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 134905) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4174, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 134905) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4174, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 134905) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4174, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Jul 2011 12:45:12	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	544,320	839,840	1.5429
25 Jul 2011 12:45:12	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	518,400	799,215	1.5417
25 Jul 2011 12:45:12	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	492,480	758,300	1.5398
25 Jul 2011 12:45:12	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	466,560	719,719	1.5426
25 Jul 2011 12:45:11	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	440,640	678,579	1.5400
10 Jul 2011 23:15:02	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	414,720	639,043	1.5409
10 Jul 2011 01:39:05	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	388,800	600,059	1.5434
09 Jul 2011 07:45:02	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	362,880	560,841	1.5455
08 Jul 2011 15:01:13	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	336,960	520,933	1.5460
08 Jul 2011 01:26:07	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	311,040	482,688	1.5519
07 Jul 2011 16:04:45	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	285,120	442,424	1.5517
07 Jul 2011 15:36:20	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	259,200	403,313	1.5560
05 Jul 2011 23:27:28	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	233,280	363,728	1.5592
05 Jul 2011 05:37:34	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	207,360	324,706	1.5659
04 Jul 2011 17:17:02	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	181,440	285,780	1.5751
04 Jul 2011 02:08:28	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	155,520	245,240	1.5769
03 Jul 2011 12:31:31	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	129,600	204,421	1.5773
02 Jul 2011 23:24:55	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	103,680	164,685	1.5884
02 Jul 2011 05:28:43	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	77,760	122,641	1.5772
30 Jun 2011 05:03:09	1140953	13012013	hadcm3n_t0ed_1940_40_007310760_0	51,840	82,100	1.5837