Task 13096254

Name	hadcm3n_yaag_1900_40_007346194_0
Workunit	7543624
Created	6 Jul 2011, 13:36:07 UTC
Sent	19 Jul 2011, 22:39:04 UTC
Report deadline	19 Oct 2011, 6:06:15 UTC
Received	13 Aug 2011, 19:32:58 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1027585
Run time	11 days 6 hours 48 min 35 sec
CPU time	10 days 5 hours 43 min 53 sec
Validate state	Invalid
Credit	5,598.72
Device peak FLOPS	2.43 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(34132,0xa07c6540) malloc: * error for object 0x811e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(34132,0xa07c6540) malloc: * error for object 0x812c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(34132,0xa07c6540) malloc: * error for object 0x812c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(34132,0xa07c6540) malloc: * error for object 0x812c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(34132,0xa07c6540) malloc: * error for object 0x812c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(34132,0xa07c6540) malloc: * error for object 0x812c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(34132,0xa07c6540) malloc: * error for object 0x812c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(34132,0xa07c6540) malloc: * error for object 0x812c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(34132,0xa07c6540) malloc: * error for object 0x812c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(34132,0xa07c6540) malloc: * error for object 0x1805204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(34132,0xa07c6540) malloc: * error for object 0x1805200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 138320) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=97225, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 138320) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=97225, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 138320) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=97225, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 138320) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=97225, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 138320) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=97225, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 138320) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=97225, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Aug 2011 14:34:55	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	466,560	863,427	1.8506
10 Aug 2011 03:51:28	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	440,640	814,608	1.8487
08 Aug 2011 03:40:18	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	414,720	765,371	1.8455
06 Aug 2011 19:52:38	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	388,800	717,041	1.8442
05 Aug 2011 06:04:36	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	362,880	668,724	1.8428
04 Aug 2011 06:35:23	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	336,960	619,794	1.8394
03 Aug 2011 10:47:27	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	311,040	571,075	1.8360
02 Aug 2011 19:37:46	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	285,120	522,229	1.8316
01 Aug 2011 12:06:36	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	259,200	473,178	1.8255
31 Jul 2011 13:47:14	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	233,280	424,131	1.8181
28 Jul 2011 02:06:45	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	207,360	375,045	1.8087
27 Jul 2011 01:50:42	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	181,440	326,139	1.7975
25 Jul 2011 23:52:23	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	155,520	279,006	1.7940
25 Jul 2011 22:24:30	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	129,600	233,070	1.7984
25 Jul 2011 21:46:15	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	103,680	186,854	1.8022
25 Jul 2011 20:36:13	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	77,760	138,052	1.7754
25 Jul 2011 19:04:02	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	51,840	89,984	1.7358
25 Jul 2011 19:04:02	1027585	13096254	hadcm3n_yaag_1900_40_007346194_0	25,920	45,182	1.7431