Task 13133172

Name	hadcm3n_yjx0_1900_40_007358670_2
Workunit	7556100
Created	8 Jul 2011, 7:48:28 UTC
Sent	8 Jul 2011, 7:54:45 UTC
Report deadline	7 Oct 2011, 15:21:56 UTC
Received	22 Jul 2011, 22:37:28 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1154447
Run time	5 days 13 hours 54 min 38 sec
CPU time	5 days 12 hours 24 min 40 sec
Validate state	Invalid
Credit	7,464.96
Device peak FLOPS	4.29 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(21832,0xa0173540) malloc: * error for object 0x2833a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21832,0xa0173540) malloc: * error for object 0x2833a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21832,0xa0173540) malloc: * error for object 0x4034204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21832,0xa0173540) malloc: * error for object 0x4845204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21832,0xa0173540) malloc: * error for object 0x2804e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21832,0xa0173540) malloc: * error for object 0x2805c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21832,0xa0173540) malloc: * error for object 0x2805c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21832,0xa0173540) malloc: * error for object 0x2805c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21832,0xa0173540) malloc: * error for object 0x2805c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21832,0xa0173540) malloc: * error for object 0x2805c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21832,0xa0173540) malloc: * error for object 0x2805c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21832,0xa0173540) malloc: * error for object 0x2805c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21832,0xa0173540) malloc: * error for object 0x2805c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21832,0xa0173540) malloc: * error for object 0x2805c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(21832,0xa0173540) malloc: * error for object 0x2805c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 137050) failed! CPDN Monitor - Quit request from BOINC... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 137050) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=571, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 137050) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=571, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 137050) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=571, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 137050) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=571, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 137050) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=571, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 137050) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=571, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Jul 2011 20:28:47	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	622,080	462,652	0.7437
25 Jul 2011 18:48:49	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	596,160	443,228	0.7435
25 Jul 2011 18:18:24	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	570,240	423,908	0.7434
25 Jul 2011 18:09:32	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	544,320	404,541	0.7432
25 Jul 2011 17:44:24	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	518,400	385,246	0.7431
25 Jul 2011 17:27:54	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	492,480	366,080	0.7433
25 Jul 2011 17:17:52	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	466,560	346,483	0.7426
25 Jul 2011 16:38:51	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	440,640	326,767	0.7416
25 Jul 2011 16:25:06	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	414,720	307,567	0.7416
25 Jul 2011 16:00:07	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	388,800	288,375	0.7417
25 Jul 2011 15:52:25	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	362,880	269,206	0.7419
25 Jul 2011 15:45:13	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	336,960	250,158	0.7424
25 Jul 2011 15:34:07	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	311,040	231,071	0.7429
25 Jul 2011 15:03:08	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	285,120	211,903	0.7432
25 Jul 2011 14:53:01	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	259,200	192,703	0.7435
25 Jul 2011 14:39:24	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	233,280	173,404	0.7433
25 Jul 2011 13:02:24	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	207,360	154,132	0.7433
25 Jul 2011 13:02:24	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	181,440	134,844	0.7432
25 Jul 2011 13:02:24	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	155,520	115,581	0.7432
25 Jul 2011 13:02:24	1154447	13133172	hadcm3n_yjx0_1900_40_007358670_2	129,600	96,496	0.7446