Task 16003076

Name	hadcm3n_3dqn_1980_40_008389806_1
Workunit	8540665
Created	4 Sep 2013, 10:22:01 UTC
Sent	4 Sep 2013, 10:57:29 UTC
Report deadline	4 Dec 2013, 18:24:40 UTC
Received	8 Nov 2013, 11:32:53 UTC
Server state	Over
Outcome	Computation error
Client state	Aborted by user
Exit status	203 (0x000000CB) EXIT_ABORTED_VIA_GUI
Computer ID	1287069
Run time	33 days 14 hours 55 min 17 sec
CPU time	18 days 10 hours 11 min 59 sec
Validate state	Invalid
Credit	10,575.36
Device peak FLOPS	2.54 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> aborted by user </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... hadcm3n_6.07_i686-apple-darwin(252,0xa0946540) malloc: * error for object 0x801004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(252,0xa0946540) malloc: * error for object 0x801000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=252, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=252, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=252, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=252, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=252, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=252, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(254,0xa0946540) malloc: * error for object 0x822c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(254,0xa0946540) malloc: * error for object 0x800a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=254, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=254, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=254, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=254, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=254, iMonCtr=1 Model crash detected, will try to restart... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=254, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( SIGSEGV: segmentation violation </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Oct 2013 12:59:44	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	881,280	1,556,789	1.7665
07 Oct 2013 22:42:44	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	855,360	1,509,855	1.7652
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	829,440	1,463,268	1.7642
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	803,520	1,417,867	1.7646
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	777,600	1,372,533	1.7651
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	751,680	1,326,947	1.7653
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	725,760	1,281,357	1.7655
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	699,840	1,235,925	1.7660
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	673,920	1,190,469	1.7665
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	648,000	1,144,964	1.7669
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	622,080	1,099,481	1.7674
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	596,160	1,054,081	1.7681
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	570,240	1,008,618	1.7688
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	544,320	962,985	1.7692
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	518,400	917,317	1.7695
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	492,480	871,307	1.7692
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	466,560	825,245	1.7688
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	440,640	779,196	1.7683
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	414,720	733,318	1.7682
07 Oct 2013 11:27:26	1287069	16003076	hadcm3n_3dqn_1980_40_008389806_1	388,800	687,435	1.7681