Task 12361714

Name	famous_vdj1_1799_200_006703299_6
Workunit	6906552
Created	4 Dec 2010, 17:42:46 UTC
Sent	4 Dec 2010, 17:51:07 UTC
Report deadline	6 Mar 2011, 1:18:18 UTC
Received	18 Jan 2011, 18:03:42 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	564402
Run time	9 days 3 hours 15 min 3 sec
CPU time	9 days 16 hours 0 min 56 sec
Validate state	Invalid
Credit	4,724.98
Device peak FLOPS	2.15 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... (15388): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy (4738): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... (23560): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... (14680): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( (32443): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Jan 2011 22:41:15	564402	12361714	famous_vdj1_1799_200_006703299_6	1,432,106	831,566	0.5807
15 Jan 2011 21:06:58	564402	12361714	famous_vdj1_1799_200_006703299_6	1,422,746	826,132	0.5807
15 Jan 2011 18:22:13	564402	12361714	famous_vdj1_1799_200_006703299_6	1,413,386	820,699	0.5807
15 Jan 2011 16:49:45	564402	12361714	famous_vdj1_1799_200_006703299_6	1,404,026	815,268	0.5807
15 Jan 2011 15:25:03	564402	12361714	famous_vdj1_1799_200_006703299_6	1,394,666	809,836	0.5807
15 Jan 2011 12:47:28	564402	12361714	famous_vdj1_1799_200_006703299_6	1,385,306	804,394	0.5807
15 Jan 2011 12:47:28	564402	12361714	famous_vdj1_1799_200_006703299_6	1,375,946	798,954	0.5807
15 Jan 2011 12:47:28	564402	12361714	famous_vdj1_1799_200_006703299_6	1,366,586	793,511	0.5807
15 Jan 2011 12:47:28	564402	12361714	famous_vdj1_1799_200_006703299_6	1,357,226	788,074	0.5807
15 Jan 2011 12:47:28	564402	12361714	famous_vdj1_1799_200_006703299_6	1,347,866	782,631	0.5806
15 Jan 2011 12:29:47	564402	12361714	famous_vdj1_1799_200_006703299_6	1,338,506	777,175	0.5806
14 Jan 2011 21:13:58	564402	12361714	famous_vdj1_1799_200_006703299_6	1,329,146	771,698	0.5806
14 Jan 2011 19:28:50	564402	12361714	famous_vdj1_1799_200_006703299_6	1,319,786	766,206	0.5806
14 Jan 2011 15:05:47	564402	12361714	famous_vdj1_1799_200_006703299_6	1,310,426	760,733	0.5805
14 Jan 2011 14:39:50	564402	12361714	famous_vdj1_1799_200_006703299_6	1,301,066	755,283	0.5805
14 Jan 2011 14:39:50	564402	12361714	famous_vdj1_1799_200_006703299_6	1,291,706	749,824	0.5805
14 Jan 2011 14:39:50	564402	12361714	famous_vdj1_1799_200_006703299_6	1,282,346	744,369	0.5805
14 Jan 2011 14:39:50	564402	12361714	famous_vdj1_1799_200_006703299_6	1,272,986	738,915	0.5805
14 Jan 2011 14:39:50	564402	12361714	famous_vdj1_1799_200_006703299_6	1,263,626	733,464	0.5804
14 Jan 2011 14:39:50	564402	12361714	famous_vdj1_1799_200_006703299_6	1,254,266	728,011	0.5804