Task 11716323

Name	famous_v4a0_1799_200_006691310_4
Workunit	6894563
Created	26 Aug 2010, 15:56:32 UTC
Sent	7 Sep 2010, 8:57:00 UTC
Report deadline	7 Dec 2010, 16:24:11 UTC
Received	8 Sep 2010, 15:01:41 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	904604
Run time
CPU time	8 hours 47 min 26 sec
Validate state	Invalid
Credit	586.84
Device peak FLOPS	2.16 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-apple-darwin
Stderr	<core_client_version>6.2.18</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (1289): No heartbeat from core client for 30 sec - exiting (1289): No heartbeat from core client for 30 sec - exiting (1289): No heartbeat from core client for 30 sec - exiting (1289): No heartbeat from core client for 30 sec - exiting (1289): No heartbeat from core client for 30 sec - exiting (1289): No heartbeat from core client for 30 sec - exiting (1289): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (1366): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (1966): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (1996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2290): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (2673): No heartbeat from core client for 30 sec - exiting (2673): No heartbeat from core client for 30 sec - exiting (2673): No heartbeat from core client for 30 sec - exiting (2673): No heartbeat from core client for 30 sec - exiting (2673): No heartbeat from core client for 30 sec - exiting (2673): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (8465): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (8659): No heartbeat from core client for 30 sec - exiting (8659): No heartbeat from core client for 30 sec - exiting (8659): No heartbeat from core client for 30 sec - exiting (8659): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8662, selfPID=8662, iMonCtr=1 Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... (8750): No heartbeat from core client for 30 sec - exiting (8750): No heartbeat from core client for 30 sec - exiting (8750): No heartbeat from core client for 30 sec - exiting (8750): No heartbeat from core client for 30 sec - exiting (8750): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (8750): No heartbeat from core client for 30 sec - exiting (8750): No heartbeat from core client for 30 sec - exiting (8750): No heartbeat from core client for 30 sec - exiting (8750): No heartbeat from core client for 30 sec - exiting (8750): No heartbeat from core client for 30 sec - exiting BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... (9075): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (9075): No heartbeat from core client for 30 sec - exiting (9075): No heartbeat from core client for 30 sec - exiting (9075): No heartbeat from core client for 30 sec - exiting (9075): No heartbeat from core client for 30 sec - exiting (9075): No heartbeat from core client for 30 sec - exiting (9075): No heartbeat from core client for 30 sec - exiting (9075): No heartbeat from core client for 30 sec - exiting (9303): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (9798): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (10317): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (10379): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (10379): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (10958): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (10988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (11145): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (11145): No heartbeat from core client for 30 sec - exiting (11145): No heartbeat from core client for 30 sec - exiting (11145): No heartbeat from core client for 30 sec - exiting (11145): No heartbeat from core client for 30 sec - exiting (11145): No heartbeat from core client for 30 sec - exiting (11145): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... (11404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (11426): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (11426): No heartbeat from core client for 30 sec - exiting (11426): No heartbeat from core client for 30 sec - exiting (11426): No heartbeat from core client for 30 sec - exiting (11426): No heartbeat from core client for 30 sec - exiting (11426): No heartbeat from core client for 30 sec - exiting (11426): No heartbeat from core client for 30 sec - exiting (11426): No heartbeat from core client for 30 sec - exiting (11426): No heartbeat from core client for 30 sec - exiting (11426): No heartbeat from core client for 30 sec - exiting (11426): No heartbeat from core client for 30 sec - exiting (11426): No heartbeat from core client for 30 sec - exiting (11426): No heartbeat from core client for 30 sec - exiting (11426): No heartbeat from core client for 30 sec - exiting (11426): No heartbeat from core client for 30 sec - exiting (11426): No heartbeat from core client for 30 sec - exiting (11438): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (11438): No heartbeat from core client for 30 sec - exiting (11438): No heartbeat from core client for 30 sec - exiting (11438): No heartbeat from core client for 30 sec - exiting (11438): No heartbeat from core client for 30 sec - exiting (11438): No heartbeat from core client for 30 sec - exiting (11438): No heartbeat from core client for 30 sec - exiting (11438): No heartbeat from core client for 30 sec - exiting (11635): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Error converting file to netcdf: /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4a0_1799_200_006691310/dataout/v4a0la#pa000001817c1+ Error converting file to netcdf: /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4a0_1799_200_006691310/dataout/v4a0la#pb000001817c1+ Error converting file to netcdf: /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4a0_1799_200_006691310/dataout/v4a0lo#pi000001817c1+ Error converting file to netcdf: /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v4a0_1799_200_006691310/dataout/v4a0lo#pj000001817c1+ MainError: 01:47:40 PM No files match the supplied pattern. MainError: 01:47:40 PM No files match the supplied pattern. (11681): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11681): No heartbeat from core client for 30 sec - exiting (11789): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (11789): No heartbeat from core client for 30 sec - exiting (11789): No heartbeat from core client for 30 sec - exiting (11789): No heartbeat from core client for 30 sec - exiting (11789): No heartbeat from core client for 30 sec - exiting (11789): No heartbeat from core client for 30 sec - exiting (11789): No heartbeat from core client for 30 sec - exiting (11789): No heartbeat from core client for 30 sec - exiting (11789): No heartbeat from core client for 30 sec - exiting (11789): No heartbeat from core client for 30 sec - exiting (11804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (11828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (11828): No heartbeat from core client for 30 sec - exiting (11828): No heartbeat from core client for 30 sec - exiting (11828): No heartbeat from core client for 30 sec - exiting (11828): No heartbeat from core client for 30 sec - exiting (11840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (12061): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy (12088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy Sorry, too many model crashes! :-( (12153): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Sep 2010 14:47:56	904604	11716323	famous_v4a0_1799_200_006691310_4	177,866	31,188	0.1753
08 Sep 2010 13:53:45	904604	11716323	famous_v4a0_1799_200_006691310_4	168,506	29,673	0.1761
08 Sep 2010 12:29:31	904604	11716323	famous_v4a0_1799_200_006691310_4	149,786	26,442	0.1765
08 Sep 2010 11:23:28	904604	11716323	famous_v4a0_1799_200_006691310_4	140,426	24,833	0.1768
08 Sep 2010 10:30:44	904604	11716323	famous_v4a0_1799_200_006691310_4	131,066	23,157	0.1767
08 Sep 2010 09:43:36	904604	11716323	famous_v4a0_1799_200_006691310_4	121,706	21,571	0.1772
08 Sep 2010 08:56:36	904604	11716323	famous_v4a0_1799_200_006691310_4	112,346	19,844	0.1766
08 Sep 2010 07:30:21	904604	11716323	famous_v4a0_1799_200_006691310_4	102,986	18,339	0.1781
08 Sep 2010 06:37:39	904604	11716323	famous_v4a0_1799_200_006691310_4	93,626	16,629	0.1776
08 Sep 2010 05:50:19	904604	11716323	famous_v4a0_1799_200_006691310_4	84,266	15,035	0.1784
07 Sep 2010 16:59:13	904604	11716323	famous_v4a0_1799_200_006691310_4	74,906	13,323	0.1779
07 Sep 2010 16:14:50	904604	11716323	famous_v4a0_1799_200_006691310_4	65,546	11,634	0.1775
07 Sep 2010 15:32:37	904604	11716323	famous_v4a0_1799_200_006691310_4	56,186	9,923	0.1766
07 Sep 2010 14:50:14	904604	11716323	famous_v4a0_1799_200_006691310_4	46,826	8,231	0.1758
07 Sep 2010 14:02:06	904604	11716323	famous_v4a0_1799_200_006691310_4	37,466	6,474	0.1728
07 Sep 2010 12:57:28	904604	11716323	famous_v4a0_1799_200_006691310_4	28,106	4,908	0.1746
07 Sep 2010 10:24:51	904604	11716323	famous_v4a0_1799_200_006691310_4	9,386	1,631	0.1738