Task 11446606

Name	famous_u4vg_1199_200_006639159_0
Workunit	6842531
Created	10 Jun 2010, 11:56:09 UTC
Sent	25 Jul 2010, 0:25:49 UTC
Report deadline	24 Oct 2010, 7:53:00 UTC
Received	2 Aug 2010, 0:49:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	998153
Run time	4 days 5 hours 0 min 7 sec
CPU time	3 days 17 hours 49 min 51 sec
Validate state	Invalid
Credit	3,736.76
Device peak FLOPS	1.48 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-apple-darwin
Stderr	<core_client_version>6.6.36</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... (376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_u4vg_1199_200_006639159/dataout/stdout_um.txt Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=383, iMonCtr=1 Model crash detected, will try to restart... (383): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_u4vg_1199_200_006639159/dataout/stdout_um.txt Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2760, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_u4vg_1199_200_006639159/dataout/stdout_um.txt Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2760, iMonCtr=1 Model crash detected, will try to restart... (2760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_u4vg_1199_200_006639159/dataout/stdout_um.txt Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2776, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( (2776): called boinc_finish BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_u4vg_1199_200_006639159/dataout/stdout_um.txt Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2788, iMonCtr=1 Model crash detected, will try to restart... (2788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (1125): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (1145): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (311): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (374): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (379): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (357): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (367): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (365): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... (294): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (323): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( (383): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Aug 2010 01:52:00	998153	11446606	famous_u4vg_1199_200_006639159_0	1,132,586	322,123	0.2844
02 Aug 2010 01:52:00	998153	11446606	famous_u4vg_1199_200_006639159_0	1,123,226	319,665	0.2846
02 Aug 2010 01:52:00	998153	11446606	famous_u4vg_1199_200_006639159_0	1,113,866	317,212	0.2848
02 Aug 2010 01:52:00	998153	11446606	famous_u4vg_1199_200_006639159_0	1,104,506	314,754	0.2850
01 Aug 2010 01:36:34	998153	11446606	famous_u4vg_1199_200_006639159_0	1,095,146	312,284	0.2852
01 Aug 2010 01:36:34	998153	11446606	famous_u4vg_1199_200_006639159_0	1,085,786	309,846	0.2854
01 Aug 2010 01:36:34	998153	11446606	famous_u4vg_1199_200_006639159_0	1,076,426	307,413	0.2856
01 Aug 2010 01:36:34	998153	11446606	famous_u4vg_1199_200_006639159_0	1,067,066	304,987	0.2858
01 Aug 2010 01:36:34	998153	11446606	famous_u4vg_1199_200_006639159_0	1,057,706	302,568	0.2861
01 Aug 2010 01:36:34	998153	11446606	famous_u4vg_1199_200_006639159_0	1,048,346	300,152	0.2863
01 Aug 2010 01:36:34	998153	11446606	famous_u4vg_1199_200_006639159_0	1,038,986	297,739	0.2866
01 Aug 2010 01:36:34	998153	11446606	famous_u4vg_1199_200_006639159_0	1,029,626	295,253	0.2868
01 Aug 2010 01:36:34	998153	11446606	famous_u4vg_1199_200_006639159_0	1,020,266	292,763	0.2869
01 Aug 2010 01:36:34	998153	11446606	famous_u4vg_1199_200_006639159_0	1,010,906	290,290	0.2872
01 Aug 2010 01:36:34	998153	11446606	famous_u4vg_1199_200_006639159_0	1,001,546	287,761	0.2873
01 Aug 2010 00:39:03	998153	11446606	famous_u4vg_1199_200_006639159_0	992,186	285,289	0.2875
31 Jul 2010 02:05:11	998153	11446606	famous_u4vg_1199_200_006639159_0	982,826	282,839	0.2878
31 Jul 2010 00:48:07	998153	11446606	famous_u4vg_1199_200_006639159_0	973,466	280,390	0.2880
31 Jul 2010 00:48:07	998153	11446606	famous_u4vg_1199_200_006639159_0	964,106	277,937	0.2883
31 Jul 2010 00:48:07	998153	11446606	famous_u4vg_1199_200_006639159_0	954,746	275,476	0.2885