Task 11657748

Name	famous_ud6y_1999_200_006649941_5
Workunit	6853313
Created	15 Aug 2010, 1:47:58 UTC
Sent	15 Aug 2010, 2:07:01 UTC
Report deadline	14 Nov 2010, 9:34:12 UTC
Received	6 Sep 2010, 4:37:25 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	3 (0x00000003) Unknown error code
Computer ID	1023690
Run time	8 days 11 hours 33 min 21 sec
CPU time	6 days 16 hours 38 min 57 sec
Validate state	Invalid
Credit	3,458.83
Device peak FLOPS	2.42 GFLOPS
Application version	UK Met Office FAMOUS v6.11 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> The system cannot find the path specified. (0x3) - exit code 3 (0x3) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:43:31 (377480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 02:06:08 (508272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:06:11 (508272): No heartbeat from core client for 30 sec - exiting Model crashed: INIT_PP: I/O Error tmp/pipe_dummy 03:44:33 (508392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:44:36 (508392): No heartbeat from core client for 30 sec - exiting 03:44:37 (508392): No heartbeat from core client for 30 sec - exiting 03:44:38 (508392): No heartbeat from core client for 30 sec - exiting BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 03:50:29 (509344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:50:30 (509344): No heartbeat from core client for 30 sec - exiting 03:50:31 (509344): No heartbeat from core client for 30 sec - exiting BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 07:31:59 (506016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 11:02:19 (508504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 12:49:35 (508992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:40:57 (46916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFOUT: Write Failed: Not enough space BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 21:07:57 (49004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Skipping gmts_generator due to netcdf error 12 - Not enough space BUFFOUT: Write Failed: Not enough space BUFFOUT: C I/O Error - Return code = 32 Model crashed: STWORK : Error in PP_FILE tmp/pipe_dummy forrtl: Not enough storage is available to process this command. forrtl: severe (39): error during read, unit 3, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_ud6y_1999_200_006649941\datain\ancil\ctldata\stasets\X01003286 Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=140832, iMonCtr=1 Model crash detected, will try to restart... : Not enough space BUFFIN: C I/O Error ferror - Unit 61 - Return code = 32 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/famous_um_6.11_windows_intelx86.exe Model crforrtl: Not enough storage is available to process this command. forrtl: severe (39): error during read, unit 22, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_ud6y_1999_200_006649941\datain\ancil\ctldata\STASHmaster\STASHmaster_O Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. BUFFOUT: Write Failed: Not enough space BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy Not enough space BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=20664, selfPID=20664, iMonCtr=1 </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	1,048,346	572,630	0.5462
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	1,038,986	567,562	0.5463
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	1,029,626	562,288	0.5461
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	1,020,266	557,221	0.5462
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	1,010,906	552,157	0.5462
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	1,001,546	547,102	0.5463
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	992,186	542,038	0.5463
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	982,826	536,982	0.5464
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	973,466	531,927	0.5464
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	964,106	526,860	0.5465
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	954,746	521,802	0.5465
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	945,386	516,736	0.5466
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	936,026	511,675	0.5466
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	926,666	506,608	0.5467
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	917,306	501,546	0.5468
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	907,946	496,476	0.5468
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	898,586	491,416	0.5469
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	889,226	486,359	0.5469
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	879,866	481,299	0.5470
06 Sep 2010 04:39:55	1023690	11657748	famous_ud6y_1999_200_006649941_5	870,506	476,243	0.5471