Task 11487172

Name	famous_ub4p_1999_200_006647268_2
Workunit	6850640
Created	10 Jun 2010, 13:06:23 UTC
Sent	11 Aug 2010, 0:57:26 UTC
Report deadline	10 Nov 2010, 8:24:37 UTC
Received	17 Aug 2010, 13:15:42 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1077167
Run time	3 days 0 hours 12 min 44 sec
CPU time	2 days 20 hours 35 min 12 sec
Validate state	Invalid
Credit	1,791.22
Device peak FLOPS	2.80 GFLOPS
Application version	UK Met Office FAMOUS v6.11 windows_intelx86
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:36:38 (7196): No heartbeat from core client for 30 sec - exiting 07:36:39 (7196): No heartbeat from core client for 30 sec - exiting 07:36:40 (7196): No heartbeat from core client for 30 sec - exiting 07:36:41 (7196): No heartbeat from core client for 30 sec - exiting 07:36:42 (7196): No heartbeat from core client for 30 sec - exiting 07:36:43 (7196): No heartbeat from core client for 30 sec - exiting 07:36:44 (7196): No heartbeat from core client for 30 sec - exiting 07:36:45 (7196): No heartbeat from core client for 30 sec - exiting 07:36:46 (7196): No heartbeat from core client for 30 sec - exiting 07:36:47 (7196): No heartbeat from core client for 30 sec - exiting 07:36:48 (7196): No heartbeat from core client for 30 sec - exiting 07:36:49 (7196): No heartbeat from core client for 30 sec - exiting 07:36:50 (7196): No heartbeat from core client for 30 sec - exiting 07:36:51 (7196): No heartbeat from core client for 30 sec - exiting 07:36:52 (7196): No heartbeat from core client for 30 sec - exiting 07:36:53 (7196): No heartbeat from core client for 30 sec - exiting 07:36:54 (7196): No heartbeat from core client for 30 sec - exiting 07:36:55 (7196): No heartbeat from core client for 30 sec - exiting 07:36:56 (7196): No heartbeat from core client for 30 sec - exiting 07:36:57 (7196): No heartbeat from core client for 30 sec - exiting 07:36:58 (7196): No heartbeat from core client for 30 sec - exiting 07:36:59 (7196): No heartbeat from core client for 30 sec - exiting 07:37:00 (7196): No heartbeat from core client for 30 sec - exiting 07:37:01 (7196): No heartbeat from core client for 30 sec - exiting 07:37:02 (7196): No heartbeat from core client for 30 sec - exiting No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8016, selfPID=8016, iMonCtr=1 Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:17:52 (2612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:59:41 (1704): No heartbeat from core client for 30 sec - exiting 22:59:42 (1704): No heartbeat from core client for 30 sec - exiting 22:59:43 (1704): No heartbeat from core client for 30 sec - exiting 22:59:44 (1704): No heartbeat from core client for 30 sec - exiting 22:59:45 (1704): No heartbeat from core client for 30 sec - exiting 22:59:46 (1704): No heartbeat from core client for 30 sec - exiting 22:59:47 (1704): No heartbeat from core client for 30 sec - exiting 22:59:48 (1704): No heartbeat from core client for 30 sec - exiting 22:59:49 (1704): No heartbeat from core client for 30 sec - exiting 22:59:50 (1704): No heartbeat from core client for 30 sec - exiting 22:59:51 (1704): No heartbeat from core client for 30 sec - exiting 22:59:52 (1704): No heartbeat from core client for 30 sec - exiting 22:59:53 (1704): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=412, selfPID=412, iMonCtr=1 Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... 23:27:30 (2276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:54:57 (4736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:20:41 (4688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:21:22 (2892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:43:33 (4340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:17:00 (3216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:31:06 (5768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:50:33 (828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:54:55 (2888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:03:13 (4632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:11:22 (3384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:12:06 (6072): No heartbeat from core client for 30 sec - exiting 05:12:07 (6072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:31:16 (1620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file D:\ProgramData\BOINC/projects/climateprediction.net/famous_ub4p_1999_200_006647268/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy cpdnmonitor: cannot open input file D:\ProgramData\BOINC/projects/climateprediction.net/famous_ub4p_1999_200_006647268/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy cpdnmonitor: cannot open input file D:\ProgramData\BOINC/projects/climateprediction.net/famous_ub4p_1999_200_006647268/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy cpdnmonitor: cannot open input file D:\ProgramData\BOINC/projects/climateprediction.net/famous_ub4p_1999_200_006647268/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy cpdnmonitor: cannot open input file D:\ProgramData\BOINC/projects/climateprediction.net/famous_ub4p_1999_200_006647268/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy cpdnmonitor: cannot open input file D:\ProgramData\BOINC/projects/climateprediction.net/famous_ub4p_1999_200_006647268/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy Sorry, too many model crashes! :-( 07:45:44 (4304): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Aug 2010 10:46:47	1077167	11487172	famous_ub4p_1999_200_006647268_2	542,906	245,454	0.4521
17 Aug 2010 01:30:36	1077167	11487172	famous_ub4p_1999_200_006647268_2	533,546	241,240	0.4521
17 Aug 2010 01:25:10	1077167	11487172	famous_ub4p_1999_200_006647268_2	524,186	236,846	0.4518
16 Aug 2010 22:52:32	1077167	11487172	famous_ub4p_1999_200_006647268_2	514,826	232,702	0.4520
16 Aug 2010 21:22:56	1077167	11487172	famous_ub4p_1999_200_006647268_2	505,466	228,438	0.4519
16 Aug 2010 11:20:53	1077167	11487172	famous_ub4p_1999_200_006647268_2	496,106	224,049	0.4516
16 Aug 2010 10:06:47	1077167	11487172	famous_ub4p_1999_200_006647268_2	486,746	219,883	0.4517
16 Aug 2010 08:40:35	1077167	11487172	famous_ub4p_1999_200_006647268_2	477,386	215,742	0.4519
16 Aug 2010 07:18:35	1077167	11487172	famous_ub4p_1999_200_006647268_2	468,026	211,315	0.4515
16 Aug 2010 07:04:04	1077167	11487172	famous_ub4p_1999_200_006647268_2	458,666	207,087	0.4515
16 Aug 2010 04:49:33	1077167	11487172	famous_ub4p_1999_200_006647268_2	449,306	202,758	0.4513
16 Aug 2010 02:14:48	1077167	11487172	famous_ub4p_1999_200_006647268_2	439,946	198,653	0.4515
16 Aug 2010 01:31:00	1077167	11487172	famous_ub4p_1999_200_006647268_2	430,586	194,302	0.4513
16 Aug 2010 00:17:32	1077167	11487172	famous_ub4p_1999_200_006647268_2	421,226	190,277	0.4517
15 Aug 2010 21:44:56	1077167	11487172	famous_ub4p_1999_200_006647268_2	411,866	186,043	0.4517
14 Aug 2010 01:40:39	1077167	11487172	famous_ub4p_1999_200_006647268_2	402,506	181,504	0.4509
13 Aug 2010 22:32:34	1077167	11487172	famous_ub4p_1999_200_006647268_2	393,146	177,319	0.4510
13 Aug 2010 19:21:55	1077167	11487172	famous_ub4p_1999_200_006647268_2	383,786	173,364	0.4517
13 Aug 2010 17:02:38	1077167	11487172	famous_ub4p_1999_200_006647268_2	374,426	169,118	0.4517
13 Aug 2010 15:48:11	1077167	11487172	famous_ub4p_1999_200_006647268_2	365,066	164,779	0.4514