Task 11825904

Name	famous_vl5r_1999_200_006713189_0
Workunit	6916442
Created	26 Aug 2010, 17:35:25 UTC
Sent	2 Nov 2010, 23:38:06 UTC
Report deadline	2 Feb 2011, 7:05:17 UTC
Received	1 Dec 2010, 11:54:07 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	960132
Run time	11 days 2 hours 7 min 11 sec
CPU time	9 days 7 hours 55 min 18 sec
Validate state	Invalid
Credit	3,767.65
Device peak FLOPS	2.33 GFLOPS
Application version	UK Met Office FAMOUS v6.11 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4012, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4152, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4952, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5636, iMonCtr=1 Model crash detected, will try to restart... 21:48:08 (5636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:48:09 (5636): No heartbeat from core client for 30 sec - exiting 21:48:10 (5636): No heartbeat from core client for 30 sec - exiting 21:48:11 (5636): No heartbeat from core client for 30 sec - exiting 21:48:12 (5636): No heartbeat from core client for 30 sec - exiting 21:48:14 (5636): No heartbeat from core client for 30 sec - exiting 21:48:15 (5636): No heartbeat from core client for 30 sec - exiting 21:48:17 (5636): No heartbeat from core client for 30 sec - exiting 21:48:18 (5636): No heartbeat from core client for 30 sec - exiting 21:48:19 (5636): No heartbeat from core client for 30 sec - exiting 21:48:20 (5636): No heartbeat from core client for 30 sec - exiting 21:48:21 (5636): No heartbeat from core client for 30 sec - exiting 21:48:22 (5636): No heartbeat from core client for 30 sec - exiting 21:48:32 (5636): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5000, selfPID=5000, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:59:23 (4996): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 23:59:24 (4996): No heartbeat from core client for 30 sec - exiting 23:59:25 (4996): No heartbeat from core client for 30 sec - exiting 23:59:26 (4996): No heartbeat from core client for 30 sec - exiting 23:59:27 (4996): No heartbeat from core client for 30 sec - exiting 23:59:28 (4996): No heartbeat from core client for 30 sec - exiting 23:59:29 (4996): No heartbeat from core client for 30 sec - exiting 23:59:30 (4996): No heartbeat from core client for 30 sec - exiting 23:59:31 (4996): No heartbeat from core client for 30 sec - exiting 23:59:32 (4996): No heartbeat from core client for 30 sec - exiting 23:59:33 (4996): No heartbeat from core client for 30 sec - exiting forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vl5r_1999_200_006713189\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6660, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vl5r_1999_200_006713189\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6660, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vl5r_1999_200_006713189\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6660, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vl5r_1999_200_006713189\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6660, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vl5r_1999_200_006713189\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6660, iMonCtr=1 Model crash detected, will try to restart... forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vl5r_1999_200_006713189\dataout\stdout_um.txt Image PC Routine Line Source famous_um_6.11_wi 008846D2 Unknown Unknown Unknown Stack trace terminated abnormally. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6660, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( 01:01:08 (6660): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Nov 2010 22:36:47	960132	11825904	famous_vl5r_1999_200_006713189_0	1,141,946	797,074	0.6980
23 Nov 2010 15:28:07	960132	11825904	famous_vl5r_1999_200_006713189_0	1,132,586	785,682	0.6937
23 Nov 2010 06:43:47	960132	11825904	famous_vl5r_1999_200_006713189_0	1,123,226	774,193	0.6893
22 Nov 2010 22:10:31	960132	11825904	famous_vl5r_1999_200_006713189_0	1,113,866	762,500	0.6846
22 Nov 2010 14:46:33	960132	11825904	famous_vl5r_1999_200_006713189_0	1,104,506	750,979	0.6799
22 Nov 2010 06:43:42	960132	11825904	famous_vl5r_1999_200_006713189_0	1,095,146	739,396	0.6752
21 Nov 2010 18:52:02	960132	11825904	famous_vl5r_1999_200_006713189_0	1,085,786	727,071	0.6696
21 Nov 2010 11:45:38	960132	11825904	famous_vl5r_1999_200_006713189_0	1,076,426	715,825	0.6650
21 Nov 2010 05:50:35	960132	11825904	famous_vl5r_1999_200_006713189_0	1,067,066	704,335	0.6601
20 Nov 2010 21:11:00	960132	11825904	famous_vl5r_1999_200_006713189_0	1,057,706	692,947	0.6551
20 Nov 2010 15:20:21	960132	11825904	famous_vl5r_1999_200_006713189_0	1,048,346	687,363	0.6557
20 Nov 2010 12:28:25	960132	11825904	famous_vl5r_1999_200_006713189_0	1,038,986	683,096	0.6575
20 Nov 2010 09:14:16	960132	11825904	famous_vl5r_1999_200_006713189_0	1,029,626	679,047	0.6595
20 Nov 2010 05:37:32	960132	11825904	famous_vl5r_1999_200_006713189_0	1,020,266	674,915	0.6615
20 Nov 2010 00:51:00	960132	11825904	famous_vl5r_1999_200_006713189_0	1,010,906	670,652	0.6634
19 Nov 2010 20:53:48	960132	11825904	famous_vl5r_1999_200_006713189_0	1,001,546	666,451	0.6654
19 Nov 2010 18:39:20	960132	11825904	famous_vl5r_1999_200_006713189_0	992,186	662,295	0.6675
19 Nov 2010 15:42:38	960132	11825904	famous_vl5r_1999_200_006713189_0	982,826	658,058	0.6696
19 Nov 2010 13:04:48	960132	11825904	famous_vl5r_1999_200_006713189_0	973,466	653,740	0.6716
19 Nov 2010 08:41:34	960132	11825904	famous_vl5r_1999_200_006713189_0	964,106	649,544	0.6737