Task 11968418

Name	famous_vkkm_599_200_006712428_5
Workunit	6915681
Created	5 Nov 2010, 0:11:45 UTC
Sent	5 Nov 2010, 0:19:27 UTC
Report deadline	4 Feb 2011, 7:46:38 UTC
Received	22 Nov 2010, 6:35:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1063040
Run time	8 days 16 hours 40 min 5 sec
CPU time	8 days 14 hours 18 min 23 sec
Validate state	Invalid
Credit	5,805.83
Device peak FLOPS	2.53 GFLOPS
Application version	UK Met Office FAMOUS v6.11 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 04:48:58 (6196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:48:59 (6196): No heartbeat from core client for 30 sec - exiting 04:49:01 (6196): No heartbeat from core client for 30 sec - exiting 04:49:02 (6196): No heartbeat from core client for 30 sec - exiting 04:49:03 (6196): No heartbeat from core client for 30 sec - exiting 04:49:04 (6196): No heartbeat from core client for 30 sec - exiting 04:49:05 (6196): No heartbeat from core client for 30 sec - exiting 04:49:06 (6196): No heartbeat from core client for 30 sec - exiting 04:49:07 (6196): No heartbeat from core client for 30 sec - exiting 04:49:08 (6196): No heartbeat from core client for 30 sec - exiting 04:49:09 (6196): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5692, selfPID=5692, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7028, selfPID=7028, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3860, selfPID=3860, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5892, selfPID=5892, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7600, selfPID=7600, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=336, selfPID=336, iMonCtr=1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=468, selfPID=468, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7148, selfPID=7148, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6056, selfPID=6056, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5524, selfPID=5524, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8108, selfPID=8108, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5648, selfPID=5648, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5184, selfPID=5184, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2184, selfPID=2184, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7808, selfPID=7808, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5540, selfPID=5540, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7888, selfPID=7888, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7908, selfPID=7908, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5904, selfPID=5904, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4968, selfPID=4968, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1340, selfPID=1340, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6248, selfPID=6248, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5544, selfPID=5544, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6668, selfPID=6668, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5368, selfPID=5368, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3336, selfPID=3336, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5888, selfPID=5888, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5392, selfPID=5392, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3016, selfPID=3016, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4824, selfPID=4824, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4112, selfPID=4112, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6636, selfPID=6636, iMonCtr=1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 C16:41:45 (3472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:41:46 (3472): No heartbeat from core client for 30 sec - exiting 16:41:48 (3472): No heartbeat from core client for 30 sec - exiting 16:41:49 (3472): No heartbeat from core client for 30 sec - exiting 16:41:50 (3472): No heartbeat from core client for 30 sec - exiting 16:41:51 (3472): No heartbeat from core client for 30 sec - exiting 16:41:52 (3472): No heartbeat from core client for 30 sec - exiting 16:41:53 (3472): No heartbeat from core client for 30 sec - exiting 16:41:54 (3472): No heartbeat from core client for 30 sec - exiting 16:41:55 (3472): No heartbeat from core client for 30 sec - exiting 16:41:56 (3472): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3548, selfPID=3548, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4832, selfPID=4832, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1408, selfPID=1408, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8692, selfPID=8692, iMonCtr=1 09:52:14 (1688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:52:15 (1688): No heartbeat from core client for 30 sec - exiting 09:52:16 (1688): No heartbeat from core client for 30 sec - exiting 09:52:17 (1688): No heartbeat from core client for 30 sec - exiting 09:52:18 (1688): No heartbeat from core client for 30 sec - exiting 09:52:20 (1688): No heartbeat from core client for 30 sec - exiting 09:52:21 (1688): No heartbeat from core client for 30 sec - exiting 09:52:22 (1688): No heartbeat from core client for 30 sec - exiting 09:52:23 (1688): No heartbeat from core client for 30 sec - exiting 09:52:24 (1688): No heartbeat from core client for 30 sec - exiting 09:52:25 (1688): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1840, selfPID=1840, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4372, selfPID=4372, iMonCtr=1 09:22:59 (4132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:23:00 (4132): No heartbeat from core client for 30 sec - exiting 09:23:01 (4132): No heartbeat from core client for 30 sec - exiting 09:23:02 (4132): No heartbeat from core client for 30 sec - exiting 09:23:03 (4132): No heartbeat from core client for 30 sec - exiting 09:23:04 (4132): No heartbeat from core client for 30 sec - exiting 09:23:05 (4132): No heartbeat from core client for 30 sec - exiting 09:23:07 (4132): No heartbeat from core client for 30 sec - exiting 09:23:08 (4132): No heartbeat from core client for 30 sec - exiting 09:23:09 (4132): No heartbeat from core client for 30 sec - exiting 09:23:10 (4132): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4256, selfPID=4256, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3528, selfPID=3528, iMonCtr=1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( 22:33:40 (3944): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Nov 2010 06:38:13	1063040	11968418	famous_vkkm_599_200_006712428_5	1,759,706	739,712	0.4204
21 Nov 2010 18:46:55	1063040	11968418	famous_vkkm_599_200_006712428_5	1,750,346	735,789	0.4204
21 Nov 2010 17:39:02	1063040	11968418	famous_vkkm_599_200_006712428_5	1,740,986	731,814	0.4203
21 Nov 2010 16:31:25	1063040	11968418	famous_vkkm_599_200_006712428_5	1,731,626	727,772	0.4203
21 Nov 2010 15:23:29	1063040	11968418	famous_vkkm_599_200_006712428_5	1,722,266	723,705	0.4202
21 Nov 2010 14:10:56	1063040	11968418	famous_vkkm_599_200_006712428_5	1,712,906	719,707	0.4202
21 Nov 2010 13:03:39	1063040	11968418	famous_vkkm_599_200_006712428_5	1,703,546	715,835	0.4202
21 Nov 2010 12:01:20	1063040	11968418	famous_vkkm_599_200_006712428_5	1,694,186	711,959	0.4202
21 Nov 2010 10:53:32	1063040	11968418	famous_vkkm_599_200_006712428_5	1,684,826	708,089	0.4203
21 Nov 2010 09:54:22	1063040	11968418	famous_vkkm_599_200_006712428_5	1,675,466	704,233	0.4203
21 Nov 2010 08:52:52	1063040	11968418	famous_vkkm_599_200_006712428_5	1,666,106	700,454	0.4204
21 Nov 2010 07:43:10	1063040	11968418	famous_vkkm_599_200_006712428_5	1,656,746	696,692	0.4205
21 Nov 2010 06:46:58	1063040	11968418	famous_vkkm_599_200_006712428_5	1,647,386	692,919	0.4206
21 Nov 2010 06:05:27	1063040	11968418	famous_vkkm_599_200_006712428_5	1,638,026	689,138	0.4207
21 Nov 2010 06:05:27	1063040	11968418	famous_vkkm_599_200_006712428_5	1,628,666	685,092	0.4206
21 Nov 2010 06:05:27	1063040	11968418	famous_vkkm_599_200_006712428_5	1,619,306	680,946	0.4205
21 Nov 2010 06:05:27	1063040	11968418	famous_vkkm_599_200_006712428_5	1,609,946	676,874	0.4204
21 Nov 2010 00:49:25	1063040	11968418	famous_vkkm_599_200_006712428_5	1,600,586	672,765	0.4203
20 Nov 2010 23:46:23	1063040	11968418	famous_vkkm_599_200_006712428_5	1,591,226	668,817	0.4203
20 Nov 2010 22:43:39	1063040	11968418	famous_vkkm_599_200_006712428_5	1,581,866	665,001	0.4204