Task 13086357

Name	hadcm3n_t2e0_1940_40_007311725_2
Workunit	7509155
Created	5 Jul 2011, 8:02:59 UTC
Sent	5 Jul 2011, 8:03:09 UTC
Report deadline	4 Oct 2011, 15:30:20 UTC
Received	21 Aug 2011, 16:39:36 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	969337
Run time	15 days 5 hours 56 min 45 sec
CPU time	15 days 2 hours 25 min 32 sec
Validate state	Invalid
Credit	8,709.12
Device peak FLOPS	2.32 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:23:55 (4612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/t2e0ko.pjf3c10 Error converting file to netcdf: dataout/t2e0ko.pif3c10 Error converting file to netcdf: dataout/t2e0ko.pff3c10 Error converting file to netcdf: dataout/t2e0ka.phf3c10 Error converting file to netcdf: dataout/t2e0ka.pgf3c10 Error converting file to netcdf: dataout/t2e0ka.pef3c10 Error converting file to netcdf: dataout/t2e0ka.pdf3c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4928, selfPID=4928, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:43:19 (5164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4468, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4468, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4468, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4292, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3932, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3932, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Aug 2011 15:16:19	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	725,760	1,264,671	1.7425
17 Aug 2011 22:42:59	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	699,840	1,221,574	1.7455
17 Aug 2011 05:11:51	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	673,920	1,177,900	1.7478
16 Aug 2011 12:05:11	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	648,000	1,133,959	1.7499
08 Aug 2011 17:36:51	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	622,080	1,090,509	1.7530
08 Aug 2011 05:07:28	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	596,160	1,045,968	1.7545
07 Aug 2011 16:17:00	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	570,240	1,000,934	1.7553
07 Aug 2011 03:00:51	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	544,320	956,101	1.7565
02 Aug 2011 18:45:19	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	518,400	910,276	1.7559
02 Aug 2011 05:56:19	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	492,480	864,600	1.7556
01 Aug 2011 17:48:18	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	466,560	819,549	1.7566
01 Aug 2011 05:04:02	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	440,640	775,831	1.7607
30 Jul 2011 17:51:45	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	414,720	732,296	1.7658
25 Jul 2011 22:29:05	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	388,800	688,594	1.7711
25 Jul 2011 22:00:24	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	362,880	644,730	1.7767
25 Jul 2011 21:10:10	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	336,960	600,849	1.7831
25 Jul 2011 19:38:43	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	311,040	555,922	1.7873
25 Jul 2011 19:22:50	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	285,120	512,795	1.7985
25 Jul 2011 19:00:04	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	259,200	467,789	1.8047
25 Jul 2011 18:45:10	969337	13086357	hadcm3n_t2e0_1940_40_007311725_2	233,280	420,511	1.8026