Task 14649525

Name	hadcm3n_1026_1940_40_007957666_1
Workunit	8112778
Created	9 May 2012, 14:43:14 UTC
Sent	10 May 2012, 21:11:45 UTC
Report deadline	10 Aug 2012, 4:38:56 UTC
Received	27 May 2012, 8:37:41 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1178719
Run time	15 days 13 hours 28 min 49 sec
CPU time	12 days 0 hours 51 min 10 sec
Validate state	Invalid
Credit	9,953.28
Device peak FLOPS	3.32 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on 1026ko.daf0c20 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:52:32 (5212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/1026ko.pjh2c10 Error converting file to netcdf: dataout/1026ko.pih2c10 Error converting file to netcdf: dataout/1026ko.pfh2c10 Error converting file to netcdf: dataout/1026ka.phh2c10 Error converting file to netcdf: dataout/1026ka.pgh2c10 Error converting file to netcdf: dataout/1026ka.peh2c10 Error converting file to netcdf: dataout/1026ka.pdh2c10 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2300, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2300, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2300, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2300, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1988, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1988, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 May 2012 21:04:21	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	829,440	1,037,365	1.2507
25 May 2012 10:34:53	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	803,520	1,006,560	1.2527
24 May 2012 23:40:50	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	777,600	975,305	1.2543
24 May 2012 10:37:06	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	751,680	942,838	1.2543
23 May 2012 23:22:15	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	725,760	909,786	1.2536
23 May 2012 13:53:22	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	699,840	877,254	1.2535
23 May 2012 02:09:03	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	673,920	844,044	1.2524
22 May 2012 14:10:47	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	648,000	810,855	1.2513
22 May 2012 02:45:55	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	622,080	777,725	1.2502
21 May 2012 15:39:31	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	596,160	745,280	1.2501
21 May 2012 03:57:34	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	570,240	715,485	1.2547
20 May 2012 15:50:58	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	544,320	683,744	1.2561
20 May 2012 04:05:37	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	518,400	652,717	1.2591
19 May 2012 16:34:19	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	492,480	619,581	1.2581
19 May 2012 03:28:22	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	466,560	585,116	1.2541
18 May 2012 16:06:51	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	440,640	550,136	1.2485
18 May 2012 06:54:07	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	414,720	514,696	1.2411
17 May 2012 21:00:50	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	388,800	479,257	1.2327
17 May 2012 09:41:23	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	362,880	443,899	1.2233
16 May 2012 20:37:22	1178719	14649525	hadcm3n_1026_1940_40_007957666_1	336,960	409,355	1.2148