Task 13393468

Name	hadcm3n_t3b2_1940_40_007446089_4
Workunit	7643592
Created	17 Sep 2011, 7:47:18 UTC
Sent	17 Sep 2011, 7:49:10 UTC
Report deadline	17 Dec 2011, 15:16:21 UTC
Received	3 Dec 2011, 9:30:00 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1169922
Run time	19 days 11 hours 17 min 51 sec
CPU time	18 days 10 hours 24 min 44 sec
Validate state	Invalid
Credit	11,508.48
Device peak FLOPS	2.39 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:04:05 (3488): Can't acquire lockfile (32) - waiting 35s 09:04:20 (5924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4760, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2388, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5940, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6912, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/t3b2ko.pjh2c10 Error converting file to netcdf: dataout/t3b2ko.pih2c10 Error converting file to netcdf: dataout/t3b2ko.pfh2c10 Error converting file to netcdf: dataout/t3b2ka.phh2c10 Error converting file to netcdf: dataout/t3b2ka.pgh2c10 Error converting file to netcdf: dataout/t3b2ka.peh2c10 Error converting file to netcdf: dataout/t3b2ka.pdh2c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Dec 2011 07:00:51	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	959,040	1,592,646	1.6607
02 Dec 2011 06:56:23	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	933,120	1,550,643	1.6618
01 Dec 2011 11:27:06	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	907,200	1,508,352	1.6626
29 Nov 2011 16:38:11	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	881,280	1,465,496	1.6629
26 Nov 2011 19:41:07	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	855,360	1,422,978	1.6636
25 Nov 2011 19:04:45	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	829,440	1,379,686	1.6634
24 Nov 2011 03:35:15	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	803,520	1,337,321	1.6643
21 Nov 2011 00:56:34	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	777,600	1,294,736	1.6650
19 Nov 2011 11:34:26	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	751,680	1,251,309	1.6647
17 Nov 2011 10:33:51	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	725,760	1,208,158	1.6647
16 Nov 2011 20:59:47	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	699,840	1,164,864	1.6645
16 Nov 2011 08:09:38	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	673,920	1,122,119	1.6651
16 Nov 2011 03:32:29	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	648,000	1,079,036	1.6652
16 Nov 2011 03:32:29	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	622,080	1,035,862	1.6652
08 Nov 2011 02:59:34	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	596,160	993,082	1.6658
06 Nov 2011 16:33:19	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	570,240	949,132	1.6644
06 Nov 2011 04:01:38	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	544,320	904,588	1.6619
05 Nov 2011 15:08:17	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	518,400	860,056	1.6591
05 Nov 2011 02:35:34	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	492,480	816,180	1.6573
04 Nov 2011 13:58:22	1169922	13393468	hadcm3n_t3b2_1940_40_007446089_4	466,560	772,442	1.6556