Task 13421774

Name	hadcm3n_u0wc_1980_40_007458085_3
Workunit	7655588
Created	26 Sep 2011, 3:21:30 UTC
Sent	26 Sep 2011, 3:25:35 UTC
Report deadline	26 Dec 2011, 10:52:46 UTC
Received	25 Oct 2011, 14:59:56 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1052475
Run time	23 days 8 hours 13 min 21 sec
CPU time	22 days 18 hours 25 min 3 sec
Validate state	Invalid
Credit	5,287.68
Device peak FLOPS	1.01 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:58:45 (16414): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12384, selfPID=12384, iMonCtr=1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/u0wcko.pjj6c10 is not a valid UM file. Error converting file to netcdf: dataout/u0wcko.pjj6c10 Error: Input file: dataout/u0wcko.pij6c10 is not a valid UM file. Error converting file to netcdf: dataout/u0wcko.pij6c10 Error: Input file: dataout/u0wcko.pfj6c10 is not a valid UM file. Error converting file to netcdf: dataout/u0wcko.pfj6c10 Error: Input file: dataout/u0wcka.phj6c10 is not a valid UM file. Error converting file to netcdf: dataout/u0wcka.phj6c10 Error: Input file: dataout/u0wcka.pgj6c10 is not a valid UM file. Error converting file to netcdf: dataout/u0wcka.pgj6c10 Error: Input file: dataout/u0wcka.pej6c10 is not a valid UM file. Error converting file to netcdf: dataout/u0wcka.pej6c10 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Oct 2011 13:15:28	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	440,640	1,914,641	4.3451
31 Oct 2011 13:15:28	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	414,720	1,806,278	4.3554
16 Oct 2011 13:35:39	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	388,800	1,693,305	4.3552
15 Oct 2011 04:13:23	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	362,880	1,584,878	4.3675
13 Oct 2011 19:56:56	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	336,960	1,476,208	4.3810
12 Oct 2011 11:12:40	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	311,040	1,367,143	4.3954
11 Oct 2011 04:36:05	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	285,120	1,258,362	4.4134
09 Oct 2011 16:03:30	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	259,200	1,145,998	4.4213
08 Oct 2011 06:50:44	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	233,280	1,029,465	4.4130
06 Oct 2011 23:20:06	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	207,360	914,916	4.4122
05 Oct 2011 13:55:28	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	181,440	800,562	4.4123
04 Oct 2011 05:18:24	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	155,520	686,377	4.4134
02 Oct 2011 21:01:41	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	129,600	572,107	4.4144
01 Oct 2011 12:21:56	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	103,680	457,673	4.4143
30 Sep 2011 03:56:44	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	77,760	343,209	4.4137
28 Sep 2011 19:41:32	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	51,840	228,597	4.4097
27 Sep 2011 11:29:57	1052475	13421774	hadcm3n_u0wc_1980_40_007458085_3	25,920	114,347	4.4115