Task 12928488

Name	hadcm3n_o2xt_1940_40_007267452_1
Workunit	7465692
Created	3 Jun 2011, 3:29:11 UTC
Sent	3 Jun 2011, 3:29:27 UTC
Report deadline	2 Sep 2011, 10:56:38 UTC
Received	4 Jul 2011, 16:07:21 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1077692
Run time	21 days 9 hours 36 min 35 sec
CPU time	20 days 14 hours 27 min 25 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.17 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.59</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:52:04 (2806): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:52:15 (2806): No heartbeat from core client for 30 sec - exiting 22:52:16 (2806): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:07:55 (2889): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:07:56 (2889): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:34:48 (2788): No heartbeat from core client for 30 sec - exiting 23:35:00 (2788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:35:01 (2788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4541, selfPID=4541, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1347, selfPID=1347, iMonCtr=1 cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o2xt_1940_40_007267452/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o2xt_1940_40_007267452/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o2xt_1940_40_007267452/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o2xt_1940_40_007267452/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o2xt_1940_40_007267452/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o2xt_1940_40_007267452/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o2xt_1940_40_007267452/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o2xt_1940_40_007267452/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o2xt_1940_40_007267452/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o2xt_1940_40_007267452/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o2xt_1940_40_007267452/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_o2xt_1940_40_007267452/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 Jul 2011 12:06:12	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	777,600	1,780,088	2.2892
02 Jul 2011 23:55:21	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	751,680	1,720,010	2.2882
02 Jul 2011 00:50:56	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	725,760	1,662,903	2.2913
01 Jul 2011 04:52:06	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	699,840	1,607,452	2.2969
30 Jun 2011 12:12:01	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	673,920	1,553,111	2.3046
29 Jun 2011 15:12:15	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	648,000	1,497,044	2.3103
28 Jun 2011 19:00:16	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	622,080	1,442,310	2.3185
27 Jun 2011 01:51:57	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	596,160	1,382,828	2.3196
26 Jun 2011 08:59:36	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	570,240	1,322,727	2.3196
25 Jun 2011 12:54:55	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	544,320	1,251,785	2.2997
24 Jun 2011 19:56:31	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	518,400	1,191,317	2.2981
24 Jun 2011 02:02:44	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	492,480	1,133,328	2.3013
23 Jun 2011 05:56:15	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	466,560	1,076,094	2.3064
22 Jun 2011 13:42:03	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	440,640	1,018,516	2.3114
21 Jun 2011 18:23:59	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	414,720	955,527	2.3040
20 Jun 2011 18:54:56	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	388,800	889,517	2.2879
20 Jun 2011 02:22:59	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	362,880	829,976	2.2872
19 Jun 2011 22:06:42	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	336,960	771,174	2.2886
19 Jun 2011 22:06:42	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	311,040	705,085	2.2669
17 Jun 2011 06:18:12	1077692	12928488	hadcm3n_o2xt_1940_40_007267452_1	285,120	645,839	2.2651