Task 12016088

Name	hadcm3igeo_w1m0_2000_80_06760969_2
Workunit	6964285
Created	12 Nov 2010, 15:31:37 UTC
Sent	14 Nov 2010, 17:13:13 UTC
Report deadline	15 May 2012, 13:56:22 UTC
Received	3 Dec 2010, 11:20:53 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	909770
Run time	12 days 19 hours 17 min 48 sec
CPU time	12 days 5 hours 43 min 4 sec
Validate state	Invalid
Credit	4,898.88
Device peak FLOPS	1.90 GFLOPS
Application version	HadCM3L Coupled Model Experiment Optimised File I/O v6.04 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> scan: 09p0_12_2000_atmos.gz scan: 09p0_12_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnpkhdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1m0_2000_80_06760969.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz CPDN Monitor - Quit request from BOINC... scan: 09p0_12_2000_atmos.gz scan: 09p0_12_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnpkhdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1m0_2000_80_06760969.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz CPDN Monitor - Quit request from BOINC... scan: 09p0_12_2000_atmos.gz scan: 09p0_12_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnpkhdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1m0_2000_80_06760969.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz CPDN Monitor - Quit request from BOINC... scan: 09p0_12_2000_atmos.gz scan: 09p0_12_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnpkhdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1m0_2000_80_06760969.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz CPDN Monitor - Quit request from BOINC... scan: 09p0_12_2000_atmos.gz scan: 09p0_12_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnpkhdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1m0_2000_80_06760969.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz CPDN Monitor - Quit request from BOINC... scan: 09p0_12_2000_atmos.gz scan: 09p0_12_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnpkhdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1m0_2000_80_06760969.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz CPDN Monitor - Quit request from BOINC... scan: 09p0_12_2000_atmos.gz scan: 09p0_12_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnpkhdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1m0_2000_80_06760969.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz CPDN Monitor - Quit request from BOINC... scan: 09p0_12_2000_atmos.gz scan: 09p0_12_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnpkhdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1m0_2000_80_06760969.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz CPDN Monitor - Quit request from BOINC... scan: 09p0_12_2000_atmos.gz scan: 09p0_12_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnpkhdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1m0_2000_80_06760969.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz CPDN Monitor - Quit request from BOINC... scan: 09p0_12_2000_atmos.gz scan: 09p0_12_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnpkhdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1m0_2000_80_06760969.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz error: cannot delete old hadcm3igeo_w1m0_2000_80_06760969/jobs/yafbg.PRESM_O error: cannot delete old hadcm3igeo_w1m0_2000_80_06760969/datain/ancil/basin.index No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=33224, selfPID=33224, iMonCtr=1 BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error in converting file dataout/w1m0fo.pjl4c10 to netcdf format. Error in converting file dataout/w1m0fo.pil4c10 to netcdf format. Error in converting file dataout/w1m0fo.pfl4c10 to netcdf format. Error in converting file dataout/w1m0fa.phl4c10 to netcdf format. Error in converting file dataout/w1m0fa.pgl4c10 to netcdf format. Error in converting file dataout/w1m0fa.pel4c10 to netcdf format. Error in converting file dataout/w1m0fa.pdl4c10 to netcdf format. scan: 09p0_12_2000_atmos.gz scan: 09p0_12_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnpkhdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1m0_2000_80_06760969.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz Model crashed: Model crashed: Model crashed: Model crashed: Model crashed: Model crashed: Sorry, too many model crashes! :-( called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Dec 2010 13:02:11	909770	12016088	hadcm3igeo_w1m0_2000_80_06760969_2	388,800	1,023,109	2.6315
01 Dec 2010 16:12:48	909770	12016088	hadcm3igeo_w1m0_2000_80_06760969_2	362,880	956,581	2.6361
30 Nov 2010 19:28:59	909770	12016088	hadcm3igeo_w1m0_2000_80_06760969_2	336,960	890,753	2.6435
29 Nov 2010 22:38:41	909770	12016088	hadcm3igeo_w1m0_2000_80_06760969_2	311,040	824,209	2.6498
29 Nov 2010 09:06:12	909770	12016088	hadcm3igeo_w1m0_2000_80_06760969_2	285,120	760,165	2.6661
28 Nov 2010 06:58:36	909770	12016088	hadcm3igeo_w1m0_2000_80_06760969_2	259,200	697,166	2.6897
27 Nov 2010 11:26:28	909770	12016088	hadcm3igeo_w1m0_2000_80_06760969_2	233,280	634,114	2.7183
26 Nov 2010 15:54:12	909770	12016088	hadcm3igeo_w1m0_2000_80_06760969_2	207,360	571,016	2.7537
25 Nov 2010 16:32:30	909770	12016088	hadcm3igeo_w1m0_2000_80_06760969_2	181,440	504,752	2.7819
24 Nov 2010 12:21:09	909770	12016088	hadcm3igeo_w1m0_2000_80_06760969_2	155,520	433,112	2.7849
23 Nov 2010 12:00:31	909770	12016088	hadcm3igeo_w1m0_2000_80_06760969_2	129,600	361,037	2.7858
22 Nov 2010 12:05:17	909770	12016088	hadcm3igeo_w1m0_2000_80_06760969_2	103,680	288,953	2.7870
21 Nov 2010 12:01:26	909770	12016088	hadcm3igeo_w1m0_2000_80_06760969_2	77,760	216,822	2.7883
20 Nov 2010 12:07:26	909770	12016088	hadcm3igeo_w1m0_2000_80_06760969_2	51,840	144,783	2.7929
19 Nov 2010 11:57:06	909770	12016088	hadcm3igeo_w1m0_2000_80_06760969_2	25,920	72,709	2.8051