Task 15479861

Name	hadcm3n_z9jd_1880_40_008255148_2
Workunit	8410272
Created	15 Dec 2012, 15:32:25 UTC
Sent	15 Dec 2012, 15:32:46 UTC
Report deadline	16 Mar 2013, 22:59:57 UTC
Received	3 Mar 2013, 15:08:57 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1148039
Run time	16 days 17 hours 20 min 1 sec
CPU time	14 days 13 hours 4 min 22 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	3.30 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/z9jdko.pj87c10 Error converting file to netcdf: dataout/z9jdko.pi87c10 Error converting file to netcdf: dataout/z9jdko.pf87c10 Error converting file to netcdf: dataout/z9jdka.ph87c10 Error converting file to netcdf: dataout/z9jdka.pg87c10 Error converting file to netcdf: dataout/z9jdka.pe87c10 Error converting file to netcdf: dataout/z9jdka.pd87c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3016, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3016, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:12:52 (3376): No heartbeat from core client for 30 sec - exiting 20:12:53 (3376): No heartbeat from core client for 30 sec - exiting 20:12:54 (3376): No heartbeat from core client for 30 sec - exiting 20:12:55 (3376): No heartbeat from core client for 30 sec - exiting 20:12:56 (3376): No heartbeat from core client for 30 sec - exiting 20:12:57 (3376): No heartbeat from core client for 30 sec - exiting 20:12:58 (3376): No heartbeat from core client for 30 sec - exiting 20:12:59 (3376): No heartbeat from core client for 30 sec - exiting 20:13:00 (3376): No heartbeat from core client for 30 sec - exiting 20:13:01 (3376): No heartbeat from core client for 30 sec - exiting 20:13:02 (3376): No heartbeat from core client for 30 sec - exiting 20:13:03 (3376): No heartbeat from core client for 30 sec - exiting 20:13:04 (3376): No heartbeat from core client for 30 sec - exiting 20:13:05 (3376): No heartbeat from core client for 30 sec - exiting 20:13:06 (3376): No heartbeat from core client for 30 sec - exiting 20:13:07 (3376): No heartbeat from core client for 30 sec - exiting 20:13:08 (3376): No heartbeat from core client for 30 sec - exiting 20:13:09 (3376): No heartbeat from core client for 30 sec - exiting 20:13:10 (3376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4064, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4064, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 17:49:27 (2956): No heartbeat from core client for 30 sec - exiting 17:49:28 (2956): No heartbeat from core client for 30 sec - exiting 17:49:29 (2956): No heartbeat from core client for 30 sec - exiting 17:49:30 (2956): No heartbeat from core client for 30 sec - exiting 17:49:31 (2956): No heartbeat from core client for 30 sec - exiting 17:49:32 (2956): No heartbeat from core client for 30 sec - exiting 17:49:33 (2956): No heartbeat from core client for 30 sec - exiting 17:49:34 (2956): No heartbeat from core client for 30 sec - exiting 17:49:35 (2956): No heartbeat from core client for 30 sec - exiting 17:49:36 (2956): No heartbeat from core client for 30 sec - exiting 17:49:37 (2956): No heartbeat from core client for 30 sec - exiting 17:49:38 (2956): No heartbeat from core client for 30 sec - exiting 17:49:39 (2956): No heartbeat from core client for 30 sec - exiting 17:49:40 (2956): No heartbeat from core client for 30 sec - exiting 17:49:41 (2956): No heartbeat from core client for 30 sec - exiting 17:49:42 (2956): No heartbeat from core client for 30 sec - exiting 17:49:43 (2956): No heartbeat from core client for 30 sec - exiting 17:49:44 (2956): No heartbeat from core client for 30 sec - exiting 17:49:45 (2956): No heartbeat from core client for 30 sec - exiting 17:49:46 (2956): No heartbeat from core client for 30 sec - exiting 17:49:47 (2956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:49:48 (2956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:45:38 (3664): No heartbeat from core client for 30 sec - exiting 19:45:39 (3664): No heartbeat from core client for 30 sec - exiting 19:45:40 (3664): No heartbeat from core client for 30 sec - exiting 19:45:41 (3664): No heartbeat from core client for 30 sec - exiting 19:45:42 (3664): No heartbeat from core client for 30 sec - exiting 19:45:43 (3664): No heartbeat from core client for 30 sec - exiting 19:45:44 (3664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:40:15 (3444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:59:51 (508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=784, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=784, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 19:44:14 (3668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/z9jdko.pjb9c10 Error converting file to netcdf: dataout/z9jdko.pib9c10 Error converting file to netcdf: dataout/z9jdko.pfb9c10 Error converting file to netcdf: dataout/z9jdka.phb9c10 Error converting file to netcdf: dataout/z9jdka.pgb9c10 Error converting file to netcdf: dataout/z9jdka.peb9c10 Error converting file to netcdf: dataout/z9jdka.pdb9c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Mar 2013 15:10:33	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	1,036,800	1,256,659	1.2121
28 Feb 2013 18:24:03	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	1,010,880	1,224,949	1.2118
24 Feb 2013 16:31:38	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	984,960	1,193,148	1.2114
23 Feb 2013 15:05:37	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	959,040	1,161,102	1.2107
21 Feb 2013 20:17:28	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	933,120	1,128,872	1.2098
18 Feb 2013 20:13:23	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	907,200	1,097,228	1.2095
16 Feb 2013 10:34:19	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	881,280	1,065,354	1.2089
11 Feb 2013 20:14:14	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	855,360	1,033,876	1.2087
08 Feb 2013 19:45:34	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	829,440	1,002,051	1.2081
04 Feb 2013 15:10:44	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	803,520	970,327	1.2076
02 Feb 2013 13:10:18	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	777,600	938,745	1.2072
31 Jan 2013 18:51:45	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	751,680	906,659	1.2062
28 Jan 2013 18:31:23	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	725,760	875,350	1.2061
27 Jan 2013 10:31:24	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	699,840	844,395	1.2066
26 Jan 2013 00:03:26	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	673,920	813,500	1.2071
24 Jan 2013 16:45:56	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	648,000	783,472	1.2091
22 Jan 2013 18:56:33	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	622,080	752,693	1.2100
20 Jan 2013 15:50:57	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	596,160	721,611	1.2104
19 Jan 2013 11:35:42	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	570,240	690,100	1.2102
17 Jan 2013 16:50:40	1148039	15479861	hadcm3n_z9jd_1880_40_008255148_2	544,320	672,938	1.2363