Task 13583913

Name	hadcm3n_y8z5_1900_40_007521943_3
Workunit	7719418
Created	2 Nov 2011, 2:49:31 UTC
Sent	2 Nov 2011, 2:50:18 UTC
Report deadline	1 Feb 2012, 10:17:29 UTC
Received	20 Dec 2011, 1:47:03 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1072042
Run time	19 days 15 hours 3 min 34 sec
CPU time	17 days 13 hours 14 min 32 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.79 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5080, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:17:51 (4588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:17:53 (4588): No heartbeat from core client for 30 sec - exiting 00:17:54 (4588): No heartbeat from core client for 30 sec - exiting 00:17:55 (4588): No heartbeat from core client for 30 sec - exiting 00:17:56 (4588): No heartbeat from core client for 30 sec - exiting 00:17:57 (4588): No heartbeat from core client for 30 sec - exiting 00:17:58 (4588): No heartbeat from core client for 30 sec - exiting 00:17:59 (4588): No heartbeat from core client for 30 sec - exiting 00:18:00 (4588): No heartbeat from core client for 30 sec - exiting 00:18:01 (4588): No heartbeat from core client for 30 sec - exiting 00:18:02 (4588): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=672, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 08:30:53 (3960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:30:54 (3960): No heartbeat from core client for 30 sec - exiting 08:30:55 (3960): No heartbeat from core client for 30 sec - exiting 08:30:56 (3960): No heartbeat from core client for 30 sec - exiting 08:30:57 (3960): No heartbeat from core client for 30 sec - exiting 08:30:58 (3960): No heartbeat from core client for 30 sec - exiting 08:30:59 (3960): No heartbeat from core client for 30 sec - exiting 08:31:00 (3960): No heartbeat from core client for 30 sec - exiting 08:31:01 (3960): No heartbeat from core client for 30 sec - exiting 08:31:02 (3960): No heartbeat from core client for 30 sec - exiting 08:31:03 (3960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: C I/O Error feof - Unit 62 - Return code = 16 BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/y8z5ko.pjc0c10 Error converting file to netcdf: dataout/y8z5ko.pic0c10 Error converting file to netcdf: dataout/y8z5ko.pfc0c10 Error converting file to netcdf: dataout/y8z5ko.pcc0c10 Error converting file to netcdf: dataout/y8z5ko.pbc0c10 Error converting file to netcdf: dataout/y8z5ko.pac0c10 Error converting file to netcdf: dataout/y8z5ka.phc0c10 Error converting file to netcdf: dataout/y8z5ka.pgc0c10 Error converting file to netcdf: dataout/y8z5ka.pec0c10 Error converting file to netcdf: dataout/y8z5ka.pdc0c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:35:00 (2508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:35:01 (2508): No heartbeat from core client for 30 sec - exiting 14:35:02 (2508): No heartbeat from core client for 30 sec - exiting 14:35:03 (2508): No heartbeat from core client for 30 sec - exiting 14:35:04 (2508): No heartbeat from core client for 30 sec - exiting 14:35:05 (2508): No heartbeat from core client for 30 sec - exiting 14:35:06 (2508): No heartbeat from core client for 30 sec - exiting 14:35:07 (2508): No heartbeat from core client for 30 sec - exiting 14:35:08 (2508): No heartbeat from core client for 30 sec - exiting 14:35:09 (2508): No heartbeat from core client for 30 sec - exiting 14:35:10 (2508): No heartbeat from core client for 30 sec - exiting 14:37:35 (4432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:37:36 (4432): No heartbeat from core client for 30 sec - exiting 14:37:37 (4432): No heartbeat from core client for 30 sec - exiting 14:37:38 (4432): No heartbeat from core client for 30 sec - exiting 14:37:39 (4432): No heartbeat from core client for 30 sec - exiting 14:37:40 (4432): No heartbeat from core client for 30 sec - exiting 14:37:41 (4432): No heartbeat from core client for 30 sec - exiting 14:37:42 (4432): No heartbeat from core client for 30 sec - exiting 14:37:43 (4432): No heartbeat from core client for 30 sec - exiting 14:37:44 (4432): No heartbeat from core client for 30 sec - exiting 14:37:45 (4432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Dec 2011 00:15:45	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	1,036,800	1,516,465	1.4626
17 Dec 2011 22:40:24	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	1,010,880	1,478,501	1.4626
16 Dec 2011 07:51:17	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	984,960	1,440,002	1.4620
15 Dec 2011 18:04:49	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	959,040	1,401,964	1.4618
14 Dec 2011 20:50:31	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	933,120	1,364,133	1.4619
13 Dec 2011 07:30:25	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	907,200	1,325,730	1.4613
12 Dec 2011 21:00:07	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	881,280	1,287,627	1.4611
11 Dec 2011 19:59:19	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	855,360	1,249,627	1.4609
10 Dec 2011 21:13:33	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	829,440	1,211,613	1.4608
09 Dec 2011 23:48:50	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	803,520	1,173,324	1.4602
09 Dec 2011 03:51:29	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	777,600	1,135,271	1.4600
08 Dec 2011 03:21:28	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	751,680	1,097,297	1.4598
07 Dec 2011 05:39:52	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	725,760	1,059,170	1.4594
05 Dec 2011 04:29:55	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	699,840	1,020,596	1.4583
03 Dec 2011 19:10:27	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	673,920	982,427	1.4578
01 Dec 2011 22:14:06	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	648,000	944,334	1.4573
01 Dec 2011 03:44:04	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	622,080	906,386	1.4570
30 Nov 2011 06:10:42	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	596,160	867,888	1.4558
29 Nov 2011 08:31:17	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	570,240	829,590	1.4548
28 Nov 2011 20:56:40	1072042	13583913	hadcm3n_y8z5_1900_40_007521943_3	544,320	791,426	1.4540