Task 15614259

Name	hadcm3n_zm7v_1880_40_008246016_4
Workunit	8401140
Created	20 Feb 2013, 6:04:54 UTC
Sent	20 Feb 2013, 6:05:37 UTC
Report deadline	22 May 2013, 13:32:48 UTC
Received	31 Mar 2013, 15:19:56 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1143689
Run time	8 days 16 hours 7 min 7 sec
CPU time	8 days 12 hours 40 min 7 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.16 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.60</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zm7vko.pj82c10 Error converting file to netcdf: dataout/zm7vko.pi82c10 Error converting file to netcdf: dataout/zm7vko.pf82c10 Error converting file to netcdf: dataout/zm7vka.ph82c10 Error converting file to netcdf: dataout/zm7vka.pg82c10 Error converting file to netcdf: dataout/zm7vka.pe82c10 Error converting file to netcdf: dataout/zm7vka.pd82c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:54:16 (4408): No heartbeat from core client for 30 sec - exiting 12:54:50 (4408): No heartbeat from core client for 30 sec - exiting 12:54:52 (4408): No heartbeat from core client for 30 sec - exiting 12:54:53 (4408): No heartbeat from core client for 30 sec - exiting 12:54:54 (4408): No heartbeat from core client for 30 sec - exiting 12:54:55 (4408): No heartbeat from core client for 30 sec - exiting 12:54:56 (4408): No heartbeat from core client for 30 sec - exiting 12:54:57 (4408): No heartbeat from core client for 30 sec - exiting 12:54:58 (4408): No heartbeat from core client for 30 sec - exiting 12:54:59 (4408): No heartbeat from core client for 30 sec - exiting 12:55:00 (4408): No heartbeat from core client for 30 sec - exiting 12:55:01 (4408): No heartbeat from core client for 30 sec - exiting 12:55:02 (4408): No heartbeat from core client for 30 sec - exiting 12:55:04 (4408): No heartbeat from core client for 30 sec - exiting 12:55:05 (4408): No heartbeat from core client for 30 sec - exiting 12:55:06 (4408): No heartbeat from core client for 30 sec - exiting 12:55:07 (4408): No heartbeat from core client for 30 sec - exiting 12:55:08 (4408): No heartbeat from core client for 30 sec - exiting 12:55:09 (4408): No heartbeat from core client for 30 sec - exiting 12:55:10 (4408): No heartbeat from core client for 30 sec - exiting 12:55:11 (4408): No heartbeat from core client for 30 sec - exiting 12:55:12 (4408): No heartbeat from core client for 30 sec - exiting 12:55:13 (4408): No heartbeat from core client for 30 sec - exiting 12:55:14 (4408): No heartbeat from core client for 30 sec - exiting 12:55:16 (4408): No heartbeat from core client for 30 sec - exiting 12:55:17 (4408): No heartbeat from core client for 30 sec - exiting 12:55:18 (4408): No heartbeat from core client for 30 sec - exiting 12:55:19 (4408): No heartbeat from core client for 30 sec - exiting 12:55:20 (4408): No heartbeat from core client for 30 sec - exiting 12:55:21 (4408): No heartbeat from core client for 30 sec - exiting 12:55:22 (4408): No heartbeat from core client for 30 sec - exiting 12:55:23 (4408): No heartbeat from core client for 30 sec - exiting 12:55:24 (4408): No heartbeat from core client for 30 sec - exiting 12:55:25 (4408): No heartbeat from core client for 30 sec - exiting 12:55:26 (4408): No heartbeat from core client for 30 sec - exiting 12:55:28 (4408): No heartbeat from core client for 30 sec - exiting 12:55:29 (4408): No heartbeat from core client for 30 sec - exiting 12:55:30 (4408): No heartbeat from core client for 30 sec - exiting 12:55:31 (4408): No heartbeat from core client for 30 sec - exiting 12:55:32 (4408): No heartbeat from core client for 30 sec - exiting 12:55:33 (4408): No heartbeat from core client for 30 sec - exiting 12:55:34 (4408): No heartbeat from core client for 30 sec - exiting 12:55:35 (4408): No heartbeat from core client for 30 sec - exiting 12:55:36 (4408): No heartbeat from core client for 30 sec - exiting 12:55:37 (4408): No heartbeat from core client for 30 sec - exiting 12:55:38 (4408): No heartbeat from core client for 30 sec - exiting 12:55:40 (4408): No heartbeat from core client for 30 sec - exiting 12:55:41 (4408): No heartbeat from core client for 30 sec - exiting 12:55:42 (4408): No heartbeat from core client for 30 sec - exiting 12:55:43 (4408): No heartbeat from core client for 30 sec - exiting 12:55:44 (4408): No heartbeat from core client for 30 sec - exiting 12:55:45 (4408): No heartbeat from core client for 30 sec - exiting 12:55:46 (4408): No heartbeat from core client for 30 sec - exiting 12:55:47 (4408): No heartbeat from core client for 30 sec - exiting 12:55:48 (4408): No heartbeat from core client for 30 sec - exiting 12:55:49 (4408): No heartbeat from core client for 30 sec - exiting 12:55:50 (4408): No heartbeat from core client for 30 sec - exiting 12:55:52 (4408): No heartbeat from core client for 30 sec - exiting 12:55:53 (4408): No heartbeat from core client for 30 sec - exiting 12:55:54 (4408): No heartbeat from core client for 30 sec - exiting 12:55:55 (4408): No heartbeat from core client for 30 sec - exiting 12:55:56 (4408): No heartbeat from core client for 30 sec - exiting 12:55:57 (4408): No heartbeat from core client for 30 sec - exiting 12:55:58 (4408): No heartbeat from core client for 30 sec - exiting 12:55:59 (4408): No heartbeat from core client for 30 sec - exiting 12:56:00 (4408): No heartbeat from core client for 30 sec - exiting 12:56:01 (4408): No heartbeat from core client for 30 sec - exiting 12:56:02 (4408): No heartbeat from core client for 30 sec - exiting 12:56:04 (4408): No heartbeat from core client for 30 sec - exiting 12:56:05 (4408): No heartbeat from core client for 30 sec - exiting 12:56:06 (4408): No heartbeat from core client for 30 sec - exiting 12:56:07 (4408): No heartbeat from core client for 30 sec - exiting 12:56:08 (4408): No heartbeat from core client for 30 sec - exiting 12:56:09 (4408): No heartbeat from core client for 30 sec - exiting 12:56:10 (4408): No heartbeat from core client for 30 sec - exiting 12:56:11 (4408): No heartbeat from core client for 30 sec - exiting 12:56:12 (4408): No heartbeat from core client for 30 sec - exiting 12:56:13 (4408): No heartbeat from core client for 30 sec - exiting 12:56:15 (4408): No heartbeat from core client for 30 sec - exiting 12:56:16 (4408): No heartbeat from core client for 30 sec - exiting 12:56:17 (4408): No heartbeat from core client for 30 sec - exiting 12:56:18 (4408): No heartbeat from core client for 30 sec - exiting 12:56:19 (4408): No heartbeat from core client for 30 sec - exiting 12:56:20 (4408): No heartbeat from core client for 30 sec - exiting 12:56:21 (4408): No heartbeat from core client for 30 sec - exiting 12:56:22 (4408): No heartbeat from core client for 30 sec - exiting 12:56:23 (4408): No heartbeat from core client for 30 sec - exiting 12:56:24 (4408): No heartbeat from core client for 30 sec - exiting 12:56:25 (4408): No heartbeat from core client for 30 sec - exiting 12:56:27 (4408): No heartbeat from core client for 30 sec - exiting 12:56:28 (4408): No heartbeat from core client for 30 sec - exiting 12:56:29 (4408): No heartbeat from core client for 30 sec - exiting 12:56:30 (4408): No heartbeat from core client for 30 sec - exiting 12:56:31 (4408): No heartbeat from core client for 30 sec - exiting 12:56:32 (4408): No heartbeat from core client for 30 sec - exiting 12:56:33 (4408): No heartbeat from core client for 30 sec - exiting 12:56:34 (4408): No heartbeat from core client for 30 sec - exiting 12:56:35 (4408): No heartbeat from core client for 30 sec - exiting 12:56:36 (4408): No heartbeat from core client for 30 sec - exiting 12:56:37 (4408): No heartbeat from core client for 30 sec - exiting 12:56:39 (4408): No heartbeat from core client for 30 sec - exiting 12:56:40 (4408): No heartbeat from core client for 30 sec - exiting 12:56:41 (4408): No heartbeat from core client for 30 sec - exiting 12:56:42 (4408): No heartbeat from core client for 30 sec - exiting 12:56:43 (4408): No heartbeat from core client for 30 sec - exiting 12:56:44 (4408): No heartbeat from core client for 30 sec - exiting 12:56:45 (4408): No heartbeat from core client for 30 sec - exiting 12:56:46 (4408): No heartbeat from core client for 30 sec - exiting 12:56:47 (4408): No heartbeat from core client for 30 sec - exiting 12:56:48 (4408): No heartbeat from core client for 30 sec - exiting 12:56:49 (4408): No heartbeat from core client for 30 sec - exiting 12:56:51 (4408): No heartbeat from core client for 30 sec - exiting 12:56:52 (4408): No heartbeat from core client for 30 sec - exiting 12:56:53 (4408): No heartbeat from core client for 30 sec - exiting 12:56:54 (4408): No heartbeat from core client for 30 sec - exiting 12:56:55 (4408): No heartbeat from core client for 30 sec - exiting 12:56:56 (4408): No heartbeat from core client for 30 sec - exiting 12:56:57 (4408): No heartbeat from core client for 30 sec - exiting 12:56:58 (4408): No heartbeat from core client for 30 sec - exiting 12:56:59 (4408): No heartbeat from core client for 30 sec - exiting 12:57:00 (4408): No heartbeat from core client for 30 sec - exiting 12:57:01 (4408): No heartbeat from core client for 30 sec - exiting 12:57:03 (4408): No heartbeat from core client for 30 sec - exiting 12:57:04 (4408): No heartbeat from core client for 30 sec - exiting 12:57:05 (4408): No heartbeat from core client for 30 sec - exiting 12:57:06 (4408): No heartbeat from core client for 30 sec - exiting 12:57:07 (4408): No heartbeat from core client for 30 sec - exiting 12:57:08 (4408): No heartbeat from core client for 30 sec - exiting 12:57:09 (4408): No heartbeat from core client for 30 sec - exiting 12:57:10 (4408): No heartbeat from core client for 30 sec - exiting 12:57:11 (4408): No heartbeat from core client for 30 sec - exiting 12:57:12 (4408): No heartbeat from core client for 30 sec - exiting 12:57:13 (4408): No heartbeat from core client for 30 sec - exiting 12:57:15 (4408): No heartbeat from core client for 30 sec - exiting 12:57:16 (4408): No heartbeat from core client for 30 sec - exiting 12:57:17 (4408): No heartbeat from core client for 30 sec - exiting 12:57:18 (4408): No heartbeat from core client for 30 sec - exiting 12:57:19 (4408): No heartbeat from core client for 30 sec - exiting 12:57:20 (4408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:57:21 (4408): No heartbeat from core client for 30 sec - exiting 12:57:22 (4408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9340, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2212, selfPID=2212, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Mar 2013 14:15:24	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	777,600	736,794	0.9475
30 Mar 2013 22:16:06	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	751,680	713,054	0.9486
30 Mar 2013 15:20:34	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	725,760	688,465	0.9486
29 Mar 2013 19:41:57	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	699,840	663,437	0.9480
17 Mar 2013 16:45:49	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	673,920	638,041	0.9468
16 Mar 2013 16:20:54	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	648,000	614,453	0.9482
13 Mar 2013 21:56:14	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	622,080	590,632	0.9494
13 Mar 2013 15:25:30	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	596,160	567,375	0.9517
10 Mar 2013 19:22:09	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	570,240	543,933	0.9539
08 Mar 2013 19:35:58	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	544,320	520,739	0.9567
03 Mar 2013 04:39:57	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	518,400	497,471	0.9596
02 Mar 2013 21:57:49	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	492,480	474,048	0.9626
27 Feb 2013 11:42:29	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	466,560	450,741	0.9661
27 Feb 2013 04:40:53	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	440,640	425,475	0.9656
25 Feb 2013 20:40:15	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	414,720	400,254	0.9651
24 Feb 2013 15:51:23	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	388,800	374,867	0.9642
24 Feb 2013 08:10:07	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	362,880	347,398	0.9573
24 Feb 2013 00:42:09	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	336,960	320,850	0.9522
23 Feb 2013 17:31:31	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	311,040	294,806	0.9478
23 Feb 2013 10:27:28	1143689	15614259	hadcm3n_zm7v_1880_40_008246016_4	285,120	268,118	0.9404