Task 15839200

Name	hadcm3n_n5fh_1920_40_008377036_1
Workunit	8527895
Created	11 Jun 2013, 16:31:06 UTC
Sent	11 Jun 2013, 16:35:02 UTC
Report deadline	11 Sep 2013, 0:02:13 UTC
Received	11 Sep 2013, 12:27:12 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1268974
Run time	15 days 7 hours 37 min 40 sec
CPU time	15 days 7 hours 37 min 40 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	3.07 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>5.8.16</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=444, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:55:26 (7244): No heartbeat from core client for 30 sec - exiting 16:55:27 (7244): No heartbeat from core client for 30 sec - exiting 16:55:28 (7244): No heartbeat from core client for 30 sec - exiting 16:55:29 (7244): No heartbeat from core client for 30 sec - exiting 16:55:30 (7244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:55:31 (7244): No heartbeat from core client for 30 sec - exiting 16:55:32 (7244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:04:08 (2972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:41:01 (776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:03:33 (5988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:20:04 (2672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:51:56 (3488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=18172, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=18172, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:06:06 (3568): No heartbeat from core client for 30 sec - exiting 08:06:07 (3568): No heartbeat from core client for 30 sec - exiting 08:06:08 (3568): No heartbeat from core client for 30 sec - exiting 08:06:09 (3568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=268, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=268, iMonCtr=1 Model crash detected, will try to restart... 09:42:29 (268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: C I/O Error feof - Unit 62 - Return code = 16 BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/n5fhko.pjg0c10 Error converting file to netcdf: dataout/n5fhko.pig0c10 Error converting file to netcdf: dataout/n5fhko.pfg0c10 Error converting file to netcdf: dataout/n5fhko.pcg0c10 Error converting file to netcdf: dataout/n5fhko.pbg0c10 Error converting file to netcdf: dataout/n5fhko.pag0c10 Error converting file to netcdf: dataout/n5fhka.phg0c10 Error converting file to netcdf: dataout/n5fhka.pgg0c10 Error converting file to netcdf: dataout/n5fhka.peg0c10 Error converting file to netcdf: dataout/n5fhka.pdg0c10 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7C911689 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... 07:25:34 (2064): No heartbeat from core client for 30 sec - exiting 07:25:36 (2064): No heartbeat from core client for 30 sec - exiting 07:25:37 (2064): No heartbeat from core client for 30 sec - exiting 07:25:38 (2064): No heartbeat from core client for 30 sec - exiting 07:25:39 (2064): No heartbeat from core client for 30 sec - exiting 07:25:40 (2064): No heartbeat from core client for 30 sec - exiting 07:25:41 (2064): No heartbeat from core client for 30 sec - exiting 07:25:42 (2064): No heartbeat from core client for 30 sec - exiting 07:25:43 (2064): No heartbeat from core client for 30 sec - exiting 07:25:44 (2064): No heartbeat from core client for 30 sec - exiting 07:25:45 (2064): No heartbeat from core client for 30 sec - exiting 07:25:46 (2064): No heartbeat from core client for 30 sec - exiting 07:25:48 (2064): No heartbeat from core client for 30 sec - exiting 07:25:49 (2064): No heartbeat from core client for 30 sec - exiting 07:25:50 (2064): No heartbeat from core client for 30 sec - exiting 07:25:51 (2064): No heartbeat from core client for 30 sec - exiting 07:25:52 (2064): No heartbeat from core client for 30 sec - exiting 07:25:53 (2064): No heartbeat from core client for 30 sec - exiting 07:25:54 (2064): No heartbeat from core client for 30 sec - exiting 07:25:55 (2064): No heartbeat from core client for 30 sec - exiting 07:25:56 (2064): No heartbeat from core client for 30 sec - exiting 07:25:57 (2064): No heartbeat from core client for 30 sec - exiting 07:25:59 (2064): No heartbeat from core client for 30 sec - exiting 07:26:00 (2064): No heartbeat from core client for 30 sec - exiting 07:26:01 (2064): No heartbeat from core client for 30 sec - exiting 07:26:02 (2064): No heartbeat from core client for 30 sec - exiting 07:26:03 (2064): No heartbeat from core client for 30 sec - exiting 07:26:04 (2064): No heartbeat from core client for 30 sec - exiting Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Sep 2013 08:38:05	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	1,036,800	1,322,906	1.2760
05 Sep 2013 21:52:41	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	1,010,880	1,292,473	1.2786
04 Sep 2013 19:04:50	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	984,960	1,257,670	1.2769
03 Sep 2013 16:18:35	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	959,040	1,223,490	1.2757
01 Sep 2013 13:51:14	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	933,120	1,204,350	1.2907
31 Aug 2013 08:14:22	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	907,200	1,170,510	1.2902
29 Aug 2013 22:01:28	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	881,280	1,136,611	1.2897
28 Aug 2013 22:37:05	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	855,360	1,103,600	1.2902
28 Aug 2013 06:47:35	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	829,440	1,069,734	1.2897
26 Aug 2013 08:19:16	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	803,520	1,034,391	1.2873
25 Aug 2013 06:17:36	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	777,600	1,001,319	1.2877
24 Aug 2013 13:39:20	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	751,680	966,298	1.2855
24 Aug 2013 00:35:58	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	725,760	932,560	1.2849
23 Aug 2013 10:48:59	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	699,840	896,606	1.2812
23 Aug 2013 10:48:59	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	673,920	863,226	1.2809
20 Aug 2013 09:43:22	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	648,000	830,613	1.2818
19 Aug 2013 21:28:58	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	622,080	799,234	1.2848
19 Aug 2013 11:40:16	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	596,160	767,735	1.2878
15 Aug 2013 09:35:54	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	570,240	735,239	1.2894
15 Aug 2013 09:35:54	1268974	15839200	hadcm3n_n5fh_1920_40_008377036_1	544,320	700,002	1.2860