Task 13589890

Name	hadcm3n_yaiw_1900_40_007520785_2
Workunit	7718260
Created	3 Nov 2011, 10:50:47 UTC
Sent	3 Nov 2011, 11:04:10 UTC
Report deadline	2 Feb 2012, 18:31:21 UTC
Received	8 Jan 2012, 21:26:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1121227
Run time	11 days 0 hours 54 min 34 sec
CPU time	9 days 21 hours 46 min 40 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.49 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2264, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4720, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1872, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2576, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1884, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2412, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5268, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 08:27:42 (2808): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 08:27:44 (2808): No heartbeat from core client for 30 sec - exiting 08:27:45 (2808): No heartbeat from core client for 30 sec - exiting 08:27:46 (2808): No heartbeat from core client for 30 sec - exiting 08:27:47 (2808): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4592, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3240, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2060, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2612, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4312, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4604, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3064, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4872, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4384, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77BE3A93 read attempt to address 0x40B6F6D4 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77E53A93 read attempt to address 0x40B6F6D4 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_yaiw_1900_40_007520785/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Jan 2012 07:49:41	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	518,400	822,341	1.5863
07 Jan 2012 19:36:28	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	492,480	780,779	1.5854
06 Jan 2012 11:48:42	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	466,560	740,624	1.5874
05 Jan 2012 21:24:51	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	440,640	699,980	1.5886
02 Jan 2012 21:58:21	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	414,720	658,015	1.5866
01 Jan 2012 21:37:01	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	388,800	616,135	1.5847
29 Dec 2011 21:05:59	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	362,880	574,137	1.5822
27 Dec 2011 14:48:52	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	336,960	531,994	1.5788
26 Dec 2011 22:49:49	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	311,040	489,966	1.5753
25 Dec 2011 18:05:13	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	285,120	447,557	1.5697
24 Dec 2011 19:03:27	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	259,200	405,559	1.5647
24 Dec 2011 02:48:51	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	233,280	363,724	1.5592
22 Dec 2011 23:03:17	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	207,360	322,650	1.5560
17 Dec 2011 19:33:11	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	181,440	281,227	1.5500
13 Dec 2011 20:54:22	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	155,520	240,036	1.5434
08 Dec 2011 12:49:12	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	129,600	198,844	1.5343
29 Nov 2011 13:06:45	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	103,680	159,325	1.5367
26 Nov 2011 13:00:37	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	77,760	120,328	1.5474
06 Nov 2011 16:59:45	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	51,840	80,643	1.5556
05 Nov 2011 03:22:56	1121227	13589890	hadcm3n_yaiw_1900_40_007520785_2	25,920	39,119	1.5092