Task 16716936

Name	hadam3p_eu_n3vn_2013_1_008799523_0
Workunit	8945501
Created	7 Jul 2014, 15:20:57 UTC
Sent	4 Aug 2014, 5:21:48 UTC
Report deadline	17 Jul 2015, 10:41:48 UTC
Received	14 Oct 2014, 5:15:09 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-2147483645 (0x80000003) Unknown error code
Computer ID	1234533
Run time	7 days 8 hours 20 min 10 sec
CPU time	2 days 19 hours 13 min 57 sec
Validate state	Invalid
Credit	1,790.21
Device peak FLOPS	3.11 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.09 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> One or more arguments are invalid (0x80000003) - exit code -2147483645 (0x80000003) </message> <stderr_txt> 12:24:10 (2036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:24:11 (2036): No heartbeat from core client for 30 sec - exiting 12:24:12 (2036): No heartbeat from core client for 30 sec - exiting 12:24:13 (2036): No heartbeat from core client for 30 sec - exiting 12:24:14 (2036): No heartbeat from core client for 30 sec - exiting 12:24:15 (2036): No heartbeat from core client for 30 sec - exiting 12:24:16 (2036): No heartbeat from core client for 30 sec - exiting 12:24:17 (2036): No heartbeat from core client for 30 sec - exiting 12:24:18 (2036): No heartbeat from core client for 30 sec - exiting 12:24:19 (2036): No heartbeat from core client for 30 sec - exiting 12:24:20 (2036): No heartbeat from core client for 30 sec - exiting 12:24:22 (2036): No heartbeat from core client for 30 sec - exiting 12:24:23 (2036): No heartbeat from core client for 30 sec - exiting 12:24:24 (2036): No heartbeat from core client for 30 sec - exiting 12:24:25 (2036): No heartbeat from core client for 30 sec - exiting 12:24:26 (2036): No heartbeat from core client for 30 sec - exiting 12:24:27 (2036): No heartbeat from core client for 30 sec - exiting 12:24:28 (2036): No heartbeat from core client for 30 sec - exiting 12:24:29 (2036): No heartbeat from core client for 30 sec - exiting 12:24:30 (2036): No heartbeat from core client for 30 sec - exiting 12:24:31 (2036): No heartbeat from core client for 30 sec - exiting 12:24:32 (2036): No heartbeat from core client for 30 sec - exiting 12:24:34 (2036): No heartbeat from core client for 30 sec - exiting 12:24:35 (2036): No heartbeat from core client for 30 sec - exiting Signal 11 received, exiting... Called boinc_finish 12:24:36 (2036): No heartbeat from core client for 30 sec - exiting 12:24:37 (2036): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Breakpoint Encountered (0x80000003) at address 0x75DA1F66 Engaging BOINC Windows Runtime Debugger... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4048, selfPID=4048, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4048, selfPID=1264, iMonCtr=1 08:04:22 (2900): No heartbeat from core client for 30 sec - exiting 08:04:23 (2900): No heartbeat from core client for 30 sec - exiting 08:04:24 (2900): No heartbeat from core client for 30 sec - exiting 08:04:25 (2900): No heartbeat from core client for 30 sec - exiting 08:04:26 (2900): No heartbeat from core client for 30 sec - exiting 08:04:27 (2900): No heartbeat from core client for 30 sec - exiting 08:04:28 (2900): No heartbeat from core client for 30 sec - exiting 08:04:29 (2900): No heartbeat from core client for 30 sec - exiting 08:04:30 (2900): No heartbeat from core client for 30 sec - exiting 08:04:32 (2900): No heartbeat from core client for 30 sec - exiting 08:04:33 (2900): No heartbeat from core client for 30 sec - exiting 08:04:34 (2900): No heartbeat from core client for 30 sec - exiting 08:04:35 (2900): No heartbeat from core client for 30 sec - exiting 08:04:36 (2900): No heartbeat from core client for 30 sec - exiting 08:04:37 (2900): No heartbeat from core client for 30 sec - exiting 08:04:38 (2900): No heartbeat from core client for 30 sec - exiting 08:04:39 (2900): No heartbeat from core client for 30 sec - exiting 08:04:40 (2900): No heartbeat from core client for 30 sec - exiting 08:04:41 (2900): No heartbeat from core client for 30 sec - exiting 08:04:43 (2900): No heartbeat from core client for 30 sec - exiting 08:04:44 (2900): No heartbeat from core client for 30 sec - exiting 08:04:45 (2900): No heartbeat from core client for 30 sec - exiting 08:04:46 (2900): No heartbeat from core client for 30 sec - exiting 08:04:47 (2900): No heartbeat from core client for 30 sec - exiting 08:04:48 (2900): No heartbeat from core client for 30 sec - exiting 08:04:49 (2900): No heartbeat from core client for 30 sec - exiting 08:04:50 (2900): No heartbeat from core client for 30 sec - exiting 08:04:51 (2900): No heartbeat from core client for 30 sec - exiting 08:04:52 (2900): No heartbeat from core client for 30 sec - exiting 08:04:53 (2900): No heartbeat from core client for 30 sec - exiting 08:04:55 (2900): No heartbeat from core client for 30 sec - exiting 08:04:56 (2900): No heartbeat from core client for 30 sec - exiting 08:04:57 (2900): No heartbeat from core client for 30 sec - exiting 08:04:58 (2900): No heartbeat from core client for 30 sec - exiting 08:04:59 (2900): No heartbeat from core client for 30 sec - exiting 08:05:00 (2900): No heartbeat from core client for 30 sec - exiting 08:05:01 (2900): No heartbeat from core client for 30 sec - exiting 08:05:02 (2900): No heartbeat from core client for 30 sec - exiting 08:05:03 (2900): No heartbeat from core client for 30 sec - exiting 08:05:04 (2900): No heartbeat from core client for 30 sec - exiting 08:05:06 (2900): No heartbeat from core client for 30 sec - exiting 08:05:07 (2900): No heartbeat from core client for 30 sec - exiting 08:05:08 (2900): No heartbeat from core client for 30 sec - exiting 08:05:09 (2900): No heartbeat from core client for 30 sec - exiting 08:05:10 (2900): No heartbeat from core client for 30 sec - exiting 08:05:11 (2900): No heartbeat from core client for 30 sec - exiting 08:05:12 (2900): No heartbeat from core client for 30 sec - exiting 08:05:13 (2900): No heartbeat from core client for 30 sec - exiting 08:05:14 (2900): No heartbeat from core client for 30 sec - exiting 08:05:15 (2900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:05:17 (2900): No heartbeat from core client for 30 sec - exiting 08:05:18 (2900): No heartbeat from core client for 30 sec - exiting 08:05:19 (2900): No heartbeat from core client for 30 sec - exiting 08:05:20 (2900): No heartbeat from core client for 30 sec - exiting 08:05:21 (2900): No heartbeat from core client for 30 sec - exiting 08:05:22 (2900): No heartbeat from core client for 30 sec - exiting 08:05:23 (2900): No heartbeat from core client for 30 sec - exiting 08:05:24 (2900): No heartbeat from core client for 30 sec - exiting 08:05:25 (2900): No heartbeat from core client for 30 sec - exiting 08:05:26 (2900): No heartbeat from core client for 30 sec - exiting 08:05:27 (2900): No heartbeat from core client for 30 sec - exiting 08:05:29 (2900): No heartbeat from core client for 30 sec - exiting 08:05:30 (2900): No heartbeat from core client for 30 sec - exiting 08:05:31 (2900): No heartbeat from core client for 30 sec - exiting 08:05:32 (2900): No heartbeat from core client for 30 sec - exiting 08:05:33 (2900): No heartbeat from core client for 30 sec - exiting 08:05:34 (2900): No heartbeat from core client for 30 sec - exiting 08:05:35 (2900): No heartbeat from core client for 30 sec - exiting 08:05:36 (2900): No heartbeat from core client for 30 sec - exiting 08:05:37 (2900): No heartbeat from core client for 30 sec - exiting 08:05:38 (2900): No heartbeat from core client for 30 sec - exiting 08:05:39 (2900): No heartbeat from core client for 30 sec - exiting 08:05:41 (2900): No heartbeat from core client for 30 sec - exiting 08:05:42 (2900): No heartbeat from core client for 30 sec - exiting 08:05:43 (2900): No heartbeat from core client for 30 sec - exiting 08:05:44 (2900): No heartbeat from core client for 30 sec - exiting 08:05:45 (2900): No heartbeat from core client for 30 sec - exiting 08:05:46 (2900): No heartbeat from core client for 30 sec - exiting 08:05:47 (2900): No heartbeat from core client for 30 sec - exiting 08:05:48 (2900): No heartbeat from core client for 30 sec - exiting 08:05:49 (2900): No heartbeat from core client for 30 sec - exiting 08:05:50 (2900): No heartbeat from core client for 30 sec - exiting 08:05:51 (2900): No heartbeat from core client for 30 sec - exiting Signal 11 received, exiting... Called boinc_finish 08:05:53 (2900): No heartbeat from core client for 30 sec - exiting 08:05:54 (2900): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Breakpoint Encountered (0x80000003) at address 0x75C71F66 Engaging BOINC Windows Runtime Debugger... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3104, selfPID=3104, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3104, selfPID=3148, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 12:39:20 (3288): No heartbeat from core client for 30 sec - exiting 12:39:31 (3288): No heartbeat from core client for 30 sec - exiting 12:39:45 (3288): No heartbeat from core client for 30 sec - exiting 12:39:55 (3288): No heartbeat from core client for 30 sec - exiting 12:40:06 (3288): No heartbeat from core client for 30 sec - exiting 12:40:20 (3288): No heartbeat from core client for 30 sec - exiting Signal 11 received, exiting... Called boinc_finish Unhandled Exception Detected... - Unhandled Exception Record - Reason: Breakpoint Encountered (0x80000003) at address 0x757D1F66 Engaging BOINC Windows Runtime Debugger... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2888, selfPID=852, iMonCtr=1 No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2888, selfPID=2888, iMonCtr=1 10:27:53 (2660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:28:02 (2660): No heartbeat from core client for 30 sec - exiting 10:28:15 (2660): No heartbeat from core client for 30 sec - exiting 10:28:24 (2660): No heartbeat from core client for 30 sec - exiting 10:28:34 (2660): No heartbeat from core client for 30 sec - exiting 10:28:46 (2660): No heartbeat from core client for 30 sec - exiting 10:28:56 (2660): No heartbeat from core client for 30 sec - exiting 10:29:05 (2660): No heartbeat from core client for 30 sec - exiting 10:29:18 (2660): No heartbeat from core client for 30 sec - exiting 10:29:27 (2660): No heartbeat from core client for 30 sec - exiting 10:29:37 (2660): No heartbeat from core client for 30 sec - exiting 10:29:49 (2660): No heartbeat from core client for 30 sec - exiting 10:29:59 (2660): No heartbeat from core client for 30 sec - exiting 10:30:08 (2660): No heartbeat from core client for 30 sec - exiting 10:30:21 (2660): No heartbeat from core client for 30 sec - exiting 10:30:30 (2660): No heartbeat from core client for 30 sec - exiting 10:30:40 (2660): No heartbeat from core client for 30 sec - exiting 10:30:52 (2660): No heartbeat from core client for 30 sec - exiting 10:30:59 (2660): No heartbeat from core client for 30 sec - exiting 10:31:08 (2660): No heartbeat from core client for 30 sec - exiting 10:31:18 (2660): No heartbeat from core client for 30 sec - exiting Signal 11 received, exiting... Called boinc_finish Unhandled Exception Detected... - Unhandled Exception Record - Reason: Breakpoint Encountered (0x80000003) at address 0x75961F66 Engaging BOINC Windows Runtime Debugger... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Unhandled Exception Detected... - Unhandled Exception Record - Reason: Breakpoint Encountered (0x80000003) at address 0x75C21F66 Engaging BOINC Windows Runtime Debugger... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakg.pipe_dummy 2048 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2640, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2168, iMonCtr=2 Signal 11 received, exiting... Called boinc_finish Unhandled Exception Detected... - Unhandled Exception Record - Reason: Breakpoint Encountered (0x80000003) at address 0x75DB1F66 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Oct 2014 02:28:23	1234533	16716936	hadam3p_eu_n3vn_2013_1_008799523_0	103,776	218,988	2.1102
01 Oct 2014 06:45:47	1234533	16716936	hadam3p_eu_n3vn_2013_1_008799523_0	92,256	194,337	2.1065
29 Sep 2014 06:04:14	1234533	16716936	hadam3p_eu_n3vn_2013_1_008799523_0	80,736	170,461	2.1113
29 Sep 2014 02:25:53	1234533	16716936	hadam3p_eu_n3vn_2013_1_008799523_0	69,216	132,934	1.9206
17 Sep 2014 07:03:15	1234533	16716936	hadam3p_eu_n3vn_2013_1_008799523_0	57,696	119,939	2.0788
15 Sep 2014 02:22:57	1234533	16716936	hadam3p_eu_n3vn_2013_1_008799523_0	46,176	94,930	2.0558
11 Sep 2014 02:08:32	1234533	16716936	hadam3p_eu_n3vn_2013_1_008799523_0	34,656	53,120	1.5328
19 Aug 2014 06:54:07	1234533	16716936	hadam3p_eu_n3vn_2013_1_008799523_0	23,136	0	0.0000