Task 15834453

Name	hadcm3n_4f69_1940_40_008302601_3
Workunit	8453736
Created	7 Jun 2013, 18:54:02 UTC
Sent	9 Jun 2013, 21:04:09 UTC
Report deadline	9 Sep 2013, 4:31:20 UTC
Received	26 Jun 2013, 18:53:25 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1236220
Run time	5 days 3 hours 2 min 14 sec
CPU time	4 days 7 hours 0 min 53 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.71 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> 15:19:58 (4900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 06:42:46 (4004): No heartbeat from core client for 30 sec - exiting 06:42:47 (4004): No heartbeat from core client for 30 sec - exiting 06:42:48 (4004): No heartbeat from core client for 30 sec - exiting 06:42:50 (4004): No heartbeat from core client for 30 sec - exiting 06:42:51 (4004): No heartbeat from core client for 30 sec - exiting 06:42:52 (4004): No heartbeat from core client for 30 sec - exiting 06:42:53 (4004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:36:59 (908): No heartbeat from core client for 30 sec - exiting 17:37:00 (908): No heartbeat from core client for 30 sec - exiting 17:37:01 (908): No heartbeat from core client for 30 sec - exiting 17:37:02 (908): No heartbeat from core client for 30 sec - exiting 17:37:03 (908): No heartbeat from core client for 30 sec - exiting 17:37:04 (908): No heartbeat from core client for 30 sec - exiting 17:37:05 (908): No heartbeat from core client for 30 sec - exiting 17:37:06 (908): No heartbeat from core client for 30 sec - exiting 17:37:07 (908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:11:47 (5496): No heartbeat from core client for 30 sec - exiting 20:11:48 (5496): No heartbeat from core client for 30 sec - exiting 20:11:49 (5496): No heartbeat from core client for 30 sec - exiting 20:11:50 (5496): No heartbeat from core client for 30 sec - exiting 20:11:51 (5496): No heartbeat from core client for 30 sec - exiting 20:11:52 (5496): No heartbeat from core client for 30 sec - exiting 20:11:53 (5496): No heartbeat from core client for 30 sec - exiting 20:11:54 (5496): No heartbeat from core client for 30 sec - exiting 20:11:55 (5496): No heartbeat from core client for 30 sec - exiting 20:11:56 (5496): No heartbeat from core client for 30 sec - exiting 20:11:57 (5496): No heartbeat from core client for 30 sec - exiting 20:11:58 (5496): No heartbeat from core client for 30 sec - exiting 20:11:59 (5496): No heartbeat from core client for 30 sec - exiting 20:12:00 (5496): No heartbeat from core client for 30 sec - exiting 20:12:01 (5496): No heartbeat from core client for 30 sec - exiting 20:12:02 (5496): No heartbeat from core client for 30 sec - exiting 20:12:03 (5496): No heartbeat from core client for 30 sec - exiting 20:12:04 (5496): No heartbeat from core client for 30 sec - exiting 20:12:05 (5496): No heartbeat from core client for 30 sec - exiting 20:12:06 (5496): No heartbeat from core client for 30 sec - exiting 20:12:07 (5496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:47:13 (4956): No heartbeat from core client for 30 sec - exiting 20:47:14 (4956): No heartbeat from core client for 30 sec - exiting 20:47:15 (4956): No heartbeat from core client for 30 sec - exiting 20:47:16 (4956): No heartbeat from core client for 30 sec - exiting 20:47:17 (4956): No heartbeat from core client for 30 sec - exiting 20:47:18 (4956): No heartbeat from core client for 30 sec - exiting 20:47:20 (4956): No heartbeat from core client for 30 sec - exiting 20:47:21 (4956): No heartbeat from core client for 30 sec - exiting 20:47:22 (4956): No heartbeat from core client for 30 sec - exiting 20:47:23 (4956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:10:30 (4292): No heartbeat from core client for 30 sec - exiting 21:10:37 (4292): No heartbeat from core client for 30 sec - exiting 21:10:38 (4292): No heartbeat from core client for 30 sec - exiting 21:10:39 (4292): No heartbeat from core client for 30 sec - exiting 21:10:40 (4292): No heartbeat from core client for 30 sec - exiting 21:10:41 (4292): No heartbeat from core client for 30 sec - exiting 21:10:42 (4292): No heartbeat from core client for 30 sec - exiting 21:10:43 (4292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:45:01 (6724): No heartbeat from core client for 30 sec - exiting 13:45:02 (6724): No heartbeat from core client for 30 sec - exiting 13:45:03 (6724): No heartbeat from core client for 30 sec - exiting 13:45:04 (6724): No heartbeat from core client for 30 sec - exiting 13:45:05 (6724): No heartbeat from core client for 30 sec - exiting 13:45:06 (6724): No heartbeat from core client for 30 sec - exiting 13:45:07 (6724): No heartbeat from core client for 30 sec - exiting 13:45:08 (6724): No heartbeat from core client for 30 sec - exiting 13:45:09 (6724): No heartbeat from core client for 30 sec - exiting 13:45:10 (6724): No heartbeat from core client for 30 sec - exiting 13:45:11 (6724): No heartbeat from core client for 30 sec - exiting 13:45:12 (6724): No heartbeat from core client for 30 sec - exiting 13:45:13 (6724): No heartbeat from core client for 30 sec - exiting 13:45:14 (6724): No heartbeat from core client for 30 sec - exiting 13:45:15 (6724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 06:49:33 (6696): No heartbeat from core client for 30 sec - exiting 06:49:34 (6696): No heartbeat from core client for 30 sec - exiting 06:49:36 (6696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:47:11 (4448): No heartbeat from core client for 30 sec - exiting 16:47:13 (4448): No heartbeat from core client for 30 sec - exiting 16:47:14 (4448): No heartbeat from core client for 30 sec - exiting 16:47:15 (4448): No heartbeat from core client for 30 sec - exiting 16:47:16 (4448): No heartbeat from core client for 30 sec - exiting 16:47:17 (4448): No heartbeat from core client for 30 sec - exiting 16:47:18 (4448): No heartbeat from core client for 30 sec - exiting 16:47:19 (4448): No heartbeat from core client for 30 sec - exiting 16:47:20 (4448): No heartbeat from core client for 30 sec - exiting 16:47:21 (4448): No heartbeat from core client for 30 sec - exiting 16:47:22 (4448): No heartbeat from core client for 30 sec - exiting 16:47:24 (4448): No heartbeat from core client for 30 sec - exiting 16:47:25 (4448): No heartbeat from core client for 30 sec - exiting 16:47:26 (4448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:11:02 (5252): No heartbeat from core client for 30 sec - exiting 06:11:03 (5252): No heartbeat from core client for 30 sec - exiting 06:11:04 (5252): No heartbeat from core client for 30 sec - exiting 06:11:05 (5252): No heartbeat from core client for 30 sec - exiting 06:11:06 (5252): No heartbeat from core client for 30 sec - exiting 06:11:07 (5252): No heartbeat from core client for 30 sec - exiting 06:11:08 (5252): No heartbeat from core client for 30 sec - exiting 06:11:10 (5252): No heartbeat from core client for 30 sec - exiting 06:11:11 (5252): No heartbeat from core client for 30 sec - exiting 06:11:12 (5252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:11:13 (5252): No heartbeat from core client for 30 sec - exiting 06:11:14 (5252): No heartbeat from core client for 30 sec - exiting 06:11:15 (5252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9324, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:53:09 (5928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:53:10 (5928): No heartbeat from core client for 30 sec - exiting 07:53:11 (5928): No heartbeat from core client for 30 sec - exiting 07:53:12 (5928): No heartbeat from core client for 30 sec - exiting 07:53:13 (5928): No heartbeat from core client for 30 sec - exiting 07:54:24 (4284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:07:48 (1236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:23:03 (6772): No heartbeat from core client for 30 sec - exiting 08:23:04 (6772): No heartbeat from core client for 30 sec - exiting 08:23:05 (6772): No heartbeat from core client for 30 sec - exiting 08:23:07 (6772): No heartbeat from core client for 30 sec - exiting 08:23:08 (6772): No heartbeat from core client for 30 sec - exiting 08:23:09 (6772): No heartbeat from core client for 30 sec - exiting 08:23:10 (6772): No heartbeat from core client for 30 sec - exiting 08:23:11 (6772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:18:39 (3592): No heartbeat from core client for 30 sec - exiting 17:18:40 (3592): No heartbeat from core client for 30 sec - exiting 17:18:42 (3592): No heartbeat from core client for 30 sec - exiting 17:18:43 (3592): No heartbeat from core client for 30 sec - exiting 17:18:44 (3592): No heartbeat from core client for 30 sec - exiting 17:18:45 (3592): No heartbeat from core client for 30 sec - exiting 17:18:46 (3592): No heartbeat from core client for 30 sec - exiting 17:18:47 (3592): No heartbeat from core client for 30 sec - exiting 17:18:48 (3592): No heartbeat from core client for 30 sec - exiting 17:18:49 (3592): No heartbeat from core client for 30 sec - exiting 17:18:50 (3592): No heartbeat from core client for 30 sec - exiting 17:18:51 (3592): No heartbeat from core client for 30 sec - exiting 17:18:52 (3592): No heartbeat from core client for 30 sec - exiting 17:18:54 (3592): No heartbeat from core client for 30 sec - exiting 17:18:55 (3592): No heartbeat from core client for 30 sec - exiting 17:18:56 (3592): No heartbeat from core client for 30 sec - exiting 17:18:57 (3592): No heartbeat from core client for 30 sec - exiting 17:18:58 (3592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:07:01 (10848): No heartbeat from core client for 30 sec - exiting 07:07:02 (10848): No heartbeat from core client for 30 sec - exiting 07:07:03 (10848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7776FF2B write attempt to address 0x401D1DD0 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77753AB3 read attempt to address 0x4094D242 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_4f69_1940_40_008302601/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Jun 2013 11:00:39	1236220	15834453	hadcm3n_4f69_1940_40_008302601_3	259,200	352,627	1.3604
25 Jun 2013 12:17:46	1236220	15834453	hadcm3n_4f69_1940_40_008302601_3	233,280	316,995	1.3589
24 Jun 2013 06:54:00	1236220	15834453	hadcm3n_4f69_1940_40_008302601_3	207,360	281,846	1.3592
22 Jun 2013 16:30:10	1236220	15834453	hadcm3n_4f69_1940_40_008302601_3	181,440	246,839	1.3604
20 Jun 2013 16:17:05	1236220	15834453	hadcm3n_4f69_1940_40_008302601_3	155,520	211,516	1.3601
18 Jun 2013 09:54:32	1236220	15834453	hadcm3n_4f69_1940_40_008302601_3	129,600	175,450	1.3538
17 Jun 2013 21:12:24	1236220	15834453	hadcm3n_4f69_1940_40_008302601_3	103,680	139,598	1.3464
15 Jun 2013 15:17:27	1236220	15834453	hadcm3n_4f69_1940_40_008302601_3	77,760	104,896	1.3490
14 Jun 2013 16:43:08	1236220	15834453	hadcm3n_4f69_1940_40_008302601_3	51,840	70,288	1.3559
12 Jun 2013 16:43:16	1236220	15834453	hadcm3n_4f69_1940_40_008302601_3	25,920	34,858	1.3448