Task 15606094

Name	hadcm3n_3lro_1940_40_008260297_2
Workunit	8415421
Created	12 Feb 2013, 14:48:23 UTC
Sent	12 Feb 2013, 14:48:29 UTC
Report deadline	14 May 2013, 22:15:40 UTC
Received	24 May 2013, 17:04:10 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1103784
Run time	16 days 22 hours 15 min 57 sec
CPU time	15 days 8 hours 46 min 51 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	3.00 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:19:18 (4240): No heartbeat from core client for 30 sec - exiting 08:19:19 (4240): No heartbeat from core client for 30 sec - exiting 08:19:20 (4240): No heartbeat from core client for 30 sec - exiting 08:19:21 (4240): No heartbeat from core client for 30 sec - exiting 08:19:22 (4240): No heartbeat from core client for 30 sec - exiting 08:19:23 (4240): No heartbeat from core client for 30 sec - exiting 08:19:24 (4240): No heartbeat from core client for 30 sec - exiting 08:19:25 (4240): No heartbeat from core client for 30 sec - exiting 08:19:26 (4240): No heartbeat from core client for 30 sec - exiting 08:19:27 (4240): No heartbeat from core client for 30 sec - exiting 08:19:28 (4240): No heartbeat from core client for 30 sec - exiting 08:19:29 (4240): No heartbeat from core client for 30 sec - exiting 08:19:30 (4240): No heartbeat from core client for 30 sec - exiting 08:19:31 (4240): No heartbeat from core client for 30 sec - exiting 08:19:32 (4240): No heartbeat from core client for 30 sec - exiting 08:19:33 (4240): No heartbeat from core client for 30 sec - exiting 08:19:34 (4240): No heartbeat from core client for 30 sec - exiting 08:19:35 (4240): No heartbeat from core client for 30 sec - exiting 08:19:36 (4240): No heartbeat from core client for 30 sec - exiting 08:19:37 (4240): No heartbeat from core client for 30 sec - exiting 08:19:38 (4240): No heartbeat from core client for 30 sec - exiting 08:19:39 (4240): No heartbeat from core client for 30 sec - exiting 08:19:40 (4240): No heartbeat from core client for 30 sec - exiting 08:19:41 (4240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:46:52 (1796): No heartbeat from core client for 30 sec - exiting 19:46:53 (1796): No heartbeat from core client for 30 sec - exiting 19:46:54 (1796): No heartbeat from core client for 30 sec - exiting 19:46:55 (1796): No heartbeat from core client for 30 sec - exiting 19:46:56 (1796): No heartbeat from core client for 30 sec - exiting 19:46:57 (1796): No heartbeat from core client for 30 sec - exiting 19:46:58 (1796): No heartbeat from core client for 30 sec - exiting 19:46:59 (1796): No heartbeat from core client for 30 sec - exiting 19:47:00 (1796): No heartbeat from core client for 30 sec - exiting 19:47:01 (1796): No heartbeat from core client for 30 sec - exiting 19:47:02 (1796): No heartbeat from core client for 30 sec - exiting 19:47:03 (1796): No heartbeat from core client for 30 sec - exiting 19:47:04 (1796): No heartbeat from core client for 30 sec - exiting 19:47:05 (1796): No heartbeat from core client for 30 sec - exiting 19:47:06 (1796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:50:09 (3860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6856, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2776, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2776, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2776, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2776, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2776, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:24:03 (2524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:21:01 (5860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:05:56 (4176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:05:57 (4176): No heartbeat from core client for 30 sec - exiting 21:05:58 (4176): No heartbeat from core client for 30 sec - exiting 21:05:59 (4176): No heartbeat from core client for 30 sec - exiting 21:06:00 (4176): No heartbeat from core client for 30 sec - exiting 21:06:01 (4176): No heartbeat from core client for 30 sec - exiting 21:06:02 (4176): No heartbeat from core client for 30 sec - exiting 21:06:03 (4176): No heartbeat from core client for 30 sec - exiting 21:06:04 (4176): No heartbeat from core client for 30 sec - exiting 21:06:05 (4176): No heartbeat from core client for 30 sec - exiting 21:06:06 (4176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3384, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77393AB3 read attempt to address 0x40F3956B Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77393AB3 read attempt to address 0x40F3956B Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3lro_1940_40_008260297/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
24 May 2013 09:48:13	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	1,036,800	1,327,277	1.2802
23 May 2013 23:24:38	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	1,010,880	1,294,091	1.2802
17 May 2013 15:07:51	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	984,960	1,260,726	1.2800
17 May 2013 04:28:46	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	959,040	1,227,940	1.2804
15 May 2013 19:50:43	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	933,120	1,194,432	1.2800
30 Apr 2013 06:49:46	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	907,200	1,160,824	1.2796
18 Apr 2013 20:28:43	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	881,280	1,127,675	1.2796
18 Apr 2013 03:59:18	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	855,360	1,094,904	1.2801
17 Apr 2013 18:03:31	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	829,440	1,061,769	1.2801
15 Apr 2013 19:42:18	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	803,520	1,028,208	1.2796
12 Apr 2013 08:59:24	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	777,600	994,671	1.2792
11 Apr 2013 12:15:08	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	751,680	961,218	1.2788
09 Apr 2013 05:33:51	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	725,760	927,486	1.2780
08 Apr 2013 19:53:32	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	699,840	894,073	1.2775
28 Mar 2013 00:50:50	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	673,920	860,712	1.2772
22 Mar 2013 12:40:14	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	648,000	827,312	1.2767
22 Mar 2013 02:37:04	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	622,080	793,921	1.2762
21 Mar 2013 17:26:01	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	596,160	760,900	1.2763
20 Mar 2013 22:56:24	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	570,240	727,044	1.2750
19 Mar 2013 19:39:41	1103784	15606094	hadcm3n_3lro_1940_40_008260297_2	544,320	693,645	1.2743