Task 15523339

Name	hadcm3n_3cu0_1940_40_008263370_1
Workunit	8418494
Created	5 Jan 2013, 4:50:52 UTC
Sent	5 Jan 2013, 4:51:17 UTC
Report deadline	6 Apr 2013, 12:18:28 UTC
Received	8 Feb 2013, 19:40:37 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1147086
Run time	16 days 23 hours 3 min 55 sec
CPU time	14 days 23 hours 1 min 25 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.31 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.60</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8352, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5772, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5960, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5960, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6624, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:55:34 (7000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:55:36 (7000): No heartbeat from core client for 30 sec - exiting 20:55:37 (7000): No heartbeat from core client for 30 sec - exiting 20:55:39 (7000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:21:35 (44300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:22:14 (44300): No heartbeat from core client for 30 sec - exiting 16:22:15 (44300): No heartbeat from core client for 30 sec - exiting 16:22:17 (44300): No heartbeat from core client for 30 sec - exiting 16:35:33 (35452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:35:45 (35452): No heartbeat from core client for 30 sec - exiting 16:35:48 (35452): No heartbeat from core client for 30 sec - exiting 16:35:49 (35452): No heartbeat from core client for 30 sec - exiting 16:35:50 (35452): No heartbeat from core client for 30 sec - exiting 16:35:51 (35452): No heartbeat from core client for 30 sec - exiting 16:35:52 (35452): No heartbeat from core client for 30 sec - exiting 16:35:53 (35452): No heartbeat from core client for 30 sec - exiting 16:35:54 (35452): No heartbeat from core client for 30 sec - exiting 16:35:55 (35452): No heartbeat from core client for 30 sec - exiting 16:35:56 (35452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1308, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5628, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77BA3AB3 read attempt to address 0x40B0FFAC Engaging BOINC Windows Runtime Debugger... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3944, selfPID=3944, iMonCtr=1 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77BA3AB3 read attempt to address 0x40B0FFAC Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3cu0_1940_40_008263370/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Feb 2013 03:30:03	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	518,400	1,257,795	2.4263
07 Feb 2013 00:40:14	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	492,480	1,194,435	2.4253
05 Feb 2013 21:30:03	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	466,560	1,130,681	2.4234
04 Feb 2013 08:57:51	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	440,640	1,066,827	2.4211
03 Feb 2013 01:51:04	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	414,720	1,002,304	2.4168
02 Feb 2013 01:08:51	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	388,800	938,904	2.4149
29 Jan 2013 20:46:29	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	362,880	874,088	2.4088
28 Jan 2013 08:24:28	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	336,960	811,201	2.4074
26 Jan 2013 12:45:08	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	311,040	747,535	2.4033
24 Jan 2013 20:26:35	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	285,120	682,789	2.3947
23 Jan 2013 08:13:07	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	259,200	624,226	2.4083
21 Jan 2013 17:31:18	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	233,280	565,157	2.4227
19 Jan 2013 22:52:34	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	207,360	501,542	2.4187
18 Jan 2013 23:28:07	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	181,440	436,977	2.4084
16 Jan 2013 19:48:33	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	155,520	372,684	2.3964
14 Jan 2013 21:25:15	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	129,600	308,907	2.3835
12 Jan 2013 10:49:44	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	103,680	246,150	2.3741
11 Jan 2013 13:00:11	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	77,760	181,830	2.3383
08 Jan 2013 17:36:32	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	51,840	119,051	2.2965
06 Jan 2013 13:52:09	1147086	15523339	hadcm3n_3cu0_1940_40_008263370_1	25,920	58,860	2.2708