Task 15542668

Name	hadcm3n_3ai1_1980_40_008283394_0
Workunit	8434529
Created	14 Jan 2013, 5:57:59 UTC
Sent	14 Jan 2013, 5:58:17 UTC
Report deadline	15 Apr 2013, 13:25:28 UTC
Received	13 Apr 2013, 15:23:19 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	871942
Run time	14 days 14 hours 58 min 26 sec
CPU time	10 days 20 hours 36 min 34 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	1.53 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 07:04:42 (4836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:15:11 (5396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:15:13 (5396): No heartbeat from core client for 30 sec - exiting 18:16:30 (4784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5480, iMonCtr=1 Model crash detected, will try to restart... 07:17:08 (6040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=488, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5368, iMonCtr=1 Model crash detected, will try to restart... 18:06:20 (1700): No heartbeat from core client for 30 sec - exiting 18:06:21 (1700): No heartbeat from core client for 30 sec - exiting 18:06:22 (1700): No heartbeat from core client for 30 sec - exiting 18:06:23 (1700): No heartbeat from core client for 30 sec - exiting 18:06:24 (1700): No heartbeat from core client for 30 sec - exiting 18:06:25 (1700): No heartbeat from core client for 30 sec - exiting 18:06:26 (1700): No heartbeat from core client for 30 sec - exiting 18:06:27 (1700): No heartbeat from core client for 30 sec - exiting 18:06:28 (1700): No heartbeat from core client for 30 sec - exiting 18:06:29 (1700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:06:30 (1700): No heartbeat from core client for 30 sec - exiting 18:06:31 (1700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4796, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5692, iMonCtr=1 Model crash detected, will try to restart... 12:53:40 (6136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:28:42 (4540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5040, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4592, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4028, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:15:01 (4384): No heartbeat from core client for 30 sec - exiting 18:15:02 (4384): No heartbeat from core client for 30 sec - exiting 18:15:03 (4384): No heartbeat from core client for 30 sec - exiting 18:15:04 (4384): No heartbeat from core client for 30 sec - exiting 18:15:05 (4384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:15:06 (4384): No heartbeat from core client for 30 sec - exiting 18:15:07 (4384): No heartbeat from core client for 30 sec - exiting 18:15:08 (4384): No heartbeat from core client for 30 sec - exiting 18:15:09 (4384): No heartbeat from core client for 30 sec - exiting 18:28:48 (4848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:28:49 (4848): No heartbeat from core client for 30 sec - exiting 18:28:50 (4848): No heartbeat from core client for 30 sec - exiting 18:28:51 (4848): No heartbeat from core client for 30 sec - exiting 18:28:52 (4848): No heartbeat from core client for 30 sec - exiting 18:28:53 (4848): No heartbeat from core client for 30 sec - exiting 18:28:54 (4848): No heartbeat from core client for 30 sec - exiting 18:28:55 (4848): No heartbeat from core client for 30 sec - exiting 18:28:56 (4848): No heartbeat from core client for 30 sec - exiting 18:28:57 (4848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 20:18:06 (4452): No heartbeat from core client for 30 sec - exiting 20:18:07 (4452): No heartbeat from core client for 30 sec - exiting 20:18:08 (4452): No heartbeat from core client for 30 sec - exiting 20:18:09 (4452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4120, iMonCtr=1 Model crash detected, will try to restart... 06:33:18 (4120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:33:19 (4120): No heartbeat from core client for 30 sec - exiting 06:33:20 (4120): No heartbeat from core client for 30 sec - exiting 06:33:21 (4120): No heartbeat from core client for 30 sec - exiting 06:33:22 (4120): No heartbeat from core client for 30 sec - exiting 06:33:23 (4120): No heartbeat from core client for 30 sec - exiting 06:33:24 (4120): No heartbeat from core client for 30 sec - exiting 06:33:25 (4120): No heartbeat from core client for 30 sec - exiting 06:33:26 (4120): No heartbeat from core client for 30 sec - exiting 09:24:31 (3444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:24:32 (3444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5900, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4364, iMonCtr=1 Model crash detected, will try to restart... 19:36:19 (4296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:36:20 (4296): No heartbeat from core client for 30 sec - exiting 19:36:21 (4296): No heartbeat from core client for 30 sec - exiting 19:37:03 (5324): No heartbeat from core client for 30 sec - exiting 19:37:04 (5324): No heartbeat from core client for 30 sec - exiting 19:37:05 (5324): No heartbeat from core client for 30 sec - exiting 19:37:06 (5324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5428, iMonCtr=1 Model crash detected, will try to restart... 20:09:28 (3780): No heartbeat from core client for 30 sec - exiting 20:09:29 (3780): No heartbeat from core client for 30 sec - exiting 20:09:30 (3780): No heartbeat from core client for 30 sec - exiting 20:09:32 (3780): No heartbeat from core client for 30 sec - exiting 20:09:33 (3780): No heartbeat from core client for 30 sec - exiting 20:09:36 (3780): No heartbeat from core client for 30 sec - exiting 20:09:38 (3780): No heartbeat from core client for 30 sec - exiting 20:09:40 (3780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:05:32 (1840): No heartbeat from core client for 30 sec - exiting 07:05:33 (1840): No heartbeat from core client for 30 sec - exiting 07:05:34 (1840): No heartbeat from core client for 30 sec - exiting 07:05:35 (1840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:05:36 (1840): No heartbeat from core client for 30 sec - exiting 07:05:37 (1840): No heartbeat from core client for 30 sec - exiting 07:05:38 (1840): No heartbeat from core client for 30 sec - exiting 07:05:39 (1840): No heartbeat from core client for 30 sec - exiting 07:05:40 (1840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3464, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3000, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 19:09:42 (4024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4300, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 13:12:48 (5200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:39:38 (4592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:39:39 (4592): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4756, iMonCtr=1 Model crash detected, will try to restart... 11:56:08 (6076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:56:09 (6076): No heartbeat from core client for 30 sec - exiting 11:56:10 (6076): No heartbeat from core client for 30 sec - exiting 11:56:11 (6076): No heartbeat from core client for 30 sec - exiting 11:56:12 (6076): No heartbeat from core client for 30 sec - exiting 11:56:13 (6076): No heartbeat from core client for 30 sec - exiting 11:56:14 (6076): No heartbeat from core client for 30 sec - exiting 11:56:15 (6076): No heartbeat from core client for 30 sec - exiting 11:56:16 (6076): No heartbeat from core client for 30 sec - exiting 11:56:17 (6076): No heartbeat from core client for 30 sec - exiting 11:56:18 (6076): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5480, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5552, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2760, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3524, iMonCtr=1 Model crash detected, will try to restart... 21:13:34 (504): No heartbeat from core client for 30 sec - exiting 21:13:35 (504): No heartbeat from core client for 30 sec - exiting 21:13:36 (504): No heartbeat from core client for 30 sec - exiting 21:13:37 (504): No heartbeat from core client for 30 sec - exiting 21:13:38 (504): No heartbeat from core client for 30 sec - exiting 21:13:39 (504): No heartbeat from core client for 30 sec - exiting 21:13:40 (504): No heartbeat from core client for 30 sec - exiting 21:13:41 (504): No heartbeat from core client for 30 sec - exiting 21:13:42 (504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:18:12 (1400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:32:25 (4924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:32:26 (4924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 11:25:36 (6044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:25:37 (6044): No heartbeat from core client for 30 sec - exiting 11:43:04 (3800): No heartbeat from core client for 30 sec - exiting 11:43:05 (3800): No heartbeat from core client for 30 sec - exiting 11:43:06 (3800): No heartbeat from core client for 30 sec - exiting 11:43:07 (3800): No heartbeat from core client for 30 sec - exiting 11:43:08 (3800): No heartbeat from core client for 30 sec - exiting 11:43:09 (3800): No heartbeat from core client for 30 sec - exiting 11:43:10 (3800): No heartbeat from core client for 30 sec - exiting 11:43:11 (3800): No heartbeat from core client for 30 sec - exiting 11:43:12 (3800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:43:13 (3800): No heartbeat from core client for 30 sec - exiting 11:43:14 (3800): No heartbeat from core client for 30 sec - exiting 11:43:15 (3800): No heartbeat from core client for 30 sec - exiting 11:43:16 (3800): No heartbeat from core client for 30 sec - exiting 11:43:17 (3800): No heartbeat from core client for 30 sec - exiting 11:43:18 (3800): No heartbeat from core client for 30 sec - exiting 11:43:19 (3800): No heartbeat from core client for 30 sec - exiting 11:43:20 (3800): No heartbeat from core client for 30 sec - exiting 11:43:21 (3800): No heartbeat from core client for 30 sec - exiting 11:43:22 (3800): No heartbeat from core client for 30 sec - exiting 19:51:24 (5496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:51:25 (5496): No heartbeat from core client for 30 sec - exiting 19:51:26 (5496): No heartbeat from core client for 30 sec - exiting 19:51:27 (5496): No heartbeat from core client for 30 sec - exiting 19:51:28 (5496): No heartbeat from core client for 30 sec - exiting 19:51:29 (5496): No heartbeat from core client for 30 sec - exiting 19:51:30 (5496): No heartbeat from core client for 30 sec - exiting 19:51:31 (5496): No heartbeat from core client for 30 sec - exiting 19:51:32 (5496): No heartbeat from core client for 30 sec - exiting 19:51:33 (5496): No heartbeat from core client for 30 sec - exiting 13:39:28 (5240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:30:01 (5400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:30:02 (5400): No heartbeat from core client for 30 sec - exiting 14:36:59 (6044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3008, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5468, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5380, iMonCtr=1 Model crash detected, will try to restart... 18:52:52 (3424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:52:53 (3424): No heartbeat from core client for 30 sec - exiting 18:52:54 (3424): No heartbeat from core client for 30 sec - exiting 19:08:22 (4040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=836, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5080, iMonCtr=1 Model crash detected, will try to restart... 20:17:43 (4460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:42:02 (5708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2520, iMonCtr=1 Model crash detected, will try to restart... 12:47:19 (5428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Apr 2013 15:25:46	871942	15542668	hadcm3n_3ai1_1980_40_008283394_0	259,200	938,185	3.6195
23 Mar 2013 17:01:01	871942	15542668	hadcm3n_3ai1_1980_40_008283394_0	233,280	847,108	3.6313
06 Mar 2013 14:40:23	871942	15542668	hadcm3n_3ai1_1980_40_008283394_0	207,360	752,181	3.6274
27 Feb 2013 17:46:26	871942	15542668	hadcm3n_3ai1_1980_40_008283394_0	181,440	656,230	3.6168
19 Feb 2013 15:10:48	871942	15542668	hadcm3n_3ai1_1980_40_008283394_0	155,520	561,515	3.6106
16 Feb 2013 07:33:41	871942	15542668	hadcm3n_3ai1_1980_40_008283394_0	129,600	468,371	3.6140
13 Feb 2013 10:41:42	871942	15542668	hadcm3n_3ai1_1980_40_008283394_0	103,680	378,944	3.6549
02 Feb 2013 18:28:39	871942	15542668	hadcm3n_3ai1_1980_40_008283394_0	77,760	288,767	3.7136
23 Jan 2013 16:35:45	871942	15542668	hadcm3n_3ai1_1980_40_008283394_0	51,840	189,652	3.6584
18 Jan 2013 13:50:29	871942	15542668	hadcm3n_3ai1_1980_40_008283394_0	25,920	95,274	3.6757