Task 16698054

Name	hadcm3n_89m2_1980_40_008721877_2
Workunit	8867855
Created	3 Jul 2014, 22:48:10 UTC
Sent	3 Jul 2014, 22:48:16 UTC
Report deadline	3 Oct 2014, 6:15:27 UTC
Received	18 Jul 2014, 21:16:26 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1143449
Run time	5 days 22 hours 52 min 45 sec
CPU time	5 days 13 hours 9 min 59 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.84 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> 08:47:48 (8584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9244, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2816, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:03:56 (4396): No heartbeat from core client for 30 sec - exiting 09:03:57 (4396): No heartbeat from core client for 30 sec - exiting 09:03:58 (4396): No heartbeat from core client for 30 sec - exiting 09:03:59 (4396): No heartbeat from core client for 30 sec - exiting 09:04:00 (4396): No heartbeat from core client for 30 sec - exiting 09:04:01 (4396): No heartbeat from core client for 30 sec - exiting 09:04:03 (4396): No heartbeat from core client for 30 sec - exiting 09:04:04 (4396): No heartbeat from core client for 30 sec - exiting 09:04:05 (4396): No heartbeat from core client for 30 sec - exiting 09:04:06 (4396): No heartbeat from core client for 30 sec - exiting 09:04:07 (4396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:56:19 (1840): No heartbeat from core client for 30 sec - exiting 08:56:20 (1840): No heartbeat from core client for 30 sec - exiting 08:56:21 (1840): No heartbeat from core client for 30 sec - exiting 08:56:22 (1840): No heartbeat from core client for 30 sec - exiting 08:56:23 (1840): No heartbeat from core client for 30 sec - exiting 08:56:24 (1840): No heartbeat from core client for 30 sec - exiting 08:56:25 (1840): No heartbeat from core client for 30 sec - exiting 08:56:26 (1840): No heartbeat from core client for 30 sec - exiting 08:56:27 (1840): No heartbeat from core client for 30 sec - exiting 08:56:28 (1840): No heartbeat from core client for 30 sec - exiting 08:56:29 (1840): No heartbeat from core client for 30 sec - exiting 08:56:30 (1840): No heartbeat from core client for 30 sec - exiting 08:56:31 (1840): No heartbeat from core client for 30 sec - exiting 08:56:32 (1840): No heartbeat from core client for 30 sec - exiting 08:56:33 (1840): No heartbeat from core client for 30 sec - exiting 08:56:34 (1840): No heartbeat from core client for 30 sec - exiting 08:56:35 (1840): No heartbeat from core client for 30 sec - exiting 08:56:36 (1840): No heartbeat from core client for 30 sec - exiting 08:56:37 (1840): No heartbeat from core client for 30 sec - exiting 08:56:38 (1840): No heartbeat from core client for 30 sec - exiting 08:56:39 (1840): No heartbeat from core client for 30 sec - exiting 08:56:40 (1840): No heartbeat from core client for 30 sec - exiting 08:56:41 (1840): No heartbeat from core client for 30 sec - exiting 08:56:42 (1840): No heartbeat from core client for 30 sec - exiting 08:56:43 (1840): No heartbeat from core client for 30 sec - exiting 08:56:44 (1840): No heartbeat from core client for 30 sec - exiting 08:56:45 (1840): No heartbeat from core client for 30 sec - exiting 08:56:46 (1840): No heartbeat from core client for 30 sec - exiting 08:56:47 (1840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6768, iMonCtr=1 Model crash detected, will try to restart... 14:25:02 (5400): No heartbeat from core client for 30 sec - exiting 14:25:03 (5400): No heartbeat from core client for 30 sec - exiting 14:25:04 (5400): No heartbeat from core client for 30 sec - exiting 14:25:05 (5400): No heartbeat from core client for 30 sec - exiting 14:25:06 (5400): No heartbeat from core client for 30 sec - exiting 14:25:07 (5400): No heartbeat from core client for 30 sec - exiting 14:25:08 (5400): No heartbeat from core client for 30 sec - exiting 14:25:09 (5400): No heartbeat from core client for 30 sec - exiting 14:25:10 (5400): No heartbeat from core client for 30 sec - exiting 14:25:11 (5400): No heartbeat from core client for 30 sec - exiting 14:25:12 (5400): No heartbeat from core client for 30 sec - exiting 14:25:13 (5400): No heartbeat from core client for 30 sec - exiting 14:25:14 (5400): No heartbeat from core client for 30 sec - exiting 14:25:15 (5400): No heartbeat from core client for 30 sec - exiting 14:25:16 (5400): No heartbeat from core client for 30 sec - exiting 14:25:17 (5400): No heartbeat from core client for 30 sec - exiting 14:25:18 (5400): No heartbeat from core client for 30 sec - exiting 14:25:19 (5400): No heartbeat from core client for 30 sec - exiting 14:25:20 (5400): No heartbeat from core client for 30 sec - exiting 14:25:21 (5400): No heartbeat from core client for 30 sec - exiting 14:25:22 (5400): No heartbeat from core client for 30 sec - exiting 14:25:23 (5400): No heartbeat from core client for 30 sec - exiting 14:25:24 (5400): No heartbeat from core client for 30 sec - exiting 14:25:25 (5400): No heartbeat from core client for 30 sec - exiting 14:25:26 (5400): No heartbeat from core client for 30 sec - exiting 14:25:27 (5400): No heartbeat from core client for 30 sec - exiting 14:25:28 (5400): No heartbeat from core client for 30 sec - exiting 14:25:29 (5400): No heartbeat from core client for 30 sec - exiting 14:25:30 (5400): No heartbeat from core client for 30 sec - exiting 14:25:31 (5400): No heartbeat from core client for 30 sec - exiting 14:25:32 (5400): No heartbeat from core client for 30 sec - exiting 14:25:33 (5400): No heartbeat from core client for 30 sec - exiting 14:25:34 (5400): No heartbeat from core client for 30 sec - exiting 14:25:35 (5400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3728, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 10:46:30 (7892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7400, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 09:06:38 (7536): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:39:08 (7984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:53:56 (6456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:54:36 (7844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Jul 2014 20:16:48	1143449	16698054	hadcm3n_89m2_1980_40_008721877_2	259,200	479,390	1.8495
17 Jul 2014 12:59:44	1143449	16698054	hadcm3n_89m2_1980_40_008721877_2	233,280	433,161	1.8568
16 Jul 2014 12:29:25	1143449	16698054	hadcm3n_89m2_1980_40_008721877_2	207,360	385,600	1.8596
15 Jul 2014 12:56:32	1143449	16698054	hadcm3n_89m2_1980_40_008721877_2	181,440	336,865	1.8566
14 Jul 2014 11:08:44	1143449	16698054	hadcm3n_89m2_1980_40_008721877_2	155,520	288,224	1.8533
11 Jul 2014 14:53:50	1143449	16698054	hadcm3n_89m2_1980_40_008721877_2	129,600	239,470	1.8478
09 Jul 2014 18:55:37	1143449	16698054	hadcm3n_89m2_1980_40_008721877_2	103,680	191,032	1.8425
08 Jul 2014 19:20:21	1143449	16698054	hadcm3n_89m2_1980_40_008721877_2	77,760	142,608	1.8340
07 Jul 2014 20:03:16	1143449	16698054	hadcm3n_89m2_1980_40_008721877_2	51,840	93,918	1.8117
04 Jul 2014 20:26:58	1143449	16698054	hadcm3n_89m2_1980_40_008721877_2	25,920	45,380	1.7508