Task 16015847

Name	hadcm3n_o1wd_1980_40_008387645_3
Workunit	8538504
Created	14 Sep 2013, 6:05:11 UTC
Sent	14 Sep 2013, 6:16:25 UTC
Report deadline	14 Dec 2013, 13:43:36 UTC
Received	17 Oct 2013, 4:02:00 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1280766
Run time	16 days 21 hours 54 min 8 sec
CPU time	15 days 8 hours 10 min 5 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.89 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7236, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:34:25 (13048): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 07:34:29 (13048): No heartbeat from core client for 30 sec - exiting 07:34:30 (13048): No heartbeat from core client for 30 sec - exiting 07:34:31 (13048): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:11:30 (6792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:11:32 (6792): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9248, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:46:34 (3372): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 06:46:36 (3372): No heartbeat from core client for 30 sec - exiting 06:46:37 (3372): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:42:11 (30464): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:48:51 (31764): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 18:48:52 (31764): No heartbeat from core client for 30 sec - exiting 18:48:53 (31764): No heartbeat from core client for 30 sec - exiting 18:48:54 (31764): No heartbeat from core client for 30 sec - exiting 18:48:55 (31764): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:14:45 (61536): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 08:10:01 (67400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:10:03 (67400): No heartbeat from core client for 30 sec - exiting 08:20:01 (68368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:26:01 (60204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:26:02 (60204): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Oct 2013 04:06:12	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	777,600	1,325,397	1.7045
16 Oct 2013 06:28:53	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	751,680	1,281,867	1.7053
15 Oct 2013 14:10:47	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	725,760	1,238,122	1.7060
14 Oct 2013 07:06:13	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	699,840	1,194,435	1.7067
13 Oct 2013 13:23:10	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	673,920	1,151,124	1.7081
12 Oct 2013 15:45:24	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	648,000	1,107,761	1.7095
11 Oct 2013 23:45:22	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	622,080	1,064,664	1.7115
10 Oct 2013 19:43:25	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	596,160	1,022,088	1.7145
10 Oct 2013 04:03:34	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	570,240	977,976	1.7150
09 Oct 2013 08:12:10	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	544,320	933,692	1.7153
08 Oct 2013 14:25:46	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	518,400	888,803	1.7145
07 Oct 2013 11:27:24	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	492,480	845,582	1.7170
07 Oct 2013 04:01:40	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	466,560	801,896	1.7187
06 Oct 2013 02:46:21	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	440,640	758,357	1.7210
05 Oct 2013 13:05:59	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	414,720	713,867	1.7213
04 Oct 2013 22:39:16	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	388,800	667,580	1.7170
04 Oct 2013 06:30:11	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	362,880	620,029	1.7086
03 Oct 2013 15:13:22	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	336,960	571,415	1.6958
26 Sep 2013 17:21:09	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	311,040	525,449	1.6893
25 Sep 2013 13:27:38	1280766	16015847	hadcm3n_o1wd_1980_40_008387645_3	285,120	482,819	1.6934