Task 15812095

Name	hadcm3n_o7e2_1940_40_008380928_0
Workunit	8531787
Created	31 May 2013, 23:33:03 UTC
Sent	15 Jun 2013, 22:42:32 UTC
Report deadline	15 Sep 2013, 6:09:43 UTC
Received	13 Sep 2013, 14:22:52 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1096394
Run time	46 days 18 hours 20 min 25 sec
CPU time	37 days 2 hours 34 min 50 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	1.68 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5544, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5544, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5544, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6064, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6064, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:24:58 (6648): No heartbeat from core client for 30 sec - exiting 14:24:59 (6648): No heartbeat from core client for 30 sec - exiting 14:25:00 (6648): No heartbeat from core client for 30 sec - exiting 14:25:01 (6648): No heartbeat from core client for 30 sec - exiting 14:25:02 (6648): No heartbeat from core client for 30 sec - exiting 14:25:03 (6648): No heartbeat from core client for 30 sec - exiting 14:25:05 (6648): No heartbeat from core client for 30 sec - exiting 14:25:06 (6648): No heartbeat from core client for 30 sec - exiting 14:25:07 (6648): No heartbeat from core client for 30 sec - exiting 14:25:08 (6648): No heartbeat from core client for 30 sec - exiting 14:25:09 (6648): No heartbeat from core client for 30 sec - exiting 14:25:10 (6648): No heartbeat from core client for 30 sec - exiting 14:25:11 (6648): No heartbeat from core client for 30 sec - exiting 14:25:12 (6648): No heartbeat from core client for 30 sec - exiting 14:25:13 (6648): No heartbeat from core client for 30 sec - exiting 14:25:14 (6648): No heartbeat from core client for 30 sec - exiting 14:25:15 (6648): No heartbeat from core client for 30 sec - exiting 14:25:16 (6648): No heartbeat from core client for 30 sec - exiting 14:25:18 (6648): No heartbeat from core client for 30 sec - exiting 14:25:19 (6648): No heartbeat from core client for 30 sec - exiting 14:25:20 (6648): No heartbeat from core client for 30 sec - exiting 14:25:21 (6648): No heartbeat from core client for 30 sec - exiting 14:25:22 (6648): No heartbeat from core client for 30 sec - exiting 14:25:23 (6648): No heartbeat from core client for 30 sec - exiting 14:25:24 (6648): No heartbeat from core client for 30 sec - exiting 14:25:25 (6648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:52:30 (4268): No heartbeat from core client for 30 sec - exiting 13:52:31 (4268): No heartbeat from core client for 30 sec - exiting 13:52:32 (4268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:28:34 (1052): No heartbeat from core client for 30 sec - exiting 02:28:36 (1052): No heartbeat from core client for 30 sec - exiting 02:28:37 (1052): No heartbeat from core client for 30 sec - exiting 02:28:38 (1052): No heartbeat from core client for 30 sec - exiting 02:28:39 (1052): No heartbeat from core client for 30 sec - exiting 02:28:40 (1052): No heartbeat from core client for 30 sec - exiting 02:28:42 (1052): No heartbeat from core client for 30 sec - exiting 02:28:43 (1052): No heartbeat from core client for 30 sec - exiting 02:28:44 (1052): No heartbeat from core client for 30 sec - exiting 02:28:45 (1052): No heartbeat from core client for 30 sec - exiting 02:28:46 (1052): No heartbeat from core client for 30 sec - exiting 02:28:47 (1052): No heartbeat from core client for 30 sec - exiting 02:28:48 (1052): No heartbeat from core client for 30 sec - exiting 02:28:49 (1052): No heartbeat from core client for 30 sec - exiting 02:28:50 (1052): No heartbeat from core client for 30 sec - exiting 02:28:51 (1052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4232, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5408, iMonCtr=1 Model crash detected, will try to restart... 16:15:46 (5648): No heartbeat from core client for 30 sec - exiting 16:15:47 (5648): No heartbeat from core client for 30 sec - exiting 16:15:48 (5648): No heartbeat from core client for 30 sec - exiting 16:15:49 (5648): No heartbeat from core client for 30 sec - exiting 16:15:50 (5648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 15:47:46 (3308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:48:55 (940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:28:50 (3440): No heartbeat from core client for 30 sec - exiting 07:28:51 (3440): No heartbeat from core client for 30 sec - exiting 07:28:52 (3440): No heartbeat from core client for 30 sec - exiting 07:28:53 (3440): No heartbeat from core client for 30 sec - exiting 07:28:54 (3440): No heartbeat from core client for 30 sec - exiting 07:28:55 (3440): No heartbeat from core client for 30 sec - exiting 07:28:57 (3440): No heartbeat from core client for 30 sec - exiting 07:28:58 (3440): No heartbeat from core client for 30 sec - exiting 07:28:59 (3440): No heartbeat from core client for 30 sec - exiting 07:29:00 (3440): No heartbeat from core client for 30 sec - exiting 07:29:01 (3440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:07:06 (6540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Sep 2013 14:26:26	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	1,036,800	3,206,083	3.0923
09 Sep 2013 18:58:33	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	1,010,880	3,130,671	3.0970
08 Sep 2013 11:17:54	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	984,960	3,044,578	3.0911
07 Sep 2013 08:03:55	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	959,040	2,960,300	3.0867
05 Sep 2013 15:05:37	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	933,120	2,878,725	3.0851
04 Sep 2013 05:10:10	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	907,200	2,797,432	3.0836
02 Sep 2013 12:30:44	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	881,280	2,710,922	3.0761
31 Aug 2013 19:34:45	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	855,360	2,626,136	3.0702
30 Aug 2013 03:19:23	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	829,440	2,538,899	3.0610
28 Aug 2013 09:38:25	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	803,520	2,456,023	3.0566
26 Aug 2013 22:52:44	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	777,600	2,371,303	3.0495
25 Aug 2013 14:04:12	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	751,680	2,284,625	3.0394
24 Aug 2013 09:23:21	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	725,760	2,200,935	3.0326
23 Aug 2013 10:06:07	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	699,840	2,118,404	3.0270
22 Aug 2013 06:03:39	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	673,920	2,039,430	3.0262
21 Aug 2013 02:50:16	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	648,000	1,958,306	3.0221
19 Aug 2013 01:08:16	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	622,080	1,869,078	3.0046
17 Aug 2013 13:35:29	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	596,160	1,784,638	2.9936
15 Aug 2013 19:09:50	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	570,240	1,702,537	2.9856
14 Aug 2013 17:09:21	1096394	15812095	hadcm3n_o7e2_1940_40_008380928_0	544,320	1,616,978	2.9706