Task 15868385

Name	hadcm3n_u42m_2020_40_008337719_2
Workunit	8488580
Created	26 Jun 2013, 23:58:20 UTC
Sent	27 Jun 2013, 0:14:55 UTC
Report deadline	26 Sep 2013, 7:42:06 UTC
Received	14 Aug 2013, 16:58:47 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1110511
Run time	13 days 1 hours 13 min 1 sec
CPU time	12 days 18 hours 55 min 25 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.44 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:54:04 (3028): No heartbeat from core client for 30 sec - exiting 07:54:05 (3028): No heartbeat from core client for 30 sec - exiting 07:54:06 (3028): No heartbeat from core client for 30 sec - exiting 07:54:07 (3028): No heartbeat from core client for 30 sec - exiting 07:54:08 (3028): No heartbeat from core client for 30 sec - exiting 07:54:09 (3028): No heartbeat from core client for 30 sec - exiting 07:54:10 (3028): No heartbeat from core client for 30 sec - exiting 07:54:11 (3028): No heartbeat from core client for 30 sec - exiting 07:54:13 (3028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3696, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5104, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:13:42 (4164): No heartbeat from core client for 30 sec - exiting 09:13:43 (4164): No heartbeat from core client for 30 sec - exiting 09:13:44 (4164): No heartbeat from core client for 30 sec - exiting 09:13:45 (4164): No heartbeat from core client for 30 sec - exiting 09:13:46 (4164): No heartbeat from core client for 30 sec - exiting 09:13:47 (4164): No heartbeat from core client for 30 sec - exiting 09:13:48 (4164): No heartbeat from core client for 30 sec - exiting 09:13:49 (4164): No heartbeat from core client for 30 sec - exiting 09:13:50 (4164): No heartbeat from core client for 30 sec - exiting 09:13:52 (4164): No heartbeat from core client for 30 sec - exiting 09:13:53 (4164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/u42mko.pjm8c10 Error converting file to netcdf: dataout/u42mko.pim8c10 Error converting file to netcdf: dataout/u42mko.pfm8c10 Error converting file to netcdf: dataout/u42mka.phm8c10 Error converting file to netcdf: dataout/u42mka.pgm8c10 Error converting file to netcdf: dataout/u42mka.pem8c10 Error converting file to netcdf: dataout/u42mka.pdm8c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:16:53 (5096): No heartbeat from core client for 30 sec - exiting 10:16:54 (5096): No heartbeat from core client for 30 sec - exiting 10:16:55 (5096): No heartbeat from core client for 30 sec - exiting 10:16:56 (5096): No heartbeat from core client for 30 sec - exiting 10:16:57 (5096): No heartbeat from core client for 30 sec - exiting 10:16:58 (5096): No heartbeat from core client for 30 sec - exiting 10:16:59 (5096): No heartbeat from core client for 30 sec - exiting 10:17:01 (5096): No heartbeat from core client for 30 sec - exiting 10:17:02 (5096): No heartbeat from core client for 30 sec - exiting 10:17:03 (5096): No heartbeat from core client for 30 sec - exiting 10:17:04 (5096): No heartbeat from core client for 30 sec - exiting 10:17:05 (5096): No heartbeat from core client for 30 sec - exiting 10:17:06 (5096): No heartbeat from core client for 30 sec - exiting 10:17:07 (5096): No heartbeat from core client for 30 sec - exiting 10:17:08 (5096): No heartbeat from core client for 30 sec - exiting 10:17:09 (5096): No heartbeat from core client for 30 sec - exiting 10:17:10 (5096): No heartbeat from core client for 30 sec - exiting 10:17:11 (5096): No heartbeat from core client for 30 sec - exiting 10:17:13 (5096): No heartbeat from core client for 30 sec - exiting 10:17:14 (5096): No heartbeat from core client for 30 sec - exiting 10:17:15 (5096): No heartbeat from core client for 30 sec - exiting 10:17:16 (5096): No heartbeat from core client for 30 sec - exiting 10:17:17 (5096): No heartbeat from core client for 30 sec - exiting 10:17:18 (5096): No heartbeat from core client for 30 sec - exiting 10:17:19 (5096): No heartbeat from core client for 30 sec - exiting 10:17:20 (5096): No heartbeat from core client for 30 sec - exiting 10:17:21 (5096): No heartbeat from core client for 30 sec - exiting 10:17:22 (5096): No heartbeat from core client for 30 sec - exiting 10:17:23 (5096): No heartbeat from core client for 30 sec - exiting 10:17:25 (5096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3912, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4512, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:13:29 (4736): No heartbeat from core client for 30 sec - exiting 08:13:31 (4736): No heartbeat from core client for 30 sec - exiting 08:13:32 (4736): No heartbeat from core client for 30 sec - exiting 08:13:33 (4736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Aug 2013 17:03:30	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	518,400	1,104,916	2.1314
14 Aug 2013 17:03:30	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	492,480	1,051,607	2.1353
14 Aug 2013 17:03:30	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	466,560	995,942	2.1346
14 Aug 2013 17:03:30	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	440,640	940,742	2.1349
14 Aug 2013 17:03:30	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	414,720	885,860	2.1360
29 Jul 2013 13:22:12	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	388,800	830,401	2.1358
26 Jul 2013 02:04:45	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	362,880	774,115	2.1333
24 Jul 2013 23:45:39	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	336,960	718,344	2.1318
23 Jul 2013 21:30:11	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	311,040	662,371	2.1295
23 Jul 2013 19:55:44	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	285,120	607,335	2.1301
23 Jul 2013 17:25:05	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	259,200	551,519	2.1278
23 Jul 2013 17:25:05	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	233,280	496,120	2.1267
23 Jul 2013 17:25:05	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	207,360	440,696	2.1253
23 Jul 2013 17:25:05	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	181,440	385,290	2.1235
11 Jul 2013 03:23:33	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	155,520	329,778	2.1205
09 Jul 2013 17:20:32	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	129,600	275,132	2.1229
06 Jul 2013 05:02:06	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	103,680	219,745	2.1195
02 Jul 2013 22:38:03	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	77,760	165,130	2.1236
02 Jul 2013 10:51:08	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	51,840	110,340	2.1285
02 Jul 2013 10:11:09	1110511	15868385	hadcm3n_u42m_2020_40_008337719_2	25,920	55,163	2.1282