Task 15935369

Name	hadcm3n_o301_1940_40_008379709_1
Workunit	8530568
Created	22 Aug 2013, 7:03:11 UTC
Sent	24 Aug 2013, 7:25:28 UTC
Report deadline	23 Nov 2013, 14:52:39 UTC
Received	14 Aug 2014, 18:55:17 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1215498
Run time	17 days 5 hours 37 min 36 sec
CPU time	15 days 3 hours 35 min 53 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	3.01 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... 20:42:02 (6456): No heartbeat from core client for 30 sec - exiting 20:42:03 (6456): No heartbeat from core client for 30 sec - exiting 20:42:04 (6456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:43:38 (2320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6388, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:01:55 (6884): No heartbeat from core client for 30 sec - exiting 09:01:56 (6884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:08:55 (6732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5288, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2256, iMonCtr=1 Model crash detected, will try to restart... 14:59:48 (4492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:16:47 (4720): No heartbeat from core client for 30 sec - exiting 16:16:48 (4720): No heartbeat from core client for 30 sec - exiting 16:16:49 (4720): No heartbeat from core client for 30 sec - exiting 16:16:51 (4720): No heartbeat from core client for 30 sec - exiting 16:16:52 (4720): No heartbeat from core client for 30 sec - exiting 16:16:53 (4720): No heartbeat from core client for 30 sec - exiting 16:16:54 (4720): No heartbeat from core client for 30 sec - exiting 16:16:55 (4720): No heartbeat from core client for 30 sec - exiting 16:16:56 (4720): No heartbeat from core client for 30 sec - exiting 16:16:57 (4720): No heartbeat from core client for 30 sec - exiting 16:16:58 (4720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:54:22 (4348): No heartbeat from core client for 30 sec - exiting 08:54:23 (4348): No heartbeat from core client for 30 sec - exiting 08:54:24 (4348): No heartbeat from core client for 30 sec - exiting 08:54:25 (4348): No heartbeat from core client for 30 sec - exiting 08:54:26 (4348): No heartbeat from core client for 30 sec - exiting 08:54:27 (4348): No heartbeat from core client for 30 sec - exiting 08:54:29 (4348): No heartbeat from core client for 30 sec - exiting 08:54:30 (4348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:54:31 (4348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 19:17:20 (2828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:55:13 (2616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:55:14 (2616): No heartbeat from core client for 30 sec - exiting 16:55:15 (2616): No heartbeat from core client for 30 sec - exiting 16:55:16 (2616): No heartbeat from core client for 30 sec - exiting 16:55:18 (2616): No heartbeat from core client for 30 sec - exiting 16:55:19 (2616): No heartbeat from core client for 30 sec - exiting 16:55:20 (2616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:00:42 (5436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:57:15 (4668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:10:35 (4204): No heartbeat from core client for 30 sec - exiting 17:10:37 (4204): No heartbeat from core client for 30 sec - exiting 17:10:38 (4204): No heartbeat from core client for 30 sec - exiting 17:10:39 (4204): No heartbeat from core client for 30 sec - exiting 17:10:40 (4204): No heartbeat from core client for 30 sec - exiting 17:10:41 (4204): No heartbeat from core client for 30 sec - exiting 17:10:42 (4204): No heartbeat from core client for 30 sec - exiting 17:10:43 (4204): No heartbeat from core client for 30 sec - exiting 17:10:44 (4204): No heartbeat from core client for 30 sec - exiting 17:10:45 (4204): No heartbeat from core client for 30 sec - exiting 17:10:47 (4204): No heartbeat from core client for 30 sec - exiting 17:10:48 (4204): No heartbeat from core client for 30 sec - exiting 17:10:49 (4204): No heartbeat from core client for 30 sec - exiting 17:10:50 (4204): No heartbeat from core client for 30 sec - exiting 17:10:51 (4204): No heartbeat from core client for 30 sec - exiting 17:10:52 (4204): No heartbeat from core client for 30 sec - exiting 17:10:53 (4204): No heartbeat from core client for 30 sec - exiting 17:10:54 (4204): No heartbeat from core client for 30 sec - exiting 17:10:55 (4204): No heartbeat from core client for 30 sec - exiting 17:10:56 (4204): No heartbeat from core client for 30 sec - exiting 17:10:57 (4204): No heartbeat from core client for 30 sec - exiting 17:10:59 (4204): No heartbeat from core client for 30 sec - exiting 17:11:00 (4204): No heartbeat from core client for 30 sec - exiting 17:11:01 (4204): No heartbeat from core client for 30 sec - exiting 17:11:02 (4204): No heartbeat from core client for 30 sec - exiting 17:11:03 (4204): No heartbeat from core client for 30 sec - exiting 17:11:04 (4204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:42:07 (5512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:50:06 (2272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:33:24 (5676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5224, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5224, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1036, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1036, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1036, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1036, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1036, iMonCtr=1 Model crash detected, will try to restart... 21:49:57 (1036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 forrtl: Es steht nicht genug Speicherplatz auf dem Datenträger zur Verfügung. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3876, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:54:19 (4468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5412, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5308, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5308, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4920, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6024, iMonCtr=1 Model crash detected, will try to restart... 08:29:41 (5928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:16:50 (5380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:32:58 (5564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:21:30 (3132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Aug 2014 17:59:24	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	1,036,800	1,308,951	1.2625
27 Jul 2014 17:46:46	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	1,010,880	1,276,002	1.2623
04 Jul 2014 12:33:22	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	984,960	1,243,019	1.2620
15 Jun 2014 16:46:55	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	959,040	1,209,162	1.2608
17 May 2014 08:58:57	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	933,120	1,175,368	1.2596
02 May 2014 19:49:26	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	907,200	1,142,071	1.2589
27 Mar 2014 20:06:02	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	881,280	1,109,216	1.2586
23 Mar 2014 13:34:35	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	855,360	1,074,034	1.2557
18 Mar 2014 19:44:51	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	829,440	1,040,253	1.2542
15 Mar 2014 15:32:47	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	803,520	1,007,171	1.2534
14 Mar 2014 21:43:28	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	777,600	974,534	1.2533
14 Mar 2014 12:24:40	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	751,680	941,831	1.2530
13 Mar 2014 16:11:59	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	725,760	908,263	1.2515
28 Feb 2014 06:45:59	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	699,840	874,209	1.2492
16 Feb 2014 10:48:54	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	673,920	840,470	1.2471
12 Feb 2014 17:09:07	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	648,000	806,175	1.2441
08 Feb 2014 15:59:40	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	622,080	772,184	1.2413
02 Feb 2014 20:53:54	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	596,160	738,182	1.2382
02 Feb 2014 09:48:40	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	570,240	704,766	1.2359
22 Jan 2014 21:22:58	1215498	15935369	hadcm3n_o301_1940_40_008379709_1	544,320	671,521	1.2337