Task 13141585

Name	hadcm3n_yemp_1900_40_007351819_2
Workunit	7549249
Created	16 Jul 2011, 3:53:11 UTC
Sent	16 Jul 2011, 3:55:03 UTC
Report deadline	15 Oct 2011, 11:22:14 UTC
Received	18 Sep 2011, 8:30:05 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1151731
Run time	36 days 17 hours 4 min 1 sec
CPU time	29 days 9 hours 40 min 50 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.62 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.6.38</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4776, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6920, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7672, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7672, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7672, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7672, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:37:38 (8896): No heartbeat from core client for 30 sec - exiting 09:37:39 (8896): No heartbeat from core client for 30 sec - exiting 09:37:40 (8896): No heartbeat from core client for 30 sec - exiting 09:37:41 (8896): No heartbeat from core client for 30 sec - exiting 09:37:42 (8896): No heartbeat from core client for 30 sec - exiting 09:37:43 (8896): No heartbeat from core client for 30 sec - exiting 09:37:44 (8896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:41:41 (532): No heartbeat from core client for 30 sec - exiting 08:41:42 (532): No heartbeat from core client for 30 sec - exiting 08:41:43 (532): No heartbeat from core client for 30 sec - exiting 08:41:44 (532): No heartbeat from core client for 30 sec - exiting 08:41:45 (532): No heartbeat from core client for 30 sec - exiting 08:41:46 (532): No heartbeat from core client for 30 sec - exiting 08:41:47 (532): No heartbeat from core client for 30 sec - exiting 08:41:48 (532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:37:10 (6016): No heartbeat from core client for 30 sec - exiting 04:37:11 (6016): No heartbeat from core client for 30 sec - exiting 04:37:12 (6016): No heartbeat from core client for 30 sec - exiting 04:37:13 (6016): No heartbeat from core client for 30 sec - exiting 04:37:14 (6016): No heartbeat from core client for 30 sec - exiting 04:37:15 (6016): No heartbeat from core client for 30 sec - exiting 04:37:17 (6016): No heartbeat from core client for 30 sec - exiting 04:37:18 (6016): No heartbeat from core client for 30 sec - exiting 04:37:19 (6016): No heartbeat from core client for 30 sec - exiting 04:37:20 (6016): No heartbeat from core client for 30 sec - exiting 04:37:21 (6016): No heartbeat from core client for 30 sec - exiting 04:37:22 (6016): No heartbeat from core client for 30 sec - exiting 04:37:23 (6016): No heartbeat from core client for 30 sec - exiting 04:37:24 (6016): No heartbeat from core client for 30 sec - exiting 04:37:25 (6016): No heartbeat from core client for 30 sec - exiting 04:37:26 (6016): No heartbeat from core client for 30 sec - exiting 04:37:27 (6016): No heartbeat from core client for 30 sec - exiting 04:37:29 (6016): No heartbeat from core client for 30 sec - exiting 04:37:30 (6016): No heartbeat from core client for 30 sec - exiting 04:37:31 (6016): No heartbeat from core client for 30 sec - exiting 04:37:32 (6016): No heartbeat from core client for 30 sec - exiting 04:37:33 (6016): No heartbeat from core client for 30 sec - exiting 04:37:34 (6016): No heartbeat from core client for 30 sec - exiting 04:37:35 (6016): No heartbeat from core client for 30 sec - exiting 04:37:36 (6016): No heartbeat from core client for 30 sec - exiting 04:37:37 (6016): No heartbeat from core client for 30 sec - exiting 04:37:38 (6016): No heartbeat from core client for 30 sec - exiting 04:37:40 (6016): No heartbeat from core client for 30 sec - exiting 04:37:41 (6016): No heartbeat from core client for 30 sec - exiting 04:37:42 (6016): No heartbeat from core client for 30 sec - exiting 04:37:43 (6016): No heartbeat from core client for 30 sec - exiting 04:37:44 (6016): No heartbeat from core client for 30 sec - exiting 04:37:45 (6016): No heartbeat from core client for 30 sec - exiting 04:37:46 (6016): No heartbeat from core client for 30 sec - exiting 04:37:47 (6016): No heartbeat from core client for 30 sec - exiting 04:37:48 (6016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:04:12 (10620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:56:33 (4204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:35:44 (6692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:35:45 (6692): No heartbeat from core client for 30 sec - exiting 14:35:46 (6692): No heartbeat from core client for 30 sec - exiting 14:35:47 (6692): No heartbeat from core client for 30 sec - exiting 14:35:48 (6692): No heartbeat from core client for 30 sec - exiting 14:35:49 (6692): No heartbeat from core client for 30 sec - exiting 14:35:50 (6692): No heartbeat from core client for 30 sec - exiting 17:14:07 (8048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:52:34 (4028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7828, selfPID=7828, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Sep 2011 23:29:08	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	1,036,800	2,539,759	2.4496
16 Sep 2011 17:46:32	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	1,010,880	2,490,191	2.4634
15 Sep 2011 10:18:51	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	984,960	2,440,728	2.4780
13 Sep 2011 22:31:22	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	959,040	2,392,122	2.4943
12 Sep 2011 14:28:09	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	933,120	2,343,109	2.5110
11 Sep 2011 00:34:39	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	907,200	2,295,359	2.5302
10 Sep 2011 04:39:02	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	881,280	2,249,174	2.5522
09 Sep 2011 10:21:16	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	855,360	2,202,332	2.5747
08 Sep 2011 17:23:15	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	829,440	2,156,878	2.6004
05 Sep 2011 14:55:10	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	803,520	2,112,780	2.6294
04 Sep 2011 21:56:17	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	777,600	2,065,348	2.6561
01 Sep 2011 11:11:34	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	751,680	2,018,298	2.6850
31 Aug 2011 15:35:25	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	725,760	1,971,049	2.7158
30 Aug 2011 13:39:26	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	699,840	1,923,100	2.7479
29 Aug 2011 21:52:15	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	673,920	1,875,284	2.7827
29 Aug 2011 06:45:19	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	648,000	1,827,304	2.8199
28 Aug 2011 12:32:05	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	622,080	1,780,212	2.8617
27 Aug 2011 21:25:46	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	596,160	1,732,351	2.9058
27 Aug 2011 04:31:46	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	570,240	1,685,774	2.9563
26 Aug 2011 07:21:58	1151731	13141585	hadcm3n_yemp_1900_40_007351819_2	544,320	1,637,370	3.0081