Task 13024013

Name	hadcm3n_t46e_1940_40_007315509_1
Workunit	7512939
Created	28 Jun 2011, 21:49:47 UTC
Sent	28 Jun 2011, 22:08:51 UTC
Report deadline	28 Sep 2011, 5:36:02 UTC
Received	4 Aug 2011, 19:55:45 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	996482
Run time	29 days 5 hours 29 min 8 sec
CPU time	23 days 23 hours 44 min 48 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.34 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.13.1</core_client_version> <![CDATA[ <stderr_txt> 03:08:48 (452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:18:39 (3480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:16:29 (2928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:16:30 (2928): No heartbeat from core client for 30 sec - exiting 02:16:31 (2928): No heartbeat from core client for 30 sec - exiting 02:17:35 (5376): No heartbeat from core client for 30 sec - exiting 02:17:36 (5376): No heartbeat from core client for 30 sec - exiting 02:17:37 (5376): No heartbeat from core client for 30 sec - exiting 02:17:38 (5376): No heartbeat from core client for 30 sec - exiting 02:17:39 (5376): No heartbeat from core client for 30 sec - exiting 02:17:40 (5376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:31:47 (2656): No heartbeat from core client for 30 sec - exiting 23:31:48 (2656): No heartbeat from core client for 30 sec - exiting 23:31:49 (2656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3344, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4504, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4788, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 03:20:14 (3116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3692, iMonCtr=1 Model crash detected, will try to restart... 06:04:06 (1844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:21:20 (3104): No heartbeat from core client for 30 sec - exiting 08:21:21 (3104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:18:59 (2232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1568, iMonCtr=1 Model crash detected, will try to restart... 12:47:20 (3196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:45:31 (4868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:44:14 (1476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:44:15 (1476): No heartbeat from core client for 30 sec - exiting 10:07:26 (3444): No heartbeat from core client for 30 sec - exiting 10:07:27 (3444): No heartbeat from core client for 30 sec - exiting 10:07:28 (3444): No heartbeat from core client for 30 sec - exiting 10:07:29 (3444): No heartbeat from core client for 30 sec - exiting 10:07:30 (3444): No heartbeat from core client for 30 sec - exiting 10:07:31 (3444): No heartbeat from core client for 30 sec - exiting 10:07:32 (3444): No heartbeat from core client for 30 sec - exiting 10:07:34 (3444): No heartbeat from core client for 30 sec - exiting 10:07:35 (3444): No heartbeat from core client for 30 sec - exiting 10:07:36 (3444): No heartbeat from core client for 30 sec - exiting 10:07:37 (3444): No heartbeat from core client for 30 sec - exiting 10:07:38 (3444): No heartbeat from core client for 30 sec - exiting 10:07:39 (3444): No heartbeat from core client for 30 sec - exiting 10:07:40 (3444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:07:41 (3444): No heartbeat from core client for 30 sec - exiting 10:07:42 (3444): No heartbeat from core client for 30 sec - exiting 10:07:43 (3444): No heartbeat from core client for 30 sec - exiting 10:07:45 (3444): No heartbeat from core client for 30 sec - exiting 10:07:46 (3444): No heartbeat from core client for 30 sec - exiting 10:07:47 (3444): No heartbeat from core client for 30 sec - exiting 10:07:48 (3444): No heartbeat from core client for 30 sec - exiting 10:07:49 (3444): No heartbeat from core client for 30 sec - exiting 10:07:50 (3444): No heartbeat from core client for 30 sec - exiting 10:07:51 (3444): No heartbeat from core client for 30 sec - exiting Ocean Restart file copy failed on t46eko.dah2230 02:35:16 (3620): No heartbeat from core client for 30 sec - exiting 02:35:17 (3620): No heartbeat from core client for 30 sec - exiting 02:35:18 (3620): No heartbeat from core client for 30 sec - exiting 02:35:19 (3620): No heartbeat from core client for 30 sec - exiting 02:35:21 (3620): No heartbeat from core client for 30 sec - exiting 02:35:22 (3620): No heartbeat from core client for 30 sec - exiting 02:35:23 (3620): No heartbeat from core client for 30 sec - exiting 02:35:24 (3620): No heartbeat from core client for 30 sec - exiting 02:35:25 (3620): No heartbeat from core client for 30 sec - exiting 02:35:26 (3620): No heartbeat from core client for 30 sec - exiting 02:35:27 (3620): No heartbeat from core client for 30 sec - exiting 02:35:28 (3620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Ocean Restart file copy failed on t46eko.dah26e0 Ocean Restart file copy failed on t46eko.dah26i0 Ocean Restart file copy failed on t46eko.dah26k0 Ocean Restart file copy failed on t46eko.dah26s0 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... 22:39:46 (3272): No heartbeat from core client for 30 sec - exiting 22:39:47 (3272): No heartbeat from core client for 30 sec - exiting 22:39:48 (3272): No heartbeat from core client for 30 sec - exiting 22:39:49 (3272): No heartbeat from core client for 30 sec - exiting 22:39:50 (3272): No heartbeat from core client for 30 sec - exiting 22:39:51 (3272): No heartbeat from core client for 30 sec - exiting 22:39:52 (3272): No heartbeat from core client for 30 sec - exiting 22:39:53 (3272): No heartbeat from core client for 30 sec - exiting 22:39:54 (3272): No heartbeat from core client for 30 sec - exiting 22:39:55 (3272): No heartbeat from core client for 30 sec - exiting 22:39:56 (3272): No heartbeat from core client for 30 sec - exiting 22:39:58 (3272): No heartbeat from core client for 30 sec - exiting 22:39:59 (3272): No heartbeat from core client for 30 sec - exiting 22:40:00 (3272): No heartbeat from core client for 30 sec - exiting 22:40:01 (3272): No heartbeat from core client for 30 sec - exiting 22:40:02 (3272): No heartbeat from core client for 30 sec - exiting 22:40:03 (3272): No heartbeat from core client for 30 sec - exiting 22:40:04 (3272): No heartbeat from core client for 30 sec - exiting 22:40:05 (3272): No heartbeat from core client for 30 sec - exiting 22:40:06 (3272): No heartbeat from core client for 30 sec - exiting 22:40:07 (3272): No heartbeat from core client for 30 sec - exiting 22:40:09 (3272): No heartbeat from core client for 30 sec - exiting 22:40:10 (3272): No heartbeat from core client for 30 sec - exiting 22:40:11 (3272): No heartbeat from core client for 30 sec - exiting 22:40:12 (3272): No heartbeat from core client for 30 sec - exiting 22:40:13 (3272): No heartbeat from core client for 30 sec - exiting 22:40:14 (3272): No heartbeat from core client for 30 sec - exiting 22:40:15 (3272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:40:16 (3272): No heartbeat from core client for 30 sec - exiting 22:40:17 (3272): No heartbeat from core client for 30 sec - exiting 22:40:18 (3272): No heartbeat from core client for 30 sec - exiting 23:19:09 (2764): No heartbeat from core client for 30 sec - exiting 23:19:11 (2764): No heartbeat from core client for 30 sec - exiting 23:19:12 (2764): No heartbeat from core client for 30 sec - exiting 23:19:13 (2764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:19:14 (2764): No heartbeat from core client for 30 sec - exiting 23:19:15 (2764): No heartbeat from core client for 30 sec - exiting 23:19:16 (2764): No heartbeat from core client for 30 sec - exiting 23:19:17 (2764): No heartbeat from core client for 30 sec - exiting 23:19:18 (2764): No heartbeat from core client for 30 sec - exiting 23:19:19 (2764): No heartbeat from core client for 30 sec - exiting 23:19:20 (2764): No heartbeat from core client for 30 sec - exiting 23:19:21 (2764): No heartbeat from core client for 30 sec - exiting 23:19:23 (2764): No heartbeat from core client for 30 sec - exiting 23:19:24 (2764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 Aug 2011 18:58:30	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	1,036,800	2,071,906	1.9984
02 Aug 2011 14:53:55	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	1,010,880	2,018,786	1.9971
02 Aug 2011 00:08:52	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	984,960	1,966,122	1.9961
01 Aug 2011 09:59:33	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	959,040	1,913,255	1.9950
31 Jul 2011 13:09:01	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	933,120	1,858,547	1.9918
30 Jul 2011 20:41:03	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	907,200	1,803,868	1.9884
30 Jul 2011 04:25:35	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	881,280	1,911,392	2.1689
29 Jul 2011 11:19:18	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	855,360	1,855,835	2.1697
28 Jul 2011 16:23:09	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	829,440	1,800,911	2.1712
27 Jul 2011 18:16:03	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	803,520	1,746,104	2.1731
26 Jul 2011 15:30:33	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	777,600	1,691,416	2.1752
26 Jul 2011 00:34:14	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	751,680	1,638,972	2.1804
25 Jul 2011 22:48:07	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	725,760	1,586,165	2.1855
25 Jul 2011 21:44:16	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	699,840	1,529,480	2.1855
25 Jul 2011 19:32:37	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	673,920	1,472,706	2.1853
25 Jul 2011 19:32:37	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	648,000	1,416,697	2.1863
25 Jul 2011 19:32:36	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	622,080	1,359,484	2.1854
25 Jul 2011 18:49:23	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	596,160	1,304,975	2.1890
25 Jul 2011 18:01:07	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	570,240	1,247,991	2.1885
25 Jul 2011 17:14:46	996482	13024013	hadcm3n_t46e_1940_40_007315509_1	544,320	1,190,355	2.1869