Task 13927575

Name	hadcm3n_t5u3_1940_40_007683670_0
Workunit	7838757
Created	16 Jan 2012, 7:07:22 UTC
Sent	16 Jan 2012, 15:01:34 UTC
Report deadline	16 Apr 2012, 22:28:45 UTC
Received	10 May 2012, 17:28:24 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1077970
Run time	34 days 22 hours 19 min 2 sec
CPU time	27 days 0 hours 16 min 54 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	1.84 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5360, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5360, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5360, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5360, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5608, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=380, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5012, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5708, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:09:21 (4400): No heartbeat from core client for 30 sec - exiting 10:09:22 (4400): No heartbeat from core client for 30 sec - exiting 10:09:23 (4400): No heartbeat from core client for 30 sec - exiting 10:09:24 (4400): No heartbeat from core client for 30 sec - exiting 10:09:25 (4400): No heartbeat from core client for 30 sec - exiting 10:09:26 (4400): No heartbeat from core client for 30 sec - exiting 10:09:28 (4400): No heartbeat from core client for 30 sec - exiting 10:09:29 (4400): No heartbeat from core client for 30 sec - exiting 10:09:30 (4400): No heartbeat from core client for 30 sec - exiting 10:09:31 (4400): No heartbeat from core client for 30 sec - exiting 10:09:32 (4400): No heartbeat from core client for 30 sec - exiting 10:09:33 (4400): No heartbeat from core client for 30 sec - exiting 10:09:34 (4400): No heartbeat from core client for 30 sec - exiting 10:09:35 (4400): No heartbeat from core client for 30 sec - exiting 10:09:36 (4400): No heartbeat from core client for 30 sec - exiting 10:09:37 (4400): No heartbeat from core client for 30 sec - exiting 10:09:38 (4400): No heartbeat from core client for 30 sec - exiting 10:09:40 (4400): No heartbeat from core client for 30 sec - exiting 10:09:41 (4400): No heartbeat from core client for 30 sec - exiting 10:09:42 (4400): No heartbeat from core client for 30 sec - exiting 10:09:43 (4400): No heartbeat from core client for 30 sec - exiting 10:09:44 (4400): No heartbeat from core client for 30 sec - exiting 10:09:45 (4400): No heartbeat from core client for 30 sec - exiting 10:09:46 (4400): No heartbeat from core client for 30 sec - exiting 10:09:47 (4400): No heartbeat from core client for 30 sec - exiting 10:09:48 (4400): No heartbeat from core client for 30 sec - exiting 10:09:49 (4400): No heartbeat from core client for 30 sec - exiting 10:09:50 (4400): No heartbeat from core client for 30 sec - exiting 10:09:52 (4400): No heartbeat from core client for 30 sec - exiting 10:09:53 (4400): No heartbeat from core client for 30 sec - exiting 10:09:54 (4400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:19:14 (5080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:23:05 (5004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:23:07 (5644): Can't acquire lockfile (32) - waiting 35s Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:49:04 (2232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:00:06 (1720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 12:40:16 (6368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:40:17 (6368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 10:18:56 (3540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:01:59 (6680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:10:04 (10612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:09:08 (21944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 May 2012 16:29:15	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	1,036,800	2,333,016	2.2502
06 May 2012 07:51:43	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	1,010,880	2,275,443	2.2510
05 May 2012 13:59:46	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	984,960	2,213,938	2.2477
04 May 2012 21:59:55	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	959,040	2,158,063	2.2502
25 Apr 2012 20:40:10	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	933,120	2,101,415	2.2520
25 Apr 2012 03:10:54	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	907,200	2,043,753	2.2528
22 Apr 2012 18:56:40	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	881,280	1,986,624	2.2542
21 Apr 2012 03:45:40	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	855,360	1,926,753	2.2526
20 Apr 2012 04:22:18	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	829,440	1,866,976	2.2509
19 Apr 2012 05:24:34	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	803,520	1,807,543	2.2495
16 Apr 2012 07:22:02	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	777,600	1,748,749	2.2489
14 Apr 2012 13:05:55	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	751,680	1,692,562	2.2517
12 Apr 2012 20:08:03	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	725,760	1,634,756	2.2525
09 Apr 2012 19:34:57	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	699,840	1,578,625	2.2557
08 Apr 2012 23:44:37	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	673,920	1,520,768	2.2566
07 Apr 2012 01:34:15	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	648,000	1,462,502	2.2569
06 Apr 2012 08:46:04	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	622,080	1,404,203	2.2573
05 Apr 2012 05:42:32	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	596,160	1,349,294	2.2633
04 Apr 2012 13:37:28	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	570,240	1,293,368	2.2681
03 Apr 2012 20:43:58	1077970	13927575	hadcm3n_t5u3_1940_40_007683670_0	544,320	1,235,974	2.2707