Task 13692078

Name	hadam3p_saf_7fid_2003_1_007577038_0
Workunit	7755168
Created	2 Dec 2011, 16:17:12 UTC
Sent	13 Dec 2011, 21:38:32 UTC
Report deadline	25 Nov 2012, 2:58:32 UTC
Received	30 Jan 2012, 0:31:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1184868
Run time	4 days 9 hours 51 min 5 sec
CPU time	3 days 9 hours 1 min 43 sec
Validate state	Invalid
Credit	1,122.82
Device peak FLOPS	1.99 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Southern Africa v6.09 windows_intelx86
Stderr	<core_client_version>7.0.2</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3180, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4032, selfPID=3224, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2820, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3344, selfPID=888, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3340, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3964, selfPID=960, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4092, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2812, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3280, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3080, selfPID=3832, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1892, iMonCtr=2 CCPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1740, selfPID=1740, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3476, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3772, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3776, selfPID=3424, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3616, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3368, selfPID=2828, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3484, selfPID=3632, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3256, selfPID=3256, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1040, selfPID=1040, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4048, selfPID=3644, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2464, selfPID=2892, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:14:39 (3908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:14:43 (2052): Can't acquire lockfile (32) - waiting 35s 22:51:12 (2052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:30:33 (3900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:22:02 (3912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3096, iMonCtr=2 Model crash detected, will try to restart... 21:11:39 (2324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:11:40 (2324): No heartbeat from core client for 30 sec - exiting 21:42:09 (1328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:42:10 (1328): No heartbeat from core client for 30 sec - exiting 21:42:11 (1328): No heartbeat from core client for 30 sec - exiting 21:42:12 (1328): No heartbeat from core client for 30 sec - exiting 21:45:16 (3636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Jan 2012 21:36:22	1184868	13692078	hadam3p_saf_7fid_2003_1_007577038_0	69,216	284,323	4.1078
18 Jan 2012 22:44:22	1184868	13692078	hadam3p_saf_7fid_2003_1_007577038_0	57,696	238,049	4.1259
08 Jan 2012 19:39:17	1184868	13692078	hadam3p_saf_7fid_2003_1_007577038_0	46,176	190,345	4.1222
04 Jan 2012 09:01:15	1184868	13692078	hadam3p_saf_7fid_2003_1_007577038_0	34,656	142,962	4.1252
27 Dec 2011 23:07:26	1184868	13692078	hadam3p_saf_7fid_2003_1_007577038_0	23,136	95,685	4.1358
20 Dec 2011 23:30:28	1184868	13692078	hadam3p_saf_7fid_2003_1_007577038_0	11,616	48,589	4.1829