Task 12328031

Name	hadam3p_pnw_ztie_1997_1_007040766_0
Workunit	7244082
Created	25 Nov 2010, 12:09:07 UTC
Sent	23 Dec 2010, 0:14:23 UTC
Report deadline	5 Dec 2011, 5:34:23 UTC
Received	1 Feb 2011, 12:44:17 UTC
Server state	Over
Outcome	No reply
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1075370
Run time	7 days 2 hours 18 min 39 sec
CPU time	4 days 18 hours 18 min 38 sec
Validate state	Invalid
Credit	2,755.56
Device peak FLOPS	2.35 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.08 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> 19:17:28 (3024): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:01:36 (6088): start_timer_thread(): CreateThread() failed, errno 0 13:01:43 (4936): start_timer_thread(): CreateThread() failed, errno 0 13:26:07 (6028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:26:14 (4652): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2148, selfPID=2148, iMonCtr=2 20:25:55 (2760): start_timer_thread(): CreateThread() failed, errno 0 20:26:05 (5036): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5036, selfPID=5036, iMonCtr=2 21:28:49 (4600): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:40:08 (3688): start_timer_thread(): CreateThread() failed, errno 0 16:40:15 (5092): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5092, selfPID=5092, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=616, selfPID=616, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=720, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3784, selfPID=3784, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2464, selfPID=4792, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 08:47:28 (2892): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 10:33:01 (4016): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4004, selfPID=4004, iMonCtr=2 11:54:09 (4768): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2260, selfPID=2260, iMonCtr=2 14:19:51 (3080): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:56:42 (1344): start_timer_thread(): CreateThread() failed, errno 0 16:56:49 (6064): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:10:41 (672): start_timer_thread(): CreateThread() failed, errno 0 19:10:48 (6084): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... GCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:36:42 (4732): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3612, selfPID=3612, iMonCtr=2 CPDN Monitor - Quit request from BOINC... GCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3908, selfPID=3908, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5924, selfPID=6000, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5680, selfPID=5680, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=388, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2856, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1448, selfPID=1448, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5940, selfPID=5940, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4180, selfPID=4180, iMonCtr=2 18:40:03 (5520): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3232, selfPID=3232, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 11:25:07 (4456): start_timer_thread(): CreateThread() failed, errno 0 11:25:13 (6072): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:38:59 (6616): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:19:55 (1384): start_timer_thread(): CreateThread() failed, errno 0 21:20:05 (7252): start_timer_thread(): CreateThread() failed, errno 0 11:08:40 (2604): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4816, selfPID=4816, iMonCtr=2 14:10:03 (7164): start_timer_thread(): CreateThread() failed, errno 0 21:17:10 (2748): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4160, selfPID=4160, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:59:36 (2124): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:19:37 (4484): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:49:51 (2976): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5012, selfPID=2140, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2096, selfPID=2096, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:48:15 (4836): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:01:10 (7648): start_timer_thread(): CreateThread() failed, errno 0 18:01:16 (3228): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:11:03 (5000): start_timer_thread(): CreateThread() failed, errno 0 12:11:10 (3712): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 14:23:39 (6968): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:53:07 (4164): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:36:19 (6036): start_timer_thread(): CreateThread() failed, errno 0 14:36:29 (5116): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5432, selfPID=5432, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:43:50 (3196): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6136, selfPID=6136, iMonCtr=2 12:21:57 (5452): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5480, selfPID=5480, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2556, selfPID=2556, iMonCtr=2 16:14:27 (4908): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2684, selfPID=2412, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 10 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=880, selfPID=880, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 13:32:51 (4440): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:43:59 (1304): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:40:42 (2864): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 08:58:50 (5316): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:07:32 (396): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:24:36 (5884): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:03:34 (1972): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:59:51 (3264): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5772, selfPID=5772, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3176, selfPID=3176, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5428, selfPID=5428, iMonCtr=2 </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Jan 2011 16:17:56	1075370	12328031	hadam3p_pnw_ztie_1997_1_007040766_0	126,816	383,242	3.0220
26 Jan 2011 23:17:55	1075370	12328031	hadam3p_pnw_ztie_1997_1_007040766_0	115,296	348,143	3.0196
24 Jan 2011 23:37:23	1075370	12328031	hadam3p_pnw_ztie_1997_1_007040766_0	103,776	313,027	3.0164
22 Jan 2011 18:27:53	1075370	12328031	hadam3p_pnw_ztie_1997_1_007040766_0	92,256	279,619	3.0309
20 Jan 2011 15:19:33	1075370	12328031	hadam3p_pnw_ztie_1997_1_007040766_0	80,736	243,083	3.0108
19 Jan 2011 11:51:27	1075370	12328031	hadam3p_pnw_ztie_1997_1_007040766_0	69,217	209,432	3.0257
19 Jan 2011 00:38:04	1075370	12328031	hadam3p_pnw_ztie_1997_1_007040766_0	69,216	209,055	3.0203
13 Jan 2011 22:17:59	1075370	12328031	hadam3p_pnw_ztie_1997_1_007040766_0	57,696	173,421	3.0058
11 Jan 2011 18:58:15	1075370	12328031	hadam3p_pnw_ztie_1997_1_007040766_0	46,176	138,354	2.9962
08 Jan 2011 00:07:12	1075370	12328031	hadam3p_pnw_ztie_1997_1_007040766_0	34,656	104,657	3.0199
04 Jan 2011 00:47:29	1075370	12328031	hadam3p_pnw_ztie_1997_1_007040766_0	23,136	70,818	3.0609
30 Dec 2010 22:09:17	1075370	12328031	hadam3p_pnw_ztie_1997_1_007040766_0	11,616	35,331	3.0416