Task 11900805

Name	hadsm3dhet2_u5eb_006726038_9
Workunit	6929381
Created	17 Sep 2010, 8:08:54 UTC
Sent	20 Sep 2010, 11:21:50 UTC
Report deadline	2 Sep 2011, 16:41:50 UTC
Received	1 Nov 2010, 8:41:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	985501
Run time	9 days 21 hours 2 min
CPU time	7 days 21 hours 17 min 7 sec
Validate state	Invalid
Credit	2,084.11
Device peak FLOPS	0.64 GFLOPS
Application version	UK Met Office HadSM3 Slab Model v6.08 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGILL: illegal instruction Stack trace (12 frames): /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x876060b] [0x4001d400] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8074f24] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x860d440] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x80710e6] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8095fe4] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c22cf] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86f5990] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c0886] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86feb0f] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x40076c76] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu(pthread_mutex_unlock+0x111)[0x804bb31] Exiting... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1125, iMonCtr=1 Model crash detected, will try to restart... SIGILL: illegal instruction Stack trace (12 frames): /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x876060b] [0x4001d400] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8074f24] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x860d440] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x80710e6] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8095fe4] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c22cf] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86f5990] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c0886] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86feb0f] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x40076c76] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu(pthread_mutex_unlock+0x111)[0x804bb31] Exiting... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1125, iMonCtr=1 Model crash detected, will try to restart... SIGILL: illegal instruction Stack trace (12 frames): /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x876060b] [0x4001d400] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8074f24] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x860d440] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x80710e6] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8095fe4] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c22cf] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86f5990] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c0886] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86feb0f] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x40076c76] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu(pthread_mutex_unlock+0x111)[0x804bb31] Exiting... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1125, iMonCtr=1 Model crash detected, will try to restart... SIGILL: illegal instruction Stack trace (12 frames): /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x876060b] [0x4001d400] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8074f24] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x860d440] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x80710e6] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8095fe4] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c22cf] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86f5990] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c0886] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86feb0f] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x40076c76] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu(pthread_mutex_unlock+0x111)[0x804bb31] Exiting... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1125, iMonCtr=1 Model crash detected, will try to restart... SIGILL: illegal instruction Stack trace (12 frames): /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x876060b] [0x4001d400] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8074f24] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x860d440] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x80710e6] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8095fe4] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c22cf] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86f5990] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c0886] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86feb0f] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x40076c76] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu(pthread_mutex_unlock+0x111)[0x804bb31] Exiting... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1125, iMonCtr=1 Model crash detected, will try to restart... SIGILL: illegal instruction Stack trace (12 frames): /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x876060b] [0x4001d400] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8074f24] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x860d440] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x80710e6] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8095fe4] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c22cf] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86f5990] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c0886] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86feb0f] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x40076c76] /var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu(pthread_mutex_unlock+0x111)[0x804bb31] Exiting... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1125, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Oct 2010 13:35:39	985501	11900805	hadsm3dhet2_u5eb_006726038_9	226,842	665,479	2.9337
29 Oct 2010 17:52:53	985501	11900805	hadsm3dhet2_u5eb_006726038_9	216,040	635,461	2.9414
27 Oct 2010 13:43:19	985501	11900805	hadsm3dhet2_u5eb_006726038_9	205,238	605,193	2.9487
23 Oct 2010 12:45:27	985501	11900805	hadsm3dhet2_u5eb_006726038_9	194,436	573,676	2.9505
22 Oct 2010 09:46:49	985501	11900805	hadsm3dhet2_u5eb_006726038_9	183,634	542,293	2.9531
19 Oct 2010 15:31:25	985501	11900805	hadsm3dhet2_u5eb_006726038_9	172,832	511,063	2.9570
18 Oct 2010 08:24:09	985501	11900805	hadsm3dhet2_u5eb_006726038_9	162,030	478,851	2.9553
15 Oct 2010 14:44:03	985501	11900805	hadsm3dhet2_u5eb_006726038_9	151,228	447,316	2.9579
14 Oct 2010 08:23:45	985501	11900805	hadsm3dhet2_u5eb_006726038_9	140,426	414,974	2.9551
12 Oct 2010 11:10:58	985501	11900805	hadsm3dhet2_u5eb_006726038_9	129,624	383,403	2.9578
11 Oct 2010 09:16:47	985501	11900805	hadsm3dhet2_u5eb_006726038_9	118,822	352,261	2.9646
10 Oct 2010 09:59:55	985501	11900805	hadsm3dhet2_u5eb_006726038_9	108,020	320,448	2.9666
10 Oct 2010 09:59:55	985501	11900805	hadsm3dhet2_u5eb_006726038_9	97,218	288,051	2.9629
06 Oct 2010 09:29:30	985501	11900805	hadsm3dhet2_u5eb_006726038_9	86,416	255,533	2.9570
05 Oct 2010 08:19:09	985501	11900805	hadsm3dhet2_u5eb_006726038_9	75,614	222,778	2.9463
03 Oct 2010 18:25:57	985501	11900805	hadsm3dhet2_u5eb_006726038_9	64,812	190,554	2.9401
02 Oct 2010 12:46:23	985501	11900805	hadsm3dhet2_u5eb_006726038_9	54,010	158,506	2.9348
30 Sep 2010 09:48:39	985501	11900805	hadsm3dhet2_u5eb_006726038_9	43,208	126,968	2.9385
28 Sep 2010 13:41:32	985501	11900805	hadsm3dhet2_u5eb_006726038_9	32,406	95,685	2.9527
23 Sep 2010 10:00:48	985501	11900805	hadsm3dhet2_u5eb_006726038_9	21,604	64,156	2.9696