Task 11459349

Name	famous_u6u7_1799_200_006641706_0
Workunit	6845078
Created	10 Jun 2010, 12:18:19 UTC
Sent	31 Jul 2010, 9:26:56 UTC
Report deadline	30 Oct 2010, 16:54:07 UTC
Received	16 Sep 2010, 9:12:06 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1024859
Run time	25 days 14 hours 55 min 2 sec
CPU time	2 days 20 hours 46 min 31 sec
Validate state	Invalid
Credit	1,389.76
Device peak FLOPS	1.88 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.6.37</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... (3406): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (17540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (18238): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (18238): No heartbeat from core client for 30 sec - exiting (24298): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (28136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting (5318): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (5318): No heartbeat from core client for 30 sec - exiting (10201): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (14209): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17503): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (10 frames): /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] /lib/libpthread.so.0[0x319368] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x816ae3d] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x816c826] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81756ab] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8325c74] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/libc.so.6(__libc_start_main+0xe5)[0x1266e5] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28320, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (10 frames): /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] /lib/libpthread.so.0[0x4a2368] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x816ae3d] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x816c826] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81756ab] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8325c74] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/libc.so.6(__libc_start_main+0xe5)[0x1fd6e5] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28320, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (10 frames): /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] /lib/libpthread.so.0[0x44a368] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x816ae3d] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x816c826] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81756ab] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8325c74] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/libc.so.6(__libc_start_main+0xe5)[0x1986e5] /home/admin/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28320, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28320, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28320, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28320, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( (28320): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Aug 2010 15:22:28	1024859	11459349	famous_u6u7_1799_200_006641706_0	421,226	243,515	0.5781
25 Aug 2010 01:35:20	1024859	11459349	famous_u6u7_1799_200_006641706_0	411,866	238,080	0.5781
24 Aug 2010 10:30:35	1024859	11459349	famous_u6u7_1799_200_006641706_0	402,506	232,677	0.5781
23 Aug 2010 19:42:57	1024859	11459349	famous_u6u7_1799_200_006641706_0	393,146	227,271	0.5781
23 Aug 2010 04:45:04	1024859	11459349	famous_u6u7_1799_200_006641706_0	383,786	221,848	0.5781
22 Aug 2010 13:52:39	1024859	11459349	famous_u6u7_1799_200_006641706_0	374,426	216,428	0.5780
21 Aug 2010 22:49:10	1024859	11459349	famous_u6u7_1799_200_006641706_0	365,066	210,957	0.5779
21 Aug 2010 07:47:00	1024859	11459349	famous_u6u7_1799_200_006641706_0	355,706	205,510	0.5778
20 Aug 2010 18:02:31	1024859	11459349	famous_u6u7_1799_200_006641706_0	346,346	200,096	0.5777
20 Aug 2010 04:37:55	1024859	11459349	famous_u6u7_1799_200_006641706_0	336,986	194,683	0.5777
19 Aug 2010 17:27:42	1024859	11459349	famous_u6u7_1799_200_006641706_0	327,626	189,265	0.5777
19 Aug 2010 04:35:17	1024859	11459349	famous_u6u7_1799_200_006641706_0	318,266	183,813	0.5775
18 Aug 2010 13:52:19	1024859	11459349	famous_u6u7_1799_200_006641706_0	308,906	178,376	0.5774
18 Aug 2010 00:07:00	1024859	11459349	famous_u6u7_1799_200_006641706_0	299,546	172,948	0.5774
17 Aug 2010 07:57:43	1024859	11459349	famous_u6u7_1799_200_006641706_0	290,186	167,518	0.5773
16 Aug 2010 16:31:50	1024859	11459349	famous_u6u7_1799_200_006641706_0	280,826	162,080	0.5772
16 Aug 2010 01:42:25	1024859	11459349	famous_u6u7_1799_200_006641706_0	271,466	156,650	0.5771
15 Aug 2010 10:36:27	1024859	11459349	famous_u6u7_1799_200_006641706_0	262,106	151,230	0.5770
14 Aug 2010 19:35:04	1024859	11459349	famous_u6u7_1799_200_006641706_0	252,746	145,774	0.5768
14 Aug 2010 04:34:33	1024859	11459349	famous_u6u7_1799_200_006641706_0	243,386	140,350	0.5767