Task 17501511

Name	hadam3p_pnw_vk21_2006_1_009230965_0
Workunit	9344076
Created	27 Nov 2014, 15:52:18 UTC
Sent	29 Nov 2014, 3:35:40 UTC
Report deadline	11 Nov 2015, 8:55:40 UTC
Received	9 Dec 2014, 12:36:52 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1337590
Run time	4 days 2 hours 20 min 12 sec
CPU time	4 days 0 hours 5 min 5 sec
Validate state	Invalid
Credit	2,009.02
Device peak FLOPS	1.50 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v7.22 i686-pc-linux-gnu
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... 18:23:21 (5641): called boinc_finish 18:23:21 (5641): called boinc_finish Signal 15 received, exiting... 18:23:21 (5642): called boinc_finish 18:23:21 (5642): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 21:17:00 (2238): called boinc_finish Signal 15 received, exiting... 21:17:00 (2239): called boinc_finish Signal 15 received, exiting... 21:17:00 (2239): called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (16 frames): ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80c8ccf] [0xf776f400] /lib/libc.so.6(getenv+0x72)[0x4e821192] /lib/libc.so.6[0x4e89a2fe] /lib/libc.so.6[0x4e89ac0f] /lib/libc.so.6(localtime_r+0x2b)[0x4e898e1b] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c4306] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c47b6] [0xf776f400] [0xf776f425] /lib/libc.so.6(nanosleep+0x46)[0x4e8a8e16] /lib/libc.so.6(usleep+0x3d)[0x4e8deaed] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80da8e0] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c5884] /lib/libpthread.so.0[0x4e9b69dc] /lib/libc.so.6(clone+0x5e)[0x4e8e7a1e] Exiting... Signal 15 received, exiting... 02:24:26 (2193): called boinc_finish Signal 15 received, exiting... 02:24:26 (2193): called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (16 frames): ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80c8ccf] [0xf7781400] /lib/libc.so.6(getenv+0x72)[0x4e821192] /lib/libc.so.6[0x4e89a2fe] /lib/libc.so.6[0x4e89ac0f] /lib/libc.so.6(localtime_r+0x2b)[0x4e898e1b] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c4306] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c47b6] [0xf7781400] [0xf7781425] /lib/libc.so.6(nanosleep+0x46)[0x4e8a8e16] /lib/libc.so.6(usleep+0x3d)[0x4e8deaed] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80da8e0] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c5884] /lib/libpthread.so.0[0x4e9b69dc] /lib/libc.so.6(clone+0x5e)[0x4e8e7a1e] Exiting... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (16 frames): ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80c8ccf] [0xf776b400] /lib/libc.so.6(getenv+0x72)[0x4e821192] /lib/libc.so.6[0x4e89a2fe] /lib/libc.so.6[0x4e89ac0f] /lib/libc.so.6(localtime_r+0x2b)[0x4e898e1b] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c4306] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c47b6] [0xf776b400] [0xf776b425] /lib/libc.so.6(nanosleep+0x46)[0x4e8a8e16] /lib/libc.so.6(usleep+0x3d)[0x4e8deaed] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80da8e0] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c5884] /lib/libpthread.so.0[0x4e9b69dc] /lib/libc.so.6(clone+0x5e)[0x4e8e7a1e] Exiting... Signal 15 received, exiting... Signal 15 received, exiting... 17:33:10 (2334): called boinc_finish Signal 15 received, exiting... 17:33:10 (2334): called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (18 frames): ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80c8ccf] [0xf77ea400] /lib/libc.so.6(getenv+0x72)[0x4e821192] /lib/libc.so.6[0x4e89a2fe] /lib/libc.so.6[0x4e89ac0f] /lib/libc.so.6(localtime_r+0x2b)[0x4e898e1b] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c4306] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c47b6] [0xf77ea400] [0xf77ea425] /lib/libc.so.6(nanosleep+0x46)[0x4e8a8e16] /lib/libc.so.6(usleep+0x3d)[0x4e8deaed] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80da8e0] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8050f76] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8051aa9] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8051d7b] /lib/libc.so.6(__libc_start_main+0xf3)[0x4e809963] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(__gxx_personality_v0+0x1bd)[0x804d331] Exiting... Signal 15 received, exiting... Signal 15 received, exiting... 21:50:24 (22160): called boinc_finish 21:50:24 (22159): called boinc_finish Signal 15 received, exiting... 21:50:24 (22159): called boinc_finish Signal 15 received, exiting... 21:50:24 (22160): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... 22:25:38 (2400): called boinc_finish 22:25:38 (2400): called boinc_finish 22:25:38 (2399): called boinc_finish 22:25:38 (2399): called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... 22:28:33 (1665): called boinc_finish 22:28:33 (1665): called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (18 frames): ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80c8ccf] [0xf7754400] /lib/libc.so.6(getenv+0x72)[0x4e821192] /lib/libc.so.6[0x4e89a2fe] /lib/libc.so.6[0x4e89ac0f] /lib/libc.so.6(localtime_r+0x2b)[0x4e898e1b] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c4306] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c47b6] [0xf7754400] [0xf7754425] /lib/libc.so.6(nanosleep+0x46)[0x4e8a8e16] /lib/libc.so.6(usleep+0x3d)[0x4e8deaed] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80da8e0] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8050f76] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8051aa9] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8051d7b] /lib/libc.so.6(__libc_start_main+0xf3)[0x4e809963] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(__gxx_personality_v0+0x1bd)[0x804d331] Exiting... Signal 15 received, exiting... Signal 15 received, exiting... 21:01:49 (2410): called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... 21:01:49 (2410): called boinc_finish 21:01:49 (2411): called boinc_finish 21:01:49 (2411): called boinc_finish Signal 15 received, exiting... 00:43:49 (2389): called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (16 frames): ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80c8ccf] [0xf7735400] /lib/libc.so.6(getenv+0x72)[0x4e821192] /lib/libc.so.6[0x4e89a2fe] /lib/libc.so.6[0x4e89ac0f] /lib/libc.so.6(localtime_r+0x2b)[0x4e898e1b] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c4306] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c47b6] [0xf7735400] [0xf7735425] /lib/libc.so.6(nanosleep+0x46)[0x4e8a8e16] /lib/libc.so.6(usleep+0x3d)[0x4e8deaed] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80da8e0] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c5884] /lib/libpthread.so.0[0x4e9b69dc] /lib/libc.so.6(clone+0x5e)[0x4e8e7a1e] Exiting... Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (20 frames): ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80c8ccf] [0xf77f3400] /lib/libc.so.6(getenv+0x72)[0x4e821192] /lib/libc.so.6[0x4e89a2fe] /lib/libc.so.6[0x4e89ac0f] /lib/libc.so.6(localtime_r+0x2b)[0x4e898e1b] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c4306] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c47b6] [0xf77f3400] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c46a0] [0xf77f3400] [0xf77f3425] /lib/libc.so.6(nanosleep+0x46)[0x4e8a8e16] /lib/libc.so.6(usleep+0x3d)[0x4e8deaed] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80da8e0] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8050f76] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8051aa9] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8051d7b] /lib/libc.so.6(__libc_start_main+0xf3)[0x4e809963] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(__gxx_personality_v0+0x1bd)[0x804d331] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (19 frames): ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80c8ccf] [0xf7745400] /lib/libc.so.6(getenv+0x72)[0x4e821192] /lib/libc.so.6[0x4e89a2fe] /lib/libc.so.6[0x4e89ac0f] /lib/libc.so.6(localtime_r+0x2b)[0x4e898e1b] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c4306] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c47b6] [0xf7745400] [0xf7745425] /lib/libc.so.6(nanosleep+0x46)[0x4e8a8e16] /lib/libc.so.6(usleep+0x3d)[0x4e8deaed] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80da8e0] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80507b5] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8050f65] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8051aa9] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8051d7b] /lib/libc.so.6(__libc_start_main+0xf3)[0x4e809963] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(__gxx_personality_v0+0x1bd)[0x804d331] Exiting... Signal 15 received, exiting... 22:56:23 (2263): called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (18 frames): ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80c8ccf] [0xf77b3cb0] /lib/libc.so.6(getenv+0x72)[0xf74613f2] /lib/libc.so.6(+0xaa43e)[0xf74da43e] /lib/libc.so.6(+0xaad4f)[0xf74dad4f] /lib/libc.so.6(localtime_r+0x2b)[0xf74d8f5b] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c4306] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80c47b6] [0xf77b3cb0] [0xf77b3cd5] /lib/libc.so.6(nanosleep+0x46)[0xf74e8ff6] /lib/libc.so.6(usleep+0x3d)[0xf751e8ed] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x80da8e0] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8050f76] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8051aa9] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8051d7b] /lib/libc.so.6(__libc_start_main+0xf3)[0xf7449b73] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(__gxx_personality_v0+0x1bd)[0x804d331] Exiting... Signal 3 received, exiting... SIGSEGV: segmentation violation 13:35:10 (1385): called boinc_finish Stack trace (7 frames): ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80c8ccf] [0xf7752cb0] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x804d6bf] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x805134f] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu[0x8051d7b] /lib/libc.so.6(__libc_start_main+0xf3)[0x4ffa1b73] ../../projects/climateprediction.net/hadam3p_pnw_7.22_i686-pc-linux-gnu(__gxx_personality_v0+0x1bd)[0x804d331] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Dec 2014 16:33:19	1337590	17501511	hadam3p_pnw_vk21_2006_1_009230965_0	92,459	330,870	3.5786
05 Dec 2014 05:41:46	1337590	17501511	hadam3p_pnw_vk21_2006_1_009230965_0	80,939	292,248	3.6107
04 Dec 2014 17:41:41	1337590	17501511	hadam3p_pnw_vk21_2006_1_009230965_0	69,419	255,042	3.6740
04 Dec 2014 06:39:11	1337590	17501511	hadam3p_pnw_vk21_2006_1_009230965_0	57,899	215,859	3.7282
03 Dec 2014 19:45:51	1337590	17501511	hadam3p_pnw_vk21_2006_1_009230965_0	46,379	178,158	3.8414
03 Dec 2014 05:18:51	1337590	17501511	hadam3p_pnw_vk21_2006_1_009230965_0	34,859	129,770	3.7227
02 Dec 2014 16:07:35	1337590	17501511	hadam3p_pnw_vk21_2006_1_009230965_0	23,339	82,113	3.5183
02 Dec 2014 04:25:57	1337590	17501511	hadam3p_pnw_vk21_2006_1_009230965_0	11,819	40,874	3.4583