Task 15652083

Name	hadcm3n_4lfw_1980_40_008325112_0
Workunit	8476247
Created	7 Mar 2013, 0:55:24 UTC
Sent	7 Mar 2013, 0:55:27 UTC
Report deadline	6 Jun 2013, 8:22:38 UTC
Received	18 Jul 2013, 13:27:42 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1217292
Run time	9 days 21 hours 29 min 21 sec
CPU time	8 days 17 hours 43 min 9 sec
Validate state	Invalid
Credit	10,575.36
Device peak FLOPS	3.32 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:54:11 (54537): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:54:12 (54537): No heartbeat from core client for 30 sec - exiting 11:54:13 (54537): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:03:56 (232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:03:57 (232): No heartbeat from core client for 30 sec - exiting 03:04:28 (232): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:51:45 (235): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:56:03 (244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:29:31 (401): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:05:54 (247): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:05:55 (247): No heartbeat from core client for 30 sec - exiting 16:05:56 (247): No heartbeat from core client for 30 sec - exiting 16:05:57 (247): No heartbeat from core client for 30 sec - exiting 16:05:58 (247): No heartbeat from core client for 30 sec - exiting 16:05:59 (247): No heartbeat from core client for 30 sec - exiting 16:06:00 (247): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... hadcm3n_6.07_i686-apple-darwin(242,0xacb282c0) malloc: * error for object 0x1292804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(242,0xacb282c0) malloc: * error for object 0x1293604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(240,0xacb282c0) malloc: * error for object 0x10d9004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(240,0xacb282c0) malloc: * error for object 0x10d9000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(240,0xacb282c0) malloc: * error for object 0x28e2004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(240,0xacb282c0) malloc: * error for object 0x28e2000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(240,0xacb282c0) malloc: * error for object 0x28de004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(243,0xacb282c0) malloc: * error for object 0xac2a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(243,0xacb282c0) malloc: * error for object 0xac2a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(243,0xacb282c0) malloc: * error for object 0xac2a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(243,0xacb282c0) malloc: * error for object 0xac2a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(243,0xacb282c0) malloc: * error for object 0xac2a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(243,0xacb282c0) malloc: * error for object 0xac2a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(243,0xacb282c0) malloc: * error for object 0xac2a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(243,0xacb282c0) malloc: * error for object 0xac2a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(243,0xacb282c0) malloc: * error for object 0xac2a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(242,0xac4db2c0) malloc: * error for object 0x2b36804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(242,0xac4db2c0) malloc: * error for object 0x2b36800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(242,0xac4db2c0) malloc: * error for object 0x2b36804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(242,0xac4db2c0) malloc: * error for object 0x2b36800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(242,0xac4db2c0) malloc: * error for object 0x2b36804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(242,0xac4db2c0) malloc: * error for object 0x2b36800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(242,0xac4db2c0) malloc: * error for object 0x2b36804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(242,0xac4db2c0) malloc: * error for object 0x2b36800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(242,0xac4db2c0) malloc: * error for object 0x2b36804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(242,0xac4db2c0) malloc: * error for object 0x2b36800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(242,0xac4db2c0) malloc: * error for object 0x2b36804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(242,0xac4db2c0) malloc: * error for object 0x2b36800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(227,0xac4db2c0) malloc: * error for object 0x8e6a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(227,0xac4db2c0) malloc: * error for object 0x8e6a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(227,0xac4db2c0) malloc: * error for object 0x28e6a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(227,0xac4db2c0) malloc: * error for object 0x28e6a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(227,0xac4db2c0) malloc: * error for object 0x28e6a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(227,0xac4db2c0) malloc: * error for object 0x28e6a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(227,0xac4db2c0) malloc: * error for object 0x28e6a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(227,0xac4db2c0) malloc: * error for object 0x28e6a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(227,0xac4db2c0) malloc: * error for object 0x28e6a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(227,0xac4db2c0) malloc: * error for object 0x28e6a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(227,0xac4db2c0) malloc: * error for object 0x28e6a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(227,0xac4db2c0) malloc: * error for object 0x28e6a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 3 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Jul 2013 16:50:09	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	881,280	748,727	0.8496
11 Jul 2013 20:44:50	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	855,360	730,367	0.8539
11 Jul 2013 02:38:24	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	829,440	710,301	0.8564
10 Jul 2013 06:15:23	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	803,520	688,014	0.8563
20 Mar 2013 19:42:44	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	777,600	666,024	0.8565
19 Mar 2013 21:59:09	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	751,680	643,995	0.8567
19 Mar 2013 14:00:45	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	725,760	621,852	0.8568
19 Mar 2013 06:58:37	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	699,840	599,426	0.8565
18 Mar 2013 22:06:16	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	673,920	577,403	0.8568
18 Mar 2013 15:00:25	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	648,000	555,314	0.8570
18 Mar 2013 07:48:26	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	622,080	533,111	0.8570
17 Mar 2013 21:14:35	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	596,160	510,836	0.8569
17 Mar 2013 11:04:31	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	570,240	488,681	0.8570
17 Mar 2013 03:02:43	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	544,320	466,204	0.8565
16 Mar 2013 16:45:58	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	518,400	443,940	0.8564
16 Mar 2013 05:57:16	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	492,480	421,701	0.8563
15 Mar 2013 17:51:45	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	466,560	399,523	0.8563
15 Mar 2013 03:47:07	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	440,640	377,401	0.8565
14 Mar 2013 17:56:30	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	414,720	355,411	0.8570
13 Mar 2013 19:48:09	1217292	15652083	hadcm3n_4lfw_1980_40_008325112_0	388,800	333,368	0.8574