Task 15822997

Name	hadcm3n_o13n_1980_40_008387220_0
Workunit	8538079
Created	3 Jun 2013, 9:13:21 UTC
Sent	7 Jun 2013, 17:55:35 UTC
Report deadline	7 Sep 2013, 1:22:46 UTC
Received	18 Jun 2013, 10:06:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1110299
Run time	7 days 12 hours 13 min 30 sec
CPU time	5 days 22 hours 24 min 19 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:50:09 (4530): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:50:11 (4530): No heartbeat from core client for 30 sec - exiting 20:50:12 (4530): No heartbeat from core client for 30 sec - exiting 20:50:13 (4530): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:10:23 (8833): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:38:42 (75996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/o13nko.pji5c10 is not a valid UM file. Error converting file to netcdf: dataout/o13nko.pji5c10 Error: Input file: dataout/o13nko.pii5c10 is not a valid UM file. Error converting file to netcdf: dataout/o13nko.pii5c10 Error: Input file: dataout/o13nko.pfi5c10 is not a valid UM file. Error converting file to netcdf: dataout/o13nko.pfi5c10 Error: Input file: dataout/o13nka.phi5c10 is not a valid UM file. Error converting file to netcdf: dataout/o13nka.phi5c10 Error: Input file: dataout/o13nka.pgi5c10 is not a valid UM file. Error converting file to netcdf: dataout/o13nka.pgi5c10 Error: Input file: dataout/o13nka.pei5c10 is not a valid UM file. Error converting file to netcdf: dataout/o13nka.pei5c10 Error: Input file: dataout/o13nka.pdi5c10 is not a valid UM file. Error converting file to netcdf: dataout/o13nka.pdi5c10 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:23:17 (7121): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:01:58 (22864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:02:34 (26472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:55:30 (56217): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:27:51 (420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:50:24 (13161): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:50:26 (13161): No heartbeat from core client for 30 sec - exiting 01:50:27 (13161): No heartbeat from core client for 30 sec - exiting 01:50:28 (13161): No heartbeat from core client for 30 sec - exiting 01:50:29 (13161): No heartbeat from core client for 30 sec - exiting 01:50:30 (13161): No heartbeat from core client for 30 sec - exiting 01:50:31 (13161): No heartbeat from core client for 30 sec - exiting 01:50:32 (13161): No heartbeat from core client for 30 sec - exiting 01:50:34 (13161): No heartbeat from core client for 30 sec - exiting 05:12:46 (28083): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x2b34204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x2b34200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x5343a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x5343a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x7315004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x7315000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x7315004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x7315000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x7315004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x7315000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x7315004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x7315000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x7315004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x7315000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x5315604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32552,0xacf142c0) malloc: * error for object 0x5315600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 3 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Jun 2013 10:09:40	1110299	15822997	hadcm3n_o13n_1980_40_008387220_0	259,200	510,747	1.9705
17 Jun 2013 09:41:18	1110299	15822997	hadcm3n_o13n_1980_40_008387220_0	233,280	463,777	1.9881
16 Jun 2013 14:18:12	1110299	15822997	hadcm3n_o13n_1980_40_008387220_0	207,360	412,765	1.9906
15 Jun 2013 13:21:36	1110299	15822997	hadcm3n_o13n_1980_40_008387220_0	181,440	362,516	1.9980
14 Jun 2013 18:25:24	1110299	15822997	hadcm3n_o13n_1980_40_008387220_0	155,520	310,236	1.9948
13 Jun 2013 22:41:50	1110299	15822997	hadcm3n_o13n_1980_40_008387220_0	129,600	258,691	1.9961
12 Jun 2013 14:51:27	1110299	15822997	hadcm3n_o13n_1980_40_008387220_0	103,680	208,385	2.0099
11 Jun 2013 18:50:43	1110299	15822997	hadcm3n_o13n_1980_40_008387220_0	77,760	155,652	2.0017
11 Jun 2013 00:24:31	1110299	15822997	hadcm3n_o13n_1980_40_008387220_0	51,840	102,104	1.9696
08 Jun 2013 19:58:10	1110299	15822997	hadcm3n_o13n_1980_40_008387220_0	25,920	52,014	2.0067