Task 15893470

Name	hadcm3n_3fy8_1980_40_008324906_3
Workunit	8476041
Created	18 Jul 2013, 10:52:11 UTC
Sent	18 Jul 2013, 15:05:24 UTC
Report deadline	17 Oct 2013, 22:32:35 UTC
Received	23 Jul 2013, 14:50:58 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1171093
Run time	2 days 16 hours 3 min 13 sec
CPU time	2 days 9 hours 16 min 56 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.79 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 1 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(65800,0xace8b2c0) malloc: * error for object 0xa90e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65800,0xace8b2c0) malloc: * error for object 0xa90e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65800,0xace8b2c0) malloc: * error for object 0xa90e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65800,0xace8b2c0) malloc: * error for object 0xa90e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65800,0xace8b2c0) malloc: * error for object 0xa90e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(65922,0xace8b2c0) malloc: * error for object 0x1332604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65922,0xace8b2c0) malloc: * error for object 0x1332600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65922,0xace8b2c0) malloc: * error for object 0x1332604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65922,0xace8b2c0) malloc: * error for object 0x1332600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65922,0xace8b2c0) malloc: * error for object 0x1332604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65922,0xace8b2c0) malloc: * error for object 0x1332600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65922,0xace8b2c0) malloc: * error for object 0x1332604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65922,0xace8b2c0) malloc: * error for object 0x1332600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65922,0xace8b2c0) malloc: * error for object 0x1332604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65922,0xace8b2c0) malloc: * error for object 0x1332600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65922,0xace8b2c0) malloc: * error for object 0x1332604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(65922,0xace8b2c0) malloc: * error for object 0x1332600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_3fy8_1980_40_008324906/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_3fy8_1980_40_008324906/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_3fy8_1980_40_008324906/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_3fy8_1980_40_008324906/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_3fy8_1980_40_008324906/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_3fy8_1980_40_008324906/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_3fy8_1980_40_008324906/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_3fy8_1980_40_008324906/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_3fy8_1980_40_008324906/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_3fy8_1980_40_008324906/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_3fy8_1980_40_008324906/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_3fy8_1980_40_008324906/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Jul 2013 22:11:44	1171093	15893470	hadcm3n_3fy8_1980_40_008324906_3	259,200	206,236	0.7957
23 Jul 2013 22:00:39	1171093	15893470	hadcm3n_3fy8_1980_40_008324906_3	233,280	185,989	0.7973
23 Jul 2013 21:54:56	1171093	15893470	hadcm3n_3fy8_1980_40_008324906_3	207,360	165,936	0.8002
23 Jul 2013 21:47:48	1171093	15893470	hadcm3n_3fy8_1980_40_008324906_3	181,440	145,871	0.8040
23 Jul 2013 21:40:29	1171093	15893470	hadcm3n_3fy8_1980_40_008324906_3	155,520	125,867	0.8093
23 Jul 2013 20:13:57	1171093	15893470	hadcm3n_3fy8_1980_40_008324906_3	129,600	105,565	0.8145
23 Jul 2013 20:02:52	1171093	15893470	hadcm3n_3fy8_1980_40_008324906_3	103,680	84,353	0.8136
23 Jul 2013 19:49:54	1171093	15893470	hadcm3n_3fy8_1980_40_008324906_3	77,760	63,130	0.8119
23 Jul 2013 19:29:40	1171093	15893470	hadcm3n_3fy8_1980_40_008324906_3	51,840	42,385	0.8176
23 Jul 2013 19:03:48	1171093	15893470	hadcm3n_3fy8_1980_40_008324906_3	25,920	21,190	0.8175