Task 15837384

Name	hadcm3n_3k7r_1940_40_008261002_4
Workunit	8416126
Created	10 Jun 2013, 9:04:17 UTC
Sent	10 Jun 2013, 9:33:42 UTC
Report deadline	9 Sep 2013, 17:00:53 UTC
Received	23 Jun 2013, 19:30:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1253077
Run time	10 days 5 hours 33 min 6 sec
CPU time	8 days 0 hours 16 min 54 sec
Validate state	Invalid
Credit	10,886.40
Device peak FLOPS	3.60 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... hadcm3n_6.07_i686-apple-darwin(24734,0xac1b5a28) malloc: * error for object 0x48d7e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24734,0xac1b5a28) malloc: * error for object 0x48d7e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24734,0xac1b5a28) malloc: * error for object 0x38dfe04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24734,0xac1b5a28) malloc: * error for object 0x38dfe00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24734,0xac1b5a28) malloc: * error for object 0x38dfe04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24734,0xac1b5a28) malloc: * error for object 0x38dfe00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24734,0xac1b5a28) malloc: * error for object 0x38dfe04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24734,0xac1b5a28) malloc: * error for object 0x38dfe00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24734,0xac1b5a28) malloc: * error for object 0x38dfe04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24734,0xac1b5a28) malloc: * error for object 0x38dfe00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24734,0xac1b5a28) malloc: * error for object 0x38dfe04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24734,0xac1b5a28) malloc: * error for object 0x38dfe00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid CPDN Monitor - Quit request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Library/Application Support/BOINC Data/slots/1/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Jun 2013 15:14:32	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	907,200	859,996	0.9480
23 Jun 2013 08:51:43	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	881,280	837,179	0.9500
23 Jun 2013 02:29:01	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	855,360	814,212	0.9519
22 Jun 2013 20:04:34	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	829,440	791,335	0.9541
22 Jun 2013 13:48:28	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	803,520	768,560	0.9565
22 Jun 2013 07:41:02	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	777,600	746,979	0.9606
22 Jun 2013 01:41:28	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	751,680	725,581	0.9653
21 Jun 2013 19:43:38	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	725,760	704,231	0.9703
21 Jun 2013 10:09:01	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	699,840	682,851	0.9757
21 Jun 2013 04:11:20	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	673,920	661,714	0.9819
20 Jun 2013 22:37:38	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	648,000	640,072	0.9878
20 Jun 2013 09:13:09	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	622,080	614,254	0.9874
20 Jun 2013 02:15:06	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	596,160	589,612	0.9890
19 Jun 2013 12:07:57	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	570,240	564,411	0.9898
19 Jun 2013 04:39:34	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	544,320	539,497	0.9911
18 Jun 2013 21:45:36	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	518,400	514,593	0.9927
18 Jun 2013 07:23:07	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	492,480	488,950	0.9928
18 Jun 2013 00:24:36	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	466,560	463,880	0.9943
17 Jun 2013 17:25:54	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	440,640	438,998	0.9963
17 Jun 2013 06:50:09	1253077	15837384	hadcm3n_3k7r_1940_40_008261002_4	414,720	413,658	0.9974