Task 12988831

Name	hadcm3n_s2ur_1940_40_007299223_1
Workunit	7496647
Created	20 Jun 2011, 17:31:34 UTC
Sent	20 Jun 2011, 17:31:45 UTC
Report deadline	20 Sep 2011, 0:58:56 UTC
Received	4 Jul 2011, 22:24:30 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1135335
Run time	5 days 12 hours 54 min 6 sec
CPU time	4 days 23 hours 52 min 28 sec
Validate state	Invalid
Credit	2,799.36
Device peak FLOPS	2.85 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.59</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:04:59 (25448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:27:52 (7698): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:27:53 (7698): No heartbeat from core client for 30 sec - exiting 21:27:54 (7698): No heartbeat from core client for 30 sec - exiting 21:27:55 (7698): No heartbeat from core client for 30 sec - exiting 21:27:56 (7698): No heartbeat from core client for 30 sec - exiting 21:27:57 (7698): No heartbeat from core client for 30 sec - exiting 21:27:58 (7698): No heartbeat from core client for 30 sec - exiting 21:27:59 (7698): No heartbeat from core client for 30 sec - exiting 21:28:00 (7698): No heartbeat from core client for 30 sec - exiting 21:28:01 (7698): No heartbeat from core client for 30 sec - exiting 21:28:02 (7698): No heartbeat from core client for 30 sec - exiting 21:28:03 (7698): No heartbeat from core client for 30 sec - exiting 21:28:04 (7698): No heartbeat from core client for 30 sec - exiting 21:28:05 (7698): No heartbeat from core client for 30 sec - exiting 21:28:06 (7698): No heartbeat from core client for 30 sec - exiting 21:28:07 (7698): No heartbeat from core client for 30 sec - exiting 21:28:08 (7698): No heartbeat from core client for 30 sec - exiting 21:28:09 (7698): No heartbeat from core client for 30 sec - exiting 21:28:10 (7698): No heartbeat from core client for 30 sec - exiting 21:28:11 (7698): No heartbeat from core client for 30 sec - exiting 21:28:12 (7698): No heartbeat from core client for 30 sec - exiting 21:28:13 (7698): No heartbeat from core client for 30 sec - exiting 21:28:14 (7698): No heartbeat from core client for 30 sec - exiting 21:28:15 (7698): No heartbeat from core client for 30 sec - exiting 21:28:16 (7698): No heartbeat from core client for 30 sec - exiting 21:28:17 (7698): No heartbeat from core client for 30 sec - exiting 21:28:18 (7698): No heartbeat from core client for 30 sec - exiting 21:28:19 (7698): No heartbeat from core client for 30 sec - exiting 21:28:20 (7698): No heartbeat from core client for 30 sec - exiting 21:28:21 (7698): No heartbeat from core client for 30 sec - exiting 21:28:22 (7698): No heartbeat from core client for 30 sec - exiting 21:28:23 (7698): No heartbeat from core client for 30 sec - exiting 21:28:24 (7698): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 Jul 2011 20:27:05	1135335	12988831	hadcm3n_s2ur_1940_40_007299223_1	233,280	429,675	1.8419
03 Jul 2011 05:04:59	1135335	12988831	hadcm3n_s2ur_1940_40_007299223_1	207,360	380,903	1.8369
01 Jul 2011 13:00:37	1135335	12988831	hadcm3n_s2ur_1940_40_007299223_1	181,440	332,223	1.8310
30 Jun 2011 08:14:20	1135335	12988831	hadcm3n_s2ur_1940_40_007299223_1	155,520	283,960	1.8259
29 Jun 2011 11:04:51	1135335	12988831	hadcm3n_s2ur_1940_40_007299223_1	129,600	236,154	1.8222
27 Jun 2011 08:38:49	1135335	12988831	hadcm3n_s2ur_1940_40_007299223_1	103,680	190,203	1.8345
23 Jun 2011 08:32:03	1135335	12988831	hadcm3n_s2ur_1940_40_007299223_1	77,760	143,588	1.8466
22 Jun 2011 13:06:42	1135335	12988831	hadcm3n_s2ur_1940_40_007299223_1	51,840	95,498	1.8422
21 Jun 2011 14:58:48	1135335	12988831	hadcm3n_s2ur_1940_40_007299223_1	25,920	48,056	1.8540