Task 13365699

Name	hadcm3n_yad5_1940_40_007450041_0
Workunit	7647544
Created	10 Sep 2011, 3:43:39 UTC
Sent	13 Sep 2011, 23:02:20 UTC
Report deadline	14 Dec 2011, 6:29:31 UTC
Received	22 Sep 2011, 2:43:14 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1052933
Run time	8 days 1 hours 37 min 26 sec
CPU time	5 days 18 hours 46 min 35 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.30 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.10.21</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 23:50:00 (86817): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:55:23 (13823): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:57:45 (13854): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:02:08 (13872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:42:55 (13902): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:55:26 (16683): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 07:28:32 (17941): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:31:54 (19430): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:37:17 (19450): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:11:31 (19485): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:14:08 (27021): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:16:35 (27032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:19:01 (27049): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:30:32 (27062): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:33:14 (29602): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:35:36 (29613): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:38:02 (29630): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:28:57 (29641): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:31:28 (32425): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:33:54 (32438): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:36:20 (32453): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:38:43 (32466): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:55:42 (32481): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:13:04 (33833): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:10:38 (33951): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:10:39 (33951): No heartbeat from core client for 30 sec - exiting 13:13:03 (44878): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:15:25 (44889): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:17:46 (44904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:06:51 (44917): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:09:17 (45630): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:11:39 (45641): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:14:05 (45658): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:16:31 (45669): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:18:51 (45686): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:07:54 (45697): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:10:15 (47676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:10:16 (47676): No heartbeat from core client for 30 sec - exiting 17:12:37 (47689): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:14:59 (47704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:17:26 (47715): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:19:52 (47732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:13:55 (47743): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:16:22 (50159): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:36:54 (50172): No heartbeat from core client for 30 sec - exiting 21:36:55 (50172): No heartbeat from core client for 30 sec - exiting 21:36:56 (50172): No heartbeat from core client for 30 sec - exiting 21:36:57 (50172): No heartbeat from core client for 30 sec - exiting 21:36:58 (50172): No heartbeat from core client for 30 sec - exiting 21:36:59 (50172): No heartbeat from core client for 30 sec - exiting 21:37:00 (50172): No heartbeat from core client for 30 sec - exiting 21:37:01 (50172): No heartbeat from core client for 30 sec - exiting 21:37:02 (50172): No heartbeat from core client for 30 sec - exiting 21:37:03 (50172): No heartbeat from core client for 30 sec - exiting 21:37:04 (50172): No heartbeat from core client for 30 sec - exiting 21:37:05 (50172): No heartbeat from core client for 30 sec - exiting 21:37:06 (50172): No heartbeat from core client for 30 sec - exiting 21:37:07 (50172): No heartbeat from core client for 30 sec - exiting 21:37:08 (50172): No heartbeat from core client for 30 sec - exiting 21:37:09 (50172): No heartbeat from core client for 30 sec - exiting 21:37:10 (50172): No heartbeat from core client for 30 sec - exiting 21:37:11 (50172): No heartbeat from core client for 30 sec - exiting 21:37:12 (50172): No heartbeat from core client for 30 sec - exiting 21:37:13 (50172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yad5_1940_40_007450041/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yad5_1940_40_007450041/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yad5_1940_40_007450041/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yad5_1940_40_007450041/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yad5_1940_40_007450041/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yad5_1940_40_007450041/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yad5_1940_40_007450041/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yad5_1940_40_007450041/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yad5_1940_40_007450041/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yad5_1940_40_007450041/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yad5_1940_40_007450041/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_yad5_1940_40_007450041/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Sep 2011 02:43:47	1052933	13365699	hadcm3n_yad5_1940_40_007450041_0	259,200	499,649	1.9277
21 Sep 2011 07:25:34	1052933	13365699	hadcm3n_yad5_1940_40_007450041_0	233,280	449,703	1.9277
20 Sep 2011 12:00:20	1052933	13365699	hadcm3n_yad5_1940_40_007450041_0	207,360	400,083	1.9294
19 Sep 2011 15:12:37	1052933	13365699	hadcm3n_yad5_1940_40_007450041_0	181,440	349,972	1.9289
18 Sep 2011 17:49:40	1052933	13365699	hadcm3n_yad5_1940_40_007450041_0	155,520	299,364	1.9249
17 Sep 2011 20:57:25	1052933	13365699	hadcm3n_yad5_1940_40_007450041_0	129,600	248,939	1.9208
17 Sep 2011 00:26:03	1052933	13365699	hadcm3n_yad5_1940_40_007450041_0	103,680	198,712	1.9166
16 Sep 2011 03:48:48	1052933	13365699	hadcm3n_yad5_1940_40_007450041_0	77,760	148,584	1.9108
15 Sep 2011 07:18:02	1052933	13365699	hadcm3n_yad5_1940_40_007450041_0	51,840	98,268	1.8956
14 Sep 2011 14:52:40	1052933	13365699	hadcm3n_yad5_1940_40_007450041_0	25,920	48,928	1.8877