Task 17540550

Name	hadam3pm2_i0ap_1959_10_009271523_0
Workunit	9364439
Created	3 Dec 2014, 14:28:24 UTC
Sent	16 Dec 2014, 9:53:56 UTC
Report deadline	28 Nov 2015, 15:13:56 UTC
Received	20 Dec 2014, 17:20:18 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1345391
Run time	3 days 3 hours 32 min 33 sec
CPU time	1 days 12 hours 29 min 41 sec
Validate state	Invalid
Credit	2,496.76
Device peak FLOPS	3.43 GFLOPS
Application version	UK Met Office HadAM3P (global only) with MOSES II landsurface scheme v7.03 i686-pc-linux-gnu
Stderr	<core_client_version>7.5.0</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... oa.pc\|9dec.nc oa.pc\|0jan.nc oa.pc\|0feb.nc oa.pc\|0mar.nc oa.pc\|0apr.nc oa.pc\|0may.nc oa.pc\|0jun.nc oa.pc\|0jul.nc oa.pc\|0aug.nc oa.pc\|0sep.nc oa.pc\|0oct.nc oa.pc\|0nov.nc oa.pe\|9dec.nc oa.pe\|0jan.nc oa.pe\|0feb.nc oa.pe\|0mar.nc oa.pe\|0apr.nc oa.pe\|0may.nc oa.pe\|0jun.nc oa.pe\|0jul.nc oa.pe\|0aug.nc oa.pe\|0sep.nc oa.pe\|0oct.nc oa.pe\|0nov.nc CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: NoSuspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc/projects/climateprediction.net/hadam3pm2_i0ap_1959_10_009271523/dataout/stdout_um.txt Image PC Routine Line Source hadam3pm2_um_7.03 0845AB9D Unknown Unknown Unknown hadam3pm2_um_7.03 08459995 Unknown Unknown Unknown hadam3pm2_um_7.03 0842865F Unknown Unknown Unknown hadam3pm2_um_7.03 083E7E1D Unknown Unknown Unknown hadam3pm2_um_7.03 083E7767 Unknown Unknown Unknown hadam3pm2_um_7.03 0841A2F9 Unknown Unknown Unknown hadam3pm2_um_7.03 08417BC7 Unknown Unknown Unknown hadam3pm2_um_7.03 08346323 Unknown Unknown Unknown hadam3pm2_um_7.03 082D4171 Unknown Unknown Unknown hadam3pm2_um_7.03 0832046D Unknown Unknown Unknown hadam3pm2_um_7.03 0834C82D Unknown Unknown Unknown libc.so.6 F0DEDC25 Unknown Unknown Unknown hadam3pm2_um_7.03 0804C821 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=17872, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc/projects/climateprediction.net/hadam3pm2_i0ap_1959_10_009271523/dataout/stdout_um.txt Image PC Routine Line Source hadam3pm2_um_7.03 0845AB9D Unknown Unknown Unknown hadam3pm2_um_7.03 08459995 Unknown Unknown Unknown hadam3pm2_um_7.03 084217:20:08 (17872): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Dec 2014 16:09:44	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	86,468	131,397	1.5196
20 Dec 2014 12:14:07	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	83,588	247,003	2.9550
20 Dec 2014 09:53:46	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	80,708	238,818	2.9590
20 Dec 2014 07:58:27	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	77,828	230,161	2.9573
20 Dec 2014 05:02:55	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	74,948	221,455	2.9548
20 Dec 2014 02:57:09	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	72,068	212,648	2.9507
19 Dec 2014 01:47:45	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	43,268	126,991	2.9350
18 Dec 2014 23:22:06	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	40,388	118,427	2.9322
18 Dec 2014 21:36:49	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	37,508	109,620	2.9226
18 Dec 2014 15:55:54	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	31,748	92,040	2.8991
18 Dec 2014 13:20:30	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	28,868	83,169	2.8810
18 Dec 2014 10:50:10	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	25,988	74,308	2.8593
18 Dec 2014 08:24:47	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	23,108	65,569	2.8375
18 Dec 2014 05:54:22	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	20,228	56,861	2.8110
18 Dec 2014 03:28:33	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	17,348	48,245	2.7810
18 Dec 2014 01:04:46	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	14,468	39,628	2.7390
17 Dec 2014 22:34:25	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	11,588	30,686	2.6481
17 Dec 2014 20:34:03	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	8,708	22,114	2.5395
17 Dec 2014 18:33:24	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	5,828	13,572	2.3288
17 Dec 2014 15:32:52	1345391	17540550	hadam3pm2_i0ap_1959_10_009271523_0	2,948	4,924	1.6703