Task 17953744

Name	hadam3pm2_pecv_1991_10_009528904_0
Workunit	9610647
Created	13 Feb 2015, 16:47:33 UTC
Sent	10 Mar 2015, 6:18:27 UTC
Report deadline	20 Feb 2016, 11:38:27 UTC
Received	28 Mar 2015, 12:53:13 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1350380
Run time	6 days 17 hours 30 min
CPU time	6 days 10 hours 36 min 9 sec
Validate state	Invalid
Credit	7,153.72
Device peak FLOPS	1.75 GFLOPS
Application version	UK Met Office HadAM3P (global only) with MOSES II landsurface scheme v7.03 i686-pc-linux-gnu
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... oa.pc\|1dec.nc oa.pc\|2jan.nc oa.pc\|2feb.nc oa.pc\|2mar.nc oa.pc\|2apr.nc oa.pc\|2may.nc oa.pc\|2jun.nc oa.pc\|2jul.nc oa.pc\|2aug.nc oa.pc\|2sep.nc oa.pc\|2oct.nc oa.pc\|2nov.nc oa.pe\|1dec.nc oa.pe\|2jan.nc oa.pe\|2feb.nc oa.pe\|2mar.nc oa.pe\|2apr.nc oa.pe\|2may.nc oa.pe\|2jun.nc oa.pe\|2jul.nc oa.pe\|2aug.nc oa.pe\|2sep.nc oa.pe\|2oct.nc oa.pe\|2nov.nc Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 06:43:09 (2628): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 22:07:22 (1405): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... oa.pc\|4dec.nc oa.pc\|5jan.nc oa.pc\|5feb.nc oa.pc\|5mar.nc oa.pc\|5apr.nc oa.pc\|5may.nc oa.pc\|5jun.nc oa.pc\|5jul.nc oa.pc\|5aug.nc oa.pc\|5sep.nc oa.pc\|5oct.nc oa.pc\|5nov.nc oa.pe\|4dec.nc oa.pe\|5jan.nc oa.pe\|5feb.nc oa.pe\|5mar.nc oa.pe\|5apr.nc oa.pe\|5may.nc oa.pe\|5jun.nc oa.pe\|5jul.nc oa.pe\|5aug.nc oa.pe\|5sep.nc oa.pe\|5oct.nc oa.pe\|5nov.nc Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 10:41:38 (3546): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... oa.pc\|6dec.nc oa.pc\|7jan.nc oa.pc\|7feb.nc oa.pc\|7mar.nc oa.pc\|7apr.nc oa.pc\|7may.nc oa.pc\|7jun.nc oa.pc\|7jul.nc oa.pc\|7aug.nc oa.pc\|7sep.nc oa.pc\|7oct.nc oa.pc\|7nov.nc oa.pe\|6dec.nc oa.pe\|7jan.nc oa.pe\|7feb.nc oa.pe\|7mar.nc oa.pe\|7apr.nc oa.pe\|7may.nc oa.pe\|7jun.nc oa.pe\|7jul.nc oa.pe\|7aug.nc oa.pe\|7sep.nc oa.pe\|7oct.nc oa.pe\|7nov.nc Signal 15 received, exiting... 18:32:25 (1483): called boinc_finish BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: go forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc/projects/climateprediction.net/hadam3pm2_pecv_1991_10_009528904/dataout/stdout_um.txt Image PC Routine Line Source hadam3pm2_um_7.03 0845AB9D Unknown Unknown Unknown hadam3pm2_um_7.03 08459995 Unknown Unknown Unknown hadam3pm2_um_7.03 0842865F Unknown Unknown Unknown hadam3pm2_um_7.03 083E7E1D Unknown Unknown Unknown hadam3pm2_um_7.03 083E7767 Unknown Unknown Unknown hadam3pm2_um_7.03 0841A2F9 Unknown Unknown Unknown hadam3pm2_um_7.03 08417BC7 Unknown Unknown Unknown hadam3pm2_um_7.03 08346323 Unknown Unknown Unknown hadam3pm2_um_7.03 082D4171 Unknown Unknown Unknown hadam3pm2_um_7.03 0832046D Unknown Unknown Unknown hadam3pm2_um_7.03 0834C82D Unknown Unknown Unknown libc.so.6 F0D27E7E Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1341, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc/projects/climateprediction.net/hadam3pm2_pecv_1991_10_009528904/dataout/stdout_um.txt Image PC Routine Line Source hadam3pm2_um_7.03 0845AB9D Unknown Unknown Unknown hadam3pm2_um_7.03 08459995 Unknown Unknown Unknown hadam3pm2_um_7.03 0842865F Unknown Unknown Unknown hadam3pm2_um_7.03 083E7E1D Unknown Unknown Unknown hadam3pm2_um_7.03 083E7767 Unknown Unknown Unknown hadam3pm2_um_7.03 0841A2F9 Unknown Unknown Unknown hadam3pm2_um_7.03 08417BC7 Unknown Unknown Unknown hadam3pm2_um_7.03 08346323 Unknown Unknown Unknown hadam3pm2_um_7.03 082D4171 Unknown Unknown Unknown hadam3pm2_um_7.03 0832046D Unknown Unknown Unknown hadam3pm2_um_7.03 0834C82D Unknown Unknown Unknown libc.so.6 F0D27E7E Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1341, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc/projects/climateprediction.net/hadam3pm2_pecv_1991_10_009528904/dataout/stdout_um.txt Image PC Routine Line Source hadam3pm2_um_7.03 0845AB9D Unknown Unknown Unknown hadam3pm2_um_7.03 08459995 Unknown Unknown Unknown hadam3pm2_um_7.03 0842865F Unknown Unknown Unknown hadam3pm2_um_7.03 083E7E1D Unknown Unknown Unknown hadam3pm2_um_7.03 083E7767 Unknown Unknown Unknown hadam3pm2_um_7.03 0841A2F9 Unknown Unknown Unknown hadam3pm2_um_7.03 08417BC7 Unknown Unknown Unknown hadam3pm2_um_7.03 08346323 Unknown Unknown Unknown hadam3pm2_um_7.03 082D4171 Unknown Unknown Unknown hadam3pm2_um_7.03 0832046D Unknown Unknown Unknown hadam3pm2_um_7.03 0834C82D Unknown Unknown Unknown libc.so.6 F0D1FE7E Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, s13:46:16 (1341): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Mar 2015 09:39:50	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	247,748	576,762	2.3280
28 Mar 2015 08:34:25	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	244,868	570,241	2.3288
27 Mar 2015 15:49:16	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	236,228	550,570	2.3307
27 Mar 2015 14:23:21	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	233,348	544,006	2.3313
26 Mar 2015 20:56:09	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	230,468	537,433	2.3319
26 Mar 2015 18:50:22	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	227,588	530,801	2.3323
26 Mar 2015 17:49:48	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	224,708	524,330	2.3334
26 Mar 2015 16:48:54	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	221,828	517,843	2.3344
26 Mar 2015 16:48:54	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	218,948	511,436	2.3359
26 Mar 2015 09:32:56	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	216,068	504,794	2.3363
26 Mar 2015 09:32:29	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	213,188	498,332	2.3375
26 Mar 2015 09:31:55	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	210,308	491,767	2.3383
26 Mar 2015 09:30:59	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	204,548	478,523	2.3394
26 Mar 2015 09:30:22	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	201,668	471,777	2.3394
26 Mar 2015 09:27:17	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	198,788	465,012	2.3392
26 Mar 2015 09:26:19	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	195,908	458,349	2.3396
26 Mar 2015 09:25:46	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	193,028	451,711	2.3401
26 Mar 2015 09:25:21	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	190,148	445,079	2.3407
26 Mar 2015 09:25:21	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	187,268	438,436	2.3412
26 Mar 2015 09:25:21	1350380	17953744	hadam3pm2_pecv_1991_10_009528904_0	184,388	431,860	2.3421