Task 14282936

Name	hadcm3n_u4rl_1980_40_007832126_1
Workunit	7987238
Created	18 Mar 2012, 14:49:52 UTC
Sent	18 Mar 2012, 14:51:38 UTC
Report deadline	17 Jun 2012, 22:18:49 UTC
Received	5 Apr 2012, 16:03:52 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1187901
Run time	11 days 5 hours 58 min 33 sec
CPU time	10 days 10 hours 10 min 50 sec
Validate state	Invalid
Credit	4,354.56
Device peak FLOPS	2.36 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1867, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_u4rl_1980_40_007832126/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08455448 Unknown Unknown Unknown hadcm3n_um_6.07_i 0829D681 Unknown Unknown Unknown hadcm3n_um_6.07_i 08340D81 Unknown Unknown Unknown hadcm3n_um_6.07_i 08194301 Unknown Unknown Unknown hadcm3n_um_6.07_i 08391957 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F8B7 Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 F75760F3 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1867, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_u4rl_1980_40_007832126/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08451069 Unknown Unknown Unknown hadcm3n_um_6.07_i 0844E937 Unknown Unknown Unknown hadcm3n_um_6.07_i 0836D10D Unknown Unknown Unknown hadcm3n_um_6.07_i 082EB086 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F66D Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 F75C00F3 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1867, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_u4rl_1980_40_007832126/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08451069 Unknown Unknown Unknown hadcm3n_um_6.07_i 0844E937 Unknown Unknown Unknown hadcm3n_um_6.07_i 0836D10D Unknown Unknown Unknown hadcm3n_um_6.07_i 082EB086 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F66D Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 F75D70F3 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1867, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Apr 2012 11:39:16	1187901	14282936	hadcm3n_u4rl_1980_40_007832126_1	362,880	954,323	2.6299
04 Apr 2012 16:00:01	1187901	14282936	hadcm3n_u4rl_1980_40_007832126_1	336,960	885,826	2.6289
03 Apr 2012 20:27:54	1187901	14282936	hadcm3n_u4rl_1980_40_007832126_1	311,040	817,047	2.6268
03 Apr 2012 00:35:41	1187901	14282936	hadcm3n_u4rl_1980_40_007832126_1	285,120	747,225	2.6207
02 Apr 2012 04:49:26	1187901	14282936	hadcm3n_u4rl_1980_40_007832126_1	259,200	677,453	2.6136
31 Mar 2012 23:05:07	1187901	14282936	hadcm3n_u4rl_1980_40_007832126_1	233,280	608,179	2.6071
31 Mar 2012 03:39:38	1187901	14282936	hadcm3n_u4rl_1980_40_007832126_1	207,360	540,003	2.6042
30 Mar 2012 08:15:49	1187901	14282936	hadcm3n_u4rl_1980_40_007832126_1	181,440	471,690	2.5997
29 Mar 2012 13:06:26	1187901	14282936	hadcm3n_u4rl_1980_40_007832126_1	155,520	403,121	2.5921
28 Mar 2012 18:09:26	1187901	14282936	hadcm3n_u4rl_1980_40_007832126_1	129,600	335,140	2.5860
27 Mar 2012 23:47:12	1187901	14282936	hadcm3n_u4rl_1980_40_007832126_1	103,680	267,029	2.5755
26 Mar 2012 09:06:37	1187901	14282936	hadcm3n_u4rl_1980_40_007832126_1	77,760	199,678	2.5679
24 Mar 2012 08:15:42	1187901	14282936	hadcm3n_u4rl_1980_40_007832126_1	51,840	132,855	2.5628
21 Mar 2012 21:20:52	1187901	14282936	hadcm3n_u4rl_1980_40_007832126_1	25,920	66,372	2.5606