Task 12811059

Name	hadcm3n_o1gu_1900_40_007197233_2
Workunit	7395513
Created	24 Apr 2011, 7:37:25 UTC
Sent	24 Apr 2011, 7:38:22 UTC
Report deadline	24 Jul 2011, 15:05:33 UTC
Received	19 Jun 2011, 10:28:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1029041
Run time	11 days 8 hours 27 min 51 sec
CPU time	8 days 18 hours 2 min 15 sec
Validate state	Invalid
Credit	2,799.36
Device peak FLOPS	1.37 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <message> El dispositivo no reconoce el comando. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 00:26:58 (4028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:26:27 (5216): No heartbeat from core client for 30 sec - exiting 22:26:29 (5216): No heartbeat from core client for 30 sec - exiting 22:26:30 (5216): No heartbeat from core client for 30 sec - exiting 22:26:31 (5216): No heartbeat from core client for 30 sec - exiting 22:26:32 (5216): No heartbeat from core client for 30 sec - exiting 22:26:33 (5216): No heartbeat from core client for 30 sec - exiting 22:26:34 (5216): No heartbeat from core client for 30 sec - exiting 22:26:35 (5216): No heartbeat from core client for 30 sec - exiting 22:26:36 (5216): No heartbeat from core client for 30 sec - exiting 22:26:37 (5216): No heartbeat from core client for 30 sec - exiting 22:26:38 (5216): No heartbeat from core client for 30 sec - exiting 22:26:39 (5216): No heartbeat from core client for 30 sec - exiting 22:26:40 (5216): No heartbeat from core client for 30 sec - exiting 22:26:41 (5216): No heartbeat from core client for 30 sec - exiting 22:26:42 (5216): No heartbeat from core client for 30 sec - exiting 22:26:43 (5216): No heartbeat from core client for 30 sec - exiting 22:26:44 (5216): No heartbeat from core client for 30 sec - exiting 22:26:45 (5216): No heartbeat from core client for 30 sec - exiting 22:26:46 (5216): No heartbeat from core client for 30 sec - exiting 22:26:47 (5216): No heartbeat from core client for 30 sec - exiting 22:26:48 (5216): No heartbeat from core client for 30 sec - exiting 22:26:49 (5216): No heartbeat from core client for 30 sec - exiting 22:26:50 (5216): No heartbeat from core client for 30 sec - exiting 22:26:51 (5216): No heartbeat from core client for 30 sec - exiting 22:26:52 (5216): No heartbeat from core client for 30 sec - exiting 22:26:53 (5216): No heartbeat from core client for 30 sec - exiting 22:26:54 (5216): No heartbeat from core client for 30 sec - exiting 22:26:55 (5216): No heartbeat from core client for 30 sec - exiting 22:26:56 (5216): No heartbeat from core client for 30 sec - exiting 22:26:57 (5216): No heartbeat from core client for 30 sec - exiting 22:26:58 (5216): No heartbeat from core client for 30 sec - exiting 22:26:59 (5216): No heartbeat from core client for 30 sec - exiting 22:27:00 (5216): No heartbeat from core client for 30 sec - exiting 22:27:01 (5216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:35:06 (2992): No heartbeat from core client for 30 sec - exiting 19:35:07 (2992): No heartbeat from core client for 30 sec - exiting 19:35:08 (2992): No heartbeat from core client for 30 sec - exiting 19:35:09 (2992): No heartbeat from core client for 30 sec - exiting 19:35:10 (2992): No heartbeat from core client for 30 sec - exiting 19:35:11 (2992): No heartbeat from core client for 30 sec - exiting 19:35:12 (2992): No heartbeat from core client for 30 sec - exiting 19:35:13 (2992): No heartbeat from core client for 30 sec - exiting 19:35:14 (2992): No heartbeat from core client for 30 sec - exiting 19:35:15 (2992): No heartbeat from core client for 30 sec - exiting 19:35:16 (2992): No heartbeat from core client for 30 sec - exiting 19:35:17 (2992): No heartbeat from core client for 30 sec - exiting 19:35:18 (2992): No heartbeat from core client for 30 sec - exiting 19:35:19 (2992): No heartbeat from core client for 30 sec - exiting 19:35:20 (2992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:35:24 (5748): Can't acquire lockfile (32) - waiting 35s Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:57:03 (3460): No heartbeat from core client for 30 sec - exiting 21:57:04 (3460): No heartbeat from core client for 30 sec - exiting 21:57:05 (3460): No heartbeat from core client for 30 sec - exiting 21:57:06 (3460): No heartbeat from core client for 30 sec - exiting 21:57:07 (3460): No heartbeat from core client for 30 sec - exiting 21:57:08 (3460): No heartbeat from core client for 30 sec - exiting 21:57:09 (3460): No heartbeat from core client for 30 sec - exiting 21:57:10 (3460): No heartbeat from core client for 30 sec - exiting 21:57:11 (3460): No heartbeat from core client for 30 sec - exiting 21:57:12 (3460): No heartbeat from core client for 30 sec - exiting 21:57:13 (3460): No heartbeat from core client for 30 sec - exiting 21:57:14 (3460): No heartbeat from core client for 30 sec - exiting 21:57:15 (3460): No heartbeat from core client for 30 sec - exiting 21:57:16 (3460): No heartbeat from core client for 30 sec - exiting 21:57:17 (3460): No heartbeat from core client for 30 sec - exiting 21:57:18 (3460): No heartbeat from core client for 30 sec - exiting 21:57:20 (3460): No heartbeat from core client for 30 sec - exiting 21:57:21 (3460): No heartbeat from core client for 30 sec - exiting 21:57:22 (3460): No heartbeat from core client for 30 sec - exiting 21:57:23 (3460): No heartbeat from core client for 30 sec - exiting 21:57:24 (3460): No heartbeat from core client for 30 sec - exiting 21:57:25 (3460): No heartbeat from core client for 30 sec - exiting 21:57:26 (3460): No heartbeat from core client for 30 sec - exiting 21:57:27 (3460): No heartbeat from core client for 30 sec - exiting 21:57:28 (3460): No heartbeat from core client for 30 sec - exiting 21:57:29 (3460): No heartbeat from core client for 30 sec - exiting 21:57:30 (3460): No heartbeat from core client for 30 sec - exiting 21:57:31 (3460): No heartbeat from core client for 30 sec - exiting 21:57:32 (3460): No heartbeat from core client for 30 sec - exiting 21:57:33 (3460): No heartbeat from core client for 30 sec - exiting 21:57:34 (3460): No heartbeat from core client for 30 sec - exiting 21:57:35 (3460): No heartbeat from core client for 30 sec - exiting 21:57:36 (3460): No heartbeat from core client for 30 sec - exiting 21:57:37 (3460): No heartbeat from core client for 30 sec - exiting 21:57:38 (3460): No heartbeat from core client for 30 sec - exiting 21:57:39 (3460): No heartbeat from core client for 30 sec - exiting 21:57:40 (3460): No heartbeat from core client for 30 sec - exiting 21:57:41 (3460): No heartbeat from core client for 30 sec - exiting 21:57:42 (3460): No heartbeat from core client for 30 sec - exiting 21:57:43 (3460): No heartbeat from core client for 30 sec - exiting 21:57:44 (3460): No heartbeat from core client for 30 sec - exiting 21:57:45 (3460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 forrtl: Espacio en disco insuficiente. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4412, iMonCtr=1 Model crash detected, will try to restart... forrtl: Espacio en disco insuficiente. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4412, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish 21:47:53 (5872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2144, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2144, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2144, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2144, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2144, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2144, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish 21:39:54 (2596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:39:55 (2596): No heartbeat from core client for 30 sec - exiting 09:36:20 (5040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 May 2011 17:14:09	1029041	12811059	hadcm3n_o1gu_1900_40_007197233_2	233,280	666,113	2.8554
09 May 2011 07:20:10	1029041	12811059	hadcm3n_o1gu_1900_40_007197233_2	207,360	592,901	2.8593
08 May 2011 09:44:41	1029041	12811059	hadcm3n_o1gu_1900_40_007197233_2	181,440	521,179	2.8725
01 May 2011 15:45:09	1029041	12811059	hadcm3n_o1gu_1900_40_007197233_2	155,520	448,689	2.8851
30 Apr 2011 16:04:26	1029041	12811059	hadcm3n_o1gu_1900_40_007197233_2	129,600	372,887	2.8772
29 Apr 2011 16:13:25	1029041	12811059	hadcm3n_o1gu_1900_40_007197233_2	103,680	296,979	2.8644
28 Apr 2011 02:13:30	1029041	12811059	hadcm3n_o1gu_1900_40_007197233_2	77,760	221,692	2.8510
27 Apr 2011 03:24:40	1029041	12811059	hadcm3n_o1gu_1900_40_007197233_2	51,840	150,322	2.8997
25 Apr 2011 08:34:52	1029041	12811059	hadcm3n_o1gu_1900_40_007197233_2	25,920	74,461	2.8727