Task 12825676

Name	hadcm3n_p430_1900_40_007222876_0
Workunit	7421116
Created	26 Apr 2011, 15:27:28 UTC
Sent	30 Apr 2011, 1:26:26 UTC
Report deadline	30 Jul 2011, 8:53:37 UTC
Received	11 Jun 2011, 13:44:03 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1091743
Run time	6 days 19 hours 11 min 41 sec
CPU time	4 days 22 hours 56 min 16 sec
Validate state	Invalid
Credit	2,799.36
Device peak FLOPS	2.93 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.6.38</core_client_version> <![CDATA[ <message> El dispositivo no reconoce el comando. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3272, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2624, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... C12:57:23 (2684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:57:25 (2684): No heartbeat from core client for 30 sec - exiting 12:57:26 (2684): No heartbeat from core client for 30 sec - exiting 12:57:27 (2684): No heartbeat from core client for 30 sec - exiting 12:57:28 (2684): No heartbeat from core client for 30 sec - exiting 12:57:29 (2684): No heartbeat from core client for 30 sec - exiting 12:57:30 (2684): No heartbeat from core client for 30 sec - exiting 12:57:31 (2684): No heartbeat from core client for 30 sec - exiting 12:57:32 (2684): No heartbeat from core client for 30 sec - exiting 12:57:33 (2684): No heartbeat from core client for 30 sec - exiting 12:57:34 (2684): No heartbeat from core client for 30 sec - exiting BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 13:36:19 (4056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2548, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3876, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2624, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2624, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=296, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4056, iMonCtr=1 Model crash detected, will try to restart... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1972, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3832, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4000, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2396, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3416, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2204, iMonCtr=1 Model crash deController::BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 12:51:08 (2444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2884, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CCPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2416, iMonCtr=1 Model crash detected, will try to restart... 16:00:49 (2372): No heartbeat from core client for 30 sec - exiting 16:00:50 (2372): No heartbeat from core client for 30 sec - exiting 16:00:51 (2372): No heartbeat from core client for 30 sec - exiting 16:00:52 (2372): No heartbeat from core client for 30 sec - exiting 16:00:53 (2372): No heartbeat from core client for 30 sec - exiting 16:00:54 (2372): No heartbeat from core client for 30 sec - exiting 16:00:55 (2372): No heartbeat from core client for 30 sec - exiting 16:00:57 (2372): No heartbeat from core client for 30 sec - exiting 16:00:58 (2372): No heartbeat from core client for 30 sec - exiting 16:00:59 (2372): No heartbeat from core client for 30 sec - exiting 16:01:00 (2372): No heartbeat from core client for 30 sec - exiting 16:01:01 (2372): No heartbeat from core client for 30 sec - exiting 16:01:02 (2372): No heartbeat from core client for 30 sec - exiting 16:01:03 (2372): No heartbeat from core client for 30 sec - exiting 16:01:04 (2372): No heartbeat from core client for 30 sec - exiting 16:01:05 (2372): No heartbeat from core client for 30 sec - exiting 16:01:06 (2372): No heartbeat from core client for 30 sec - exiting 16:01:07 (2372): No heartbeat from core client for 30 sec - exiting 16:01:08 (2372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3064, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2980, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3324, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3040, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2244, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3024, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2636, iMonCtr=1 Model crash detected, will try to restart... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3852, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3852, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2796, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3004, iMonCtr=1 Model crash detected, will try to restart... 20:30:51 (2332): No heartbeat from core client for 30 sec - exiting 20:30:52 (2332): No heartbeat from core client for 30 sec - exiting 20:30:53 (2332): No heartbeat from core client for 30 sec - exiting 20:30:54 (2332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1564, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2972, iMonCtr=1 Model crash detected, will try to restart... 13:54:58 (2940): No heartbeat from core client for 30 sec - exiting 13:54:59 (2940): No heartbeat from core client for 30 sec - exiting 13:55:01 (2940): No heartbeat from core client for 30 sec - exiting 13:55:02 (2940): No heartbeat from core client for 30 sec - exiting 13:55:03 (2940): No heartbeat from core client for 30 sec - exiting 13:55:04 (2940): No heartbeat from core client for 30 sec - exiting 13:55:05 (2940): No heartbeat from core client for 30 sec - exiting 13:55:06 (2940): No heartbeat from core client for 30 sec - exiting 13:55:07 (2940): No heartbeat from core client for 30 sec - exiting 13:55:08 (2940): No heartbeat from core client for 30 sec - exiting 13:55:09 (2940): No heartbeat from core client for 30 sec - exiting 13:55:10 (2940): No heartbeat from core client for 30 sec - exiting 13:55:11 (2940): No heartbeat from core client for 30 sec - exiting 13:55:12 (2940): No heartbeat from core client for 30 sec - exiting 13:55:14 (2940): No heartbeat from core client for 30 sec - exiting 13:55:15 (2940): No heartbeat from core client for 30 sec - exiting 13:55:16 (2940): No heartbeat from core client for 30 sec - exiting 13:55:17 (2940): No heartbeat from core client for 30 sec - exiting 13:55:18 (2940): No heartbeat from core client for 30 sec - exiting 13:55:19 (2940): No heartbeat from core client for 30 sec - exiting 13:55:20 (2940): No heartbeat from core client for 30 sec - exiting 13:55:21 (2940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA 18:56:30 (2280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Jun 2011 20:49:43	1091743	12825676	hadcm3n_p430_1900_40_007222876_0	233,280	388,946	1.6673
04 Jun 2011 23:41:53	1091743	12825676	hadcm3n_p430_1900_40_007222876_0	207,360	352,488	1.6999
28 May 2011 19:58:07	1091743	12825676	hadcm3n_p430_1900_40_007222876_0	181,440	305,406	1.6832
25 May 2011 02:59:56	1091743	12825676	hadcm3n_p430_1900_40_007222876_0	155,520	259,481	1.6685
20 May 2011 20:00:35	1091743	12825676	hadcm3n_p430_1900_40_007222876_0	129,600	212,064	1.6363
19 May 2011 17:38:15	1091743	12825676	hadcm3n_p430_1900_40_007222876_0	103,680	187,509	1.8085
15 May 2011 22:31:54	1091743	12825676	hadcm3n_p430_1900_40_007222876_0	77,760	141,489	1.8196
11 May 2011 02:15:49	1091743	12825676	hadcm3n_p430_1900_40_007222876_0	51,840	94,500	1.8229
05 May 2011 18:31:59	1091743	12825676	hadcm3n_p430_1900_40_007222876_0	25,920	46,036	1.7761