Task 15554419

Name	hadcm3n_t31r_1940_40_008281426_2
Workunit	8432561
Created	22 Jan 2013, 4:22:26 UTC
Sent	22 Jan 2013, 4:22:29 UTC
Report deadline	23 Apr 2013, 11:49:40 UTC
Received	9 Mar 2013, 16:05:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	985115
Run time	20 days 19 hours 5 min 9 sec
CPU time	19 days 19 hours 48 min 38 sec
Validate state	Invalid
Credit	10,575.36
Device peak FLOPS	1.61 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> El dispositivo no reconoce el comando. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5100, iMonCtr=1 Model crash detected, will try to restart... 15:39:34 (4732): No heartbeat from core client for 30 sec - exiting 15:39:36 (4732): No heartbeat from core client for 30 sec - exiting 15:39:37 (4732): No heartbeat from core client for 30 sec - exiting 15:39:38 (4732): No heartbeat from core client for 30 sec - exiting 15:39:39 (4732): No heartbeat from core client for 30 sec - exiting 15:39:40 (4732): No heartbeat from core client for 30 sec - exiting 15:39:41 (4732): No heartbeat from core client for 30 sec - exiting 15:39:42 (4732): No heartbeat from core client for 30 sec - exiting 15:39:43 (4732): No heartbeat from core client for 30 sec - exiting 15:39:44 (4732): No heartbeat from core client for 30 sec - exiting 15:39:45 (4732): No heartbeat from core client for 30 sec - exiting 15:39:46 (4732): No heartbeat from core client for 30 sec - exiting 15:39:48 (4732): No heartbeat from core client for 30 sec - exiting 15:39:49 (4732): No heartbeat from core client for 30 sec - exiting 15:39:50 (4732): No heartbeat from core client for 30 sec - exiting 15:39:51 (4732): No heartbeat from core client for 30 sec - exiting 15:39:52 (4732): No heartbeat from core client for 30 sec - exiting 15:39:53 (4732): No heartbeat from core client for 30 sec - exiting 15:39:54 (4732): No heartbeat from core client for 30 sec - exiting 15:39:55 (4732): No heartbeat from core client for 30 sec - exiting 15:39:56 (4732): No heartbeat from core client for 30 sec - exiting 15:39:57 (4732): No heartbeat from core client for 30 sec - exiting 15:39:58 (4732): No heartbeat from core client for 30 sec - exiting 15:40:00 (4732): No heartbeat from core client for 30 sec - exiting 15:40:01 (4732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:41:11 (7272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:44:01 (4060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 15:51:15 (7348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:26:53 (5056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2228, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 10:25:45 (7544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:27:29 (6156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4704, iMonCtr=1 Model crash detected, will try to restart... 11:35:03 (7424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4480, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 21:16:24 (7168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:16:30 (7168): No heartbeat from core client for 30 sec - exiting 21:16:31 (7168): No heartbeat from core client for 30 sec - exiting 21:16:32 (7168): No heartbeat from core client for 30 sec - exiting 21:16:34 (7168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Mar 2013 13:20:03	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	881,280	1,732,649	1.9661
08 Mar 2013 16:05:49	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	855,360	1,680,579	1.9648
07 Mar 2013 16:35:47	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	829,440	1,628,293	1.9631
05 Mar 2013 21:22:21	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	803,520	1,575,666	1.9610
04 Mar 2013 23:07:54	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	777,600	1,523,336	1.9590
26 Feb 2013 23:39:34	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	751,680	1,473,085	1.9597
25 Feb 2013 16:52:13	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	725,760	1,422,004	1.9593
23 Feb 2013 16:15:50	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	699,840	1,371,703	1.9600
22 Feb 2013 19:52:21	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	673,920	1,320,364	1.9592
21 Feb 2013 22:44:44	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	648,000	1,268,591	1.9577
20 Feb 2013 22:42:19	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	622,080	1,217,901	1.9578
20 Feb 2013 08:28:02	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	596,160	1,167,406	1.9582
19 Feb 2013 11:23:49	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	570,240	1,116,577	1.9581
18 Feb 2013 11:18:52	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	544,320	1,064,774	1.9562
17 Feb 2013 12:27:43	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	518,400	1,013,557	1.9552
16 Feb 2013 13:55:20	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	492,480	963,993	1.9574
15 Feb 2013 17:06:04	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	466,560	914,074	1.9592
14 Feb 2013 13:20:14	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	440,640	864,972	1.9630
13 Feb 2013 15:15:36	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	414,720	814,678	1.9644
12 Feb 2013 18:51:13	985115	15554419	hadcm3n_t31r_1940_40_008281426_2	388,800	764,054	1.9652