Task 15617775

Name	hadcm3n_zlvy_1880_40_008250567_2
Workunit	8405691
Created	21 Feb 2013, 13:56:10 UTC
Sent	21 Feb 2013, 13:56:17 UTC
Report deadline	23 May 2013, 21:23:28 UTC
Received	24 Apr 2013, 8:20:03 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1253957
Run time	17 days 4 hours 50 min 29 sec
CPU time	7 days 18 hours 47 min 11 sec
Validate state	Invalid
Credit	9,953.28
Device peak FLOPS	2.28 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:00:06 (5328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:30:37 (5528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2516, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5064, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5064, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5064, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2612, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 03:26:16 (4652): No heartbeat from core client for 30 sec - exiting 03:26:17 (4652): No heartbeat from core client for 30 sec - exiting 03:26:18 (4652): No heartbeat from core client for 30 sec - exiting 03:26:19 (4652): No heartbeat from core client for 30 sec - exiting 03:26:20 (4652): No heartbeat from core client for 30 sec - exiting 03:26:21 (4652): No heartbeat from core client for 30 sec - exiting 03:26:23 (4652): No heartbeat from core client for 30 sec - exiting 03:26:24 (4652): No heartbeat from core client for 30 sec - exiting 03:26:25 (4652): No heartbeat from core client for 30 sec - exiting 03:26:26 (4652): No heartbeat from core client for 30 sec - exiting 03:26:27 (4652): No heartbeat from core client for 30 sec - exiting 03:26:28 (4652): No heartbeat from core client for 30 sec - exiting 03:26:29 (4652): No heartbeat from core client for 30 sec - exiting 03:26:30 (4652): No heartbeat from core client for 30 sec - exiting 03:26:31 (4652): No heartbeat from core client for 30 sec - exiting 03:26:32 (4652): No heartbeat from core client for 30 sec - exiting 03:26:34 (4652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5104, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5104, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5104, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3580, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=220, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Mar 2013 16:05:53	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	829,440	639,899	0.7715
31 Mar 2013 03:17:43	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	803,520	595,302	0.7409
30 Mar 2013 13:30:01	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	777,600	550,765	0.7083
29 Mar 2013 23:56:24	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	751,680	800,934	1.0655
29 Mar 2013 10:41:37	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	725,760	756,303	1.0421
28 Mar 2013 16:06:13	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	699,840	710,782	1.0156
28 Mar 2013 03:21:53	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	673,920	665,691	0.9878
27 Mar 2013 13:45:07	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	648,000	619,695	0.9563
27 Mar 2013 00:08:29	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	622,080	573,835	0.9224
26 Mar 2013 12:22:19	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	596,160	530,148	0.8893
21 Mar 2013 20:28:34	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	570,240	949,876	1.6657
21 Mar 2013 07:48:58	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	544,320	905,531	1.6636
20 Mar 2013 19:26:19	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	518,400	860,969	1.6608
20 Mar 2013 06:06:22	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	492,480	816,493	1.6579
19 Mar 2013 11:59:51	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	466,560	772,148	1.6550
18 Mar 2013 23:56:52	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	440,640	728,751	1.6538
18 Mar 2013 11:54:39	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	414,720	685,567	1.6531
17 Mar 2013 23:36:14	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	388,800	642,262	1.6519
17 Mar 2013 11:24:35	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	362,880	598,838	1.6502
16 Mar 2013 23:21:38	1253957	15617775	hadcm3n_zlvy_1880_40_008250567_2	336,960	555,452	1.6484