Task 13026300

Name	hadcm3n_t5vo_1940_40_007316352_1
Workunit	7513782
Created	29 Jun 2011, 2:42:30 UTC
Sent	29 Jun 2011, 2:49:27 UTC
Report deadline	28 Sep 2011, 10:16:38 UTC
Received	22 Jul 2011, 12:48:44 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1307343
Run time	19 days 7 hours 13 min 49 sec
CPU time	14 days 10 hours 40 min 29 sec
Validate state	Invalid
Credit	10,575.36
Device peak FLOPS	2.69 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3244, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1776, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1328, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3068, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1336, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3084, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4376, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Jul 2011 19:19:16	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	881,280	1,574,344	1.7864
25 Jul 2011 18:50:11	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	855,360	1,527,808	1.7862
25 Jul 2011 18:10:48	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	829,440	1,481,552	1.7862
25 Jul 2011 17:49:48	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	803,520	1,435,228	1.7862
25 Jul 2011 17:27:36	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	777,600	1,388,284	1.7853
25 Jul 2011 16:38:27	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	751,680	1,341,674	1.7849
25 Jul 2011 16:01:43	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	725,760	1,294,755	1.7840
25 Jul 2011 15:42:10	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	699,840	1,248,054	1.7833
25 Jul 2011 14:52:28	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	673,920	1,201,138	1.7823
25 Jul 2011 14:25:03	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	648,000	1,154,423	1.7815
25 Jul 2011 14:25:03	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	622,080	1,108,259	1.7815
25 Jul 2011 14:25:03	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	596,160	1,061,527	1.7806
25 Jul 2011 14:25:03	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	570,240	1,014,997	1.7799
25 Jul 2011 14:25:03	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	544,320	968,885	1.7800
25 Jul 2011 14:25:03	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	518,400	922,465	1.7794
25 Jul 2011 14:25:03	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	492,480	876,348	1.7795
25 Jul 2011 14:25:02	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	466,560	829,861	1.7787
10 Jul 2011 22:04:35	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	440,640	783,421	1.7779
10 Jul 2011 07:26:15	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	414,720	736,663	1.7763
09 Jul 2011 18:55:51	1080220	13026300	hadcm3n_t5vo_1940_40_007316352_1	388,800	690,263	1.7754