Task 13633504

Name	hadcm3n_yjeu_1900_40_007525304_4
Workunit	7722779
Created	15 Nov 2011, 16:49:33 UTC
Sent	18 Nov 2011, 21:33:37 UTC
Report deadline	18 Feb 2012, 5:00:48 UTC
Received	15 Dec 2011, 3:34:16 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1090056
Run time	15 days 0 hours 59 min 14 sec
CPU time	11 days 21 hours 22 min 35 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.35 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3760, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3760, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3928, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3552, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=756, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Dec 2011 00:10:24	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	518,400	1,011,422	1.9510
13 Dec 2011 00:07:44	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	492,480	964,866	1.9592
11 Dec 2011 13:43:04	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	466,560	917,464	1.9664
10 Dec 2011 01:19:16	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	440,640	868,693	1.9714
09 Dec 2011 03:41:28	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	414,720	817,060	1.9701
08 Dec 2011 06:47:36	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	388,800	765,882	1.9699
06 Dec 2011 22:22:04	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	362,880	716,805	1.9753
05 Dec 2011 13:22:16	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	336,960	667,256	1.9802
04 Dec 2011 18:22:37	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	311,040	615,566	1.9791
03 Dec 2011 21:46:58	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	285,120	561,651	1.9699
02 Dec 2011 12:48:05	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	259,200	510,403	1.9691
01 Dec 2011 05:34:40	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	233,280	458,461	1.9653
30 Nov 2011 06:05:41	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	207,360	406,798	1.9618
28 Nov 2011 12:36:40	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	181,440	357,279	1.9691
26 Nov 2011 12:25:30	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	155,520	305,233	1.9627
26 Nov 2011 02:38:01	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	129,600	253,743	1.9579
26 Nov 2011 02:38:01	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	103,680	203,175	1.9596
24 Nov 2011 06:56:05	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	77,760	153,973	1.9801
23 Nov 2011 16:04:02	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	51,840	102,009	1.9678
22 Nov 2011 02:07:14	1090056	13633504	hadcm3n_yjeu_1900_40_007525304_4	25,920	52,134	2.0113