Task 17661255

Name	hadcm3n_xcze_1940_40_009153120_2
Workunit	9283456
Created	30 Dec 2014, 23:45:13 UTC
Sent	31 Dec 2014, 0:38:54 UTC
Report deadline	1 Apr 2015, 8:06:05 UTC
Received	18 Jan 2015, 21:44:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1257519
Run time	8 days 7 hours 45 min 4 sec
CPU time	7 days 9 hours 23 min 49 sec
Validate state	Invalid
Credit	9,953.28
Device peak FLOPS	3.73 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.4.27</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 09:16:19 (5184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:15:18 (3560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5236, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4972, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Jan 2015 15:59:15	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	829,440	638,640	0.7700
18 Jan 2015 02:16:08	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	803,520	620,245	0.7719
17 Jan 2015 20:27:23	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	777,600	601,850	0.7740
17 Jan 2015 04:14:07	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	751,680	583,543	0.7763
16 Jan 2015 03:11:03	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	725,760	565,129	0.7787
15 Jan 2015 02:11:06	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	699,840	546,803	0.7813
14 Jan 2015 01:30:18	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	673,920	528,671	0.7845
13 Jan 2015 01:17:15	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	648,000	510,582	0.7879
12 Jan 2015 01:06:07	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	622,080	492,206	0.7912
11 Jan 2015 19:30:24	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	596,160	473,998	0.7951
11 Jan 2015 13:54:35	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	570,240	455,844	0.7994
10 Jan 2015 23:17:09	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	544,320	437,425	0.8036
10 Jan 2015 17:46:24	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	518,400	419,255	0.8087
10 Jan 2015 05:34:24	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	492,480	401,613	0.8155
10 Jan 2015 00:09:00	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	466,560	383,634	0.8223
09 Jan 2015 18:53:15	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	440,640	366,454	0.8316
09 Jan 2015 14:09:53	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	414,720	349,108	0.8418
09 Jan 2015 01:00:26	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	388,800	329,937	0.8486
08 Jan 2015 18:59:39	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	362,880	309,971	0.8542
08 Jan 2015 16:22:50	1257519	17661255	hadcm3n_xcze_1940_40_009153120_2	336,960	289,831	0.8601