Task 19316858

Name	hadcm3n_sdp7_198012_480_352_010334096_0
Workunit	10334096
Created	26 Feb 2016, 22:45:51 UTC
Sent	27 Feb 2016, 2:51:43 UTC
Report deadline	8 Feb 2017, 8:11:43 UTC
Received	17 Mar 2016, 22:25:56 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1367823
Run time	1 days 2 hours 12 min 44 sec
CPU time	21 hours 37 min 34 sec
Validate state	Invalid
Credit	9,642.24
Device peak FLOPS	4.11 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 18:52:06 (9072): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:38:21 (9636): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:44:28 (10852): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:56:15 (1204): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:53:09 (10824): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:14:43 (900): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:53:42 (7508): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:54:40 (1240): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:41:45 (5236): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 12:35:39 (6844): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:34:50 (5192): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:41:32 (7492): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 14:44:30 (2548): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 16:23:02 (3568): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 13:26:26 (8664): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 12:41:24 (9432): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:20:40 (11948): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:31:05 (4324): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:08:09 (8228): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 20:06:02 (10308): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 11:35:49 (1292): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 12:43:24 (6908): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:07:09 (9480): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 14:07:29 (5704): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:36:29 (7640): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:41:33 (4280): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:32:37 (4924): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 13:18:27 (14588): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:37:55 (5900): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 11:07:44 (10276): start_timer_thread(): CreateThread() failed, errno 0 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 11:15:39 (92): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 13:59:27 (13652): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 14:25:20 (8636): start_timer_thread(): CreateThread() failed, errno 0 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 14:33:46 (14568): start_timer_thread(): CreateThread() failed, errno 0 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 14:34:10 (5364): start_timer_thread(): CreateThread() failed, errno 0 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 14:34:34 (12448): start_timer_thread(): CreateThread() failed, errno 0 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 14:34:58 (1240): start_timer_thread(): CreateThread() failed, errno 0 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 14:35:22 (13980): start_timer_thread(): CreateThread() failed, errno 0 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Mar 2016 23:56:14	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	803,520	87,193	0.1085
16 Mar 2016 14:50:05	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	777,600	119,350	0.1535
16 Mar 2016 06:31:26	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	751,680	91,520	0.1218
15 Mar 2016 21:25:09	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	725,760	78,507	0.1082
14 Mar 2016 17:47:39	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	699,840	150,425	0.2149
14 Mar 2016 16:44:47	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	673,920	123,456	0.1832
14 Mar 2016 16:44:47	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	648,000	96,607	0.1491
14 Mar 2016 16:44:47	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	622,080	132,277	0.2126
14 Mar 2016 16:44:47	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	596,160	106,242	0.1782
14 Mar 2016 16:44:47	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	570,240	79,873	0.1401
14 Mar 2016 16:44:47	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	544,320	57,368	0.1054
14 Mar 2016 16:44:47	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	518,400	30,260	0.0584
14 Mar 2016 16:44:47	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	492,480	2,903	0.0059
09 Mar 2016 00:23:31	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	466,560	161,932	0.3471
08 Mar 2016 14:58:56	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	440,640	135,074	0.3065
08 Mar 2016 07:50:20	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	414,720	108,413	0.2614
07 Mar 2016 22:45:15	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	388,800	82,370	0.2119
07 Mar 2016 14:19:44	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	362,880	55,808	0.1538
07 Mar 2016 06:49:49	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	336,960	29,670	0.0881
06 Mar 2016 20:52:05	1367823	19316858	hadcm3n_sdp7_198012_480_352_010334096_0	311,040	3,056	0.0098