Task 15491083

Name	hadcm3n_3k83_1940_40_008262262_1
Workunit	8417386
Created	21 Dec 2012, 2:07:25 UTC
Sent	28 Dec 2012, 16:55:17 UTC
Report deadline	30 Mar 2013, 0:22:28 UTC
Received	4 May 2013, 15:10:57 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1252118
Run time	7 days 14 hours 21 min 30 sec
CPU time	6 days 9 hours 19 min 1 sec
Validate state	Invalid
Credit	4,976.64
Device peak FLOPS	2.74 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3032, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4920, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5324, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5712, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4444, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5272, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 13:01:37 (6760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:26:23 (5720): No heartbeat from core client for 30 sec - exiting 14:26:24 (5720): No heartbeat from core client for 30 sec - exiting 14:26:25 (5720): No heartbeat from core client for 30 sec - exiting 14:26:26 (5720): No heartbeat from core client for 30 sec - exiting 14:26:27 (5720): No heartbeat from core client for 30 sec - exiting 14:26:28 (5720): No heartbeat from core client for 30 sec - exiting 14:26:29 (5720): No heartbeat from core client for 30 sec - exiting 14:26:30 (5720): No heartbeat from core client for 30 sec - exiting 14:26:31 (5720): No heartbeat from core client for 30 sec - exiting 14:26:32 (5720): No heartbeat from core client for 30 sec - exiting 14:26:33 (5720): No heartbeat from core client for 30 sec - exiting 14:26:34 (5720): No heartbeat from core client for 30 sec - exiting 14:26:35 (5720): No heartbeat from core client for 30 sec - exiting 14:26:36 (5720): No heartbeat from core client for 30 sec - exiting 14:26:37 (5720): No heartbeat from core client for 30 sec - exiting 14:26:38 (5720): No heartbeat from core client for 30 sec - exiting 14:26:39 (5720): No heartbeat from core client for 30 sec - exiting 14:26:40 (5720): No heartbeat from core client for 30 sec - exiting 14:26:41 (5720): No heartbeat from core client for 30 sec - exiting 14:26:42 (5720): No heartbeat from core client for 30 sec - exiting 14:26:43 (5720): No heartbeat from core client for 30 sec - exiting 14:26:44 (5720): No heartbeat from core client for 30 sec - exiting 14:26:45 (5720): No heartbeat from core client for 30 sec - exiting 14:26:46 (5720): No heartbeat from core client for 30 sec - exiting 14:26:47 (5720): No heartbeat from core client for 30 sec - exiting 14:26:48 (5720): No heartbeat from core client for 30 sec - exiting 14:26:49 (5720): No heartbeat from core client for 30 sec - exiting 14:26:50 (5720): No heartbeat from core client for 30 sec - exiting 14:26:51 (5720): No heartbeat from core client for 30 sec - exiting 14:26:52 (5720): No heartbeat from core client for 30 sec - exiting 14:26:53 (5720): No heartbeat from core client for 30 sec - exiting 14:26:54 (5720): No heartbeat from core client for 30 sec - exiting 14:26:55 (5720): No heartbeat from core client for 30 sec - exiting 14:26:56 (5720): No heartbeat from core client for 30 sec - exiting 14:26:57 (5720): No heartbeat from core client for 30 sec - exiting 14:26:58 (5720): No heartbeat from core client for 30 sec - exiting 14:26:59 (5720): No heartbeat from core client for 30 sec - exiting 14:27:00 (5720): No heartbeat from core client for 30 sec - exiting 14:27:01 (5720): No heartbeat from core client for 30 sec - exiting 14:27:02 (5720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:27:03 (5720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 May 2013 09:21:16	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	414,720	557,766	1.3449
28 Apr 2013 12:57:25	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	388,800	522,480	1.3438
28 Apr 2013 02:34:11	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	362,880	487,500	1.3434
21 Mar 2013 18:26:32	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	336,960	452,914	1.3441
15 Mar 2013 20:01:49	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	311,040	418,092	1.3442
15 Mar 2013 08:18:22	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	285,120	383,445	1.3449
05 Mar 2013 11:56:43	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	259,200	349,330	1.3477
05 Mar 2013 01:18:46	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	233,280	315,182	1.3511
11 Feb 2013 17:46:33	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	207,360	279,469	1.3477
04 Feb 2013 17:01:09	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	181,440	244,252	1.3462
30 Jan 2013 18:26:08	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	155,520	209,369	1.3463
24 Jan 2013 21:02:09	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	129,600	174,339	1.3452
22 Jan 2013 22:00:57	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	103,680	139,180	1.3424
17 Jan 2013 20:15:59	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	77,760	104,243	1.3406
10 Jan 2013 18:29:05	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	51,840	70,433	1.3587
08 Jan 2013 17:36:27	1252118	15491083	hadcm3n_3k83_1940_40_008262262_1	25,920	36,174	1.3956