Task 13141296

Name	hadcm3n_yeua_1900_40_007352092_2
Workunit	7549522
Created	16 Jul 2011, 1:19:10 UTC
Sent	16 Jul 2011, 1:22:30 UTC
Report deadline	15 Oct 2011, 8:49:41 UTC
Received	27 Sep 2011, 3:18:57 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1064622
Run time	8 days 13 hours 22 min 15 sec
CPU time	8 days 13 hours 22 min 15 sec
Validate state	Invalid
Credit	5,287.68
Device peak FLOPS	2.81 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3896, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:31:43 (3408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:22:51 (3400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:13:03 (3460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:40:38 (1004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not runnin12:24:19 (2536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:47:19 (3212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:18:14 (3336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:24:19 (3396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:10:27 (4652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:08:32 (3136): No heartbeat from core client for 30 sec - exiting 11:08:33 (3136): No heartbeat from core client for 30 sec - exiting 11:08:34 (3136): No heartbeat from core client for 30 sec - exiting 11:08:35 (3136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:13:00 (3468): No heartbeat from core client for 30 sec - exiting 22:13:02 (3468): No heartbeat from core client for 30 sec - exiting 22:13:03 (3468): No heartbeat from core client for 30 sec - exiting 22:13:04 (3468): No heartbeat from core client for 30 sec - exiting 22:13:05 (3468): No heartbeat from core client for 30 sec - exiting 22:13:06 (3468): No heartbeat from core client for 30 sec - exiting 22:13:07 (3468): No heartbeat from core client for 30 sec - exiting 22:13:08 (3468): No heartbeat from core client for 30 sec - exiting 22:13:09 (3468): No heartbeat from core client for 30 sec - exiting 22:13:10 (3468): No heartbeat from core client for 30 sec - exiting 22:13:11 (3468): No heartbeat from core client for 30 sec - exiting 22:13:13 (3468): No heartbeat from core client for 30 sec - exiting 22:13:14 (3468): No heartbeat from core client for 30 sec - exiting 22:13:15 (3468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:13:16 (3468): No heartbeat from core client for 30 sec - exiting 22:13:17 (3468): No heartbeat from core client for 30 sec - exiting 22:13:18 (3468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:22:33 (3380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3756, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5720, iMonCtr=1 Model crash detected, will try to restart... 21:16:01 (3444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Sep 2011 22:25:43	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	440,640	736,878	1.6723
23 Sep 2011 14:40:25	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	414,720	693,464	1.6721
19 Sep 2011 04:25:17	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	388,800	650,152	1.6722
17 Sep 2011 01:46:40	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	362,880	606,587	1.6716
14 Sep 2011 20:20:54	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	336,960	562,712	1.6700
03 Sep 2011 18:26:24	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	311,040	518,848	1.6681
02 Sep 2011 03:16:33	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	285,120	475,369	1.6673
28 Aug 2011 00:45:42	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	259,200	432,147	1.6672
25 Aug 2011 02:16:42	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	233,280	388,980	1.6674
15 Aug 2011 02:38:52	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	207,360	346,151	1.6693
11 Aug 2011 04:11:10	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	181,440	302,677	1.6682
08 Aug 2011 12:57:08	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	155,520	259,525	1.6688
03 Aug 2011 02:17:13	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	129,600	215,937	1.6662
30 Jul 2011 05:34:52	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	103,680	172,912	1.6677
28 Jul 2011 20:09:44	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	77,760	129,818	1.6695
25 Jul 2011 22:46:41	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	51,840	86,694	1.6723
25 Jul 2011 20:23:11	1064622	13141296	hadcm3n_yeua_1900_40_007352092_2	25,920	43,410	1.6748