Task 16227777

Name	hadcm3n_7i3r_1980_40_008434746_1
Workunit	8585602
Created	14 Jan 2014, 4:59:02 UTC
Sent	14 Jan 2014, 4:59:56 UTC
Report deadline	15 Apr 2014, 12:27:07 UTC
Received	23 Feb 2014, 11:52:52 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1264739
Run time	8 days 2 hours 49 min 8 sec
CPU time	8 days 1 hours 24 min 9 sec
Validate state	Invalid
Credit	6,531.84
Device peak FLOPS	2.63 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> Das Gerät erkennt den Befehl nicht. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4876, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3504, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 19:36:20 (4648): No heartbeat from core client for 30 sec - exiting 19:36:21 (4648): No heartbeat from core client for 30 sec - exiting 19:36:22 (4648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:56:35 (4652): No heartbeat from core client for 30 sec - exiting 05:56:36 (4652): No heartbeat from core client for 30 sec - exiting 05:56:38 (4652): No heartbeat from core client for 30 sec - exiting 05:56:39 (4652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:06:38 (4536): No heartbeat from core client for 30 sec - exiting 00:06:39 (4536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:54:51 (4424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:16:01 (3688): No heartbeat from core client for 30 sec - exiting 15:16:02 (3688): No heartbeat from core client for 30 sec - exiting 15:16:03 (3688): No heartbeat from core client for 30 sec - exiting 15:16:04 (3688): No heartbeat from core client for 30 sec - exiting 15:16:06 (3688): No heartbeat from core client for 30 sec - exiting 15:16:07 (3688): No heartbeat from core client for 30 sec - exiting 15:16:08 (3688): No heartbeat from core client for 30 sec - exiting 15:16:09 (3688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:00:14 (4184): No heartbeat from core client for 30 sec - exiting 06:00:15 (4184): No heartbeat from core client for 30 sec - exiting 06:00:16 (4184): No heartbeat from core client for 30 sec - exiting 06:00:17 (4184): No heartbeat from core client for 30 sec - exiting 06:00:18 (4184): No heartbeat from core client for 30 sec - exiting 06:00:19 (4184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:00:21 (4184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 12:45:41 (3124): No heartbeat from core client for 30 sec - exiting 12:45:42 (3124): No heartbeat from core client for 30 sec - exiting 12:45:43 (3124): No heartbeat from core client for 30 sec - exiting 12:45:44 (3124): No heartbeat from core client for 30 sec - exiting 12:45:45 (3124): No heartbeat from core client for 30 sec - exiting 12:45:46 (3124): No heartbeat from core client for 30 sec - exiting 12:45:47 (3124): No heartbeat from core client for 30 sec - exiting 12:45:48 (3124): No heartbeat from core client for 30 sec - exiting 12:45:49 (3124): No heartbeat from core client for 30 sec - exiting 12:45:51 (3124): No heartbeat from core client for 30 sec - exiting 12:45:52 (3124): No heartbeat from core client for 30 sec - exiting 12:45:53 (3124): No heartbeat from core client for 30 sec - exiting 12:45:54 (3124): No heartbeat from core client for 30 sec - exiting 12:45:55 (3124): No heartbeat from core client for 30 sec - exiting 12:45:56 (3124): No heartbeat from core client for 30 sec - exiting 12:45:57 (3124): No heartbeat from core client for 30 sec - exiting 12:45:58 (3124): No heartbeat from core client for 30 sec - exiting 12:45:59 (3124): No heartbeat from core client for 30 sec - exiting 12:46:00 (3124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:30:50 (4236): No heartbeat from core client for 30 sec - exiting 20:30:51 (4236): No heartbeat from core client for 30 sec - exiting 20:30:52 (4236): No heartbeat from core client for 30 sec - exiting 20:30:53 (4236): No heartbeat from core client for 30 sec - exiting 20:30:54 (4236): No heartbeat from core client for 30 sec - exiting 20:30:55 (4236): No heartbeat from core client for 30 sec - exiting 20:30:56 (4236): No heartbeat from core client for 30 sec - exiting 20:30:57 (4236): No heartbeat from core client for 30 sec - exiting 20:30:58 (4236): No heartbeat from core client for 30 sec - exiting 20:30:59 (4236): No heartbeat from core client for 30 sec - exiting 20:31:00 (4236): No heartbeat from core client for 30 sec - exiting 20:31:02 (4236): No heartbeat from core client for 30 sec - exiting 20:31:03 (4236): No heartbeat from core client for 30 sec - exiting 20:31:04 (4236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/7i3rko.pjk1c10 Error converting file to netcdf: dataout/7i3rko.pik1c10 Error converting file to netcdf: dataout/7i3rko.pfk1c10 Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2672, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4364, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2552, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6072, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6072, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6072, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6072, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Feb 2014 12:03:16	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	544,320	672,416	1.2353
16 Feb 2014 10:58:56	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	518,400	645,047	1.2443
16 Feb 2014 03:07:51	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	492,480	619,376	1.2577
15 Feb 2014 19:51:34	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	466,560	593,178	1.2714
15 Feb 2014 05:36:19	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	440,640	561,254	1.2737
14 Feb 2014 19:34:34	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	414,720	526,101	1.2686
14 Feb 2014 09:51:06	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	388,800	491,074	1.2631
14 Feb 2014 00:09:23	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	362,880	456,188	1.2571
08 Feb 2014 12:01:28	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	336,960	419,242	1.2442
08 Feb 2014 03:34:45	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	311,040	388,849	1.2502
07 Feb 2014 18:17:23	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	285,120	354,864	1.2446
04 Feb 2014 18:38:00	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	259,200	325,216	1.2547
03 Feb 2014 23:02:16	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	233,280	293,896	1.2598
31 Jan 2014 05:02:40	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	207,360	257,809	1.2433
27 Jan 2014 23:09:19	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	181,440	221,248	1.2194
26 Jan 2014 03:16:08	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	155,520	183,799	1.1818
25 Jan 2014 17:38:32	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	129,600	149,490	1.1535
25 Jan 2014 09:07:14	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	103,680	119,038	1.1481
25 Jan 2014 00:35:39	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	77,760	88,141	1.1335
24 Jan 2014 16:12:35	1264739	16227777	hadcm3n_7i3r_1980_40_008434746_1	51,840	57,244	1.1042