Task 16644728

Name	hadcm3n_8f4g_1980_40_008729019_2
Workunit	8874997
Created	20 May 2014, 6:20:07 UTC
Sent	20 May 2014, 6:25:30 UTC
Report deadline	19 Aug 2014, 13:52:41 UTC
Received	30 Jul 2014, 20:54:20 UTC
Server state	Over
Outcome	Computation error
Client state	Done
Exit status	22 (0x00000016) Unknown error code
Computer ID	895753
Run time	15 days 4 hours 33 min 11 sec
CPU time	15 days 4 hours 33 min 11 sec
Validate state	Invalid
Credit	7,776.00
Device peak FLOPS	2.05 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>5.10.45</core_client_version> <![CDATA[ <message> La periferica non riconosce il comando. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:48:58 (608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:00:20 (7800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:00:21 (7800): No heartbeat from core client for 30 sec - exiting 13:00:22 (7800): No heartbeat from core client for 30 sec - exiting 13:00:23 (7800): No heartbeat from core client for 30 sec - exiting 13:00:24 (7800): No heartbeat from core client for 30 sec - exiting 13:00:25 (7800): No heartbeat from core client for 30 sec - exiting 13:00:26 (7800): No heartbeat from core client for 30 sec - exiting 13:00:27 (7800): No heartbeat from core client for 30 sec - exiting 13:00:28 (7800): No heartbeat from core client for 30 sec - exiting 13:00:29 (7800): No heartbeat from core client for 30 sec - exiting 13:00:30 (7800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/8f4gko.pjj6c10 Error converting file to netcdf: dataout/8f4gko.pij6c10 Error converting file to netcdf: dataout/8f4gko.pfj6c10 Error converting file to netcdf: dataout/8f4gka.phj6c10 Error converting file to netcdf: dataout/8f4gka.pgj6c10 Error converting file to netcdf: dataout/8f4gka.pej6c10 Error converting file to netcdf: dataout/8f4gka.pdj6c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:28:11 (1604): No heartbeat from core client for 30 sec - exiting 07:28:12 (1604): No heartbeat from core client for 30 sec - exiting 07:28:13 (1604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:28:14 (1604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:27:52 (3824): No heartbeat from core client for 30 sec - exiting 08:27:53 (3824): No heartbeat from core client for 30 sec - exiting 08:27:54 (3824): No heartbeat from core client for 30 sec - exiting 08:27:55 (3824): No heartbeat from core client for 30 sec - exiting 08:27:56 (3824): No heartbeat from core client for 30 sec - exiting 08:27:57 (3824): No heartbeat from core client for 30 sec - exiting 08:27:58 (3824): No heartbeat from core client for 30 sec - exiting 08:28:00 (3824): No heartbeat from core client for 30 sec - exiting 08:28:01 (3824): No heartbeat from core client for 30 sec - exiting 08:28:02 (3824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:13:28 (3852): No heartbeat from core client for 30 sec - exiting 08:13:29 (3852): No heartbeat from core client for 30 sec - exiting 08:13:30 (3852): No heartbeat from core client for 30 sec - exiting 08:13:31 (3852): No heartbeat from core client for 30 sec - exiting 08:13:32 (3852): No heartbeat from core client for 30 sec - exiting 08:13:33 (3852): No heartbeat from core client for 30 sec - exiting 08:13:34 (3852): No heartbeat from core client for 30 sec - exiting 08:13:36 (3852): No heartbeat from core client for 30 sec - exiting 08:13:37 (3852): No heartbeat from core client for 30 sec - exiting 08:13:38 (3852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:28:32 (3496): No heartbeat from core client for 30 sec - exiting 08:28:33 (3496): No heartbeat from core client for 30 sec - exiting 08:28:34 (3496): No heartbeat from core client for 30 sec - exiting 08:28:36 (3496): No heartbeat from core client for 30 sec - exiting 08:28:37 (3496): No heartbeat from core client for 30 sec - exiting 08:28:38 (3496): No heartbeat from core client for 30 sec - exiting 08:28:39 (3496): No heartbeat from core client for 30 sec - exiting 08:28:40 (3496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:48:22 (2384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:48:38 (2384):12:52:50 (2332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:52:51 (2332): No heartbeat from core client for 30 sec - exiting 12:52:52 (2332): No heartbeat from core client for 30 sec - exiting 12:52:53 (2332): No heartbeat from core client for 30 sec - exiting 12:52:54 (2332): No heartbeat from core client for 30 sec - exiting 12:52:55 (2332): No heartbeat from core client for 30 sec - exiting 12:52:56 (2332): No heartbeat from core client for 30 sec - exiting 12:52:57 (2332): No heartbeat from core client for 30 sec - exiting 12:52:58 (2332): No heartbeat from core client for 30 sec - exiting 12:52:59 (2332): No heartbeat from core client for 30 sec - exiting 12:53:00 (2332): No heartbeat from core client for 30 sec - exiting 12:57:14 (2744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:57:20 (2744): No heartbeat from core client for 30 sec - exiting 12:57:21 (2744): No heartbeat from core client for 30 sec - exiting 12:57:22 (2744): No heartbeat from core client for 30 sec - exiting 12:57:23 (2744): No heartbeat from core client for 30 sec - exiting 12:57:24 (2744): No heartbeat from core client for 30 sec - exiting 12:57:25 (2744): No heartbeat from core client for 30 sec - exiting 12:57:26 (2744): No heartbeat from core client for 30 sec - exiting 12:57:27 (2744): No heartbeat from core client for 30 sec - exiting 12:57:28 (2744): No heartbeat from core client for 30 sec - exiting 12:57:29 (2744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:46:19 (3852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:46:20 (3852): No heartbeat from core client for 30 sec - exiting 07:46:21 (3852): No heartbeat from core client for 30 sec - exiting 07:46:22 (3852): No heartbeat from core client for 30 sec - exiting 07:46:23 (3852): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 07:57:45 (5916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:57:46 (5916): No heartbeat from core client for 30 sec - exiting 07:57:47 (5916): No heartbeat from core client for 30 sec - exiting 08:03:50 (1932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:10:30 (5420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:10:31 (5420): No heartbeat from core client for 30 sec - exiting 08:10:32 (5420): No heartbeat from core client for 30 sec - exiting 08:10:33 (5420): No heartbeat from core client for 30 sec - exiting 08:10:34 (5420):CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:39:11 (3716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:50:32 (1320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:53:39 (4032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:53:40 (4032): No heartbeat from core client for 30 sec - exiting 08:53:41 (4032): No heartbeat from core client for 30 sec - exiting 08:53:42 (4032): No heartbeat from core client for 30 sec - exiting 08:53:43 (4032): No heartbeat from core client for 30 sec - exiting 08:53:44 (4032): No heartbeat from core client for 30 sec - exiting 09:04:17 (268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:04:18 (268): No heartbeat from core client for 30 sec - exiting 09:04:19 (268): No heartbeat from core client for 30 sec - exiting 09:04:20 (268): No heartbeat from core client for 30 sec - exiting 09:04:21 (268): No heartbeat from core client for 30 sec - exiting 09:04:22 (268): No heartbeat from core client for 30 sec - exiting 09:04:23 (268): No heartbeat from core client for 30 sec - exiting 09:04:24 (268): No heartbeat from core client for 30 sec - exiting 09:04:25 (268): No heartbeat from core client for 30 sec - exiting 09:04:26 (268): No heartbeat from core client for 30 sec - exiting 09:04:27 (268): No heartbeat from core client for 30 sec - exiting forrtl: Accesso negato. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4988, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 16:31:55 (3844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:36:13 (5520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:36:14 (5520): No heartbeat from core client for 30 sec - exiting 16:36:16 (5520): No heartbeat from core client for 30 sec - exiting 16:36:17 (5520): No heartbeat from core client for 30 sec - exiting 16:36:18 (5520): No heartbeat from core client for 30 sec - exiting 16:36:19 (5520): No heartbeat from core client for 30 sec - exiting 16:36:20 (5520): No heartbeat from core client for 30 sec - exiting 16:36:21 (5520): No heartbeat from core client for 30 sec - exiting 16:36:22 (5520): No heartbeat from core client for 30 sec - exiting 16:36:23 (5520): No heartbeat from core client for 30 sec - exiting 16:36:24 (5520): No heartbeat from core client for 30 sec - exiting 16:44:59 (4016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:45:00 (4016): No heartbeat from core client for 30 sec - exiting 16:45:01 (4016): No heartbeat from core client for 30 sec - exiting 16:45:02 (4016): No heartbeat from core client for 30 sec - exiting 16:45:03 (4016): No heartbeat from core client for 30 sec - exiting 16:45:04 (4016): No heartbeat from core client for 30 sec - exiting 16:53:07 (4612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4796, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4796, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4796, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4796, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4796, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4796, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Jul 2014 05:23:34	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	648,000	1,275,228	1.9679
24 Jul 2014 13:42:21	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	622,080	1,221,822	1.9641
22 Jul 2014 14:13:39	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	596,160	1,167,571	1.9585
21 Jul 2014 06:52:09	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	570,240	1,115,351	1.9559
20 Jul 2014 14:29:23	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	544,320	1,066,451	1.9592
17 Jul 2014 17:36:09	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	518,400	1,018,027	1.9638
17 Jul 2014 02:54:10	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	492,480	968,062	1.9657
16 Jul 2014 12:49:46	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	466,560	918,407	1.9685
11 Jul 2014 18:07:33	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	440,640	869,839	1.9740
09 Jul 2014 00:23:33	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	414,720	819,040	1.9749
03 Jul 2014 18:43:00	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	388,800	769,455	1.9791
28 Jun 2014 18:33:17	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	362,880	718,599	1.9803
23 Jun 2014 15:49:23	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	336,960	667,030	1.9796
20 Jun 2014 12:17:08	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	311,040	616,711	1.9827
19 Jun 2014 11:01:55	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	285,120	565,988	1.9851
17 Jun 2014 11:41:20	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	259,200	514,794	1.9861
16 Jun 2014 21:04:35	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	233,280	463,909	1.9886
13 Jun 2014 02:31:20	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	207,360	412,685	1.9902
12 Jun 2014 11:09:38	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	181,440	360,755	1.9883
11 Jun 2014 13:07:26	895753	16644728	hadcm3n_8f4g_1980_40_008729019_2	155,520	308,631	1.9845