Task 13649136

Name	hadcm3n_y9fy_1900_40_007345096_2
Workunit	7542526
Created	20 Nov 2011, 12:39:49 UTC
Sent	20 Nov 2011, 12:58:06 UTC
Report deadline	19 Feb 2012, 20:25:17 UTC
Received	18 Jan 2012, 21:53:41 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1168944
Run time	20 days 8 hours 18 min 46 sec
CPU time	19 days 16 hours 27 min 55 sec
Validate state	Invalid
Credit	11,819.52
Device peak FLOPS	2.42 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> Il dispositivo non riconosce il comando. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2300, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:50:53 (2808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:54:46 (760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:34:28 (2424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:54:03 (2536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... C19:52:20 (3088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:25:49 (2384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:11:15 (2560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:18:20 (3472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:56:45 (4056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:00:47 (3896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:52:19 (652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:34:44 (3968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:28:28 (3512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:01:21 (1272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:14:17 (2604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:14:18 (2604): No heartbeat from core client for 30 sec - exiting 18:14:19 (2604): No heartbeat from core client for 30 sec - exiting 18:14:21 (2604): No heartbeat from core client for 30 sec - exiting 18:14:22 (2604): No heartbeat from core client for 30 sec - exiting 18:14:23 (2604): No heartbeat from core client for 30 sec - exiting 18:14:24 (2604): No heartbeat from core client for 30 sec - exiting 18:14:25 (2604): No heartbeat from core client for 30 sec - exiting 18:14:26 (2604): No heartbeat from core client for 30 sec - exiting 18:14:27 (2604): No heartbeat from core client for 30 sec - exiting 18:14:28 (2604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3488, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/y9fyko.pjc4c10 Error converting file to netcdf: dataout/y9fyko.pic4c10 Error converting file to netcdf: dataout/y9fyko.pfc4c10 Error converting file to netcdf: dataout/y9fyka.phc4c10 Error converting file to netcdf: dataout/y9fyka.pgc4c10 Error converting file to netcdf: dataout/y9fyka.pec4c10 Error converting file to netcdf: dataout/y9fyka.pdc4c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:19:06 (2420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3908, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 15:16:52 (1300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:50:16 (2428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:40:09 (3604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:54:06 (5112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:14:26 (3132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:09:44 (5072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:52:11 (2320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:21:35 (296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Jan 2012 20:00:45	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	984,960	1,700,877	1.7268
14 Jan 2012 21:53:02	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	959,040	1,656,016	1.7267
13 Jan 2012 20:12:15	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	933,120	1,610,227	1.7256
10 Jan 2012 21:15:59	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	907,200	1,564,506	1.7245
08 Jan 2012 16:36:41	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	881,280	1,519,343	1.7240
07 Jan 2012 13:58:20	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	855,360	1,472,908	1.7220
06 Jan 2012 12:03:52	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	829,440	1,427,130	1.7206
05 Jan 2012 09:59:48	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	803,520	1,382,082	1.7200
04 Jan 2012 09:16:19	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	777,600	1,337,124	1.7196
03 Jan 2012 08:21:10	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	751,680	1,291,771	1.7185
02 Jan 2012 10:05:05	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	725,760	1,247,138	1.7184
01 Jan 2012 09:37:27	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	699,840	1,201,825	1.7173
31 Dec 2011 08:21:20	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	673,920	1,156,869	1.7166
29 Dec 2011 22:11:32	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	648,000	1,111,736	1.7156
29 Dec 2011 06:08:08	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	622,080	1,066,676	1.7147
27 Dec 2011 06:37:06	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	596,160	1,022,397	1.7150
25 Dec 2011 20:22:20	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	570,240	978,200	1.7154
24 Dec 2011 20:20:25	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	544,320	934,200	1.7163
23 Dec 2011 20:06:38	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	518,400	889,607	1.7161
19 Dec 2011 21:34:50	1168944	13649136	hadcm3n_y9fy_1900_40_007345096_2	492,480	844,863	1.7155