Task 13013212

Name	hadcm3n_t1j9_1940_40_007311250_0
Workunit	7508680
Created	27 Jun 2011, 22:24:13 UTC
Sent	27 Jun 2011, 22:24:55 UTC
Report deadline	27 Sep 2011, 5:52:06 UTC
Received	15 Jul 2011, 22:08:34 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1070382
Run time	6 days 2 hours 38 min 22 sec
CPU time	4 days 19 hours 8 min 32 sec
Validate state	Invalid
Credit	2,488.32
Device peak FLOPS	1.65 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5104, iMonCtr=1 Model crash detected, will try to restart... 14:44:52 (4004): No heartbeat from core client for 30 sec - exiting 14:44:54 (4004): No heartbeat from core client for 30 sec - exiting 14:44:55 (4004): No heartbeat from core client for 30 sec - exiting 14:44:56 (4004): No heartbeat from core client for 30 sec - exiting 14:44:57 (4004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:12:07 (2904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:46:18 (6132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:46:20 (6132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 17:52:33 (5776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:33:25 (3256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:05:03 (5056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:02:16 (6816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:09:44 (976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:45:56 (2640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:49:31 (3720): No heartbeat from core client for 30 sec - exiting 10:49:33 (3720): No heartbeat from core client for 30 sec - exiting 10:49:34 (3720): No heartbeat from core client for 30 sec - exiting 10:49:35 (3720): No heartbeat from core client for 30 sec - exiting 10:49:36 (3720): No heartbeat from core client for 30 sec - exiting 10:49:37 (3720): No heartbeat from core client for 30 sec - exiting 10:49:38 (3720): No heartbeat from core client for 30 sec - exiting 10:49:39 (3720): No heartbeat from core client for 30 sec - exiting 10:49:40 (3720): No heartbeat from core client for 30 sec - exiting 10:49:42 (3720): No heartbeat from core client for 30 sec - exiting 10:49:43 (3720): No heartbeat from core client for 30 sec - exiting 10:49:44 (3720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:49:45 (3720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 14:13:56 (1940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:45:01 (5080): No heartbeat from core client for 30 sec - exiting 07:45:02 (5080): No heartbeat from core client for 30 sec - exiting 07:45:03 (5080): No heartbeat from core client for 30 sec - exiting 07:45:04 (5080): No heartbeat from core client for 30 sec - exiting 07:45:05 (5080): No heartbeat from core client for 30 sec - exiting 07:45:06 (5080): No heartbeat from core client for 30 sec - exiting 07:45:07 (5080): No heartbeat from core client for 30 sec - exiting 07:45:08 (5080): No heartbeat from core client for 30 sec - exiting 07:45:09 (5080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:12:31 (4440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:20:54 (6520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 11:10:46 (4120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:00:22 (4436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:00:23 (4436): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 12:40:17 (5116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:00:50 (6312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:51:41 (6352): No heartbeat from core client for 30 sec - exiting 07:51:49 (6352): No heartbeat from core client for 30 sec - exiting 07:51:50 (6352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: INITDUMP: Wrong no of ocean prognostic fields tmp/pipe_dummy 2048 Model crashed: INITDUMP: Wrong no of ocean prognostic fields tmp/pipe_dummy 2048 Model crashed: INITDUMP: Wrong no of ocean prognostic fields tmp/pipe_dummy 2048 Model crashed: INITDUMP: Wrong no of ocean prognostic fields tmp/pipe_dummy 2048 Model crashed: INITDUMP: Wrong no of ocean prognostic fields tmp/pipe_dummy 2048 Model crashed: INITDUMP: Wrong no of ocean prognostic fields tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish 12:05:42 (872): No heartbeat from core client for 30 sec - exiting 12:05:44 (872): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Jul 2011 14:47:33	1070382	13013212	hadcm3n_t1j9_1940_40_007311250_0	207,360	374,664	1.8068
25 Jul 2011 14:47:33	1070382	13013212	hadcm3n_t1j9_1940_40_007311250_0	181,440	330,183	1.8198
25 Jul 2011 14:47:33	1070382	13013212	hadcm3n_t1j9_1940_40_007311250_0	155,520	277,942	1.7872
08 Jul 2011 04:32:10	1070382	13013212	hadcm3n_t1j9_1940_40_007311250_0	129,600	232,560	1.7944
07 Jul 2011 20:13:52	1070382	13013212	hadcm3n_t1j9_1940_40_007311250_0	103,680	185,834	1.7924
05 Jul 2011 22:01:51	1070382	13013212	hadcm3n_t1j9_1940_40_007311250_0	77,760	144,522	1.8586
03 Jul 2011 19:39:05	1070382	13013212	hadcm3n_t1j9_1940_40_007311250_0	51,840	86,671	1.6719
29 Jun 2011 07:27:47	1070382	13013212	hadcm3n_t1j9_1940_40_007311250_0	25,920	44,631	1.7219