Task 12995575

Name	hadcm3n_o5h3_1940_40_007301763_1
Workunit	7499187
Created	22 Jun 2011, 14:57:49 UTC
Sent	22 Jun 2011, 14:58:04 UTC
Report deadline	21 Sep 2011, 22:25:15 UTC
Received	3 Aug 2011, 19:06:56 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1052964
Run time	8 days 7 hours 29 min 15 sec
CPU time	7 days 17 hours 20 min 58 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.60 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> 06:55:25 (1516): No heartbeat from core client for 30 sec - exiting 06:55:26 (1516): No heartbeat from core client for 30 sec - exiting 06:55:27 (1516): No heartbeat from core client for 30 sec - exiting 06:55:28 (1516): No heartbeat from core client for 30 sec - exiting 06:55:29 (1516): No heartbeat from core client for 30 sec - exiting 06:55:30 (1516): No heartbeat from core client for 30 sec - exiting 06:55:31 (1516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:36:26 (5896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:12:36 (5084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:40:32 (1476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:16:24 (4520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:16:25 (4520): No heartbeat from core client for 30 sec - exiting 08:01:26 (2980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:02:03 (6548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:02:46 (2300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:05:04 (7656): No heartbeat from core client for 30 sec - exiting 08:05:05 (7656): No heartbeat from core client for 30 sec - exiting 08:05:06 (7656): No heartbeat from core client for 30 sec - exiting 08:05:07 (7656): No heartbeat from core client for 30 sec - exiting 08:05:08 (7656): No heartbeat from core client for 30 sec - exiting 08:05:09 (7656): No heartbeat from core client for 30 sec - exiting 08:05:10 (7656): No heartbeat from core client for 30 sec - exiting 08:05:11 (7656): No heartbeat from core client for 30 sec - exiting 08:05:12 (7656): No heartbeat from core client for 30 sec - exiting 08:05:13 (7656): No heartbeat from core client for 30 sec - exiting 08:05:14 (7656): No heartbeat from core client for 30 sec - exiting 08:05:15 (7656): No heartbeat from core client for 30 sec - exiting 08:05:16 (7656): No heartbeat from core client for 30 sec - exiting 08:05:17 (7656): No heartbeat from core client for 30 sec - exiting 08:05:18 (7656): No heartbeat from core client for 30 sec - exiting 08:05:19 (7656): No heartbeat from core client for 30 sec - exiting 08:05:20 (7656): No heartbeat from core client for 30 sec - exiting 08:05:21 (7656): No heartbeat from core client for 30 sec - exiting 08:05:22 (7656): No heartbeat from core client for 30 sec - exiting 08:05:23 (7656): No heartbeat from core client for 30 sec - exiting 08:05:24 (7656): No heartbeat from core client for 30 sec - exiting 08:05:25 (7656): No heartbeat from core client for 30 sec - exiting 08:05:26 (7656): No heartbeat from core client for 30 sec - exiting 08:05:27 (7656): No heartbeat from core client for 30 sec - exiting 08:05:28 (7656): No heartbeat from core client for 30 sec - exiting 08:05:29 (7656): No heartbeat from core client for 30 sec - exiting 08:05:30 (7656): No heartbeat from core client for 30 sec - exiting 08:05:31 (7656): No heartbeat from core client for 30 sec - exiting 08:05:32 (7656): No heartbeat from core client for 30 sec - exiting 08:05:33 (7656): No heartbeat from core client for 30 sec - exiting 08:05:34 (7656): No heartbeat from core client for 30 sec - exiting 08:05:35 (7656): No heartbeat from core client for 30 sec - exiting 08:05:36 (7656): No heartbeat from core client for 30 sec - exiting 08:05:37 (7656): No heartbeat from core client for 30 sec - exiting 08:05:38 (7656): No heartbeat from core client for 30 sec - exiting 08:05:39 (7656): No heartbeat from core client for 30 sec - exiting 08:05:40 (7656): No heartbeat from core client for 30 sec - exiting 08:06:12 (7656): No heartbeat from core client for 30 sec - exiting 08:06:13 (7656): No heartbeat from core client for 30 sec - exiting 08:06:14 (7656): No heartbeat from core client for 30 sec - exiting 08:06:15 (7656): No heartbeat from core client for 30 sec - exiting 08:06:16 (7656): No heartbeat from core client for 30 sec - exiting 08:06:17 (7656): No heartbeat from core client for 30 sec - exiting 08:06:18 (7656): No heartbeat from core client for 30 sec - exiting 08:06:19 (7656): No heartbeat from core client for 30 sec - exiting 08:06:20 (7656): No heartbeat from core client for 30 sec - exiting 08:06:21 (7656): No heartbeat from core client for 30 sec - exiting 08:06:22 (7656): No heartbeat from core client for 30 sec - exiting 08:06:23 (7656): No heartbeat from core client for 30 sec - exiting 08:06:24 (7656): No heartbeat from core client for 30 sec - exiting 08:06:25 (7656): No heartbeat from core client for 30 sec - exiting 08:06:26 (7656): No heartbeat from core client for 30 sec - exiting 08:06:27 (7656): No heartbeat from core client for 30 sec - exiting 08:06:28 (7656): No heartbeat from core client for 30 sec - exiting 08:06:29 (7656): No heartbeat from core client for 30 sec - exiting 08:06:30 (7656): No heartbeat from core client for 30 sec - exiting 08:06:31 (7656): No heartbeat from core client for 30 sec - exiting 08:06:32 (7656): No heartbeat from core client for 30 sec - exiting 08:06:33 (7656): No heartbeat from core client for 30 sec - exiting 08:06:34 (7656): No heartbeat from core client for 30 sec - exiting 08:06:35 (7656): No heartbeat from core client for 30 sec - exiting 08:06:36 (7656): No heartbeat from core client for 30 sec - exiting 08:06:37 (7656): No heartbeat from core client for 30 sec - exiting 08:06:38 (7656): No heartbeat from core client for 30 sec - exiting 08:06:39 (7656): No heartbeat from core client for 30 sec - exiting 08:06:40 (7656): No heartbeat from core client for 30 sec - exiting 08:06:41 (7656): No heartbeat from core client for 30 sec - exiting 08:06:42 (7656): No heartbeat from core client for 30 sec - exiting 08:06:43 (7656): No heartbeat from core client for 30 sec - exiting 08:06:44 (7656): No heartbeat from core client for 30 sec - exiting 08:06:45 (7656): No heartbeat from core client for 30 sec - exiting 08:06:46 (7656): No heartbeat from core client for 30 sec - exiting 08:06:47 (7656): No heartbeat from core client for 30 sec - exiting 08:06:48 (7656): No heartbeat from core client for 30 sec - exiting 08:06:49 (7656): No heartbeat from core client for 30 sec - exiting 08:06:50 (7656): No heartbeat from core client for 30 sec - exiting 08:06:51 (7656): No heartbeat from core client for 30 sec - exiting 08:06:52 (7656): No heartbeat from core client for 30 sec - exiting 08:06:53 (7656): No heartbeat from core client for 30 sec - exiting 08:06:54 (7656): No heartbeat from core client for 30 sec - exiting 08:06:55 (7656): No heartbeat from core client for 30 sec - exiting 08:06:56 (7656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:08:21 (6856): No heartbeat from core client for 30 sec - exiting 08:08:22 (6856): No heartbeat from core client for 30 sec - exiting 08:08:23 (6856): No heartbeat from core client for 30 sec - exiting 08:08:24 (6856): No heartbeat from core client for 30 sec - exiting 08:08:25 (6856): No heartbeat from core client for 30 sec - exiting 08:08:26 (6856): No heartbeat from core client for 30 sec - exiting 08:08:27 (6856): No heartbeat from core client for 30 sec - exiting 08:08:28 (6856): No heartbeat from core client for 30 sec - exiting 08:08:29 (6856): No heartbeat from core client for 30 sec - exiting 08:08:30 (6856): No heartbeat from core client for 30 sec - exiting 08:08:31 (6856): No heartbeat from core client for 30 sec - exiting 08:08:32 (6856): No heartbeat from core client for 30 sec - exiting 08:08:33 (6856): No heartbeat from core client for 30 sec - exiting 08:08:34 (6856): No heartbeat from core client for 30 sec - exiting 08:08:35 (6856): No heartbeat from core client for 30 sec - exiting 08:08:36 (6856): No heartbeat from core client for 30 sec - exiting 08:08:37 (6856): No heartbeat from core client for 30 sec - exiting 08:08:38 (6856): No heartbeat from core client for 30 sec - exiting 08:08:39 (6856): No heartbeat from core client for 30 sec - exiting 08:08:40 (6856): No heartbeat from core client for 30 sec - exiting 08:08:41 (6856): No heartbeat from core client for 30 sec - exiting 08:08:42 (6856): No heartbeat from core client for 30 sec - exiting 08:08:43 (6856): No heartbeat from core client for 30 sec - exiting 08:08:44 (6856): No heartbeat from core client for 30 sec - exiting 08:08:45 (6856): No heartbeat from core client for 30 sec - exiting 08:08:46 (6856): No heartbeat from core client for 30 sec - exiting 08:08:47 (6856): No heartbeat from core client for 30 sec - exiting 08:08:48 (6856): No heartbeat from core client for 30 sec - exiting 08:09:19 (6856): No heartbeat from core client for 30 sec - exiting 08:09:20 (6856): No heartbeat from core client for 30 sec - exiting 08:09:21 (6856): No heartbeat from core client for 30 sec - exiting 08:09:22 (6856): No heartbeat from core client for 30 sec - exiting 08:09:23 (6856): No heartbeat from core client for 30 sec - exiting 08:09:24 (6856): No heartbeat from core client for 30 sec - exiting 08:09:25 (6856): No heartbeat from core client for 30 sec - exiting 08:09:26 (6856): No heartbeat from core client for 30 sec - exiting 08:09:27 (6856): No heartbeat from core client for 30 sec - exiting 08:09:28 (6856): No heartbeat from core client for 30 sec - exiting 08:09:29 (6856): No heartbeat from core client for 30 sec - exiting 08:09:30 (6856): No heartbeat from core client for 30 sec - exiting 08:09:31 (6856): No heartbeat from core client for 30 sec - exiting 08:09:32 (6856): No heartbeat from core client for 30 sec - exiting 08:09:33 (6856): No heartbeat from core client for 30 sec - exiting 08:09:34 (6856): No heartbeat from core client for 30 sec - exiting 08:09:35 (6856): No heartbeat from core client for 30 sec - exiting 08:09:36 (6856): No heartbeat from core client for 30 sec - exiting 08:09:37 (6856): No heartbeat from core client for 30 sec - exiting 08:09:38 (6856): No heartbeat from core client for 30 sec - exiting 08:09:39 (6856): No heartbeat from core client for 30 sec - exiting 08:09:40 (6856): No heartbeat from core client for 30 sec - exiting 08:09:41 (6856): No heartbeat from core client for 30 sec - exiting 08:09:42 (6856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:48:11 (7680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:51:19 (2828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:30:34 (3672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:30:35 (3672): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:04:02 (2184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5948, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 17:51:56 (1620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:45:03 (6736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:45:47 (7696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:59:39 (5724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:59:40 (5724): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1080, iMonCtr=1 Model crash detected, will try to restart... 22:18:01 (6292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:18:02 (6292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9756, iMonCtr=1 Model crash detected, will try to restart... 05:49:20 (4516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4236, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6124, iMonCtr=1 Model crash detected, will try to restart... 07:16:17 (5784): No heartbeat from core client for 30 sec - exiting 07:16:18 (5784): No heartbeat from core client for 30 sec - exiting 07:16:19 (5784): No heartbeat from core client for 30 sec - exiting 07:16:20 (5784): No heartbeat from core client for 30 sec - exiting 07:16:21 (5784): No heartbeat from core client for 30 sec - exiting 07:16:22 (5784): No heartbeat from core client for 30 sec - exiting 07:16:23 (5784): No heartbeat from core client for 30 sec - exiting 07:16:24 (5784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:02:24 (7180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:31:58 (2276): No heartbeat from core client for 30 sec - exiting 06:31:59 (2276): No heartbeat from core client for 30 sec - exiting 06:32:00 (2276): No heartbeat from core client for 30 sec - exiting 06:32:01 (2276): No heartbeat from core client for 30 sec - exiting 06:32:02 (2276): No heartbeat from core client for 30 sec - exiting 06:32:03 (2276): No heartbeat from core client for 30 sec - exiting 06:32:04 (2276): No heartbeat from core client for 30 sec - exiting 06:32:05 (2276): No heartbeat from core client for 30 sec - exiting 06:32:06 (2276): No heartbeat from core client for 30 sec - exiting 06:32:07 (2276): No heartbeat from core client for 30 sec - exiting 06:32:08 (2276): No heartbeat from core client for 30 sec - exiting 06:32:10 (2276): No heartbeat from core client for 30 sec - exiting 06:32:11 (2276): No heartbeat from core client for 30 sec - exiting 06:32:12 (2276): No heartbeat from core client for 30 sec - exiting 06:32:13 (2276): No heartbeat from core client for 30 sec - exiting 06:32:14 (2276): No heartbeat from core client for 30 sec - exiting 06:32:15 (2276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:16:20 (4124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6764, iMonCtr=1 Model crash detected, will try to restart... 19:47:34 (5784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:55:58 (4348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:32:56 (5248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:33:50 (7436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:48:16 (4780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:42:58 (688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:20:02 (5584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:59:24 (6256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:00:34 (6612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:33:01 (6876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:42:00 (2232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:31:18 (5368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76E567A7 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Cannot serialize file D:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o5h3_1940_40_007301763/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Aug 2011 19:07:09	1052964	12995575	hadcm3n_o5h3_1940_40_007301763_1	259,200	667,244	2.5742
01 Aug 2011 05:19:35	1052964	12995575	hadcm3n_o5h3_1940_40_007301763_1	233,280	606,624	2.6004
26 Jul 2011 03:58:44	1052964	12995575	hadcm3n_o5h3_1940_40_007301763_1	207,360	543,889	2.6229
25 Jul 2011 19:49:51	1052964	12995575	hadcm3n_o5h3_1940_40_007301763_1	181,440	478,483	2.6371
25 Jul 2011 19:26:58	1052964	12995575	hadcm3n_o5h3_1940_40_007301763_1	155,520	426,039	2.7394
25 Jul 2011 17:33:16	1052964	12995575	hadcm3n_o5h3_1940_40_007301763_1	129,600	355,746	2.7450
25 Jul 2011 16:25:01	1052964	12995575	hadcm3n_o5h3_1940_40_007301763_1	103,680	281,044	2.7107
25 Jul 2011 16:06:26	1052964	12995575	hadcm3n_o5h3_1940_40_007301763_1	77,760	210,902	2.7122
05 Jul 2011 07:33:24	1052964	12995575	hadcm3n_o5h3_1940_40_007301763_1	51,840	139,391	2.6889
25 Jun 2011 05:01:02	1052964	12995575	hadcm3n_o5h3_1940_40_007301763_1	25,920	63,980	2.4684