Task 12898714

Name	hadcm3n_p7fc_1900_40_007227208_2
Workunit	7425448
Created	20 May 2011, 16:57:53 UTC
Sent	20 May 2011, 16:58:06 UTC
Report deadline	20 Aug 2011, 0:25:17 UTC
Received	15 Jul 2011, 14:35:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-2 (0xFFFFFFFE) Unknown error code
Computer ID	1037994
Run time	33 days 8 hours 13 min 49 sec
CPU time	28 days 16 hours 31 min 23 sec
Validate state	Invalid
Credit	8,398.08
Device peak FLOPS	1.58 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code -2 (0xfffffffe) </message> <stderr_txt> 05:34:48 (2892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:36:46 (9084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:47:24 (3708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:52:20 (2120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:01:18 (7832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:06:54 (9568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:07:57 (6300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:09:22 (8052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:10:08 (3224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:14:46 (7664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:35:45 (4532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:36:41 (9200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:38:42 (2132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:40:53 (8064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:41:39 (2292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:45:11 (6180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:45:52 (3880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:50:35 (5164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:51:30 (4460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:06:54 (7072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:15:00 (4396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:29:34 (6456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:32:13 (9072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:33:25 (8300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:34:11 (8032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:35:41 (9912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:36:27 (5204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:40:02 (6332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:41:35 (1860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:42:21 (3624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:43:06 (1528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:44:22 (6352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:46:27 (8712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:47:43 (5224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:48:29 (7632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:49:29 (5832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:56:26 (320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:57:22 (7972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:07:12 (3224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:08:01 (9284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:08:47 (6156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:17:01 (7036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:20:22 (4588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:24:40 (1748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:25:33 (2080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:33:54 (9672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:38:48 (8936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:39:47 (4520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:40:25 (8268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:44:50 (5008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:51:35 (2720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:58:20 (1284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:04:13 (5852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:10:06 (4264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:13:22 (4872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:14:25 (5780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:15:29 (8212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:18:02 (6976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:19:02 (9024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:21:49 (9732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:22:35 (8692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:24:35 (7060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:25:37 (2208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:26:22 (10176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:29:39 (5056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:09:20 (7528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:10:16 (4664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:17:18 (6348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:21:55 (4028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:28:01 (4256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:33:54 (7024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:39:57 (8080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:45:35 (7188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:52:18 (7760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:55:41 (8620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:03:31 (7928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:07:58 (6556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:10:12 (10080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:11:12 (4444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:11:59 (10112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:19:44 (8512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:23:26 (6780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:30:08 (8300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:32:48 (9992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:33:26 (9092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:34:12 (6564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:40:08 (8596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:46:57 (9128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:51:24 (3436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:59:36 (4956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:01:33 (3728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:07:27 (4584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:12:22 (3964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:14:43 (7136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:17:51 (9108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:19:07 (7056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:20:03 (4784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:20:41 (7560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:23:46 (4748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:28:06 (2280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:34:42 (260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:41:47 (5508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:01:52 (10792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:29:58 (15980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:53:47 (28900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:54:44 (36544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:53:14 (5660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:14:07 (10940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:15:01 (24984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:39:24 (22072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:40:48 (7708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:03:42 (4260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:16:24 (24612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:12:38 (37708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:13:30 (27572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:36:35 (6848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:32:31 (36916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:33:25 (12448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:31:58 (14660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:57:42 (27116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:45:33 (17300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:46:27 (16712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:47:22 (8904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:17:45 (21296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:04:46 (16776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:05:40 (18972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:41:56 (12584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:22:54 (29948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:23:29 (21108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:27:13 (28476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:28:10 (31876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:39:16 (30264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:52:44 (4060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:06:41 (16824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:18:50 (19352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:31:16 (33644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:43:18 (11384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:00:24 (10968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:12:42 (30484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:17:01 (30020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:29:41 (18112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:43:00 (13564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:00:19 (29340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:07:15 (38420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:08:09 (4368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:16:39 (10360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/p7fcko.pja4c10 Error converting file to netcdf: dataout/p7fcko.pia4c10 Error converting file to netcdf: dataout/p7fcko.pfa4c10 Error converting file to netcdf: dataout/p7fcka.pha4c10 Error converting file to netcdf: dataout/p7fcka.pga4c10 Error converting file to netcdf: dataout/p7fcka.pea4c10 Error converting file to netcdf: dataout/p7fcka.pda4c10 13:18:58 (30592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 13:33:05 (28088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:46:39 (33408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:11:59 (38380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:27:03 (28756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:42:18 (6972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:04:29 (29588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:08:23 (25480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:22:30 (11100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:37:49 (31628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:52:09 (34184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:13:17 (3384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:14:11 (25456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:29:55 (17176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:48:52 (1476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:04:53 (24476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:21:23 (30872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:36:22 (15200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:50:37 (10124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:51:12 (38392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:09:37 (37828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:26:15 (15292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:42:15 (28924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:46:22 (27680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:04:18 (34644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:22:39 (6140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:39:39 (32244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:56:04 (14236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:15:10 (24104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:24:52 (38796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:10:53 (28332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:11:51 (33880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:48:45 (31372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:34:34 (11520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:35:15 (15328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:06:50 (34340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:51:37 (10392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:52:32 (11708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:27:37 (11696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:11:56 (13260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:12:51 (10812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:47:56 (26644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:34:08 (32960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:35:03 (35900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:36:29 (7212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:37:23 (26328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:38:25 (25720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:39:11 (19024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:39:51 (34032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:40:42 (34236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:41:29 (31308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:44:45 (34848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:45:49 (23348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:46:51 (6920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:47:43 (30000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:18:30 (34180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:19:25 (31788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:20:27 (15576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:21:13 (29520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:21:11 (32236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:25:45 (20800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:26:46 (33344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:27:32 (37548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:29:02 (34764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:04:41 (11456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:05:34 (25428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Could not create shared memory region hadcm3n_p7fc_1900_40_007227208, 4677032 Error in creating shared memory region! Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:14:43 (15008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:15:30 (20200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:31:20 (11748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:32:35 (6456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:33:26 (10336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:34:10 (11288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:48:32 (4036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:50:18 (1448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:21:38 (1940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:45:40 (6660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:23:57 (5792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:24:39 (5016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:05:16 (10116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:43:12 (10392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:44:08 (12176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:21:32 (10220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:50:36 (6076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:12:47 (8936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:13:34 (10556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:13:35 (10556): No heartbeat from core client for 30 sec - exiting 21:13:36 (10556): No heartbeat from core client for 30 sec - exiting 21:13:37 (10556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 11:47:29 (4524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:53:23 (19988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:31:40 (4308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:34:06 (9012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:01:23 (4588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:24:03 (18660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:17:52 (13044): No heartbeat from core client for 30 sec - exiting 22:17:53 (13044): No heartbeat from core client for 30 sec - exiting 22:17:54 (13044): No heartbeat from core client for 30 sec - exiting 22:17:55 (13044): No heartbeat from core client for 30 sec - exiting 22:17:56 (13044): No heartbeat from core client for 30 sec - exiting 22:17:57 (13044): No heartbeat from core client for 30 sec - exiting 22:17:58 (13044): No heartbeat from core client for 30 sec - exiting 22:17:59 (13044): No heartbeat from core client for 30 sec - exiting 22:18:00 (13044): No heartbeat from core client for 30 sec - exiting 22:18:01 (13044): No heartbeat from core client for 30 sec - exiting 22:18:02 (13044): No heartbeat from core client for 30 sec - exiting 22:18:03 (13044): No heartbeat from core client for 30 sec - exiting 22:18:04 (13044): No heartbeat from core client for 30 sec - exiting 22:18:05 (13044): No heartbeat from core client for 30 sec - exiting 22:18:06 (13044): No heartbeat from core client for 30 sec - exiting 22:18:07 (13044): No heartbeat from core client for 30 sec - exiting 22:18:08 (13044): No heartbeat from core client for 30 sec - exiting 22:18:09 (13044): No heartbeat from core client for 30 sec - exiting 22:18:10 (13044): No heartbeat from core client for 30 sec - exiting 22:18:11 (13044): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812AFB Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 07/12/11 22:18:11 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : CPDN Monitor - No 'heartbeat' from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812AFB Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 07/12/11 22:19:02 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : Loaded Library : C:\Program Files\BOINC\\dbghelp.dll Loaded Library : C:\Program Files\BOINC\\symsrv.dll Loaded Library : C:\Program Files\BOINC\\srcsrv.dll LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_p7fc_1900_40_007227208;C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 7c940000 0009f000 C:\WINDOWS\system32\ntdll.dll (5.1.2600.6055) (-exported- Symbols Loaded) Linked PDB Filename : ntdll.pdb File Version : 5.1.2600.6055 (xpsp_sp3_gdr.101209-1647) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.6055 ModLoad: 7c800000 00133000 C:\WINDOWS\system32\kernel32.dll (5.1.2600.5781) (-exported- Symbols Loaded) Linked PDB Filename : kernel32.pdb File Version : 5.1.2600.5781 (xpsp_sp3_gdr.090321-1317) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.5781 ModLoad: 77cf0000 00090000 C:\WINDOWS\system32\USER32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : user32.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.5512 ModLoad: 77ed0000 00049000 C:\WINDOWS\system32\GDI32.dll (5.1.2600.5698) (-exported- Symbols Loaded) Linked PDB Filename : gdi32.pdb File Version : 5.1.2600.5698 (xpsp_sp3_gdr.081022-1932) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5698 ModLoad: 77d80000 000a9000 C:\WINDOWS\system32\ADVAPI32.dll (5.1.2600.5755) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 5.1.2600.5755 (xpsp_sp3_gdr.090206-1234) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.5755 ModLoad: 77e30000 00093000 C:\WINDOWS\system32\RPCRT4.dll (5.1.2600.6022) (-exported- Symbols Loaded) Linked PDB Filename : rpcrt4.pdb File Version : 5.1.2600.6022 (xpsp_sp3_gdr.100813-1643) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.6022 ModLoad: 77fa0000 00011000 C:\WINDOWS\system32\Secur32.dll (5.1.2600.5834) (-exported- Symbols Loaded) Linked PDB Filename : secur32.pdb File Version : 5.1.2600.5834 (xpsp_sp3_gdr.090624-1305) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5834 ModLoad: 5c8d0000 000cc000 C:\WINDOWS\system32\OPENGL32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : opengl32.pdb File Version : 5.1.2600.5512 (xpsp.080413-0845) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5512 ModLoad: 77bc0000 00058000 C:\WINDOWS\system32\msvcrt.dll (7.0.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.2600.5512 (xpsp.080413-2111) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 7.0.2600.5512 ModLoad: 679f0000 00020000 C:\WINDOWS\system32\GLU32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : glu32.pdb File Version : 5.1.2600.5512 (xpsp.080413-0845) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.5512 ModLoad: 736b0000 0004b000 C:\WINDOWS\system32\DDRAW.dll (5.3.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : ddraw.pdb File Version : 5.03.2600.5512 (xpsp.080413-0845) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.03.2600.5512 ModLoad: 73b10000 00006000 C:\WINDOWS\system32\DCIMAN32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : dciman32.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5512 ModLoad: 76c40000 00028000 C:\WINDOWS\system32\imagehlp.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5512 ModLoad: 762e0000 0001d000 C:\WINDOWS\system32\IMM32.DLL (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : imm32.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5512 ModLoad: 60740000 00009000 C:\WINDOWS\system32\LPK.DLL (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : lpk.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5512 ModLoad: 73f80000 0006b000 C:\WINDOWS\system32\USP10.dll (1.420.2600.5969) (-exported- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0420.2600.5969 (xpsp_sp3_gdr.100416-1716) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0420.2600.5969 ModLoad: 33ae0000 00115000 C:\Program Files\BOINC\dbghelp.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 33d00000 00048000 C:\Program Files\BOINC\symsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : symsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 33d50000 0003b000 C:\Program Files\BOINC\srcsrv.dll (6.8.4.0) (-nosymbols- Symbols Loaded) Linked PDB Filename : srcsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 77bb0000 00008000 C:\WINDOWS\system32\version.dll (5.1.2600.5512) (-nosymbols- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5512 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 4, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 60, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 65660, QuotaPeakPagedPoolUsage: 65660 QuotaNonPagedPoolUsage: 2296, QuotaPeakNonPagedPoolUsage: 2296 - Virtual Memory Usage - VirtualSize: 873959424, PeakVirtualSize: 873959424 - Pagefile Usage - PagefileUsage: 32833536, PeakPagefileUsage: 106201088 - Working Set Size - WorkingSetSize: 2543616, PeakWorkingSetSize: 2543616, PageFaultCount: 616 * Dump of thread ID 14464 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 156250.000000, User Time: 156250.000000, Wait Time: 2594459.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812AFB - Registers - eax=1132fa04 ebx=00000001 ecx=00000000 edx=009fce98 esi=1132fa8c edi=fffffffe eip=7c812afb esp=1132fa00 ebp=1132fa54 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 1132fa54 007bfa90 e06d7363 00000001 00000003 1132fa80 kernel32!RaiseException+0x0 1132fa8c 007bbbcb 1132fa9c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132faa8 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 1132fb10 00408130 1132fb24 000523af 1132fb24 00052311 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ffc0 7c817077 00000000 0012f100 7ffdf000 e06d7363 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fff0 00000000 007bde7f 00000000 00000000 00000000 kernel32!RegisterWaitForInputIdle+0x0 * Dump of thread ID 15956 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 2594456.000000 - Registers - eax=002160ec ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=325bff70 eip=7c94e514 esp=325bff40 ebp=325bff98 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 325bff98 7c802455 00000064 00000000 325bffb4 00749924 ntdll!KiFastSystemCallRet+0x0 325bffa8 00749924 00000064 325bffec 7c80b729 00000000 kernel32!Sleep+0x0 325bffb4 7c80b729 00000000 735c6c61 625f6d68 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 325bffec 00000000 00749910 00000000 00000000 32ad0000 kernel32!GetModuleFileNameA+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=308, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812AFB Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 07/12/11 22:20:03 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : Loaded Library : C:\Program Files\BOINC\\dbghelp.dll Loaded Library : C:\Program Files\BOINC\\symsrv.dll Loaded Library : C:\Program Files\BOINC\\srcsrv.dll LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_p7fc_1900_40_007227208;C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 7c940000 0009f000 C:\WINDOWS\system32\ntdll.dll (5.1.2600.6055) (-exported- Symbols Loaded) Linked PDB Filename : ntdll.pdb File Version : 5.1.2600.6055 (xpsp_sp3_gdr.101209-1647) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.6055 ModLoad: 7c800000 00133000 C:\WINDOWS\system32\kernel32.dll (5.1.2600.5781) (-exported- Symbols Loaded) Linked PDB Filename : kernel32.pdb File Version : 5.1.2600.5781 (xpsp_sp3_gdr.090321-1317) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.5781 ModLoad: 77cf0000 00090000 C:\WINDOWS\system32\USER32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : user32.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.5512 ModLoad: 77ed0000 00049000 C:\WINDOWS\system32\GDI32.dll (5.1.2600.5698) (-exported- Symbols Loaded) Linked PDB Filename : gdi32.pdb File Version : 5.1.2600.5698 (xpsp_sp3_gdr.081022-1932) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5698 ModLoad: 77d80000 000a9000 C:\WINDOWS\system32\ADVAPI32.dll (5.1.2600.5755) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 5.1.2600.5755 (xpsp_sp3_gdr.090206-1234) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.5755 ModLoad: 77e30000 00093000 C:\WINDOWS\system32\RPCRT4.dll (5.1.2600.6022) (-exported- Symbols Loaded) Linked PDB Filename : rpcrt4.pdb File Version : 5.1.2600.6022 (xpsp_sp3_gdr.100813-1643) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.6022 ModLoad: 77fa0000 00011000 C:\WINDOWS\system32\Secur32.dll (5.1.2600.5834) (-exported- Symbols Loaded) Linked PDB Filename : secur32.pdb File Version : 5.1.2600.5834 (xpsp_sp3_gdr.090624-1305) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5834 ModLoad: 5c8d0000 000cc000 C:\WINDOWS\system32\OPENGL32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : opengl32.pdb File Version : 5.1.2600.5512 (xpsp.080413-0845) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5512 ModLoad: 77bc0000 00058000 C:\WINDOWS\system32\msvcrt.dll (7.0.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.2600.5512 (xpsp.080413-2111) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 7.0.2600.5512 ModLoad: 679f0000 00020000 C:\WINDOWS\system32\GLU32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : glu32.pdb File Version : 5.1.2600.5512 (xpsp.080413-0845) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.5512 ModLoad: 736b0000 0004b000 C:\WINDOWS\system32\DDRAW.dll (5.3.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : ddraw.pdb File Version : 5.03.2600.5512 (xpsp.080413-0845) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.03.2600.5512 ModLoad: 73b10000 00006000 C:\WINDOWS\system32\DCIMAN32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : dciman32.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5512 ModLoad: 76c40000 00028000 C:\WINDOWS\system32\imagehlp.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5512 ModLoad: 762e0000 0001d000 C:\WINDOWS\system32\IMM32.DLL (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : imm32.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5512 ModLoad: 60740000 00009000 C:\WINDOWS\system32\LPK.DLL (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : lpk.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5512 ModLoad: 73f80000 0006b000 C:\WINDOWS\system32\USP10.dll (1.420.2600.5969) (-exported- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0420.2600.5969 (xpsp_sp3_gdr.100416-1716) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0420.2600.5969 ModLoad: 33ae0000 00115000 C:\Program Files\BOINC\dbghelp.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 33d00000 00048000 C:\Program Files\BOINC\symsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : symsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 33d50000 0003b000 C:\Program Files\BOINC\srcsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : srcsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 77bb0000 00008000 C:\WINDOWS\system32\version.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 5.1.2600.5512 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 5, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 61, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 65660, QuotaPeakPagedPoolUsage: 65660 QuotaNonPagedPoolUsage: 2296, QuotaPeakNonPagedPoolUsage: 2296 - Virtual Memory Usage - VirtualSize: 873959424, PeakVirtualSize: 873959424 - Pagefile Usage - PagefileUsage: 32833536, PeakPagefileUsage: 106201088 - Working Set Size - WorkingSetSize: 2547712, PeakWorkingSetSize: 2547712, PageFaultCount: 617 * Dump of thread ID 14936 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 312500.000000, User Time: 0.000000, Wait Time: 2598331.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812AFB - Registers - eax=1132fa04 ebx=00000001 ecx=00000000 edx=009fce98 esi=1132fa8c edi=fffffffe eip=7c812afb esp=1132fa00 ebp=1132fa54 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 1132fa54 007bfa90 e06d7363 00000001 00000003 1132fa80 kernel32!RaiseException+0x0 1132fa8c 007bbbcb 1132fa9c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132faa8 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 1132fb10 00408130 1132fb24 000523af 1132fb24 00052311 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ffc0 7c817077 00000000 011801c0 7ffdf000 e06d7363 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fff0 00000000 007bde7f 00000000 00000000 00000000 kernel32!RegisterWaitForInputIdle+0x0 * Dump of thread ID 14536 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 2598330.000000 - Registers - eax=0000003c ebx=00000000 ecx=00000069 edx=00000001 esi=00000000 edi=325bff70 eip=7c94e514 esp=325bff40 ebp=325bff98 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 325bff98 7c802455 00000064 00000000 325bffb4 00749924 ntdll!KiFastSystemCallRet+0x0 325bffa8 00749924 00000064 325bffec 7c80b729 00000000 kernel32!Sleep+0x0 325bffb4 7c80b729 00000000 735c6c61 625f6d68 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 325bffec 00000000 00749910 00000000 00000000 32ad0000 kernel32!GetModuleFileNameA+0x0 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=308, iMonCtr=1 Model crash detected, will try to restart... Could not launch model process. Last Error=1455 Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Jul 2011 13:01:42	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	699,840	2,473,960	3.5350
10 Jul 2011 08:11:28	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	673,920	2,402,890	3.5655
09 Jul 2011 10:25:55	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	648,000	2,332,465	3.5995
08 Jul 2011 14:45:22	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	622,080	2,263,470	3.6386
07 Jul 2011 15:59:29	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	596,160	2,194,065	3.6803
05 Jul 2011 17:15:42	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	570,240	2,126,486	3.7291
02 Jul 2011 23:55:25	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	544,320	2,058,014	3.7809
01 Jul 2011 21:18:13	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	518,400	1,989,235	3.8373
28 Jun 2011 17:17:25	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	492,480	1,941,608	3.9425
28 Jun 2011 00:07:59	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	466,560	1,904,256	4.0815
27 Jun 2011 12:55:10	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	440,640	1,866,874	4.2367
27 Jun 2011 01:46:56	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	414,720	1,828,668	4.4094
26 Jun 2011 14:11:24	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	388,800	1,791,105	4.6068
26 Jun 2011 02:19:10	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	362,880	1,754,209	4.8341
25 Jun 2011 00:08:55	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	336,960	1,717,340	5.0966
01 Jun 2011 22:32:27	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	311,040	811,090	2.6077
31 May 2011 18:53:20	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	285,120	743,146	2.6064
30 May 2011 19:18:31	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	259,200	676,468	2.6098
29 May 2011 22:15:32	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	233,280	609,554	2.6130
28 May 2011 23:19:43	1037994	12898714	hadcm3n_p7fc_1900_40_007227208_2	207,360	541,417	2.6110