Task 15827595

Name	hadcm3n_3b3k_2020_40_008390232_0
Workunit	8541091
Created	4 Jun 2013, 3:20:11 UTC
Sent	4 Jun 2013, 10:23:51 UTC
Report deadline	3 Sep 2013, 17:51:02 UTC
Received	2 Dec 2013, 20:50:18 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1269200
Run time	12 days 2 hours 25 min 45 sec
CPU time	11 days 22 hours 46 min 57 sec
Validate state	Invalid
Credit	6,531.84
Device peak FLOPS	2.69 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:24:37 (24898): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:26:19 (5221): No heartbeat from core client for 30 sec - exiting 16:26:21 (5221):CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting 16:27:21 (5221): No heartbeat from core client for 30 sec - exiting 16:27:22 (5221): No heartbeat from core client for 30 sec - exiting 16:27:23 (5221): No heartbeat from core client for 30 sec - exiting 16:27:24 (5221): No heartbeat from core client for 30 sec - exiting 16:27:25 (5221): No heartbeat from core client for 30 sec - exiting 16:27:26 (5221): No heartbeat from core client for 30 sec - exiting 16:27:27 (5221): No heartbeat from core client for 30 sec - exiting 16:29:44 (5291): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:29:45 (5291): No heartbeat from core client for 30 sec - exiting 16:29:46 (5291): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 16:34:10 (5367): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:38:33 (5409): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:42:21 (5443): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:45:11 (5485): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:52:17 (5540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:52:18 (5540): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 16:55:33 (5616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:55:43 (5616): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 16:59:42 (5651): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:02:06 (5725): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:12:22 (5767): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:13:07 (5849): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:16:42 (5892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:16:43 (5892): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 17:40:25 (5971): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:59:35 (6132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:07:43 (6300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:15:32 (6360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:18:53 (6440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:19:28 (6500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:28:42 (6560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:29:23 (6607): No heartbeat from core client for 30 sec - exiting 18:30:22 (6607): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:43:55 (25576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:48:26 (18529): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:53:34 (18593): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:56:56 (18645): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:57:11 (18645): No heartbeat from core client for 30 sec - exiting 20:57:14 (18645): No heartbeat from core client for 30 sec - exiting 20:57:15 (18645): No heartbeat from core client for 30 sec - exiting 20:57:16 (18645): No heartbeat from core client for 30 sec - exiting 20:57:17 (18645): No heartbeat from core client for 30 sec - exiting 20:57:18 (18645): No heartbeat from core client for 30 sec - exiting 20:57:19 (18645): No heartbeat from core client for 30 sec - exiting 20:57:20 (18645): No heartbeat from core client for 30 sec - exiting 20:57:21 (18645): No heartbeat from core client for 30 sec - exiting 20:57:22 (18645): No heartbeat from core client for 30 sec - exiting 20:57:23 (18645): No heartbeat from core client for 30 sec - exiting 20:57:24 (18645): No heartbeat from core client for 30 sec - exiting 20:57:25 (18645): No heartbeat from core client for 30 sec - exiting 20:57:26 (18645): No heartbeat from core client for 30 sec - exiting 20:57:27 (18645): No heartbeat from core client for 30 sec - exiting 20:57:28 (18645): No heartbeat from core client for 30 sec - exiting 20:57:29 (18645): No heartbeat from core client for 30 sec - exiting 20:57:30 (18645): No heartbeat from core client for 30 sec - exiting 20:57:31 (18645): No heartbeat from core client for 30 sec - exiting 20:57:32 (18645): No heartbeat from core client for 30 sec - exiting 20:57:33 (18645): No heartbeat from core client for 30 sec - exiting 20:57:34 (18645): No heartbeat from core client for 30 sec - exiting 20:57:35 (18645): No heartbeat from core client for 30 sec - exiting 20:57:36 (18645): No heartbeat from core client for 30 sec - exiting 20:57:37 (18645): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:01:41 (18704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:01:43 (18704): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 21:05:14 (18770): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:29:13 (7493): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:30:07 (29460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:35:00 (29525): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:40:15 (29695): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:16:16 (29783): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:20:28 (30617): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 02:23:09 (30667): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:24:03 (30667): No heartbeat from core client for 30 sec - exiting 02:24:09 (30667): No heartbeat from core client for 30 sec - exiting 02:24:10 (30667): No heartbeat from core client for 30 sec - exiting 02:24:11 (30667): No heartbeat from core client for 30 sec - exiting 02:24:12 (30667): No heartbeat from core client for 30 sec - exiting 02:24:13 (30667): No heartbeat from core client for 30 sec - exiting 02:24:14 (30667): No heartbeat from core client for 30 sec - exiting 02:24:15 (30667): No heartbeat from core client for 30 sec - exiting 02:24:16 (30667): No heartbeat from core client for 30 sec - exiting 02:24:17 (30667): No heartbeat from core client for 30 sec - exiting 02:24:18 (30667): No heartbeat from core client for 30 sec - exiting 02:24:19 (30667): No heartbeat from core client for 30 sec - exiting 02:24:20 (30667): No heartbeat from core client for 30 sec - exiting 02:24:21 (30667): No heartbeat from core client for 30 sec - exiting 02:24:22 (30667): No heartbeat from core client for 30 sec - exiting 02:24:23 (30667): No heartbeat from core client for 30 sec - exiting 02:24:24 (30667): No heartbeat from core client for 30 sec - exiting 02:24:25 (30667): No heartbeat from core client for 30 sec - exiting 02:24:26 (30667): No heartbeat from core client for 30 sec - exiting 02:24:27 (30667): No heartbeat from core client for 30 sec - exiting 02:24:28 (30667): No heartbeat from core client for 30 sec - exiting 02:24:29 (30667): No heartbeat from core client for 30 sec - exiting 02:24:30 (30667): No heartbeat from core client for 30 sec - exiting 02:24:31 (30667): No heartbeat from core client for 30 sec - exiting 02:24:32 (30667): No heartbeat from core client for 30 sec - exiting 02:24:33 (30667): No heartbeat from core client for 30 sec - exiting 02:24:34 (30667): No heartbeat from core client for 30 sec - exiting 02:25:43 (30751): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:25:45 (30751): No heartbeat from core client for 30 sec - exiting 02:25:46 (30751): No heartbeat from core client for 30 sec - exiting 02:25:47 (30751): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 02:29:46 (30767): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:29:47 (30767): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 02:34:07 (31034): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:34:09 (31034): No heartbeat from core client for 30 sec - exiting 02:35:55 (31077): No heartbeat from core client for 30 sec - exiting 02:35:57 (31077): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 2 received, exiting... Signal 2 received, exiting... Called boinc_finish 18:20:12 (9938): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:08:18 (31527): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:25:06 (31931): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:47:01 (32081): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:47:37 (32282): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:47:38 (32282): No heartbeat from core client for 30 sec - exiting 19:47:39 (32282): No heartbeat from core client for 30 sec - exiting 19:47:40 (32282): No heartbeat from core client for 30 sec - exiting 19:47:41 (32282): No heartbeat from core client for 30 sec - exiting 19:47:42 (32282): No heartbeat from core client for 30 sec - exiting 19:47:43 (32282): No heartbeat from core client for 30 sec - exiting 19:47:44 (32282): No heartbeat from core client for 30 sec - exiting 19:47:45 (32282): No heartbeat from core client for 30 sec - exiting 19:47:46 (32282): No heartbeat from core client for 30 sec - exiting 19:47:47 (32282): No heartbeat from core client for 30 sec - exiting 19:47:48 (32282): No heartbeat from core client for 30 sec - exiting 19:47:49 (32282): No heartbeat from core client for 30 sec - exiting 19:47:50 (32282): No heartbeat from core client for 30 sec - exiting 19:47:51 (32282): No heartbeat from core client for 30 sec - exiting 19:47:52 (32282): No heartbeat from core client for 30 sec - exiting 19:47:53 (32282): No heartbeat from core client for 30 sec - exiting 19:47:54 (32282): No heartbeat from core client for 30 sec - exiting 19:47:55 (32282): No heartbeat from core client for 30 sec - exiting 19:47:56 (32282): No heartbeat from core client for 30 sec - exiting 19:47:57 (32282): No heartbeat from core client for 30 sec - exiting 20:05:44 (32300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:08:52 (32699): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:45:13 (32728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:17:30 (607): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:30:46 (891): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:40:03 (1016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:43:56 (1093): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:45:30 (1122): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:45:31 (1122): No heartbeat from core client for 30 sec - exiting 21:45:32 (1122): No heartbeat from core client for 30 sec - exiting 21:45:33 (1122): No heartbeat from core client for 30 sec - exiting 21:45:34 (1122): No heartbeat from core client for 30 sec - exiting 21:45:35 (1122): No heartbeat from core client for 30 sec - exiting 21:45:36 (1122): No heartbeat from core client for 30 sec - exiting 21:45:37 (1122): No heartbeat from core client for 30 sec - exiting 21:45:38 (1122): No heartbeat from core client for 30 sec - exiting 21:45:39 (1122): No heartbeat from core client for 30 sec - exiting 21:45:40 (1122): No heartbeat from core client for 30 sec - exiting 21:45:41 (1122): No heartbeat from core client for 30 sec - exiting 21:45:42 (1122): No heartbeat from core client for 30 sec - exiting 21:45:43 (1122): No heartbeat from core client for 30 sec - exiting 21:45:44 (1122): No heartbeat from core client for 30 sec - exiting 21:45:45 (1122): No heartbeat from core client for 30 sec - exiting 21:45:46 (1122): No heartbeat from core client for 30 sec - exiting 21:45:47 (1122): No heartbeat from core client for 30 sec - exiting 21:45:48 (1122): No heartbeat from core client for 30 sec - exiting 21:45:49 (1122): No heartbeat from core client for 30 sec - exiting SIGABRT: abort called Stack trace (9 frames): /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf773d400] linux-gate.so.1(__kernel_vsyscall+0x5)[0xf773d425] /lib/libc.so.6(gsignal+0x4f)[0xf757731f] /lib/libc.so.6(abort+0x143)[0xf7578c03] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf5)[0xf75623d5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1224, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7718400] linux-gate.so.1(__kernel_vsyscall+0x5)[0xf7718425] /lib/libc.so.6(gsignal+0x4f)[0xf755231f] /lib/libc.so.6(abort+0x143)[0xf7553c03] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf5)[0xf753d3d5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1224, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77c4400] linux-gate.so.1(__kernel_vsyscall+0x5)[0xf77c4425] /lib/libc.so.6(gsignal+0x4f)[0xf75fe31f] /lib/libc.so.6(abort+0x143)[0xf75ffc03] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf5)[0xf75e93d5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1224, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7721400] linux-gate.so.1(__kernel_vsyscall+0x5)[0xf7721425] /lib/libc.so.6(gsignal+0x4f)[0xf755b31f] /lib/libc.so.6(abort+0x143)[0xf755cc03] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf5)[0xf75463d5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1224, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7784400] linux-gate.so.1(__kernel_vsyscall+0x5)[0xf7784425] /lib/libc.so.6(gsignal+0x4f)[0xf75be31f] /lib/libc.so.6(abort+0x143)[0xf75bfc03] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf5)[0xf75a93d5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1224, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7799400] linux-gate.so.1(__kernel_vsyscall+0x5)[0xf7799425] /lib/libc.so.6(gsignal+0x4f)[0xf75d331f] /lib/libc.so.6(abort+0x143)[0xf75d4c03] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/cognac/bin/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf5)[0xf75be3d5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1224, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Dec 2013 08:48:55	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	544,320	998,784	1.8349
29 Nov 2013 12:59:53	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	518,400	952,098	1.8366
28 Nov 2013 20:16:25	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	492,480	906,055	1.8398
28 Nov 2013 07:22:04	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	466,560	860,069	1.8434
17 Oct 2013 15:51:14	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	440,640	812,377	1.8436
17 Oct 2013 01:54:58	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	414,720	762,517	1.8386
16 Oct 2013 12:00:57	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	388,800	712,964	1.8338
15 Oct 2013 22:16:21	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	362,880	664,110	1.8301
15 Oct 2013 08:25:44	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	336,960	614,763	1.8244
14 Oct 2013 18:17:19	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	311,040	564,196	1.8139
10 Jul 2013 14:22:05	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	285,120	516,478	1.8114
06 Jul 2013 18:00:10	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	259,200	469,812	1.8125
06 Jul 2013 05:02:52	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	233,280	423,454	1.8152
06 Jul 2013 04:19:48	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	207,360	377,921	1.8225
26 Jun 2013 16:54:02	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	181,440	331,335	1.8261
26 Jun 2013 03:47:23	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	155,520	283,716	1.8243
07 Jun 2013 04:33:40	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	129,600	236,124	1.8219
06 Jun 2013 15:22:16	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	103,680	188,790	1.8209
06 Jun 2013 02:08:17	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	77,760	141,549	1.8203
05 Jun 2013 12:52:12	1269200	15827595	hadcm3n_3b3k_2020_40_008390232_0	51,840	94,312	1.8193