Task 13956066

Name	hadcm3n_yf1i_1940_40_007693766_1
Workunit	7848874
Created	23 Jan 2012, 18:47:07 UTC
Sent	23 Jan 2012, 18:47:16 UTC
Report deadline	24 Apr 2012, 2:14:27 UTC
Received	22 Feb 2012, 19:59:37 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	859190
Run time	17 days 13 hours 9 min 48 sec
CPU time	15 days 23 hours 53 min 13 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.01 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.17</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Signal 15 received, exiting... Called boinc_finish 10:34:36 (2651): No heartbeat from core client for 30 sec - exiting 10:34:37 (2651): No heartbeat from core client for 30 sec - exiting 10:34:38 (2651): No heartbeat from core client for 30 sec - exiting 10:34:39 (2651): No heartbeat from core client for 30 sec - exiting 10:34:40 (2651): No heartbeat from core client for 30 sec - exiting 10:34:41 (2651): No heartbeat from core client for 30 sec - exiting 10:34:42 (2651): No heartbeat from core client for 30 sec - exiting 10:34:43 (2651): No heartbeat from core client for 30 sec - exiting 10:34:44 (2651): No heartbeat from core client for 30 sec - exiting 10:34:45 (2651): No heartbeat from core client for 30 sec - exiting 10:34:46 (2651): No heartbeat from core client for 30 sec - exiting 10:34:47 (2651): No heartbeat from core client for 30 sec - exiting 10:34:48 (2651): No heartbeat from core client for 30 sec - exiting 10:34:49 (2651): No heartbeat from core client for 30 sec - exiting 10:34:50 (2651): No heartbeat from core client for 30 sec - exiting 10:34:51 (2651): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... Called boinc_finish 12:38:47 (2681): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish 07:44:37 (3598): No heartbeat from core client for 30 sec - exiting 07:44:39 (3598): No heartbeat from core client for 30 sec - exiting 07:44:40 (3598): No heartbeat from core client for 30 sec - exiting 07:44:41 (3598): No heartbeat from core client for 30 sec - exiting 07:44:42 (3598): No heartbeat from core client for 30 sec - exiting 07:44:43 (3598): No heartbeat from core client for 30 sec - exiting 07:44:44 (3598): No heartbeat from core client for 30 sec - exiting 07:44:45 (3598): No heartbeat from core client for 30 sec - exiting 07:44:46 (3598): No heartbeat from core client for 30 sec - exiting 07:44:47 (3598): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Signal 15 received, exiting... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish 03:04:22 (2662): No heartbeat from core client for 30 sec - exiting 03:04:23 (2662): No heartbeat from core client for 30 sec - exiting 03:04:24 (2662): No heartbeat from core client for 30 sec - exiting 03:04:25 (2662): No heartbeat from core client for 30 sec - exiting 03:04:26 (2662): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... Called boinc_finish 04:08:19 (2398): No heartbeat from core client for 30 sec - exiting 04:08:20 (2398): No heartbeat from core client for 30 sec - exiting 04:08:21 (2398): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish 10:36:27 (2725): No heartbeat from core client for 30 sec - exiting 10:36:28 (2725): No heartbeat from core client for 30 sec - exiting 10:36:29 (2725): No heartbeat from core client for 30 sec - exiting 10:36:30 (2725): No heartbeat from core client for 30 sec - exiting 10:36:31 (2725): No heartbeat from core client for 30 sec - exiting 10:36:32 (2725): No heartbeat from core client for 30 sec - exiting 10:36:33 (2725): No heartbeat from core client for 30 sec - exiting 10:36:34 (2725): No heartbeat from core client for 30 sec - exiting 10:36:35 (2725): No heartbeat from core client for 30 sec - exiting 10:36:36 (2725): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish 04:50:49 (2542): No heartbeat from core client for 30 sec - exiting 04:50:50 (2542): No heartbeat from core client for 30 sec - exiting 04:50:51 (2542): No heartbeat from core client for 30 sec - exiting 04:50:52 (2542): No heartbeat from core client for 30 sec - exiting 04:50:53 (2542): No heartbeat from core client for 30 sec - exiting 04:50:54 (2542): No heartbeat from core client for 30 sec - exiting 04:50:55 (2542): No heartbeat from core client for 30 sec - exiting 04:50:56 (2542): No heartbeat from core client for 30 sec - exiting 04:50:57 (2542): No heartbeat from core client for 30 sec - exiting 04:50:58 (2542): No heartbeat from core client for 30 sec - exiting 04:50:59 (2542): No heartbeat from core client for 30 sec - exiting 04:51:00 (2542): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 1 received, exiting... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:31:43 (2689): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:31:44 (2689): No heartbeat from core client for 30 sec - exiting 07:31:45 (2689): No heartbeat from core client for 30 sec - exiting 07:31:46 (2689): No heartbeat from core client for 30 sec - exiting 07:31:47 (2689): No heartbeat from core client for 30 sec - exiting Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish 05:20:15 (2544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:23:27 (2736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish 10:29:53 (3200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:38:47 (3129): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish 10:40:15 (2413): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (29 frames): /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xb782a400] /lib/tls/i686/cmov/libc.so.6(cfree+0x31)[0xb76f9e91] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f6a51] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f6a7d] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839df50] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83a5fb2] [0xb782a400] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f6a51] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f6a7d] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839df50] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83a5fb2] [0xb782a400] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8068d89] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8063b30] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81ca51e] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8309a29] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x828d3b0] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8090808] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x808a417] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8299901] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x821c545] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8219da1] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838720b] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83995c6] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb76a0bd6] /home/hans/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Signal 15 received, exiting... Called boinc_finish 15:02:20 (2876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:02:22 (2876): No heartbeat from core client for 30 sec - exiting 15:02:23 (2876): No heartbeat from core client for 30 sec - exiting 15:02:24 (2876): No heartbeat from core client for 30 sec - exiting 15:02:25 (2876): No heartbeat from core client for 30 sec - exiting 15:02:26 (2876): No heartbeat from core client for 30 sec - exiting 15:02:27 (2876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish 10:43:05 (3049): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... Called boinc_finish 13:10:12 (3101): No heartbeat from core client for 30 sec - exiting 13:10:13 (3101): No heartbeat from core client for 30 sec - exiting 13:10:14 (3101): No heartbeat from core client for 30 sec - exiting 13:10:15 (3101): No heartbeat from core client for 30 sec - exiting 13:10:16 (3101): No heartbeat from core client for 30 sec - exiting 13:10:17 (3101): No heartbeat from core client for 30 sec - exiting 13:10:18 (3101): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish 10:14:35 (2886): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... Called boinc_finish 14:09:32 (3164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish 15:23:09 (2487): No heartbeat from core client for 30 sec - exiting 15:23:10 (2487): No heartbeat from core client for 30 sec - exiting 15:23:11 (2487): No heartbeat from core client for 30 sec - exiting 15:23:12 (2487): No heartbeat from core client for 30 sec - exiting 15:23:13 (2487): No heartbeat from core client for 30 sec - exiting 15:23:14 (2487): No heartbeat from core client for 30 sec - exiting 15:23:15 (2487): No heartbeat from core client for 30 sec - exiting 15:23:16 (2487): No heartbeat from core client for 30 sec - exiting 15:23:17 (2487): No heartbeat from core client for 30 sec - exiting 15:23:18 (2487): No heartbeat from core client for 30 sec - exiting 15:23:19 (2487): No heartbeat from core client for 30 sec - exiting 15:23:20 (2487): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 1 received, exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4524, iMonCtr=1 Model crash detected, will try to restart... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish 08:41:42 (2780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish 10:26:24 (2817): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... Called boinc_finish * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09514a30 *** ======= Backtrace: ========= /lib/tls/i686/cmov/libc.so.6(+0x6b591)[0xb760f591] /lib/tls/i686/cmov/libc.so.6(+0x6cde8)[0xb7610de8] /lib/tls/i686/cmov/libc.so.6(cfree+0x6d)[0xb7613ecd] /usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0xb77fe741] /usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0xb77fe79d] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb75babd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 08:05 21185851 /home/hans/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rwxp 0009b000 08:05 21185851 /home/hans/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rwxp 00000000 00:00 0 094bf000-09525000 rwxp 00000000 00:00 0 [heap] b7000000-b7021000 rwxp 00000000 00:00 0 b7021000-b7100000 ---p 00000000 00:00 0 b7127000-b75a2000 rwxs 00000000 08:05 23183473 /home/hans/BOINC/slots/5/137830 b75a2000-b75a4000 rwxp 00000000 00:00 0 b75a4000-b76f7000 r-xp 00000000 08:05 5660682 /lib/tls/i686/cmov/libc-2.11.1.so b76f7000-b76f8000 ---p 00153000 08:05 5660682 /lib/tls/i686/cmov/libc-2.11.1.so b76f8000-b76fa000 r-xp 00153000 08:05 5660682 /lib/tls/i686/cmov/libc-2.11.1.so b76fa000-b76fb000 rwxp 00155000 08:05 5660682 /lib/tls/i686/cmov/libc-2.11.1.so b76fb000-b76fe000 rwxp 00000000 00:00 0 b76fe000-b771b000 r-xp 00000000 08:05 5636179 /lib/libgcc_s.so.1 b771b000-b771c000 r-xp 0001c000 08:05 5636179 /lib/libgcc_s.so.1 b771c000-b771d000 rwxp 0001d000 08:05 5636179 /lib/libgcc_s.so.1 b771d000-b7741000 r-xp 00000000 08:05 5660690 /lib/tls/i686/cmov/libm-2.11.1.so b7741000-b7742000 r-xp 00023000 08:05 5660690 /lib/tls/i686/cmov/libm-2.11.1.so b7742000-b7743000 rwxp 00024000 08:05 5660690 /lib/tls/i686/cmov/libm-2.11.1.so b7743000-b782c000 r-xp 00000000 08:05 11110936 /usr/lib/libstdc++.so.6.0.13 b782c000-b782d000 ---p 000e9000 08:05 11110936 /usr/lib/libstdc++.so.6.0.13 b782d000-b7831000 r-xp 000e9000 08:05 11110936 /usr/lib/libstdc++.so.6.0.13 b7831000-b7832000 rwxp 000ed000 08:05 11110936 /usr/lib/libstdc++.so.6.0.13 b7832000-b783a000 rwxp 00000000 00:00 0 b783a000-b783c000 r-xp 00000000 08:05 5660688 /lib/tls/i686/cmov/libdl-2.11.1.so b783c000-b783d000 r-xp 00001000 08:05 5660688 /lib/tls/i686/cmov/libdl-2.11.1.so b783d000-b783e000 rwxp 00002000 08:05 5660688 /lib/tls/i686/cmov/libdl-2.11.1.so b783e000-b7853000 r-xp 00000000 08:05 5660708 /lib/tls/i686/cmov/libpthread-2.11.1.so b7853000-b7854000 r-xp 00014000 08:05 5660708 /lib/tls/i686/cmov/libpthread-2.11.1.so b7854000-b7855000 rwxp 00015000 08:05 5660708 /lib/tls/i686/cmov/libpthread-2.11.1.so b7855000-b7857000 rwxp 00000000 00:00 0 b7868000-b7869000 rwxp 00000000 00:00 0 b7869000-b786a000 ---p 00000000 00:00 0 b786a000-b786d000 rwxp 00000000 00:00 0 b786d000-b786f000 rwxs 00000000 08:05 23183382 /home/hans/BOINC/slots/5/boinc_mmap_file b786f000-b7871000 rwxp 00000000 00:00 0 b7871000-b7872000 r-xp 00000000 00:00 0 [vdso] b7872000-b788d000 r-xp 00000000 08:05 5636121 /lib/ld-2.11.1.so b788d000-b788e000 r-xp 0001a000 08:05 5636121 /lib/ld-2.11.1.so b788e000-b788f000 rwxp 0001b000 08:05 5636121 /lib/ld-2.11.1.so bfb42000-bfbb2000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (19 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xb7871400] [0xb7871430] /lib/tls/i686/cmov/libc.so.6(gsignal+0x51)[0xb75ce651] /lib/tls/i686/cmov/libc.so.6(abort+0x182)[0xb75d1a82] /lib/tls/i686/cmov/libc.so.6(+0x6149d)[0xb760549d] /lib/tls/i686/cmov/libc.so.6(+0x6b591)[0xb760f591] /lib/tls/i686/cmov/libc.so.6(+0x6cde8)[0xb7610de8] /lib/tls/i686/cmov/libc.so.6(cfree+0x6d)[0xb7613ecd] /usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0xb77fe741] /usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0xb77fe79d] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb75babd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Feb 2012 20:02:48	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	1,036,800	1,381,987	1.3329
21 Feb 2012 21:54:13	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	1,010,880	1,347,354	1.3329
21 Feb 2012 04:01:19	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	984,960	1,312,790	1.3328
20 Feb 2012 18:38:14	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	959,040	1,278,180	1.3328
19 Feb 2012 21:45:19	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	933,120	1,243,601	1.3327
19 Feb 2012 11:00:33	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	907,200	1,209,121	1.3328
19 Feb 2012 00:27:26	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	881,280	1,174,674	1.3329
18 Feb 2012 14:10:55	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	855,360	1,140,324	1.3332
18 Feb 2012 03:25:28	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	829,440	1,105,721	1.3331
17 Feb 2012 16:49:31	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	803,520	1,070,845	1.3327
16 Feb 2012 23:25:20	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	777,600	1,035,633	1.3318
16 Feb 2012 01:45:48	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	751,680	1,001,099	1.3318
15 Feb 2012 04:46:34	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	725,760	966,531	1.3318
14 Feb 2012 17:49:59	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	699,840	931,992	1.3317
13 Feb 2012 21:38:45	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	673,920	897,598	1.3319
13 Feb 2012 01:41:45	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	648,000	862,897	1.3316
12 Feb 2012 11:21:26	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	622,080	828,350	1.3316
11 Feb 2012 17:09:10	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	596,160	793,835	1.3316
11 Feb 2012 02:17:03	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	570,240	759,140	1.3313
10 Feb 2012 05:34:54	859190	13956066	hadcm3n_yf1i_1940_40_007693766_1	544,320	724,671	1.3313