Task 21744084

Name	hadcm3s_hf14_190012_240_835_011892376_0
Workunit	11892376
Created	25 Sep 2019, 10:39:40 UTC
Sent	25 Sep 2019, 14:49:23 UTC
Report deadline	6 Sep 2020, 20:09:23 UTC
Received	28 Oct 2019, 1:28:28 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1489850
Run time	11 days 18 hours 44 min 31 sec
CPU time	11 days 17 hours 20 min 36 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.74 GFLOPS
Application version	UK Met Office HadCM3 short v8.36 i686-pc-linux-gnu
Peak working set size	167.26 MB
Peak swap size	215.76 MB
Peak disk usage	99.19 MB
Stderr	<core_client_version>7.16.3</core_client_version> <![CDATA[ <stderr_txt> Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 04:44:10 (20779): called boinc_finish(193) SIGSEGV: segmentation violation Stack trace (11 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3s_um_8.36_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x84ff4f7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7efc950] /lib/i386-linux-gnu/libc.so.6(+0x147f66)[0xf7d28f66] /var/lib/boinc-client/projects/climateprediction.net/hadcm3s_um_8.36_i686-pc-linux-gnu[0x8498da5] /var/lib/boinc-client/projects/climateprediction.net/hadcm3s_um_8.36_i686-pc-linux-gnu[0x80f3057] /var/lib/boinc-client/projects/climateprediction.net/hadcm3s_um_8.36_i686-pc-linux-gnu[0x80863d0] /var/lib/boinc-client/projects/climateprediction.net/hadcm3s_um_8.36_i686-pc-linux-gnu[0x8479d6e] /var/lib/boinc-client/projects/climateprediction.net/hadcm3s_um_8.36_i686-pc-linux-gnu[0x8494feb] /var/lib/boinc-client/projects/climateprediction.net/hadcm3s_um_8.36_i686-pc-linux-gnu[0x848be04] /var/lib/boinc-client/projects/climateprediction.net/hadcm3s_um_8.36_i686-pc-linux-gnu[0x8496bad] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0xf7bff751] Exiting... Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80bc177] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f44950] /lib/i386-linux-gnu/libc.so.6(getenv+0x99)[0xf7a630c9] /lib/i386-linux-gnu/libc.so.6(+0xb538d)[0xf7ae338d] /lib/i386-linux-gnu/libc.so.6(+0xb571b)[0xf7ae371b] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x17)[0xf7ae1857] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7632] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7d80] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7e71] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f44950] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7a00] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f44950] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f44939] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x4b)[0xf7af361b] /lib/i386-linux-gnu/libc.so.6(usleep+0x41)[0xf7b26cd1] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80cead5] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x804f747] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x804fe7a] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80515bd] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80517bd] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0xf7a4c751] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x804c7a1] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... MainError: 04:30:23 AM No files match the supplied pattern. Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... MainError: 04:30:23 AM No files match the supplied pattern. MainError: 12:45:39 AM No files match the supplied pattern. MainError: 12:45:39 AM No files match the supplied pattern. MainError: 01:16:25 AM No files match the supplied pattern. MainError: 01:16:25 AM No files match the supplied pattern. MainError: 01:49:30 PM No files match the supplied pattern. MainError: 01:49:30 PM No files match the supplied pattern. MainError: 03:10:02 AM No files match the supplied pattern. MainError: 03:10:02 AM No files match the supplied pattern. MainError: 04:07:31 PM No files match the supplied pattern. MainError: 04:07:31 PM No files match the supplied pattern. MainError: 04:58:52 AM No files match the supplied pattern. MainError: 04:58:52 AM No files match the supplied pattern. MainError: 06:28:54 PM No files match the supplied pattern. MainError: 06:28:54 PM No files match the supplied pattern. MainError: 08:14:29 AM No files match the supplied pattern. MainError: 08:14:29 AM No files match the supplied pattern. MainError: 09:44:38 PM No files match the supplied pattern. MainError: 09:44:38 PM No files match the supplied pattern. MainError: 10:13:03 AM No files match the supplied pattern. MainError: 10:13:03 AM No files match the supplied pattern. MainError: 11:15:22 PM No files match the supplied pattern. MainError: 11:15:22 PM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... MainError: 12:03:40 AM No files match the supplied pattern. MainError: 12:03:40 AM No files match the supplied pattern. Processing restart Year 1920 Month 12 Day 1 MainError: 12:31:21 AM No files match the supplied pattern. MainError: 12:31:21 AM No files match the supplied pattern. 19:31:46 (1939): called boinc_finish(0) </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadcm3s_hf14_190012_240_835_011892376_0_r183056782_1.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hf14_190012_240_835_011892376_0_r183056782_2.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hf14_190012_240_835_011892376_0_r183056782_3.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hf14_190012_240_835_011892376_0_r183056782_4.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hf14_190012_240_835_011892376_0_r183056782_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hf14_190012_240_835_011892376_0_r183056782_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Oct 2019 01:30:33	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	518,400	1,012,826	1.9538
27 Oct 2019 12:19:09	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	492,480	967,986	1.9655
27 Oct 2019 00:10:35	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	466,560	922,525	1.9773
26 Oct 2019 11:05:04	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	440,640	876,095	1.9882
25 Oct 2019 21:54:29	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	414,720	831,242	2.0043
25 Oct 2019 08:42:03	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	388,800	782,656	2.0130
24 Oct 2019 18:34:15	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	362,880	733,147	2.0204
24 Oct 2019 05:59:16	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	336,960	684,571	2.0316
23 Oct 2019 16:09:27	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	311,040	638,336	2.0523
23 Oct 2019 03:14:09	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	285,120	591,735	2.0754
22 Oct 2019 13:53:01	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	259,200	544,590	2.1010
22 Oct 2019 01:18:41	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	233,280	499,442	2.1410
21 Oct 2019 12:48:50	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	207,360	454,452	2.1916
14 Oct 2019 15:47:18	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	190,080	423,361	2.2273
14 Oct 2019 14:49:52	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	187,920	419,165	2.2306
14 Oct 2019 13:47:06	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	185,760	414,993	2.2340
14 Oct 2019 12:44:44	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	183,600	410,835	2.2377
14 Oct 2019 11:46:39	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	181,440	406,686	2.2414
14 Oct 2019 09:46:29	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	179,280	402,527	2.2452
14 Oct 2019 08:44:08	1489850	21744084	hadcm3s_hf14_190012_240_835_011892376_0	177,120	398,320	2.2489