climateprediction.net home page
Task 20658355

Task 20658355

Name wah2_afr50_k1se_201512_13_644_011206980_0
Workunit 11206980
Created 11 Aug 2017, 11:27:46 UTC
Sent 11 Aug 2017, 14:46:04 UTC
Report deadline 24 Jul 2018, 20:06:04 UTC
Received 2 Sep 2017, 10:55:41 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 194 (0x000000C2) EXIT_ABORTED_BY_CLIENT
Computer ID 1292474
Run time 3 days 18 hours 24 min 24 sec
CPU time 3 days 12 hours 30 min 21 sec
Validate state Invalid
Credit 6,859.15
Device peak FLOPS 4.01 GFLOPS
Application version Weather At Home 2 (wah2) v8.24
windows_intelx86
Peak working set size 219.74 MB
Peak swap size 183.40 MB
Peak disk usage 16.14 MB
Stderr
<core_client_version>7.6.33</core_client_version>
<![CDATA[
<message>
finish file present too long
</message>
<stderr_txt>
eckPID=7072, selfPID=13768, iMonCtr=1
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7072, selfPID=7072, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
05:08:51 (4120): start_timer_thread(): CreateThread() failed, errno 0
05:08:53 (15936): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4120, selfPID=4120, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
20:19:33 (11940): start_timer_thread(): CreateThread() failed, errno 0
20:19:34 (11668): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11940, selfPID=11668, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
03:44:57 (15264): start_timer_thread(): CreateThread() failed, errno 0
03:44:59 (3144): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
10:38:36 (12236): start_timer_thread(): CreateThread() failed, errno 0
10:38:38 (15132): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12236, selfPID=12236, iMonCtr=1
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12236, selfPID=15132, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
16:09:07 (11340): start_timer_thread(): CreateThread() failed, errno 0
16:09:09 (9492): start_timer_thread(): CreateThread() failed, errno 0
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11340, selfPID=11340, iMonCtr=1
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11340, selfPID=9492, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
20:52:08 (10668): start_timer_thread(): CreateThread() failed, errno 0
20:52:09 (14208): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10668, selfPID=10668, iMonCtr=1
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10668, selfPID=14208, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:00:31 (1312): start_timer_thread(): CreateThread() failed, errno 0
14:00:33 (16920): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
18:16:41 (7104): start_timer_thread(): CreateThread() failed, errno 0
18:16:43 (2980): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
19:41:25 (10848): start_timer_thread(): CreateThread() failed, errno 0
19:41:26 (3536): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10848, selfPID=10848, iMonCtr=1
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10848, selfPID=3536, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
10:15:38 (10980): start_timer_thread(): CreateThread() failed, errno 0
10:15:40 (16048): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10980, selfPID=10980, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10980, selfPID=16048, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
15:12:16 (13604): start_timer_thread(): CreateThread() failed, errno 0
15:12:18 (3452): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11844, selfPID=11844, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11844, selfPID=7776, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9996, selfPID=9996, iMonCtr=1
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9996, selfPID=3876, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13604, selfPID=13604, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13604, selfPID=7736, iMonCtr=1
17:23:33 (1672): start_timer_thread(): CreateThread() failed, errno 0
17:23:35 (13320): start_timer_thread(): CreateThread() failed, errno 0
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1672, selfPID=1672, iMonCtr=1
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1672, selfPID=13320, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
20:11:36 (3988): start_timer_thread(): CreateThread() failed, errno 0
20:11:37 (12920): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3988, selfPID=3988, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3988, selfPID=12920, iMonCtr=1
09:49:16 (11184): start_timer_thread(): CreateThread() failed, errno 0
09:49:18 (10056): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11184, selfPID=11184, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11184, selfPID=10056, iMonCtr=1
10:48:06 (10276): start_timer_thread(): CreateThread() failed, errno 0
10:48:07 (8996): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10276, selfPID=8996, iMonCtr=1
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10276, selfPID=10276, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=964, selfPID=3436, iMonCtr=1
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=964, selfPID=964, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
18:59:23 (14220): start_timer_thread(): CreateThread() failed, errno 0
18:59:24 (3360): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
19:31:19 (2836): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
20:01:52 (4384): start_timer_thread(): CreateThread() failed, errno 0
20:01:53 (3900): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:48:30 (7000): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
09:10:47 (9428): start_timer_thread(): CreateThread() failed, errno 0
09:10:48 (7344): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9428, selfPID=7344, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
10:23:55 (13276): start_timer_thread(): CreateThread() failed, errno 0
10:23:56 (5028): start_timer_thread(): CreateThread() failed, errno 0
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13276, selfPID=5028, iMonCtr=1
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13276, selfPID=13276, iMonCtr=1
10:30:29 (12268): start_timer_thread(): CreateThread() failed, errno 0
10:30:31 (11764): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12268, selfPID=11764, iMonCtr=1
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12268, selfPID=12268, iMonCtr=1
22:03:24 (5668): start_timer_thread(): CreateThread() failed, errno 0
22:03:25 (15052): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5668, selfPID=5668, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5668, selfPID=15052, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
09:05:40 (11664): start_timer_thread(): CreateThread() failed, errno 0
09:05:41 (8940): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11664, selfPID=8940, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
16:09:07 (2512): start_timer_thread(): CreateThread() failed, errno 0
16:09:09 (14624): start_timer_thread(): CreateThread() failed, errno 0
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2512, selfPID=2512, iMonCtr=1
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2512, selfPID=14624, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
18:53:04 (6352): start_timer_thread(): CreateThread() failed, errno 0
18:53:05 (12012): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6352, selfPID=12012, iMonCtr=1
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6352, selfPID=6352, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
20:20:40 (6108): start_timer_thread(): CreateThread() failed, errno 0
20:20:41 (7224): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
22:15:53 (11328): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
13:16:01 (3548): start_timer_thread(): CreateThread() failed, errno 0
13:16:03 (11668): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3548, selfPID=3548, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
15:05:54 (4468): start_timer_thread(): CreateThread() failed, errno 0
15:05:55 (15216): start_timer_thread(): CreateThread() failed, errno 0
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4468, selfPID=4468, iMonCtr=1
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4468, selfPID=15216, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
18:45:44 (12728): start_timer_thread(): CreateThread() failed, errno 0
18:45:46 (9472): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12728, selfPID=12728, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
21:06:55 (12508): start_timer_thread(): CreateThread() failed, errno 0
21:06:56 (9012): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12508, selfPID=9012, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
07:59:25 (2688): start_timer_thread(): CreateThread() failed, errno 0
07:59:27 (4316): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
10:30:56 (14400): start_timer_thread(): CreateThread() failed, errno 0
10:30:57 (13692): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
12:21:38 (4584): start_timer_thread(): CreateThread() failed, errno 0
12:21:39 (1560): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
12:52:12 (10508): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=14700, iMonCtr=2
Model crash detected, will try to restart...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10508, iMonCtr=2
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
15:34:59 (14724): start_timer_thread(): CreateThread() failed, errno 0
15:35:01 (10836): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14724, selfPID=14724, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14724, selfPID=10836, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
16:09:03 (1304): start_timer_thread(): CreateThread() failed, errno 0
16:09:04 (1572): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1304, selfPID=1572, iMonCtr=1
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1304, selfPID=1304, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
16:32:55 (13164): start_timer_thread(): CreateThread() failed, errno 0
16:32:57 (10284): start_timer_thread(): CreateThread() failed, errno 0
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13164, selfPID=10284, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
23:31:40 (11876): start_timer_thread(): CreateThread() failed, errno 0
23:31:42 (7368): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11876, selfPID=7368, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
13:43:46 (12504): start_timer_thread(): CreateThread() failed, errno 0
13:43:47 (15624): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12504, selfPID=15624, iMonCtr=1
16:19:29 (16176): start_timer_thread(): CreateThread() failed, errno 0
16:19:30 (15272): start_timer_thread(): CreateThread() failed, errno 0
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=16176, selfPID=15272, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
18:54:41 (14956): start_timer_thread(): CreateThread() failed, errno 0
18:54:42 (15072): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14956, selfPID=14956, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
21:08:33 (9368): start_timer_thread(): CreateThread() failed, errno 0
21:08:35 (12276): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9368, selfPID=12276, iMonCtr=1
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9368, selfPID=9368, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
01:18:02 (15880): start_timer_thread(): CreateThread() failed, errno 0
01:18:03 (2304): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=15880, selfPID=2304, iMonCtr=1
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=15880, selfPID=15880, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
05:33:46 (14496): start_timer_thread(): CreateThread() failed, errno 0
05:33:48 (11236): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14496, selfPID=14496, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13868, selfPID=13868, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13868, selfPID=15652, iMonCtr=1
16:06:11 (10708): start_timer_thread(): CreateThread() failed, errno 0
16:06:12 (15084): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
20:01:15 (7092): start_timer_thread(): CreateThread() failed, errno 0
20:01:17 (15376): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:26:06 (10136): start_timer_thread(): CreateThread() failed, errno 0
21:26:08 (10980): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
10:04:05 (15716): start_timer_thread(): CreateThread() failed, errno 0
10:04:08 (12900): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=15716, selfPID=15716, iMonCtr=1
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=15716, selfPID=12900, iMonCtr=1
23:42:27 (13168): start_timer_thread(): CreateThread() failed, errno 0
23:42:28 (10772): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13168, selfPID=13168, iMonCtr=1
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13168, selfPID=10772, iMonCtr=1
11:53:30 (10836): start_timer_thread(): CreateThread() failed, errno 0
11:53:31 (13040): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
13:38:45 (8540): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8540, iMonCtr=2
CPDN Monitor - Quit request from BOINC...
14:52:52 (3776): start_timer_thread(): CreateThread() failed, errno 0
14:52:53 (10016): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3776, selfPID=3776, iMonCtr=1
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3776, selfPID=10016, iMonCtr=1
20:14:16 (8084): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8084, selfPID=8084, iMonCtr=1
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8084, selfPID=8044, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
09:57:40 (11416): start_timer_thread(): CreateThread() failed, errno 0
09:57:41 (7228): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11416, selfPID=7228, iMonCtr=1
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11416, selfPID=11416, iMonCtr=1
16:14:40 (9624): start_timer_thread(): CreateThread() failed, errno 0
16:14:42 (1620): start_timer_thread(): CreateThread() failed, errno 0
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1620, selfPID=14904, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
19:59:24 (15284): start_timer_thread(): CreateThread() failed, errno 0
19:59:25 (15136): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:50:34 (10156): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10156, iMonCtr=2
CPDN Monitor - Quit request from BOINC...
13:43:08 (16360): start_timer_thread(): CreateThread() failed, errno 0
13:43:09 (9448): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=16360, selfPID=9448, iMonCtr=1
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=16360, selfPID=16360, iMonCtr=1
20:33:45 (4668): start_timer_thread(): CreateThread() failed, errno 0
20:33:47 (4616): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4668, selfPID=4668, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8924, iMonCtr=2
Model crash detected, will try to restart...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1080, iMonCtr=2
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9380, selfPID=9380, iMonCtr=1
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9380, selfPID=14136, iMonCtr=1
15:57:46 (11316): start_timer_thread(): CreateThread() failed, errno 0
15:57:47 (11220): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
23:25:56 (10880): start_timer_thread(): CreateThread() failed, errno 0
23:25:57 (1804): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10880, selfPID=10880, iMonCtr=1
09:40:04 (11348): start_timer_thread(): CreateThread() failed, errno 0
09:40:05 (9232): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11348, selfPID=9232, iMonCtr=1
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11348, selfPID=11348, iMonCtr=1
11:51:08 (10552): start_timer_thread(): CreateThread() failed, errno 0
11:51:09 (10708): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10552, selfPID=10552, iMonCtr=1
14:46:00 (13544): start_timer_thread(): CreateThread() failed, errno 0
14:46:02 (11148): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13544, selfPID=13544, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
18:51:52 (1992): start_timer_thread(): CreateThread() failed, errno 0
18:51:54 (12108): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1992, selfPID=1992, iMonCtr=1
20:08:57 (13596): start_timer_thread(): CreateThread() failed, errno 0
20:08:58 (12956): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6748, iMonCtr=2
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
03:03:22 (14320): start_timer_thread(): CreateThread() failed, errno 0
03:03:23 (1068): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14320, selfPID=14320, iMonCtr=1
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14320, selfPID=1068, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
10:01:54 (12736): start_timer_thread(): CreateThread() failed, errno 0
10:01:56 (10424): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
12:26:01 (5948): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
12:57:14 (236): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
13:27:46 (13628): start_timer_thread(): CreateThread() failed, errno 0
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10336, iMonCtr=2
Model crash detected, will try to restart...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13628, iMonCtr=2
Leaving CPDN_ain::Monitor...
16:30:45 (10336): called boinc_finish(0)
CPDN Monitor - Abort request from BOINC...


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x02AAC12C

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.7.0


Dump Timestamp    : 09/02/17 18:55:05
Install Directory : 
Data Directory    : D:\boinc
Project Symstore  : 
LoadLibraryA( D:\boinc\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( D:\boinc\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( D:\boinc\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( D:\boinc\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: D:\boinc\projects\climateprediction.net;D:\boinc\projects\climateprediction.net


ModLoad: 0000000000400000 0000000000152000 D:\boinc\projects\climateprediction.net\wah2_8.24_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000077480000 000000000018e000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.15063.447) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.0

ModLoad: 0000000075650000 00000000000d0000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.15063.296) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.0

ModLoad: 0000000075db0000 00000000001c2000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.15063.502) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 10.0.15063.502 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.502

ModLoad: 000000006ef60000 0000000000093000 C:\WINDOWS\SYSTEM32\apphelp.dll (6.2.15063.447) (-exported- Symbols Loaded)
    Linked PDB Filename   : apphelp.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.0

ModLoad: 0000000075720000 000000000013c000 C:\WINDOWS\System32\USER32.dll (6.2.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.0

ModLoad: 00000000752c0000 0000000000016000 C:\WINDOWS\System32\win32u.dll (6.2.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : wwin32u.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.0

ModLoad: 00000000753a0000 0000000000021000 C:\WINDOWS\System32\GDI32.dll (6.2.15063.483) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 10.0.15063.483 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.483

ModLoad: 0000000076720000 0000000000158000 C:\WINDOWS\System32\gdi32full.dll (6.2.15063.413) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32full.pdb
    File Version          : 10.0.15063.413 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.413

ModLoad: 0000000075450000 0000000000079000 C:\WINDOWS\System32\msvcp_win.dll (6.2.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcp_win.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.0

ModLoad: 00000000763f0000 0000000000118000 C:\WINDOWS\System32\ucrtbase.dll (6.2.15063.447) (-exported- Symbols Loaded)
    Linked PDB Filename   : ucrtbase.pdb
    File Version          : 10.0.15063.447 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.447

ModLoad: 00000000765f0000 0000000000077000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.0

ModLoad: 00000000752e0000 00000000000bd000 C:\WINDOWS\System32\msvcrt.dll (7.0.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 7.0.15063.0

ModLoad: 0000000075d10000 0000000000041000 C:\WINDOWS\System32\sechost.dll (6.2.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.0

ModLoad: 0000000077120000 00000000000c0000 C:\WINDOWS\System32\RPCRT4.dll (6.2.15063.447) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.0

ModLoad: 0000000073f50000 0000000000020000 C:\WINDOWS\System32\SspiCli.dll (6.2.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.0

ModLoad: 0000000073f40000 000000000000a000 C:\WINDOWS\System32\CRYPTBASE.dll (6.2.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.0

ModLoad: 0000000076910000 0000000000057000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.15063.138) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.15063.138 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.138

ModLoad: 00000000770e0000 0000000000025000 C:\WINDOWS\System32\IMM32.DLL (6.2.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : wimm32.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.0

ModLoad: 0000000073030000 000000000015d000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.15063.447) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.15063.447 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.447

ModLoad: 0000000073e80000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.15063.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.15063.0 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : MicrosoftR WindowsR Operating System
    Product Version       : 10.0.15063.0



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 4663, Write: 0, Other 26548

- I/O Transfers Counters -
Read: 0, Write: 2485, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 98328, QuotaPeakPagedPoolUsage: 98400
QuotaNonPagedPoolUsage: 8896, QuotaPeakNonPagedPoolUsage: 9184

- Virtual Memory Usage -
VirtualSize: 59482112, PeakVirtualSize: 59482112

- Pagefile Usage -
PagefileUsage: 3649536, PeakPagefileUsage: 3649536

- Working Set Size -
WorkingSetSize: 6787072, PeakWorkingSetSize: 6787072, PageFaultCount: 2271

*** Dump of thread ID 13276 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 2968750.000000, User Time: 9375000.000000, Wait Time: 6495776.000000

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x02AAC12C

- Registers -
eax=00000065 ebx=004b63b9 ecx=0019d8e8 edx=02aac12c esi=0019d6a0 edi=0019d918
eip=0048ee46 esp=0019d634 ebp=0019d8c4
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202

- Callstack -
ChildEBP RetAddr  Args to Child
0019d8c4 0048161c 0019d8e8 004b63b8 00000000 0019d918 wah2_8.24_windows_intelx86!+0x0 
0019d908 0044fa52 02aac12c 004b63b8 004b6484 00000421 wah2_8.24_windows_intelx86!+0x0 
0019d928 00496e90 0000000b 00000000 00000000 00000000 wah2_8.24_windows_intelx86!+0x0 
0019d940 00485ca1 c0000005 0019d974 00490c54 00000000 wah2_8.24_windows_intelx86!+0x0 
0019ff80 75668744 0029f000 75668720 6d60eae7 0019ffdc wah2_8.24_windows_intelx86!+0x0 
0019ff94 774e582d 0029f000 56e653d9 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
0019ffdc 774e57fd ffffffff 7750638f 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
0019ffec 00000000 00485cce 0029f000 00000000 78746341 ntdll!RtlGetAppContainerNamedObjectPath+0x0 

*** Dump of thread ID 4496 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 6494280.000000

- Registers -
eax=00000000 ebx=007e8008 ecx=00000000 edx=00000000 esi=007ef710 edi=007ef8d0
eip=774f396c esp=0077fd7c ebp=0077ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
0077ff80 75668744 007e8008 75668720 6d0eeae7 0077ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
0077ff94 774e582d 007e8008 568853d9 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
0077ffdc 774e57fd ffffffff 7750638f 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
0077ffec 00000000 774ba470 007e8008 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 

*** Dump of thread ID 7364 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 6494280.000000

- Registers -
eax=00000000 ebx=007e8008 ecx=00000000 edx=00000000 esi=007f10c0 edi=007f1280
eip=774f396c esp=009dfd7c ebp=009dff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
009dff80 75668744 007e8008 75668720 6de4eae7 009dffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
009dff94 774e582d 007e8008 566253d9 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
009dffdc 774e57fd ffffffff 7750638f 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
009dffec 00000000 774ba470 007e8008 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 

*** Dump of thread ID 14496 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 6494280.000000

- Registers -
eax=774ba470 ebx=007e8008 ecx=00000000 edx=00000000 esi=007f2638 edi=00000000
eip=774f396c esp=00b1fd7c ebp=00b1ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
00b1ff80 75668744 007e8008 75668720 6dc8eae7 00b1ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
00b1ff94 774e582d 007e8008 564e53d9 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
00b1ffdc 774e57fd ffffffff 7750638f 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
00b1ffec 00000000 774ba470 007e8008 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 

*** Dump of thread ID 7496 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 6495775.000000

- Registers -
eax=00000000 ebx=029dfd1c ecx=00000000 edx=00000000 esi=00000000 edi=00000001
eip=774f205c esp=029dfcdc ebp=029dfd40
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
029dfd40 75ea351f 00000064 00000000 029dff80 00478b44 ntdll!NtDelayExecution+0x0 
029dfd50 00478b44 00000064 00478b20 00478b20 00000000 KERNELBASE!Sleep+0x0 
029dff80 75668744 00000000 75668720 6fe4eae7 029dffdc wah2_8.24_windows_intelx86!+0x0 
029dff94 774e582d 00000000 546253d9 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
029dffdc 774e57fd ffffffff 7750638f 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 
029dffec 00000000 00478b20 00000000 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
02 Sep 2017 03:48:41 1292474 20658355 wah2_afr50_k1se_201512_13_644_011206980_0 103,979 291,624 2.8046
01 Sep 2017 06:16:43 1292474 20658355 wah2_afr50_k1se_201512_13_644_011206980_0 92,459 260,817 2.8209
30 Aug 2017 12:10:48 1292474 20658355 wah2_afr50_k1se_201512_13_644_011206980_0 80,939 228,398 2.8219
28 Aug 2017 04:02:52 1292474 20658355 wah2_afr50_k1se_201512_13_644_011206980_0 69,419 196,588 2.8319
25 Aug 2017 13:49:21 1292474 20658355 wah2_afr50_k1se_201512_13_644_011206980_0 57,899 167,284 2.8892
22 Aug 2017 11:38:00 1292474 20658355 wah2_afr50_k1se_201512_13_644_011206980_0 46,379 132,511 2.8571
20 Aug 2017 00:49:32 1292474 20658355 wah2_afr50_k1se_201512_13_644_011206980_0 34,859 98,935 2.8381
15 Aug 2017 13:34:29 1292474 20658355 wah2_afr50_k1se_201512_13_644_011206980_0 23,339 66,759 2.8604
13 Aug 2017 21:39:42 1292474 20658355 wah2_afr50_k1se_201512_13_644_011206980_0 11,819 32,982 2.7906


©2024 climateprediction.net