climateprediction.net home page
Task 22444722

Task 22444722

Name oifs_43r3_bl_a25p_2016092300_20_1018_12291124_0
Workunit 12291124
Created 13 Jun 2024, 0:05:23 UTC
Sent 13 Jun 2024, 13:07:45 UTC
Report deadline 12 Aug 2024, 13:07:45 UTC
Received 13 Jun 2024, 21:52:11 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1548614
Run time 5 hours 5 min 59 sec
CPU time 5 hours 5 min 59 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 6.71 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,016.34 MB
Peak swap size 5,981.07 MB
Peak disk usage 1,344.52 MB
Stderr
<core_client_version>7.22.1</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
ctory: /home/robbie/slots/16/ICMGGh7zg+000372
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000372
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000372
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:32:05 STEP  373 H=  93:15 +CPU= 20.115
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:32:46 STEP  374 H=  93:30 +CPU= 20.383
  16:33:08 STEP  375 H=  93:45 +CPU= 20.808
  16:33:29 STEP  376 H=  94:00 +CPU= 21.269
  16:33:50 STEP  377 H=  94:15 +CPU= 20.833
  16:34:11 STEP  378 H=  94:30 +CPU= 20.822
  16:34:31 STEP  379 H=  94:45 +CPU= 20.687
  16:34:53 STEP  380 H=  95:00 +CPU= 21.282
  16:35:14 STEP  381 H=  95:15 +CPU= 20.882
  16:35:34 STEP  382 H=  95:30 +CPU= 20.697
  16:35:55 STEP  383 H=  95:45 +CPU= 20.914
  16:36:26 STEP  384 H=  96:00 +CPU= 30.046
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000384
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000384
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000384
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000288
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000288
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000288
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000300
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000300
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000300
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000312
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000312
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000312
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000324
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000324
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000324
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000336
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000336
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000336
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000348
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000348
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000348
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000360
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000360
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000360
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000372
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000372
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000372
Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a25p_2016092300_20_1018_12291124_0_r1343717625_3.zip
  16:36:47 STEP  385 H=  96:15 +CPU= 20.895
Uploading the intermediate file: upload_file_3.zip
  16:37:08 STEP  386 H=  96:30 +CPU= 20.746
  16:37:29 STEP  387 H=  96:45 +CPU= 20.759
  16:37:50 STEP  388 H=  97:00 +CPU= 21.199
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:38:31 STEP  389 H=  97:15 +CPU= 20.066
  16:38:52 STEP  390 H=  97:30 +CPU= 20.911
  16:39:13 STEP  391 H=  97:45 +CPU= 20.783
  16:39:35 STEP  392 H=  98:00 +CPU= 21.401
  16:39:56 STEP  393 H=  98:15 +CPU= 20.891
  16:40:16 STEP  394 H=  98:30 +CPU= 20.629
  16:40:37 STEP  395 H=  98:45 +CPU= 20.908
  16:41:03 STEP  396 H=  99:00 +CPU= 26.037
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000396
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000396
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000396
  16:41:24 STEP  397 H=  99:15 +CPU= 21.043
  16:41:45 STEP  398 H=  99:30 +CPU= 20.841
  16:42:06 STEP  399 H=  99:45 +CPU= 20.701
  16:42:27 STEP  400 H= 100:00 +CPU= 21.147
  16:42:48 STEP  401 H= 100:15 +CPU= 20.739
  16:43:09 STEP  402 H= 100:30 +CPU= 20.653
  16:43:30 STEP  403 H= 100:45 +CPU= 20.768
  16:43:51 STEP  404 H= 101:00 +CPU= 21.121
  16:44:12 STEP  405 H= 101:15 +CPU= 20.929
  16:44:33 STEP  406 H= 101:30 +CPU= 20.889
  16:44:53 STEP  407 H= 101:45 +CPU= 20.756
  16:45:19 STEP  408 H= 102:00 +CPU= 25.861
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000408
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000408
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000408
  16:45:40 STEP  409 H= 102:15 +CPU= 20.895
  16:46:01 STEP  410 H= 102:30 +CPU= 20.445
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:46:41 STEP  411 H= 102:45 +CPU= 19.301
  16:47:02 STEP  412 H= 103:00 +CPU= 21.197
  16:47:23 STEP  413 H= 103:15 +CPU= 20.730
  16:47:44 STEP  414 H= 103:30 +CPU= 20.648
  16:48:05 STEP  415 H= 103:45 +CPU= 20.663
  16:48:26 STEP  416 H= 104:00 +CPU= 21.127
  16:48:47 STEP  417 H= 104:15 +CPU= 20.701
  16:49:08 STEP  418 H= 104:30 +CPU= 20.804
  16:49:29 STEP  419 H= 104:45 +CPU= 20.709
  16:49:55 STEP  420 H= 105:00 +CPU= 26.010
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000420
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000420
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000420
  16:50:16 STEP  421 H= 105:15 +CPU= 21.015
  16:50:37 STEP  422 H= 105:30 +CPU= 20.925
  16:50:57 STEP  423 H= 105:45 +CPU= 20.826
  16:51:19 STEP  424 H= 106:00 +CPU= 21.280
  16:51:40 STEP  425 H= 106:15 +CPU= 20.898
  16:52:01 STEP  426 H= 106:30 +CPU= 20.832
  16:52:21 STEP  427 H= 106:45 +CPU= 20.712
  16:52:43 STEP  428 H= 107:00 +CPU= 21.236
  16:53:04 STEP  429 H= 107:15 +CPU= 20.951
  16:53:25 STEP  430 H= 107:30 +CPU= 20.691
  16:53:46 STEP  431 H= 107:45 +CPU= 20.909
  16:54:16 STEP  432 H= 108:00 +CPU= 30.016
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000432
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000432
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000432
  16:54:37 STEP  433 H= 108:15 +CPU= 20.805
  16:54:58 STEP  434 H= 108:30 +CPU= 20.861
  16:55:19 STEP  435 H= 108:45 +CPU= 20.951
  16:55:40 STEP  436 H= 109:00 +CPU= 21.253
  16:56:01 STEP  437 H= 109:15 +CPU= 20.897
  16:56:22 STEP  438 H= 109:30 +CPU= 20.850
  16:56:43 STEP  439 H= 109:45 +CPU= 20.872
  16:57:04 STEP  440 H= 110:00 +CPU= 21.148
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:57:45 STEP  441 H= 110:15 +CPU= 20.469
  16:58:06 STEP  442 H= 110:30 +CPU= 20.693
  16:58:27 STEP  443 H= 110:45 +CPU= 20.814
  16:58:53 STEP  444 H= 111:00 +CPU= 25.898
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000444
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000444
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000444
  16:59:14 STEP  445 H= 111:15 +CPU= 20.779
  16:59:35 STEP  446 H= 111:30 +CPU= 20.789
  16:59:55 STEP  447 H= 111:45 +CPU= 20.720
  17:00:17 STEP  448 H= 112:00 +CPU= 21.171
  17:00:38 STEP  449 H= 112:15 +CPU= 20.831
  17:00:59 STEP  450 H= 112:30 +CPU= 20.700
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:01:40 STEP  451 H= 112:45 +CPU= 20.309
  17:02:01 STEP  452 H= 113:00 +CPU= 21.166
  17:02:22 STEP  453 H= 113:15 +CPU= 20.755
  17:02:43 STEP  454 H= 113:30 +CPU= 20.633
  17:03:03 STEP  455 H= 113:45 +CPU= 20.703
  17:03:29 STEP  456 H= 114:00 +CPU= 25.865
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000456
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000456
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000456
  17:03:50 STEP  457 H= 114:15 +CPU= 20.734
  17:04:11 STEP  458 H= 114:30 +CPU= 20.935
  17:04:32 STEP  459 H= 114:45 +CPU= 21.121
  17:04:54 STEP  460 H= 115:00 +CPU= 21.352
  17:05:14 STEP  461 H= 115:15 +CPU= 20.860
  17:05:35 STEP  462 H= 115:30 +CPU= 20.675
  17:05:56 STEP  463 H= 115:45 +CPU= 20.842
  17:06:18 STEP  464 H= 116:00 +CPU= 21.196
  17:06:38 STEP  465 H= 116:15 +CPU= 20.667
  17:06:59 STEP  466 H= 116:30 +CPU= 20.749
  17:07:20 STEP  467 H= 116:45 +CPU= 20.769
  17:07:46 STEP  468 H= 117:00 +CPU= 25.945
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000468
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000468
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000468
  17:08:07 STEP  469 H= 117:15 +CPU= 20.638
  17:08:28 STEP  470 H= 117:30 +CPU= 20.869
  17:08:49 STEP  471 H= 117:45 +CPU= 20.942
  17:09:10 STEP  472 H= 118:00 +CPU= 21.171
  17:09:31 STEP  473 H= 118:15 +CPU= 20.759
  17:09:51 STEP  474 H= 118:30 +CPU= 20.676
  17:10:12 STEP  475 H= 118:45 +CPU= 20.721
  17:10:33 STEP  476 H= 119:00 +CPU= 21.258
  17:10:54 STEP  477 H= 119:15 +CPU= 20.656
  17:11:15 STEP  478 H= 119:30 +CPU= 20.879
  17:11:36 STEP  479 H= 119:45 +CPU= 20.966
  17:12:06 STEP  480 H= 120:00 +CPU= 30.009
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000480
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000480
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000480
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000384
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000384
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000384
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000396
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000396
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000396
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000408
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000408
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000408
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000420
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000420
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000420
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000432
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000432
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000432
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000444
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000444
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000444
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000456
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000456
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000456
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000468
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000468
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000468
Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a25p_2016092300_20_1018_12291124_0_r1343717625_4.zip
Uploading the intermediate file: upload_file_4.zip
  17:12:28 STEP  481 H= 120:15 +CPU= 20.903
  17:12:48 STEP  482 H= 120:30 +CPU= 20.696
  17:13:09 STEP  483 H= 120:45 +CPU= 20.835
  17:13:31 STEP  484 H= 121:00 +CPU= 21.555
  17:13:52 STEP  485 H= 121:15 +CPU= 20.745
  17:14:13 STEP  486 H= 121:30 +CPU= 20.970
  17:14:34 STEP  487 H= 121:45 +CPU= 20.712
  17:14:55 STEP  488 H= 122:00 +CPU= 21.309
  17:15:16 STEP  489 H= 122:15 +CPU= 20.863
  17:15:37 STEP  490 H= 122:30 +CPU= 20.704
  17:15:57 STEP  491 H= 122:45 +CPU= 20.672
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:16:43 STEP  492 H= 123:00 +CPU= 25.102
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000492
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000492
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000492
  17:17:04 STEP  493 H= 123:15 +CPU= 20.752
  17:17:25 STEP  494 H= 123:30 +CPU= 20.744
  17:17:46 STEP  495 H= 123:45 +CPU= 20.873
  17:18:07 STEP  496 H= 124:00 +CPU= 21.253
  17:18:28 STEP  497 H= 124:15 +CPU= 20.732
  17:18:49 STEP  498 H= 124:30 +CPU= 20.809
  17:19:10 STEP  499 H= 124:45 +CPU= 20.805
  17:19:31 STEP  500 H= 125:00 +CPU= 21.225
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:20:12 STEP  501 H= 125:15 +CPU= 19.665
  17:20:33 STEP  502 H= 125:30 +CPU= 20.780
  17:20:53 STEP  503 H= 125:45 +CPU= 20.743
  17:21:19 STEP  504 H= 126:00 +CPU= 25.855
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000504
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000504
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000504
  17:21:40 STEP  505 H= 126:15 +CPU= 20.883
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:22:22 STEP  506 H= 126:30 +CPU= 20.258
  17:22:42 STEP  507 H= 126:45 +CPU= 20.680
  17:23:04 STEP  508 H= 127:00 +CPU= 21.255
  17:23:25 STEP  509 H= 127:15 +CPU= 20.752
  17:23:45 STEP  510 H= 127:30 +CPU= 20.680
  17:24:06 STEP  511 H= 127:45 +CPU= 20.876
  17:24:27 STEP  512 H= 128:00 +CPU= 21.167
  17:24:48 STEP  513 H= 128:15 +CPU= 20.806
  17:25:09 STEP  514 H= 128:30 +CPU= 20.807
  17:25:30 STEP  515 H= 128:45 +CPU= 20.729
  17:25:55 STEP  516 H= 129:00 +CPU= 25.408
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000516
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000516
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000516
  17:26:36 STEP  517 H= 129:15 +CPU= 19.788
  17:26:57 STEP  518 H= 129:30 +CPU= 20.717
  17:27:18 STEP  519 H= 129:45 +CPU= 20.710
  17:27:39 STEP  520 H= 130:00 +CPU= 21.190
  17:28:00 STEP  521 H= 130:15 +CPU= 20.689
  17:28:20 STEP  522 H= 130:30 +CPU= 20.716
  17:28:41 STEP  523 H= 130:45 +CPU= 20.763
  17:29:02 STEP  524 H= 131:00 +CPU= 21.098
  17:29:23 STEP  525 H= 131:15 +CPU= 20.727
  17:29:44 STEP  526 H= 131:30 +CPU= 20.878
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:30:25 STEP  527 H= 131:45 +CPU= 20.324
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:31:16 STEP  528 H= 132:00 +CPU= 29.185
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000528
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000528
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000528
  17:31:36 STEP  529 H= 132:15 +CPU= 20.601
  17:31:57 STEP  530 H= 132:30 +CPU= 20.825
  17:32:18 STEP  531 H= 132:45 +CPU= 20.749
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:32:59 STEP  532 H= 133:00 +CPU= 20.029
  17:33:20 STEP  533 H= 133:15 +CPU= 20.790
  17:33:41 STEP  534 H= 133:30 +CPU= 20.806
  17:34:02 STEP  535 H= 133:45 +CPU= 20.760
  17:34:23 STEP  536 H= 134:00 +CPU= 21.313
  17:34:44 STEP  537 H= 134:15 +CPU= 20.833
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:35:25 STEP  538 H= 134:30 +CPU= 19.451
  17:35:45 STEP  539 H= 134:45 +CPU= 20.690
  17:36:11 STEP  540 H= 135:00 +CPU= 25.717
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000540
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000540
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000540
  17:36:32 STEP  541 H= 135:15 +CPU= 20.682
  17:36:53 STEP  542 H= 135:30 +CPU= 20.745
  17:37:14 STEP  543 H= 135:45 +CPU= 20.757
  17:37:35 STEP  544 H= 136:00 +CPU= 21.193
  17:37:56 STEP  545 H= 136:15 +CPU= 20.864
  17:38:16 STEP  546 H= 136:30 +CPU= 20.771
  17:38:37 STEP  547 H= 136:45 +CPU= 20.617
  17:38:58 STEP  548 H= 137:00 +CPU= 21.206
  17:39:19 STEP  549 H= 137:15 +CPU= 20.624
  17:39:40 STEP  550 H= 137:30 +CPU= 20.991
  17:40:01 STEP  551 H= 137:45 +CPU= 20.877
  17:40:27 STEP  552 H= 138:00 +CPU= 25.879
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000552
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000552
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000552
  17:40:48 STEP  553 H= 138:15 +CPU= 20.754
  17:41:09 STEP  554 H= 138:30 +CPU= 20.833
  17:41:30 STEP  555 H= 138:45 +CPU= 20.764
  17:41:51 STEP  556 H= 139:00 +CPU= 21.149
  17:42:11 STEP  557 H= 139:15 +CPU= 20.631
  17:42:32 STEP  558 H= 139:30 +CPU= 20.715
  17:42:53 STEP  559 H= 139:45 +CPU= 20.700
  17:43:14 STEP  560 H= 140:00 +CPU= 21.066
  17:43:35 STEP  561 H= 140:15 +CPU= 20.804
  17:43:56 STEP  562 H= 140:30 +CPU= 20.907
  17:44:17 STEP  563 H= 140:45 +CPU= 20.764
  17:44:43 STEP  564 H= 141:00 +CPU= 25.916
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000564
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000564
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000564
  17:45:04 STEP  565 H= 141:15 +CPU= 20.864
  17:45:24 STEP  566 H= 141:30 +CPU= 20.706
  17:45:45 STEP  567 H= 141:45 +CPU= 20.899
  17:46:06 STEP  568 H= 142:00 +CPU= 21.038
  17:46:27 STEP  569 H= 142:15 +CPU= 20.721
  17:46:48 STEP  570 H= 142:30 +CPU= 20.617
  17:47:09 STEP  571 H= 142:45 +CPU= 20.826
  17:47:30 STEP  572 H= 143:00 +CPU= 21.246
  17:47:51 STEP  573 H= 143:15 +CPU= 20.924
  17:48:12 STEP  574 H= 143:30 +CPU= 20.744
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:48:52 STEP  575 H= 143:45 +CPU= 19.687
  17:49:22 STEP  576 H= 144:00 +CPU= 29.819
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000576
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000576
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000576
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000480
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000480
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000480
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000492
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000492
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000492
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000504
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000504
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000504
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000516
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000516
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000516
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000528
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000528
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000528
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000540
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000540
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000540
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000552
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000552
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000552
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000564
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000564
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000564
Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a25p_2016092300_20_1018_12291124_0_r1343717625_5.zip
Uploading the intermediate file: upload_file_5.zip
  17:49:43 STEP  577 H= 144:15 +CPU= 20.745
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:50:25 STEP  578 H= 144:30 +CPU= 20.246
  17:50:46 STEP  579 H= 144:45 +CPU= 20.907
  17:51:07 STEP  580 H= 145:00 +CPU= 21.145
  17:51:28 STEP  581 H= 145:15 +CPU= 20.756
  17:51:49 STEP  582 H= 145:30 +CPU= 20.870
  17:52:09 STEP  583 H= 145:45 +CPU= 20.842
  17:52:31 STEP  584 H= 146:00 +CPU= 21.152
  17:52:51 STEP  585 H= 146:15 +CPU= 20.638
  17:53:12 STEP  586 H= 146:30 +CPU= 20.787
  17:53:33 STEP  587 H= 146:45 +CPU= 20.766
  17:53:59 STEP  588 H= 147:00 +CPU= 25.805
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000588
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000588
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000588
  17:54:20 STEP  589 H= 147:15 +CPU= 20.702
  17:54:41 STEP  590 H= 147:30 +CPU= 20.801
  17:55:01 STEP  591 H= 147:45 +CPU= 20.672
  17:55:23 STEP  592 H= 148:00 +CPU= 21.080
  17:55:43 STEP  593 H= 148:15 +CPU= 20.711
  17:56:04 STEP  594 H= 148:30 +CPU= 20.757
  17:56:25 STEP  595 H= 148:45 +CPU= 20.826
  17:56:46 STEP  596 H= 149:00 +CPU= 21.287
  17:57:07 STEP  597 H= 149:15 +CPU= 20.693
  17:57:28 STEP  598 H= 149:30 +CPU= 21.013
  17:57:49 STEP  599 H= 149:45 +CPU= 20.737
  17:58:15 STEP  600 H= 150:00 +CPU= 25.854
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000600
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000600
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000600
  17:58:35 STEP  601 H= 150:15 +CPU= 20.659
  17:58:56 STEP  602 H= 150:30 +CPU= 20.866
  17:59:17 STEP  603 H= 150:45 +CPU= 20.785
  17:59:39 STEP  604 H= 151:00 +CPU= 21.291
  17:59:59 STEP  605 H= 151:15 +CPU= 20.808
  18:00:20 STEP  606 H= 151:30 +CPU= 20.635
  18:00:41 STEP  607 H= 151:45 +CPU= 20.868
  18:01:02 STEP  608 H= 152:00 +CPU= 21.236
  18:01:23 STEP  609 H= 152:15 +CPU= 20.618
  18:01:44 STEP  610 H= 152:30 +CPU= 20.785
  18:02:05 STEP  611 H= 152:45 +CPU= 20.819
  18:02:31 STEP  612 H= 153:00 +CPU= 25.757
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000612
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000612
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000612
  18:02:51 STEP  613 H= 153:15 +CPU= 20.756
  18:03:12 STEP  614 H= 153:30 +CPU= 20.730
  18:03:33 STEP  615 H= 153:45 +CPU= 20.718
  18:03:54 STEP  616 H= 154:00 +CPU= 21.245
  18:04:15 STEP  617 H= 154:15 +CPU= 20.718
  18:04:36 STEP  618 H= 154:30 +CPU= 20.737
  18:04:56 STEP  619 H= 154:45 +CPU= 20.806
  18:05:17 STEP  620 H= 155:00 +CPU= 21.074
  18:05:38 STEP  621 H= 155:15 +CPU= 20.722
  18:05:59 STEP  622 H= 155:30 +CPU= 20.704
  18:06:20 STEP  623 H= 155:45 +CPU= 20.861
  18:06:50 STEP  624 H= 156:00 +CPU= 29.933
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000624
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000624
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000624
  18:07:11 STEP  625 H= 156:15 +CPU= 21.000
  18:07:32 STEP  626 H= 156:30 +CPU= 20.658
  18:07:53 STEP  627 H= 156:45 +CPU= 20.823
  18:08:14 STEP  628 H= 157:00 +CPU= 21.209
  18:08:35 STEP  629 H= 157:15 +CPU= 20.771
  18:08:56 STEP  630 H= 157:30 +CPU= 20.752
  18:09:16 STEP  631 H= 157:45 +CPU= 20.798
  18:09:38 STEP  632 H= 158:00 +CPU= 21.139
  18:09:58 STEP  633 H= 158:15 +CPU= 20.754
  18:10:19 STEP  634 H= 158:30 +CPU= 20.901
  18:10:40 STEP  635 H= 158:45 +CPU= 20.839
  18:11:06 STEP  636 H= 159:00 +CPU= 25.816
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000636
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000636
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000636
  18:11:27 STEP  637 H= 159:15 +CPU= 20.773
  18:11:48 STEP  638 H= 159:30 +CPU= 20.752
  18:12:09 STEP  639 H= 159:45 +CPU= 20.681
  18:12:30 STEP  640 H= 160:00 +CPU= 21.242
  18:12:51 STEP  641 H= 160:15 +CPU= 20.712
  18:13:12 STEP  642 H= 160:30 +CPU= 20.880
  18:13:32 STEP  643 H= 160:45 +CPU= 20.800
  18:13:53 STEP  644 H= 161:00 +CPU= 21.045
  18:14:14 STEP  645 H= 161:15 +CPU= 20.753
  18:14:35 STEP  646 H= 161:30 +CPU= 20.692
  18:14:56 STEP  647 H= 161:45 +CPU= 20.802
  18:15:22 STEP  648 H= 162:00 +CPU= 25.834
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000648
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000648
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000648
  18:16:03 STEP  649 H= 162:15 +CPU= 19.680
  18:16:23 STEP  650 H= 162:30 +CPU= 20.672
  18:16:44 STEP  651 H= 162:45 +CPU= 20.744
  18:17:05 STEP  652 H= 163:00 +CPU= 21.176
  18:17:26 STEP  653 H= 163:15 +CPU= 20.663
  18:17:47 STEP  654 H= 163:30 +CPU= 21.014
  18:18:08 STEP  655 H= 163:45 +CPU= 20.936
  18:18:29 STEP  656 H= 164:00 +CPU= 21.128
  18:18:50 STEP  657 H= 164:15 +CPU= 20.636
  18:19:11 STEP  658 H= 164:30 +CPU= 20.732
  18:19:31 STEP  659 H= 164:45 +CPU= 20.746
  18:19:57 STEP  660 H= 165:00 +CPU= 25.914
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000660
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000660
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000660
  18:20:18 STEP  661 H= 165:15 +CPU= 20.686
  18:20:39 STEP  662 H= 165:30 +CPU= 20.865
  18:21:00 STEP  663 H= 165:45 +CPU= 20.931
  18:21:21 STEP  664 H= 166:00 +CPU= 21.335
  18:21:43 STEP  665 H= 166:15 +CPU= 20.978
  18:22:05 STEP  666 H= 166:30 +CPU= 20.825
  18:22:26 STEP  667 H= 166:45 +CPU= 20.684
  18:22:47 STEP  668 H= 167:00 +CPU= 21.106
  18:23:07 STEP  669 H= 167:15 +CPU= 20.697
  18:23:28 STEP  670 H= 167:30 +CPU= 20.634
  18:23:49 STEP  671 H= 167:45 +CPU= 20.746
  18:24:19 STEP  672 H= 168:00 +CPU= 29.871
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000672
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000672
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000672
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000576
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000576
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000576
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000588
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000588
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000588
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000600
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000600
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000600
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000612
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000612
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000612
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000624
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000624
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000624
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000636
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000636
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000636
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000648
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000648
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000648
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000660
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000660
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000660
Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a25p_2016092300_20_1018_12291124_0_r1343717625_6.zip
Uploading the intermediate file: upload_file_6.zip
  18:24:36 STEP  673 H= 168:15 +CPU= 17.015
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:24:55 STEP  674 H= 168:30 +CPU= 10.690
  18:25:16 STEP  675 H= 168:45 +CPU= 20.677
  18:25:37 STEP  676 H= 169:00 +CPU= 21.131
  18:25:58 STEP  677 H= 169:15 +CPU= 20.682
  18:26:19 STEP  678 H= 169:30 +CPU= 20.703
  18:26:40 STEP  679 H= 169:45 +CPU= 20.925
  18:27:01 STEP  680 H= 170:00 +CPU= 21.054
  18:27:21 STEP  681 H= 170:15 +CPU= 20.758
  18:27:42 STEP  682 H= 170:30 +CPU= 20.845
  18:28:03 STEP  683 H= 170:45 +CPU= 20.861
  18:28:29 STEP  684 H= 171:00 +CPU= 25.985
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000684
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000684
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000684
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:29:10 STEP  685 H= 171:15 +CPU= 20.011
  18:29:31 STEP  686 H= 171:30 +CPU= 20.746
  18:29:52 STEP  687 H= 171:45 +CPU= 20.666
  18:30:13 STEP  688 H= 172:00 +CPU= 21.179
  18:30:34 STEP  689 H= 172:15 +CPU= 20.749
  18:30:55 STEP  690 H= 172:30 +CPU= 20.905
  18:31:15 STEP  691 H= 172:45 +CPU= 20.714
  18:31:37 STEP  692 H= 173:00 +CPU= 21.287
  18:31:58 STEP  693 H= 173:15 +CPU= 20.793
  18:32:18 STEP  694 H= 173:30 +CPU= 20.660
  18:32:39 STEP  695 H= 173:45 +CPU= 20.762
  18:33:05 STEP  696 H= 174:00 +CPU= 25.999
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000696
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000696
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000696
  18:33:26 STEP  697 H= 174:15 +CPU= 20.675
  18:33:47 STEP  698 H= 174:30 +CPU= 20.797
  18:34:07 STEP  699 H= 174:45 +CPU= 20.697
  18:34:29 STEP  700 H= 175:00 +CPU= 21.163
  18:34:49 STEP  701 H= 175:15 +CPU= 20.898
  18:35:10 STEP  702 H= 175:30 +CPU= 20.923
  18:35:31 STEP  703 H= 175:45 +CPU= 20.906
  18:35:53 STEP  704 H= 176:00 +CPU= 21.100
  18:36:13 STEP  705 H= 176:15 +CPU= 20.789
  18:36:34 STEP  706 H= 176:30 +CPU= 20.839
  18:36:55 STEP  707 H= 176:45 +CPU= 20.648
  18:37:21 STEP  708 H= 177:00 +CPU= 25.742
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000708
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000708
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000708
  18:37:42 STEP  709 H= 177:15 +CPU= 20.879
  18:38:03 STEP  710 H= 177:30 +CPU= 20.866
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:38:43 STEP  711 H= 177:45 +CPU= 19.617
  18:39:04 STEP  712 H= 178:00 +CPU= 20.719
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:39:46 STEP  713 H= 178:15 +CPU= 20.249
  18:40:07 STEP  714 H= 178:30 +CPU= 20.860
  18:40:27 STEP  715 H= 178:45 +CPU= 20.703
  18:40:48 STEP  716 H= 179:00 +CPU= 21.235
  18:41:09 STEP  717 H= 179:15 +CPU= 20.659
  18:41:30 STEP  718 H= 179:30 +CPU= 20.635
  18:41:50 STEP  719 H= 179:45 +CPU= 20.551
  18:42:21 STEP  720 H= 180:00 +CPU= 29.806
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000720
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000720
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000720
  18:42:41 STEP  721 H= 180:15 +CPU= 20.690
  18:43:02 STEP  722 H= 180:30 +CPU= 20.649
  18:43:23 STEP  723 H= 180:45 +CPU= 20.764
  18:43:44 STEP  724 H= 181:00 +CPU= 21.091
  18:44:05 STEP  725 H= 181:15 +CPU= 20.776
  18:44:25 STEP  726 H= 181:30 +CPU= 20.691
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:45:07 STEP  727 H= 181:45 +CPU= 20.319
  18:45:28 STEP  728 H= 182:00 +CPU= 20.805
  18:45:48 STEP  729 H= 182:15 +CPU= 20.753
  18:46:09 STEP  730 H= 182:30 +CPU= 20.861
  18:46:30 STEP  731 H= 182:45 +CPU= 20.901
  18:46:56 STEP  732 H= 183:00 +CPU= 25.800
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000732
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000732
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000732
  18:47:17 STEP  733 H= 183:15 +CPU= 20.865
  18:47:38 STEP  734 H= 183:30 +CPU= 20.845
  18:47:59 STEP  735 H= 183:45 +CPU= 20.803
  18:48:20 STEP  736 H= 184:00 +CPU= 21.100
  18:48:41 STEP  737 H= 184:15 +CPU= 20.952
  18:49:02 STEP  738 H= 184:30 +CPU= 20.670
  18:49:22 STEP  739 H= 184:45 +CPU= 20.818
  18:49:44 STEP  740 H= 185:00 +CPU= 21.135
  18:50:05 STEP  741 H= 185:15 +CPU= 20.946
  18:50:25 STEP  742 H= 185:30 +CPU= 20.773
  18:50:46 STEP  743 H= 185:45 +CPU= 20.734
  18:51:12 STEP  744 H= 186:00 +CPU= 25.970
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000744
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000744
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000744
  18:51:33 STEP  745 H= 186:15 +CPU= 20.937
  18:51:54 STEP  746 H= 186:30 +CPU= 20.884
  18:52:15 STEP  747 H= 186:45 +CPU= 20.794
  18:52:36 STEP  748 H= 187:00 +CPU= 21.170
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:53:17 STEP  749 H= 187:15 +CPU= 19.879
  18:53:38 STEP  750 H= 187:30 +CPU= 20.586
  18:53:58 STEP  751 H= 187:45 +CPU= 20.712
  18:54:19 STEP  752 H= 188:00 +CPU= 21.033
  18:54:40 STEP  753 H= 188:15 +CPU= 20.750
  18:55:01 STEP  754 H= 188:30 +CPU= 20.788
  18:55:22 STEP  755 H= 188:45 +CPU= 20.759
  18:55:48 STEP  756 H= 189:00 +CPU= 25.723
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000756
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000756
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000756
  18:56:09 STEP  757 H= 189:15 +CPU= 20.927
  18:56:30 STEP  758 H= 189:30 +CPU= 20.940
  18:56:51 STEP  759 H= 189:45 +CPU= 20.926
  18:57:12 STEP  760 H= 190:00 +CPU= 21.124
  18:57:33 STEP  761 H= 190:15 +CPU= 20.943
  18:57:54 STEP  762 H= 190:30 +CPU= 20.963
  18:58:14 STEP  763 H= 190:45 +CPU= 20.625
  18:58:36 STEP  764 H= 191:00 +CPU= 21.276
  18:58:56 STEP  765 H= 191:15 +CPU= 20.729
  18:59:17 STEP  766 H= 191:30 +CPU= 20.932
  18:59:38 STEP  767 H= 191:45 +CPU= 20.891
  19:00:08 STEP  768 H= 192:00 +CPU= 29.910
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000768
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000768
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000768
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000672
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000672
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000672
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000684
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000684
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000684
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000696
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000696
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000696
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000708
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000708
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000708
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000720
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000720
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000720
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000732
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000732
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000732
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000744
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000744
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000744
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000756
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000756
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000756
Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a25p_2016092300_20_1018_12291124_0_r1343717625_7.zip
Uploading the intermediate file: upload_file_7.zip
  19:00:30 STEP  769 H= 192:15 +CPU= 20.913
  19:00:50 STEP  770 H= 192:30 +CPU= 20.843
  19:01:11 STEP  771 H= 192:45 +CPU= 20.750
  19:01:32 STEP  772 H= 193:00 +CPU= 21.181
  19:01:53 STEP  773 H= 193:15 +CPU= 20.709
  19:02:14 STEP  774 H= 193:30 +CPU= 20.690
  19:02:35 STEP  775 H= 193:45 +CPU= 20.822
  19:02:56 STEP  776 H= 194:00 +CPU= 21.161
  19:03:17 STEP  777 H= 194:15 +CPU= 20.807
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:03:58 STEP  778 H= 194:30 +CPU= 20.241
  19:04:19 STEP  779 H= 194:45 +CPU= 20.471
  19:04:45 STEP  780 H= 195:00 +CPU= 25.835
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000780
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000780
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000780
  19:05:05 STEP  781 H= 195:15 +CPU= 20.557
  19:05:26 STEP  782 H= 195:30 +CPU= 20.556
  19:05:47 STEP  783 H= 195:45 +CPU= 20.601
  19:06:08 STEP  784 H= 196:00 +CPU= 21.002
  19:06:29 STEP  785 H= 196:15 +CPU= 20.760
  19:06:49 STEP  786 H= 196:30 +CPU= 20.594
  19:07:10 STEP  787 H= 196:45 +CPU= 20.627
  19:07:31 STEP  788 H= 197:00 +CPU= 21.307
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:08:12 STEP  789 H= 197:15 +CPU= 19.906
  19:08:33 STEP  790 H= 197:30 +CPU= 20.562
  19:08:54 STEP  791 H= 197:45 +CPU= 20.929
  19:09:20 STEP  792 H= 198:00 +CPU= 26.003
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000792
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000792
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000792
  19:09:41 STEP  793 H= 198:15 +CPU= 20.694
  19:10:01 STEP  794 H= 198:30 +CPU= 20.725
  19:10:22 STEP  795 H= 198:45 +CPU= 20.688
  19:10:43 STEP  796 H= 199:00 +CPU= 20.970
  19:11:04 STEP  797 H= 199:15 +CPU= 20.845
  19:11:25 STEP  798 H= 199:30 +CPU= 20.722
  19:11:46 STEP  799 H= 199:45 +CPU= 20.947
  19:12:07 STEP  800 H= 200:00 +CPU= 21.312
  19:12:28 STEP  801 H= 200:15 +CPU= 20.686
  19:12:49 STEP  802 H= 200:30 +CPU= 20.772
  19:13:10 STEP  803 H= 200:45 +CPU= 20.723
  19:13:35 STEP  804 H= 201:00 +CPU= 25.683
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000804
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000804
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000804
  19:13:56 STEP  805 H= 201:15 +CPU= 20.753
  19:14:17 STEP  806 H= 201:30 +CPU= 20.666
  19:14:38 STEP  807 H= 201:45 +CPU= 20.771
  19:14:59 STEP  808 H= 202:00 +CPU= 21.114
  19:15:20 STEP  809 H= 202:15 +CPU= 20.833
  19:15:41 STEP  810 H= 202:30 +CPU= 20.758
  19:16:01 STEP  811 H= 202:45 +CPU= 20.630
  19:16:23 STEP  812 H= 203:00 +CPU= 21.135
  19:16:43 STEP  813 H= 203:15 +CPU= 20.704
  19:17:04 STEP  814 H= 203:30 +CPU= 20.724
  19:17:25 STEP  815 H= 203:45 +CPU= 20.606
  19:17:55 STEP  816 H= 204:00 +CPU= 29.777
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000816
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000816
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000816
  19:18:15 STEP  817 H= 204:15 +CPU= 20.729
  19:18:36 STEP  818 H= 204:30 +CPU= 20.684
  19:18:57 STEP  819 H= 204:45 +CPU= 20.802
  19:19:18 STEP  820 H= 205:00 +CPU= 21.168
  19:19:39 STEP  821 H= 205:15 +CPU= 20.603
  19:20:00 STEP  822 H= 205:30 +CPU= 20.830
  19:20:20 STEP  823 H= 205:45 +CPU= 20.685
  19:20:42 STEP  824 H= 206:00 +CPU= 21.312
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:21:23 STEP  825 H= 206:15 +CPU= 20.335
  19:21:44 STEP  826 H= 206:30 +CPU= 20.708
  19:22:05 STEP  827 H= 206:45 +CPU= 20.676
  19:22:30 STEP  828 H= 207:00 +CPU= 25.774
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000828
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000828
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000828
  19:22:51 STEP  829 H= 207:15 +CPU= 20.774
  19:23:12 STEP  830 H= 207:30 +CPU= 20.707
  19:23:33 STEP  831 H= 207:45 +CPU= 20.799
  19:23:54 STEP  832 H= 208:00 +CPU= 21.082
  19:24:17 STEP  833 H= 208:15 +CPU= 22.661
  19:24:40 STEP  834 H= 208:30 +CPU= 21.747
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:25:21 STEP  835 H= 208:45 +CPU= 20.509
  19:25:45 STEP  836 H= 209:00 +CPU= 22.642
  19:26:08 STEP  837 H= 209:15 +CPU= 22.481
  19:26:29 STEP  838 H= 209:30 +CPU= 21.366
  19:26:51 STEP  839 H= 209:45 +CPU= 21.555
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:27:39 STEP  840 H= 210:00 +CPU= 26.332
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000840
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000840
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000840
  19:28:01 STEP  841 H= 210:15 +CPU= 22.153
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a25p
(argv4) batchid: 1018
(argv5) wuid: 12291124
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /home/robbie/slots/16
Project directory is: /home/robbie/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124
..mkdir for temp folder for results failed
Copying: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /home/robbie/slots/16/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /home/robbie/slots/16/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_799e61a8ea8a8d5e53b469a55a2744e0 to: /home/robbie/slots/16/oifs_43r3_bl_a25p_2016092300_20_1018_12291124.zip
Unzipping the namelist zip file: /home/robbie/slots/16/oifs_43r3_bl_a25p_2016092300_20_1018_12291124.zip
ic_ancil_file: ic_ancil_12291124
ifsdata_file: ifsdata_12291124
climate_data_file: clim_data_12291124
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /home/robbie/slots/16/ic_ancil_12291124.zip
Unzipping the IC ancils zip file: /home/robbie/slots/16/ic_ancil_12291124.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ba8765fdf72061374855111149e0f2a4 to: /home/robbie/slots/16/ifsdata/ifsdata_12291124.zip
Unzipping the ifsdata_zip file: /home/robbie/slots/16/ifsdata/ifsdata_12291124.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/robbie/slots/16/159l_2/clim_data_12291124.zip
Unzipping the climate data zip file: /home/robbie/slots/16/159l_2/clim_data_12291124.zip
Checking for progress XML file: /home/robbie/slots/16/progress_file_12291124.xml
Opened progress file ok : /home/robbie/slots/16/progress_file_12291124.xml
-- Model is restarting --
Adjusting last_iter, 842, to previous model restart step.
Creating progress file: /home/robbie/slots/16/progress_file_12291124.xml
last_cpu_time: 17914
upload_file_number: 8
last_iter: 817
last_upload: 691200
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a25p_2016092300_20_1018_12291124_0_r1343717625
The child process has been launched with process id: 1531
Executing the command: /home/robbie/slots/16/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at 0x1)
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:30:56 STEP  816 H= 204:00 +CPU= 22.039
  19:31:17 STEP  817 H= 204:15 +CPU= 21.839
  19:31:39 STEP  818 H= 204:30 +CPU= 21.728
  19:32:01 STEP  819 H= 204:45 +CPU= 21.638
  19:32:24 STEP  820 H= 205:00 +CPU= 22.780
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:33:03 STEP  821 H= 205:15 +CPU= 17.602
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:33:46 STEP  822 H= 205:30 +CPU= 20.540
  19:34:08 STEP  823 H= 205:45 +CPU= 21.313
  19:34:28 STEP  824 H= 206:00 +CPU= 20.455
  19:34:49 STEP  825 H= 206:15 +CPU= 20.716
  19:35:09 STEP  826 H= 206:30 +CPU= 19.766
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:35:48 STEP  827 H= 206:45 +CPU= 17.765
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:37:12 STEP  828 H= 207:00 +CPU= 22.448
Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000828
Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000828
Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000828
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:13:21 STEP  829 H= 207:15 +CPU= 18.569
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:14:01 STEP  830 H= 207:30 +CPU= 18.894
  20:14:19 STEP  831 H= 207:45 +CPU= 16.149
  20:16:46 STEP  832 H= 208:00 +CPU= 13.603
  20:16:59 STEP  833 H= 208:15 +CPU= 12.014
  20:17:13 STEP  834 H= 208:30 +CPU= 11.601
  20:17:25 STEP  835 H= 208:45 +CPU= 11.668
  20:17:37 STEP  836 H= 209:00 +CPU= 11.792
  21:49:48 STEP  837 H= 209:15 +CPU= 22.486
  21:50:07 STEP  838 H= 209:30 +CPU= 10.460
  21:50:17 STEP  839 H= 209:45 +CPU= 10.368
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
..The child process has been killed with signal: 9
>>> Printing last 70 lines from file: NODE.001_01

ISTEP=     840: Reset RSOLINC= 1361.4482 RIP0= 1361.4482
 SU_GHGCLIM, RCARDI,RCH4,RN2O,RCFC11,RCFC12,RNO2:   6.179466599480000E-004
  1.088035170184667E-006  4.990317592110001E-007  3.778213382954999E-009
  2.097581753513333E-009  7.940283000000001E-011
 suecaec NAERMACC =  1
 UPDT RAESHDU =  2000.0
 SUECAEC: GLOBAL AVERAGE VOLCANIC SULPHATE FOR (KINDAT,KMINUT)     20160923
        1170  9.999999999999998E-005
 NORMS AT NSTEP CNT4  840
 SPECTRAL NORMS -   LOG(PREHYDS)    0.115260884535244E+02
 LEV     VORTICITY             DIVERGENCE           TEMPERATURE           KINETIC ENERGY    
 AVE 0.837199945155559E-06 0.187344209400869E-07 0.217437694554772E+03 0.462863193667014E+01
 GPNORM  HUMIDITY            AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.659859636017025E-03 0.100000000000000E-07 0.867830843596815E-02
 GPNORM  LIQUID WATER        AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM  ICE WATER           AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM SNOW                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM RAIN                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM  CLOUD FRACTION      AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CLW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CIW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CC                   AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM RFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.511005718465802E-13 0.000000000000000E+00 0.208070111402550E-11
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.219780219780221E-02 0.000000000000000E+00 0.100000000000000E+00
 NSTEP =   840 STEPO   A00000000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   840 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000840 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+000840
 MPI-TASK:   1 -    310566276 BYTES IN      6 RECORDS TRANSFERRED IN    0.0007 SECONDS443666.1 Mbytes/s, TOTAL TIME=    0.1955( 0.4%)
 NSTEP =   840 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+000840 MODE=w
 IO-STREAM CLOSED - ./ICMSHh7zg+000840
 MPI-TASK:   1 -    707399250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0210 SECONDS 33685.7 Mbytes/s, TOTAL TIME=    0.2814( 7.5%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000840 MODE=w
 IO-STREAM CLOSED - ./ICMUAh7zg+000840
 MPI-TASK:   1 -   4969060416 BYTES IN    125 RECORDS TRANSFERRED IN    0.0075 SECONDS662541.4 Mbytes/s, TOTAL TIME=    0.1318( 5.7%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000840 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000840
 MPI-TASK:   1 -           10 BYTES IN      2 RECORDS TRANSFERRED IN    0.0004 SECONDS     0.0 Mbytes/s, TOTAL TIME=    0.0031(12.9%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000840 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000840
 MPI-TASK:   1 -    301986547 BYTES IN      4 RECORDS TRANSFERRED IN    0.0004 SECONDS754966.4 Mbytes/s, TOTAL TIME=    0.0048( 8.3%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =   840 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 21:50:07 0AAA00AAA STEPO      839    10.459   10.459   15.197      7:01    140:42 0.18149607650051E-07       2GB       0MB
 21:50:47 A00000000 STEPO      840    10.489   10.489   40.260      7:12    141:22 0.18734420940087E-07       2GB       0MB
 21:50:47 0AA000000 STEPO      840     0.026    0.026    0.383      7:12    141:22 0.18734420940087E-07       2GB       0MB
 21:50:50 FULLPOS-B DYNFPOS    840     1.171    1.171    2.251      7:13    141:24 0.18734420940087E-07       2GB       0MB
 21:50:52 FULLPOS-V DYNFPOS    840     2.404    2.404    2.709      7:15    141:27 0.18734420940087E-07       2GB       0MB
 21:50:53 FULLPOS-T DYNFPOS    840     0.301    0.301    0.307      7:15    141:27 0.18734420940087E-07       2GB       0MB
 21:50:53 0AAA00AAA STEPO      840     0.296    0.296    0.296      7:16    141:28 0.18734420940087E-07       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "     816",
 CTIME   = "00081200      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,         804, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /home/robbie/slots/16/progress_file_12291124.xml

<running_values>
  <last_cpu_time>18356.420000</last_cpu_time>
  <upload_file_number>8</upload_file_number>
  <last_iter>840</last_iter>
  <last_upload>691200</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 cpdn.org