Name | oifs_43r3_bl_a25p_2016092300_20_1018_12291124_0 |
Workunit | 12291124 |
Created | 13 Jun 2024, 0:05:23 UTC |
Sent | 13 Jun 2024, 13:07:45 UTC |
Report deadline | 12 Aug 2024, 13:07:45 UTC |
Received | 13 Jun 2024, 21:52:11 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 1 (0x00000001) Unknown error code |
Computer ID | 1548614 |
Run time | 5 hours 5 min 59 sec |
CPU time | 5 hours 5 min 59 sec |
Validate state | Invalid |
Credit | 0.00 |
Device peak FLOPS | 6.71 GFLOPS |
Application version | OpenIFS 43r3 Baroclinic Lifecycle v1.13 x86_64-pc-linux-gnu |
Peak working set size | 5,016.34 MB |
Peak swap size | 5,981.07 MB |
Peak disk usage | 1,344.52 MB |
Stderr | <core_client_version>7.22.1</core_client_version> <![CDATA[ <message> process exited with code 1 (0x1, -255)</message> <stderr_txt> ctory: /home/robbie/slots/16/ICMGGh7zg+000372 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000372 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000372 Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:32:05 STEP 373 H= 93:15 +CPU= 20.115 Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:32:46 STEP 374 H= 93:30 +CPU= 20.383 16:33:08 STEP 375 H= 93:45 +CPU= 20.808 16:33:29 STEP 376 H= 94:00 +CPU= 21.269 16:33:50 STEP 377 H= 94:15 +CPU= 20.833 16:34:11 STEP 378 H= 94:30 +CPU= 20.822 16:34:31 STEP 379 H= 94:45 +CPU= 20.687 16:34:53 STEP 380 H= 95:00 +CPU= 21.282 16:35:14 STEP 381 H= 95:15 +CPU= 20.882 16:35:34 STEP 382 H= 95:30 +CPU= 20.697 16:35:55 STEP 383 H= 95:45 +CPU= 20.914 16:36:26 STEP 384 H= 96:00 +CPU= 30.046 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000384 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000384 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000384 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000288 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000288 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000288 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000300 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000300 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000300 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000312 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000312 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000312 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000324 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000324 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000324 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000336 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000336 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000336 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000348 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000348 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000348 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000360 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000360 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000360 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000372 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000372 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000372 Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a25p_2016092300_20_1018_12291124_0_r1343717625_3.zip 16:36:47 STEP 385 H= 96:15 +CPU= 20.895 Uploading the intermediate file: upload_file_3.zip 16:37:08 STEP 386 H= 96:30 +CPU= 20.746 16:37:29 STEP 387 H= 96:45 +CPU= 20.759 16:37:50 STEP 388 H= 97:00 +CPU= 21.199 Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:38:31 STEP 389 H= 97:15 +CPU= 20.066 16:38:52 STEP 390 H= 97:30 +CPU= 20.911 16:39:13 STEP 391 H= 97:45 +CPU= 20.783 16:39:35 STEP 392 H= 98:00 +CPU= 21.401 16:39:56 STEP 393 H= 98:15 +CPU= 20.891 16:40:16 STEP 394 H= 98:30 +CPU= 20.629 16:40:37 STEP 395 H= 98:45 +CPU= 20.908 16:41:03 STEP 396 H= 99:00 +CPU= 26.037 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000396 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000396 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000396 16:41:24 STEP 397 H= 99:15 +CPU= 21.043 16:41:45 STEP 398 H= 99:30 +CPU= 20.841 16:42:06 STEP 399 H= 99:45 +CPU= 20.701 16:42:27 STEP 400 H= 100:00 +CPU= 21.147 16:42:48 STEP 401 H= 100:15 +CPU= 20.739 16:43:09 STEP 402 H= 100:30 +CPU= 20.653 16:43:30 STEP 403 H= 100:45 +CPU= 20.768 16:43:51 STEP 404 H= 101:00 +CPU= 21.121 16:44:12 STEP 405 H= 101:15 +CPU= 20.929 16:44:33 STEP 406 H= 101:30 +CPU= 20.889 16:44:53 STEP 407 H= 101:45 +CPU= 20.756 16:45:19 STEP 408 H= 102:00 +CPU= 25.861 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000408 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000408 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000408 16:45:40 STEP 409 H= 102:15 +CPU= 20.895 16:46:01 STEP 410 H= 102:30 +CPU= 20.445 Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:46:41 STEP 411 H= 102:45 +CPU= 19.301 16:47:02 STEP 412 H= 103:00 +CPU= 21.197 16:47:23 STEP 413 H= 103:15 +CPU= 20.730 16:47:44 STEP 414 H= 103:30 +CPU= 20.648 16:48:05 STEP 415 H= 103:45 +CPU= 20.663 16:48:26 STEP 416 H= 104:00 +CPU= 21.127 16:48:47 STEP 417 H= 104:15 +CPU= 20.701 16:49:08 STEP 418 H= 104:30 +CPU= 20.804 16:49:29 STEP 419 H= 104:45 +CPU= 20.709 16:49:55 STEP 420 H= 105:00 +CPU= 26.010 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000420 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000420 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000420 16:50:16 STEP 421 H= 105:15 +CPU= 21.015 16:50:37 STEP 422 H= 105:30 +CPU= 20.925 16:50:57 STEP 423 H= 105:45 +CPU= 20.826 16:51:19 STEP 424 H= 106:00 +CPU= 21.280 16:51:40 STEP 425 H= 106:15 +CPU= 20.898 16:52:01 STEP 426 H= 106:30 +CPU= 20.832 16:52:21 STEP 427 H= 106:45 +CPU= 20.712 16:52:43 STEP 428 H= 107:00 +CPU= 21.236 16:53:04 STEP 429 H= 107:15 +CPU= 20.951 16:53:25 STEP 430 H= 107:30 +CPU= 20.691 16:53:46 STEP 431 H= 107:45 +CPU= 20.909 16:54:16 STEP 432 H= 108:00 +CPU= 30.016 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000432 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000432 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000432 16:54:37 STEP 433 H= 108:15 +CPU= 20.805 16:54:58 STEP 434 H= 108:30 +CPU= 20.861 16:55:19 STEP 435 H= 108:45 +CPU= 20.951 16:55:40 STEP 436 H= 109:00 +CPU= 21.253 16:56:01 STEP 437 H= 109:15 +CPU= 20.897 16:56:22 STEP 438 H= 109:30 +CPU= 20.850 16:56:43 STEP 439 H= 109:45 +CPU= 20.872 16:57:04 STEP 440 H= 110:00 +CPU= 21.148 Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:57:45 STEP 441 H= 110:15 +CPU= 20.469 16:58:06 STEP 442 H= 110:30 +CPU= 20.693 16:58:27 STEP 443 H= 110:45 +CPU= 20.814 16:58:53 STEP 444 H= 111:00 +CPU= 25.898 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000444 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000444 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000444 16:59:14 STEP 445 H= 111:15 +CPU= 20.779 16:59:35 STEP 446 H= 111:30 +CPU= 20.789 16:59:55 STEP 447 H= 111:45 +CPU= 20.720 17:00:17 STEP 448 H= 112:00 +CPU= 21.171 17:00:38 STEP 449 H= 112:15 +CPU= 20.831 17:00:59 STEP 450 H= 112:30 +CPU= 20.700 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:01:40 STEP 451 H= 112:45 +CPU= 20.309 17:02:01 STEP 452 H= 113:00 +CPU= 21.166 17:02:22 STEP 453 H= 113:15 +CPU= 20.755 17:02:43 STEP 454 H= 113:30 +CPU= 20.633 17:03:03 STEP 455 H= 113:45 +CPU= 20.703 17:03:29 STEP 456 H= 114:00 +CPU= 25.865 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000456 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000456 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000456 17:03:50 STEP 457 H= 114:15 +CPU= 20.734 17:04:11 STEP 458 H= 114:30 +CPU= 20.935 17:04:32 STEP 459 H= 114:45 +CPU= 21.121 17:04:54 STEP 460 H= 115:00 +CPU= 21.352 17:05:14 STEP 461 H= 115:15 +CPU= 20.860 17:05:35 STEP 462 H= 115:30 +CPU= 20.675 17:05:56 STEP 463 H= 115:45 +CPU= 20.842 17:06:18 STEP 464 H= 116:00 +CPU= 21.196 17:06:38 STEP 465 H= 116:15 +CPU= 20.667 17:06:59 STEP 466 H= 116:30 +CPU= 20.749 17:07:20 STEP 467 H= 116:45 +CPU= 20.769 17:07:46 STEP 468 H= 117:00 +CPU= 25.945 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000468 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000468 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000468 17:08:07 STEP 469 H= 117:15 +CPU= 20.638 17:08:28 STEP 470 H= 117:30 +CPU= 20.869 17:08:49 STEP 471 H= 117:45 +CPU= 20.942 17:09:10 STEP 472 H= 118:00 +CPU= 21.171 17:09:31 STEP 473 H= 118:15 +CPU= 20.759 17:09:51 STEP 474 H= 118:30 +CPU= 20.676 17:10:12 STEP 475 H= 118:45 +CPU= 20.721 17:10:33 STEP 476 H= 119:00 +CPU= 21.258 17:10:54 STEP 477 H= 119:15 +CPU= 20.656 17:11:15 STEP 478 H= 119:30 +CPU= 20.879 17:11:36 STEP 479 H= 119:45 +CPU= 20.966 17:12:06 STEP 480 H= 120:00 +CPU= 30.009 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000480 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000480 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000480 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000384 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000384 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000384 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000396 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000396 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000396 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000408 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000408 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000408 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000420 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000420 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000420 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000432 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000432 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000432 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000444 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000444 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000444 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000456 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000456 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000456 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000468 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000468 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000468 Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a25p_2016092300_20_1018_12291124_0_r1343717625_4.zip Uploading the intermediate file: upload_file_4.zip 17:12:28 STEP 481 H= 120:15 +CPU= 20.903 17:12:48 STEP 482 H= 120:30 +CPU= 20.696 17:13:09 STEP 483 H= 120:45 +CPU= 20.835 17:13:31 STEP 484 H= 121:00 +CPU= 21.555 17:13:52 STEP 485 H= 121:15 +CPU= 20.745 17:14:13 STEP 486 H= 121:30 +CPU= 20.970 17:14:34 STEP 487 H= 121:45 +CPU= 20.712 17:14:55 STEP 488 H= 122:00 +CPU= 21.309 17:15:16 STEP 489 H= 122:15 +CPU= 20.863 17:15:37 STEP 490 H= 122:30 +CPU= 20.704 17:15:57 STEP 491 H= 122:45 +CPU= 20.672 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:16:43 STEP 492 H= 123:00 +CPU= 25.102 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000492 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000492 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000492 17:17:04 STEP 493 H= 123:15 +CPU= 20.752 17:17:25 STEP 494 H= 123:30 +CPU= 20.744 17:17:46 STEP 495 H= 123:45 +CPU= 20.873 17:18:07 STEP 496 H= 124:00 +CPU= 21.253 17:18:28 STEP 497 H= 124:15 +CPU= 20.732 17:18:49 STEP 498 H= 124:30 +CPU= 20.809 17:19:10 STEP 499 H= 124:45 +CPU= 20.805 17:19:31 STEP 500 H= 125:00 +CPU= 21.225 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:20:12 STEP 501 H= 125:15 +CPU= 19.665 17:20:33 STEP 502 H= 125:30 +CPU= 20.780 17:20:53 STEP 503 H= 125:45 +CPU= 20.743 17:21:19 STEP 504 H= 126:00 +CPU= 25.855 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000504 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000504 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000504 17:21:40 STEP 505 H= 126:15 +CPU= 20.883 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:22:22 STEP 506 H= 126:30 +CPU= 20.258 17:22:42 STEP 507 H= 126:45 +CPU= 20.680 17:23:04 STEP 508 H= 127:00 +CPU= 21.255 17:23:25 STEP 509 H= 127:15 +CPU= 20.752 17:23:45 STEP 510 H= 127:30 +CPU= 20.680 17:24:06 STEP 511 H= 127:45 +CPU= 20.876 17:24:27 STEP 512 H= 128:00 +CPU= 21.167 17:24:48 STEP 513 H= 128:15 +CPU= 20.806 17:25:09 STEP 514 H= 128:30 +CPU= 20.807 17:25:30 STEP 515 H= 128:45 +CPU= 20.729 17:25:55 STEP 516 H= 129:00 +CPU= 25.408 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000516 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000516 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000516 17:26:36 STEP 517 H= 129:15 +CPU= 19.788 17:26:57 STEP 518 H= 129:30 +CPU= 20.717 17:27:18 STEP 519 H= 129:45 +CPU= 20.710 17:27:39 STEP 520 H= 130:00 +CPU= 21.190 17:28:00 STEP 521 H= 130:15 +CPU= 20.689 17:28:20 STEP 522 H= 130:30 +CPU= 20.716 17:28:41 STEP 523 H= 130:45 +CPU= 20.763 17:29:02 STEP 524 H= 131:00 +CPU= 21.098 17:29:23 STEP 525 H= 131:15 +CPU= 20.727 17:29:44 STEP 526 H= 131:30 +CPU= 20.878 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:30:25 STEP 527 H= 131:45 +CPU= 20.324 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:31:16 STEP 528 H= 132:00 +CPU= 29.185 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000528 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000528 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000528 17:31:36 STEP 529 H= 132:15 +CPU= 20.601 17:31:57 STEP 530 H= 132:30 +CPU= 20.825 17:32:18 STEP 531 H= 132:45 +CPU= 20.749 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:32:59 STEP 532 H= 133:00 +CPU= 20.029 17:33:20 STEP 533 H= 133:15 +CPU= 20.790 17:33:41 STEP 534 H= 133:30 +CPU= 20.806 17:34:02 STEP 535 H= 133:45 +CPU= 20.760 17:34:23 STEP 536 H= 134:00 +CPU= 21.313 17:34:44 STEP 537 H= 134:15 +CPU= 20.833 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:35:25 STEP 538 H= 134:30 +CPU= 19.451 17:35:45 STEP 539 H= 134:45 +CPU= 20.690 17:36:11 STEP 540 H= 135:00 +CPU= 25.717 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000540 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000540 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000540 17:36:32 STEP 541 H= 135:15 +CPU= 20.682 17:36:53 STEP 542 H= 135:30 +CPU= 20.745 17:37:14 STEP 543 H= 135:45 +CPU= 20.757 17:37:35 STEP 544 H= 136:00 +CPU= 21.193 17:37:56 STEP 545 H= 136:15 +CPU= 20.864 17:38:16 STEP 546 H= 136:30 +CPU= 20.771 17:38:37 STEP 547 H= 136:45 +CPU= 20.617 17:38:58 STEP 548 H= 137:00 +CPU= 21.206 17:39:19 STEP 549 H= 137:15 +CPU= 20.624 17:39:40 STEP 550 H= 137:30 +CPU= 20.991 17:40:01 STEP 551 H= 137:45 +CPU= 20.877 17:40:27 STEP 552 H= 138:00 +CPU= 25.879 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000552 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000552 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000552 17:40:48 STEP 553 H= 138:15 +CPU= 20.754 17:41:09 STEP 554 H= 138:30 +CPU= 20.833 17:41:30 STEP 555 H= 138:45 +CPU= 20.764 17:41:51 STEP 556 H= 139:00 +CPU= 21.149 17:42:11 STEP 557 H= 139:15 +CPU= 20.631 17:42:32 STEP 558 H= 139:30 +CPU= 20.715 17:42:53 STEP 559 H= 139:45 +CPU= 20.700 17:43:14 STEP 560 H= 140:00 +CPU= 21.066 17:43:35 STEP 561 H= 140:15 +CPU= 20.804 17:43:56 STEP 562 H= 140:30 +CPU= 20.907 17:44:17 STEP 563 H= 140:45 +CPU= 20.764 17:44:43 STEP 564 H= 141:00 +CPU= 25.916 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000564 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000564 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000564 17:45:04 STEP 565 H= 141:15 +CPU= 20.864 17:45:24 STEP 566 H= 141:30 +CPU= 20.706 17:45:45 STEP 567 H= 141:45 +CPU= 20.899 17:46:06 STEP 568 H= 142:00 +CPU= 21.038 17:46:27 STEP 569 H= 142:15 +CPU= 20.721 17:46:48 STEP 570 H= 142:30 +CPU= 20.617 17:47:09 STEP 571 H= 142:45 +CPU= 20.826 17:47:30 STEP 572 H= 143:00 +CPU= 21.246 17:47:51 STEP 573 H= 143:15 +CPU= 20.924 17:48:12 STEP 574 H= 143:30 +CPU= 20.744 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:48:52 STEP 575 H= 143:45 +CPU= 19.687 17:49:22 STEP 576 H= 144:00 +CPU= 29.819 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000576 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000576 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000576 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000480 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000480 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000480 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000492 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000492 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000492 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000504 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000504 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000504 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000516 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000516 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000516 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000528 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000528 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000528 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000540 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000540 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000540 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000552 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000552 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000552 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000564 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000564 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000564 Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a25p_2016092300_20_1018_12291124_0_r1343717625_5.zip Uploading the intermediate file: upload_file_5.zip 17:49:43 STEP 577 H= 144:15 +CPU= 20.745 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:50:25 STEP 578 H= 144:30 +CPU= 20.246 17:50:46 STEP 579 H= 144:45 +CPU= 20.907 17:51:07 STEP 580 H= 145:00 +CPU= 21.145 17:51:28 STEP 581 H= 145:15 +CPU= 20.756 17:51:49 STEP 582 H= 145:30 +CPU= 20.870 17:52:09 STEP 583 H= 145:45 +CPU= 20.842 17:52:31 STEP 584 H= 146:00 +CPU= 21.152 17:52:51 STEP 585 H= 146:15 +CPU= 20.638 17:53:12 STEP 586 H= 146:30 +CPU= 20.787 17:53:33 STEP 587 H= 146:45 +CPU= 20.766 17:53:59 STEP 588 H= 147:00 +CPU= 25.805 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000588 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000588 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000588 17:54:20 STEP 589 H= 147:15 +CPU= 20.702 17:54:41 STEP 590 H= 147:30 +CPU= 20.801 17:55:01 STEP 591 H= 147:45 +CPU= 20.672 17:55:23 STEP 592 H= 148:00 +CPU= 21.080 17:55:43 STEP 593 H= 148:15 +CPU= 20.711 17:56:04 STEP 594 H= 148:30 +CPU= 20.757 17:56:25 STEP 595 H= 148:45 +CPU= 20.826 17:56:46 STEP 596 H= 149:00 +CPU= 21.287 17:57:07 STEP 597 H= 149:15 +CPU= 20.693 17:57:28 STEP 598 H= 149:30 +CPU= 21.013 17:57:49 STEP 599 H= 149:45 +CPU= 20.737 17:58:15 STEP 600 H= 150:00 +CPU= 25.854 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000600 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000600 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000600 17:58:35 STEP 601 H= 150:15 +CPU= 20.659 17:58:56 STEP 602 H= 150:30 +CPU= 20.866 17:59:17 STEP 603 H= 150:45 +CPU= 20.785 17:59:39 STEP 604 H= 151:00 +CPU= 21.291 17:59:59 STEP 605 H= 151:15 +CPU= 20.808 18:00:20 STEP 606 H= 151:30 +CPU= 20.635 18:00:41 STEP 607 H= 151:45 +CPU= 20.868 18:01:02 STEP 608 H= 152:00 +CPU= 21.236 18:01:23 STEP 609 H= 152:15 +CPU= 20.618 18:01:44 STEP 610 H= 152:30 +CPU= 20.785 18:02:05 STEP 611 H= 152:45 +CPU= 20.819 18:02:31 STEP 612 H= 153:00 +CPU= 25.757 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000612 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000612 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000612 18:02:51 STEP 613 H= 153:15 +CPU= 20.756 18:03:12 STEP 614 H= 153:30 +CPU= 20.730 18:03:33 STEP 615 H= 153:45 +CPU= 20.718 18:03:54 STEP 616 H= 154:00 +CPU= 21.245 18:04:15 STEP 617 H= 154:15 +CPU= 20.718 18:04:36 STEP 618 H= 154:30 +CPU= 20.737 18:04:56 STEP 619 H= 154:45 +CPU= 20.806 18:05:17 STEP 620 H= 155:00 +CPU= 21.074 18:05:38 STEP 621 H= 155:15 +CPU= 20.722 18:05:59 STEP 622 H= 155:30 +CPU= 20.704 18:06:20 STEP 623 H= 155:45 +CPU= 20.861 18:06:50 STEP 624 H= 156:00 +CPU= 29.933 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000624 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000624 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000624 18:07:11 STEP 625 H= 156:15 +CPU= 21.000 18:07:32 STEP 626 H= 156:30 +CPU= 20.658 18:07:53 STEP 627 H= 156:45 +CPU= 20.823 18:08:14 STEP 628 H= 157:00 +CPU= 21.209 18:08:35 STEP 629 H= 157:15 +CPU= 20.771 18:08:56 STEP 630 H= 157:30 +CPU= 20.752 18:09:16 STEP 631 H= 157:45 +CPU= 20.798 18:09:38 STEP 632 H= 158:00 +CPU= 21.139 18:09:58 STEP 633 H= 158:15 +CPU= 20.754 18:10:19 STEP 634 H= 158:30 +CPU= 20.901 18:10:40 STEP 635 H= 158:45 +CPU= 20.839 18:11:06 STEP 636 H= 159:00 +CPU= 25.816 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000636 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000636 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000636 18:11:27 STEP 637 H= 159:15 +CPU= 20.773 18:11:48 STEP 638 H= 159:30 +CPU= 20.752 18:12:09 STEP 639 H= 159:45 +CPU= 20.681 18:12:30 STEP 640 H= 160:00 +CPU= 21.242 18:12:51 STEP 641 H= 160:15 +CPU= 20.712 18:13:12 STEP 642 H= 160:30 +CPU= 20.880 18:13:32 STEP 643 H= 160:45 +CPU= 20.800 18:13:53 STEP 644 H= 161:00 +CPU= 21.045 18:14:14 STEP 645 H= 161:15 +CPU= 20.753 18:14:35 STEP 646 H= 161:30 +CPU= 20.692 18:14:56 STEP 647 H= 161:45 +CPU= 20.802 18:15:22 STEP 648 H= 162:00 +CPU= 25.834 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000648 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000648 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000648 18:16:03 STEP 649 H= 162:15 +CPU= 19.680 18:16:23 STEP 650 H= 162:30 +CPU= 20.672 18:16:44 STEP 651 H= 162:45 +CPU= 20.744 18:17:05 STEP 652 H= 163:00 +CPU= 21.176 18:17:26 STEP 653 H= 163:15 +CPU= 20.663 18:17:47 STEP 654 H= 163:30 +CPU= 21.014 18:18:08 STEP 655 H= 163:45 +CPU= 20.936 18:18:29 STEP 656 H= 164:00 +CPU= 21.128 18:18:50 STEP 657 H= 164:15 +CPU= 20.636 18:19:11 STEP 658 H= 164:30 +CPU= 20.732 18:19:31 STEP 659 H= 164:45 +CPU= 20.746 18:19:57 STEP 660 H= 165:00 +CPU= 25.914 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000660 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000660 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000660 18:20:18 STEP 661 H= 165:15 +CPU= 20.686 18:20:39 STEP 662 H= 165:30 +CPU= 20.865 18:21:00 STEP 663 H= 165:45 +CPU= 20.931 18:21:21 STEP 664 H= 166:00 +CPU= 21.335 18:21:43 STEP 665 H= 166:15 +CPU= 20.978 18:22:05 STEP 666 H= 166:30 +CPU= 20.825 18:22:26 STEP 667 H= 166:45 +CPU= 20.684 18:22:47 STEP 668 H= 167:00 +CPU= 21.106 18:23:07 STEP 669 H= 167:15 +CPU= 20.697 18:23:28 STEP 670 H= 167:30 +CPU= 20.634 18:23:49 STEP 671 H= 167:45 +CPU= 20.746 18:24:19 STEP 672 H= 168:00 +CPU= 29.871 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000672 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000672 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000672 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000576 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000576 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000576 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000588 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000588 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000588 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000600 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000600 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000600 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000612 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000612 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000612 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000624 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000624 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000624 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000636 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000636 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000636 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000648 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000648 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000648 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000660 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000660 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000660 Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a25p_2016092300_20_1018_12291124_0_r1343717625_6.zip Uploading the intermediate file: upload_file_6.zip 18:24:36 STEP 673 H= 168:15 +CPU= 17.015 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:24:55 STEP 674 H= 168:30 +CPU= 10.690 18:25:16 STEP 675 H= 168:45 +CPU= 20.677 18:25:37 STEP 676 H= 169:00 +CPU= 21.131 18:25:58 STEP 677 H= 169:15 +CPU= 20.682 18:26:19 STEP 678 H= 169:30 +CPU= 20.703 18:26:40 STEP 679 H= 169:45 +CPU= 20.925 18:27:01 STEP 680 H= 170:00 +CPU= 21.054 18:27:21 STEP 681 H= 170:15 +CPU= 20.758 18:27:42 STEP 682 H= 170:30 +CPU= 20.845 18:28:03 STEP 683 H= 170:45 +CPU= 20.861 18:28:29 STEP 684 H= 171:00 +CPU= 25.985 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000684 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000684 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000684 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:29:10 STEP 685 H= 171:15 +CPU= 20.011 18:29:31 STEP 686 H= 171:30 +CPU= 20.746 18:29:52 STEP 687 H= 171:45 +CPU= 20.666 18:30:13 STEP 688 H= 172:00 +CPU= 21.179 18:30:34 STEP 689 H= 172:15 +CPU= 20.749 18:30:55 STEP 690 H= 172:30 +CPU= 20.905 18:31:15 STEP 691 H= 172:45 +CPU= 20.714 18:31:37 STEP 692 H= 173:00 +CPU= 21.287 18:31:58 STEP 693 H= 173:15 +CPU= 20.793 18:32:18 STEP 694 H= 173:30 +CPU= 20.660 18:32:39 STEP 695 H= 173:45 +CPU= 20.762 18:33:05 STEP 696 H= 174:00 +CPU= 25.999 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000696 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000696 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000696 18:33:26 STEP 697 H= 174:15 +CPU= 20.675 18:33:47 STEP 698 H= 174:30 +CPU= 20.797 18:34:07 STEP 699 H= 174:45 +CPU= 20.697 18:34:29 STEP 700 H= 175:00 +CPU= 21.163 18:34:49 STEP 701 H= 175:15 +CPU= 20.898 18:35:10 STEP 702 H= 175:30 +CPU= 20.923 18:35:31 STEP 703 H= 175:45 +CPU= 20.906 18:35:53 STEP 704 H= 176:00 +CPU= 21.100 18:36:13 STEP 705 H= 176:15 +CPU= 20.789 18:36:34 STEP 706 H= 176:30 +CPU= 20.839 18:36:55 STEP 707 H= 176:45 +CPU= 20.648 18:37:21 STEP 708 H= 177:00 +CPU= 25.742 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000708 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000708 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000708 18:37:42 STEP 709 H= 177:15 +CPU= 20.879 18:38:03 STEP 710 H= 177:30 +CPU= 20.866 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:38:43 STEP 711 H= 177:45 +CPU= 19.617 18:39:04 STEP 712 H= 178:00 +CPU= 20.719 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:39:46 STEP 713 H= 178:15 +CPU= 20.249 18:40:07 STEP 714 H= 178:30 +CPU= 20.860 18:40:27 STEP 715 H= 178:45 +CPU= 20.703 18:40:48 STEP 716 H= 179:00 +CPU= 21.235 18:41:09 STEP 717 H= 179:15 +CPU= 20.659 18:41:30 STEP 718 H= 179:30 +CPU= 20.635 18:41:50 STEP 719 H= 179:45 +CPU= 20.551 18:42:21 STEP 720 H= 180:00 +CPU= 29.806 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000720 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000720 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000720 18:42:41 STEP 721 H= 180:15 +CPU= 20.690 18:43:02 STEP 722 H= 180:30 +CPU= 20.649 18:43:23 STEP 723 H= 180:45 +CPU= 20.764 18:43:44 STEP 724 H= 181:00 +CPU= 21.091 18:44:05 STEP 725 H= 181:15 +CPU= 20.776 18:44:25 STEP 726 H= 181:30 +CPU= 20.691 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:45:07 STEP 727 H= 181:45 +CPU= 20.319 18:45:28 STEP 728 H= 182:00 +CPU= 20.805 18:45:48 STEP 729 H= 182:15 +CPU= 20.753 18:46:09 STEP 730 H= 182:30 +CPU= 20.861 18:46:30 STEP 731 H= 182:45 +CPU= 20.901 18:46:56 STEP 732 H= 183:00 +CPU= 25.800 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000732 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000732 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000732 18:47:17 STEP 733 H= 183:15 +CPU= 20.865 18:47:38 STEP 734 H= 183:30 +CPU= 20.845 18:47:59 STEP 735 H= 183:45 +CPU= 20.803 18:48:20 STEP 736 H= 184:00 +CPU= 21.100 18:48:41 STEP 737 H= 184:15 +CPU= 20.952 18:49:02 STEP 738 H= 184:30 +CPU= 20.670 18:49:22 STEP 739 H= 184:45 +CPU= 20.818 18:49:44 STEP 740 H= 185:00 +CPU= 21.135 18:50:05 STEP 741 H= 185:15 +CPU= 20.946 18:50:25 STEP 742 H= 185:30 +CPU= 20.773 18:50:46 STEP 743 H= 185:45 +CPU= 20.734 18:51:12 STEP 744 H= 186:00 +CPU= 25.970 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000744 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000744 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000744 18:51:33 STEP 745 H= 186:15 +CPU= 20.937 18:51:54 STEP 746 H= 186:30 +CPU= 20.884 18:52:15 STEP 747 H= 186:45 +CPU= 20.794 18:52:36 STEP 748 H= 187:00 +CPU= 21.170 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:53:17 STEP 749 H= 187:15 +CPU= 19.879 18:53:38 STEP 750 H= 187:30 +CPU= 20.586 18:53:58 STEP 751 H= 187:45 +CPU= 20.712 18:54:19 STEP 752 H= 188:00 +CPU= 21.033 18:54:40 STEP 753 H= 188:15 +CPU= 20.750 18:55:01 STEP 754 H= 188:30 +CPU= 20.788 18:55:22 STEP 755 H= 188:45 +CPU= 20.759 18:55:48 STEP 756 H= 189:00 +CPU= 25.723 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000756 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000756 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000756 18:56:09 STEP 757 H= 189:15 +CPU= 20.927 18:56:30 STEP 758 H= 189:30 +CPU= 20.940 18:56:51 STEP 759 H= 189:45 +CPU= 20.926 18:57:12 STEP 760 H= 190:00 +CPU= 21.124 18:57:33 STEP 761 H= 190:15 +CPU= 20.943 18:57:54 STEP 762 H= 190:30 +CPU= 20.963 18:58:14 STEP 763 H= 190:45 +CPU= 20.625 18:58:36 STEP 764 H= 191:00 +CPU= 21.276 18:58:56 STEP 765 H= 191:15 +CPU= 20.729 18:59:17 STEP 766 H= 191:30 +CPU= 20.932 18:59:38 STEP 767 H= 191:45 +CPU= 20.891 19:00:08 STEP 768 H= 192:00 +CPU= 29.910 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000768 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000768 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000768 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000672 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000672 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000672 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000684 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000684 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000684 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000696 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000696 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000696 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000708 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000708 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000708 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000720 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000720 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000720 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000732 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000732 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000732 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000744 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000744 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000744 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMGGh7zg+000756 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMSHh7zg+000756 Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124/ICMUAh7zg+000756 Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a25p_2016092300_20_1018_12291124_0_r1343717625_7.zip Uploading the intermediate file: upload_file_7.zip 19:00:30 STEP 769 H= 192:15 +CPU= 20.913 19:00:50 STEP 770 H= 192:30 +CPU= 20.843 19:01:11 STEP 771 H= 192:45 +CPU= 20.750 19:01:32 STEP 772 H= 193:00 +CPU= 21.181 19:01:53 STEP 773 H= 193:15 +CPU= 20.709 19:02:14 STEP 774 H= 193:30 +CPU= 20.690 19:02:35 STEP 775 H= 193:45 +CPU= 20.822 19:02:56 STEP 776 H= 194:00 +CPU= 21.161 19:03:17 STEP 777 H= 194:15 +CPU= 20.807 Suspend request received from the BOINC client, suspending the child process Resuming the child process 19:03:58 STEP 778 H= 194:30 +CPU= 20.241 19:04:19 STEP 779 H= 194:45 +CPU= 20.471 19:04:45 STEP 780 H= 195:00 +CPU= 25.835 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000780 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000780 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000780 19:05:05 STEP 781 H= 195:15 +CPU= 20.557 19:05:26 STEP 782 H= 195:30 +CPU= 20.556 19:05:47 STEP 783 H= 195:45 +CPU= 20.601 19:06:08 STEP 784 H= 196:00 +CPU= 21.002 19:06:29 STEP 785 H= 196:15 +CPU= 20.760 19:06:49 STEP 786 H= 196:30 +CPU= 20.594 19:07:10 STEP 787 H= 196:45 +CPU= 20.627 19:07:31 STEP 788 H= 197:00 +CPU= 21.307 Suspend request received from the BOINC client, suspending the child process Resuming the child process 19:08:12 STEP 789 H= 197:15 +CPU= 19.906 19:08:33 STEP 790 H= 197:30 +CPU= 20.562 19:08:54 STEP 791 H= 197:45 +CPU= 20.929 19:09:20 STEP 792 H= 198:00 +CPU= 26.003 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000792 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000792 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000792 19:09:41 STEP 793 H= 198:15 +CPU= 20.694 19:10:01 STEP 794 H= 198:30 +CPU= 20.725 19:10:22 STEP 795 H= 198:45 +CPU= 20.688 19:10:43 STEP 796 H= 199:00 +CPU= 20.970 19:11:04 STEP 797 H= 199:15 +CPU= 20.845 19:11:25 STEP 798 H= 199:30 +CPU= 20.722 19:11:46 STEP 799 H= 199:45 +CPU= 20.947 19:12:07 STEP 800 H= 200:00 +CPU= 21.312 19:12:28 STEP 801 H= 200:15 +CPU= 20.686 19:12:49 STEP 802 H= 200:30 +CPU= 20.772 19:13:10 STEP 803 H= 200:45 +CPU= 20.723 19:13:35 STEP 804 H= 201:00 +CPU= 25.683 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000804 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000804 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000804 19:13:56 STEP 805 H= 201:15 +CPU= 20.753 19:14:17 STEP 806 H= 201:30 +CPU= 20.666 19:14:38 STEP 807 H= 201:45 +CPU= 20.771 19:14:59 STEP 808 H= 202:00 +CPU= 21.114 19:15:20 STEP 809 H= 202:15 +CPU= 20.833 19:15:41 STEP 810 H= 202:30 +CPU= 20.758 19:16:01 STEP 811 H= 202:45 +CPU= 20.630 19:16:23 STEP 812 H= 203:00 +CPU= 21.135 19:16:43 STEP 813 H= 203:15 +CPU= 20.704 19:17:04 STEP 814 H= 203:30 +CPU= 20.724 19:17:25 STEP 815 H= 203:45 +CPU= 20.606 19:17:55 STEP 816 H= 204:00 +CPU= 29.777 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000816 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000816 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000816 19:18:15 STEP 817 H= 204:15 +CPU= 20.729 19:18:36 STEP 818 H= 204:30 +CPU= 20.684 19:18:57 STEP 819 H= 204:45 +CPU= 20.802 19:19:18 STEP 820 H= 205:00 +CPU= 21.168 19:19:39 STEP 821 H= 205:15 +CPU= 20.603 19:20:00 STEP 822 H= 205:30 +CPU= 20.830 19:20:20 STEP 823 H= 205:45 +CPU= 20.685 19:20:42 STEP 824 H= 206:00 +CPU= 21.312 Suspend request received from the BOINC client, suspending the child process Resuming the child process 19:21:23 STEP 825 H= 206:15 +CPU= 20.335 19:21:44 STEP 826 H= 206:30 +CPU= 20.708 19:22:05 STEP 827 H= 206:45 +CPU= 20.676 19:22:30 STEP 828 H= 207:00 +CPU= 25.774 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000828 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000828 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000828 19:22:51 STEP 829 H= 207:15 +CPU= 20.774 19:23:12 STEP 830 H= 207:30 +CPU= 20.707 19:23:33 STEP 831 H= 207:45 +CPU= 20.799 19:23:54 STEP 832 H= 208:00 +CPU= 21.082 19:24:17 STEP 833 H= 208:15 +CPU= 22.661 19:24:40 STEP 834 H= 208:30 +CPU= 21.747 Suspend request received from the BOINC client, suspending the child process Resuming the child process 19:25:21 STEP 835 H= 208:45 +CPU= 20.509 19:25:45 STEP 836 H= 209:00 +CPU= 22.642 19:26:08 STEP 837 H= 209:15 +CPU= 22.481 19:26:29 STEP 838 H= 209:30 +CPU= 21.366 19:26:51 STEP 839 H= 209:45 +CPU= 21.555 Suspend request received from the BOINC client, suspending the child process Resuming the child process 19:27:39 STEP 840 H= 210:00 +CPU= 26.332 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000840 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000840 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000840 19:28:01 STEP 841 H= 210:15 +CPU= 22.153 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu (argv1) start_date: 2016092300 (argv2) exptid: h7zg (argv3) unique_member_id: a25p (argv4) batchid: 1018 (argv5) wuid: 12291124 (argv6) fclen: 20 (argv7) app_name: oifs_43r3_bl (argv8) nthreads: 1 Working directory is: /home/robbie/slots/16 Project directory is: /home/robbie/projects/climateprediction.net/ app name: oifs_43r3_bl version: 1.13 Location of temp folder: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12291124 ..mkdir for temp folder for results failed Copying: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /home/robbie/slots/16/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /home/robbie/slots/16/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_799e61a8ea8a8d5e53b469a55a2744e0 to: /home/robbie/slots/16/oifs_43r3_bl_a25p_2016092300_20_1018_12291124.zip Unzipping the namelist zip file: /home/robbie/slots/16/oifs_43r3_bl_a25p_2016092300_20_1018_12291124.zip ic_ancil_file: ic_ancil_12291124 ifsdata_file: ifsdata_12291124 climate_data_file: clim_data_12291124 horiz_resolution: 159 vert_resolution: 91 grid_type: l_2 upload_interval: 96 utstep: 900 nfrres: restart dump frequency (steps) 48 Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /home/robbie/slots/16/ic_ancil_12291124.zip Unzipping the IC ancils zip file: /home/robbie/slots/16/ic_ancil_12291124.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ba8765fdf72061374855111149e0f2a4 to: /home/robbie/slots/16/ifsdata/ifsdata_12291124.zip Unzipping the ifsdata_zip file: /home/robbie/slots/16/ifsdata/ifsdata_12291124.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/robbie/slots/16/159l_2/clim_data_12291124.zip Unzipping the climate data zip file: /home/robbie/slots/16/159l_2/clim_data_12291124.zip Checking for progress XML file: /home/robbie/slots/16/progress_file_12291124.xml Opened progress file ok : /home/robbie/slots/16/progress_file_12291124.xml -- Model is restarting -- Adjusting last_iter, 842, to previous model restart step. Creating progress file: /home/robbie/slots/16/progress_file_12291124.xml last_cpu_time: 17914 upload_file_number: 8 last_iter: 817 last_upload: 691200 model_completed: 0 total_length_of_simulation: 1728000 result_base_name: oifs_43r3_bl_a25p_2016092300_20_1018_12291124_0_r1343717625 The child process has been launched with process id: 1531 Executing the command: /home/robbie/slots/16/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440 [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil)) [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil)) [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil)) [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil)) [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at 0x1) [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil)) [EC_DRHOOK:robbie-Intel-Z490:1:1:1531:1531] [20240613:192925:1718303365:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 19:30:56 STEP 816 H= 204:00 +CPU= 22.039 19:31:17 STEP 817 H= 204:15 +CPU= 21.839 19:31:39 STEP 818 H= 204:30 +CPU= 21.728 19:32:01 STEP 819 H= 204:45 +CPU= 21.638 19:32:24 STEP 820 H= 205:00 +CPU= 22.780 Suspend request received from the BOINC client, suspending the child process Resuming the child process 19:33:03 STEP 821 H= 205:15 +CPU= 17.602 Suspend request received from the BOINC client, suspending the child process Resuming the child process 19:33:46 STEP 822 H= 205:30 +CPU= 20.540 19:34:08 STEP 823 H= 205:45 +CPU= 21.313 19:34:28 STEP 824 H= 206:00 +CPU= 20.455 19:34:49 STEP 825 H= 206:15 +CPU= 20.716 19:35:09 STEP 826 H= 206:30 +CPU= 19.766 Suspend request received from the BOINC client, suspending the child process Resuming the child process 19:35:48 STEP 827 H= 206:45 +CPU= 17.765 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 19:37:12 STEP 828 H= 207:00 +CPU= 22.448 Moving to projects directory: /home/robbie/slots/16/ICMGGh7zg+000828 Moving to projects directory: /home/robbie/slots/16/ICMSHh7zg+000828 Moving to projects directory: /home/robbie/slots/16/ICMUAh7zg+000828 Suspend request received from the BOINC client, suspending the child process Resuming the child process 20:13:21 STEP 829 H= 207:15 +CPU= 18.569 Suspend request received from the BOINC client, suspending the child process Resuming the child process 20:14:01 STEP 830 H= 207:30 +CPU= 18.894 20:14:19 STEP 831 H= 207:45 +CPU= 16.149 20:16:46 STEP 832 H= 208:00 +CPU= 13.603 20:16:59 STEP 833 H= 208:15 +CPU= 12.014 20:17:13 STEP 834 H= 208:30 +CPU= 11.601 20:17:25 STEP 835 H= 208:45 +CPU= 11.668 20:17:37 STEP 836 H= 209:00 +CPU= 11.792 21:49:48 STEP 837 H= 209:15 +CPU= 22.486 21:50:07 STEP 838 H= 209:30 +CPU= 10.460 21:50:17 STEP 839 H= 209:45 +CPU= 10.368 Suspend request received from the BOINC client, suspending the child process Resuming the child process ..The child process has been killed with signal: 9 >>> Printing last 70 lines from file: NODE.001_01 ISTEP= 840: Reset RSOLINC= 1361.4482 RIP0= 1361.4482 SU_GHGCLIM, RCARDI,RCH4,RN2O,RCFC11,RCFC12,RNO2: 6.179466599480000E-004 1.088035170184667E-006 4.990317592110001E-007 3.778213382954999E-009 2.097581753513333E-009 7.940283000000001E-011 suecaec NAERMACC = 1 UPDT RAESHDU = 2000.0 SUECAEC: GLOBAL AVERAGE VOLCANIC SULPHATE FOR (KINDAT,KMINUT) 20160923 1170 9.999999999999998E-005 NORMS AT NSTEP CNT4 840 SPECTRAL NORMS - LOG(PREHYDS) 0.115260884535244E+02 LEV VORTICITY DIVERGENCE TEMPERATURE KINETIC ENERGY AVE 0.837199945155559E-06 0.187344209400869E-07 0.217437694554772E+03 0.462863193667014E+01 GPNORM HUMIDITY AVERAGE MINIMUM MAXIMUM AVE 0.659859636017025E-03 0.100000000000000E-07 0.867830843596815E-02 GPNORM LIQUID WATER AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM ICE WATER AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM SNOW AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM RAIN AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM CLOUD FRACTION AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM CLW AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM CIW AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM CC AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM RFL AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM SFL AVERAGE MINIMUM MAXIMUM AVE 0.511005718465802E-13 0.000000000000000E+00 0.208070111402550E-11 GPNORM CRFL AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM CSFL AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM PFRC AVERAGE MINIMUM MAXIMUM AVE 0.219780219780221E-02 0.000000000000000E+00 0.100000000000000E+00 NSTEP = 840 STEPO A00000000 MAXGPFV : MAX. VALUE = 0.000000000000000E+000 MAXGPFV : MAX. VALUE = 0.000000000000000E+000 NSTEP = 840 SCAN2M_HPOS P IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ICMGGh7zg+000840 MODE=w IO-STREAM CLOSED - ICMGGh7zg+000840 MPI-TASK: 1 - 310566276 BYTES IN 6 RECORDS TRANSFERRED IN 0.0007 SECONDS443666.1 Mbytes/s, TOTAL TIME= 0.1955( 0.4%) NSTEP = 840 STEPO 0AA000000 IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMSHh7zg+000840 MODE=w IO-STREAM CLOSED - ./ICMSHh7zg+000840 MPI-TASK: 1 - 707399250 BYTES IN 202 RECORDS TRANSFERRED IN 0.0210 SECONDS 33685.7 Mbytes/s, TOTAL TIME= 0.2814( 7.5%) IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMUAh7zg+000840 MODE=w IO-STREAM CLOSED - ./ICMUAh7zg+000840 MPI-TASK: 1 - 4969060416 BYTES IN 125 RECORDS TRANSFERRED IN 0.0075 SECONDS662541.4 Mbytes/s, TOTAL TIME= 0.1318( 5.7%) STEPO_FPOS WAS : B00000F00E IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMUAh7zg+000840 MODE=a IO-STREAM CLOSED - ./ICMUAh7zg+000840 MPI-TASK: 1 - 10 BYTES IN 2 RECORDS TRANSFERRED IN 0.0004 SECONDS 0.0 Mbytes/s, TOTAL TIME= 0.0031(12.9%) STEPO_FPOS WAS : V00000000Y IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMUAh7zg+000840 MODE=a IO-STREAM CLOSED - ./ICMUAh7zg+000840 MPI-TASK: 1 - 301986547 BYTES IN 4 RECORDS TRANSFERRED IN 0.0004 SECONDS754966.4 Mbytes/s, TOTAL TIME= 0.0048( 8.3%) STEPO_FPOS WAS : T00000000M NSTEP = 840 STEPO 0AAA00AAA ------------------------------------------------ CNT0 not found; string returned was: 'STEPO' >>> Printing last 8 lines from file: ifs.stat 21:50:07 0AAA00AAA STEPO 839 10.459 10.459 15.197 7:01 140:42 0.18149607650051E-07 2GB 0MB 21:50:47 A00000000 STEPO 840 10.489 10.489 40.260 7:12 141:22 0.18734420940087E-07 2GB 0MB 21:50:47 0AA000000 STEPO 840 0.026 0.026 0.383 7:12 141:22 0.18734420940087E-07 2GB 0MB 21:50:50 FULLPOS-B DYNFPOS 840 1.171 1.171 2.251 7:13 141:24 0.18734420940087E-07 2GB 0MB 21:50:52 FULLPOS-V DYNFPOS 840 2.404 2.404 2.709 7:15 141:27 0.18734420940087E-07 2GB 0MB 21:50:53 FULLPOS-T DYNFPOS 840 0.301 0.301 0.307 7:15 141:27 0.18734420940087E-07 2GB 0MB 21:50:53 0AAA00AAA STEPO 840 0.296 0.296 0.296 7:16 141:28 0.18734420940087E-07 2GB 0MB ------------------------------------------------ >>> Printing last 11 lines from file: rcf CSTEP = " 816", CTIME = "00081200 ", NSTEPLPP = 201, 202, 49, 228226, 228227, 2*0, 804, 2*0, IPRGPNSRES = 1, IPRGPEWRES = 1, IPRTRWRES = 1, IPRTRVRES = 1, GMASS0 = 0.000000000000000E+000, GMASSI = 0.000000000000000E+000 / ------------------------------------------------ >>> Printing last 8 lines from file: /home/robbie/slots/16/progress_file_12291124.xml <running_values> <last_cpu_time>18356.420000</last_cpu_time> <upload_file_number>8</upload_file_number> <last_iter>840</last_iter> <last_upload>691200</last_upload> <model_completed>0</model_completed> </running_values> ------------------------------------------------ ..Failed, model did not complete successfully </stderr_txt> ]]> |
No trickles! |
---|
©2024 cpdn.org