climateprediction.net home page
Task 22270328

Task 22270328

Name oifs_43r3_ps_0399_1999050100_123_968_12185043_0
Workunit 12185043
Created 22 Dec 2022, 18:43:19 UTC
Sent 22 Dec 2022, 19:02:36 UTC
Report deadline 21 Jan 2023, 19:02:36 UTC
Received 23 Dec 2022, 20:33:17 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 194 (0x000000C2) EXIT_ABORTED_BY_CLIENT
Computer ID 1517859
Run time 16 hours 31 min 27 sec
CPU time 16 hours 13 min 47 sec
Validate state Invalid
Credit 7,257.60
Device peak FLOPS 6.98 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,792.37 MB
Peak swap size 4,974.19 MB
Peak disk usage 1,972.59 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Process still present 5 min after writing finish file; aborting</message>
<stderr_txt>
_12185043_0_r1895724334_94.zip
Uploading the intermediate file: upload_file_94.zip
  11:12:40 STEP 2281 H=2281:00 +CPU= 16.951
  11:12:58 STEP 2282 H=2282:00 +CPU= 17.866
  11:13:26 STEP 2283 H=2283:00 +CPU= 27.962
  11:13:45 STEP 2284 H=2284:00 +CPU= 17.795
  11:14:02 STEP 2285 H=2285:00 +CPU= 17.136
  11:14:30 STEP 2286 H=2286:00 +CPU= 27.617
  11:14:45 STEP 2287 H=2287:00 +CPU= 15.220
  11:15:03 STEP 2288 H=2288:00 +CPU= 17.356
  11:15:42 STEP 2289 H=2289:00 +CPU= 38.387
  11:16:04 STEP 2290 H=2290:00 +CPU= 21.050
  11:16:41 STEP 2291 H=2291:00 +CPU= 37.090
  11:17:11 STEP 2292 H=2292:00 +CPU= 28.947
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002292
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002292
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002292
  11:17:29 STEP 2293 H=2293:00 +CPU= 17.685
  11:17:46 STEP 2294 H=2294:00 +CPU= 16.874
  11:18:13 STEP 2295 H=2295:00 +CPU= 26.615
  11:18:31 STEP 2296 H=2296:00 +CPU= 18.112
  11:18:53 STEP 2297 H=2297:00 +CPU= 21.403
  11:19:21 STEP 2298 H=2298:00 +CPU= 27.879
  11:19:47 STEP 2299 H=2299:00 +CPU= 25.468
  11:20:04 STEP 2300 H=2300:00 +CPU= 16.793
  11:20:30 STEP 2301 H=2301:00 +CPU= 25.382
  11:20:46 STEP 2302 H=2302:00 +CPU= 16.112
  11:21:03 STEP 2303 H=2303:00 +CPU= 16.719
  11:21:34 STEP 2304 H=2304:00 +CPU= 29.862
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002304
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002304
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002304
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002280
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002280
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002280
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002292
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002292
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002292
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_95.zip
Uploading the intermediate file: upload_file_95.zip
  11:21:51 STEP 2305 H=2305:00 +CPU= 17.513
  11:22:08 STEP 2306 H=2306:00 +CPU= 16.093
  11:22:34 STEP 2307 H=2307:00 +CPU= 25.360
  11:22:52 STEP 2308 H=2308:00 +CPU= 18.084
  11:23:09 STEP 2309 H=2309:00 +CPU= 16.815
  11:23:36 STEP 2310 H=2310:00 +CPU= 26.797
  11:23:53 STEP 2311 H=2311:00 +CPU= 16.077
  11:24:09 STEP 2312 H=2312:00 +CPU= 16.167
  11:24:36 STEP 2313 H=2313:00 +CPU= 25.883
  11:24:52 STEP 2314 H=2314:00 +CPU= 16.212
  11:25:09 STEP 2315 H=2315:00 +CPU= 16.949
  11:25:39 STEP 2316 H=2316:00 +CPU= 28.756
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002316
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002316
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002316
  11:25:57 STEP 2317 H=2317:00 +CPU= 17.559
  11:26:14 STEP 2318 H=2318:00 +CPU= 17.178
  11:26:41 STEP 2319 H=2319:00 +CPU= 27.066
  11:27:00 STEP 2320 H=2320:00 +CPU= 18.201
  11:27:19 STEP 2321 H=2321:00 +CPU= 18.880
  11:27:45 STEP 2322 H=2322:00 +CPU= 25.662
  11:28:02 STEP 2323 H=2323:00 +CPU= 16.505
  11:28:20 STEP 2324 H=2324:00 +CPU= 17.567
  11:28:46 STEP 2325 H=2325:00 +CPU= 25.629
  11:29:02 STEP 2326 H=2326:00 +CPU= 15.428
  11:29:18 STEP 2327 H=2327:00 +CPU= 16.054
  11:29:47 STEP 2328 H=2328:00 +CPU= 28.924
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002328
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002328
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002328
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002304
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002304
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002304
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002316
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002316
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002316
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_96.zip
Uploading the intermediate file: upload_file_96.zip
  11:30:04 STEP 2329 H=2329:00 +CPU= 15.717
  11:30:20 STEP 2330 H=2330:00 +CPU= 16.480
  11:30:47 STEP 2331 H=2331:00 +CPU= 26.364
  11:31:03 STEP 2332 H=2332:00 +CPU= 15.379
  11:31:19 STEP 2333 H=2333:00 +CPU= 15.845
  11:31:45 STEP 2334 H=2334:00 +CPU= 25.568
  11:32:00 STEP 2335 H=2335:00 +CPU= 15.676
  11:32:19 STEP 2336 H=2336:00 +CPU= 18.827
  11:32:46 STEP 2337 H=2337:00 +CPU= 25.678
  11:33:03 STEP 2338 H=2338:00 +CPU= 16.664
  11:33:20 STEP 2339 H=2339:00 +CPU= 17.116
  11:33:52 STEP 2340 H=2340:00 +CPU= 30.670
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002340
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002340
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002340
  11:34:08 STEP 2341 H=2341:00 +CPU= 16.091
  11:34:26 STEP 2342 H=2342:00 +CPU= 16.749
  11:34:52 STEP 2343 H=2343:00 +CPU= 26.387
  11:35:09 STEP 2344 H=2344:00 +CPU= 16.814
  11:35:26 STEP 2345 H=2345:00 +CPU= 16.638
  11:35:53 STEP 2346 H=2346:00 +CPU= 26.292
  11:36:10 STEP 2347 H=2347:00 +CPU= 17.140
  11:36:27 STEP 2348 H=2348:00 +CPU= 16.326
  11:36:53 STEP 2349 H=2349:00 +CPU= 25.157
  11:37:10 STEP 2350 H=2350:00 +CPU= 17.245
  11:37:26 STEP 2351 H=2351:00 +CPU= 15.395
  11:37:56 STEP 2352 H=2352:00 +CPU= 29.367
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002352
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002352
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002352
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002328
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002328
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002328
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002340
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002340
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002340
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_97.zip
Uploading the intermediate file: upload_file_97.zip
  11:38:15 STEP 2353 H=2353:00 +CPU= 18.740
  11:38:34 STEP 2354 H=2354:00 +CPU= 18.664
  11:39:07 STEP 2355 H=2355:00 +CPU= 31.691
  11:39:22 STEP 2356 H=2356:00 +CPU= 15.752
  11:39:39 STEP 2357 H=2357:00 +CPU= 16.660
  11:40:06 STEP 2358 H=2358:00 +CPU= 26.266
  11:40:22 STEP 2359 H=2359:00 +CPU= 15.145
  11:40:38 STEP 2360 H=2360:00 +CPU= 16.018
  11:41:07 STEP 2361 H=2361:00 +CPU= 27.579
  11:41:23 STEP 2362 H=2362:00 +CPU= 16.343
  11:41:39 STEP 2363 H=2363:00 +CPU= 15.720
  11:42:08 STEP 2364 H=2364:00 +CPU= 27.794
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002364
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002364
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002364
  11:42:27 STEP 2365 H=2365:00 +CPU= 19.435
  11:42:46 STEP 2366 H=2366:00 +CPU= 17.760
  11:43:11 STEP 2367 H=2367:00 +CPU= 25.183
  11:43:29 STEP 2368 H=2368:00 +CPU= 17.163
  11:43:45 STEP 2369 H=2369:00 +CPU= 16.605
  11:44:12 STEP 2370 H=2370:00 +CPU= 26.844
  11:44:29 STEP 2371 H=2371:00 +CPU= 16.596
  11:44:46 STEP 2372 H=2372:00 +CPU= 16.447
  11:45:12 STEP 2373 H=2373:00 +CPU= 25.533
  11:45:37 STEP 2374 H=2374:00 +CPU= 24.803
  11:45:56 STEP 2375 H=2375:00 +CPU= 18.367
  11:46:25 STEP 2376 H=2376:00 +CPU= 29.057
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002376
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002376
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002376
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002352
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002352
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002352
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002364
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002364
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002364
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_98.zip
Uploading the intermediate file: upload_file_98.zip
  11:46:42 STEP 2377 H=2377:00 +CPU= 16.263
  11:46:59 STEP 2378 H=2378:00 +CPU= 16.710
  11:47:23 STEP 2379 H=2379:00 +CPU= 24.515
  11:47:41 STEP 2380 H=2380:00 +CPU= 16.876
  11:47:56 STEP 2381 H=2381:00 +CPU= 15.095
  11:48:22 STEP 2382 H=2382:00 +CPU= 25.208
  11:48:38 STEP 2383 H=2383:00 +CPU= 16.216
  11:48:55 STEP 2384 H=2384:00 +CPU= 16.157
  11:49:20 STEP 2385 H=2385:00 +CPU= 25.274
  11:49:38 STEP 2386 H=2386:00 +CPU= 17.400
  11:49:54 STEP 2387 H=2387:00 +CPU= 15.328
  11:50:21 STEP 2388 H=2388:00 +CPU= 27.530
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002388
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002388
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002388
  11:50:39 STEP 2389 H=2389:00 +CPU= 16.895
  11:50:56 STEP 2390 H=2390:00 +CPU= 17.012
  11:51:22 STEP 2391 H=2391:00 +CPU= 25.454
  11:51:39 STEP 2392 H=2392:00 +CPU= 16.842
  11:51:56 STEP 2393 H=2393:00 +CPU= 15.916
  11:52:25 STEP 2394 H=2394:00 +CPU= 29.022
  11:52:41 STEP 2395 H=2395:00 +CPU= 15.928
  11:52:58 STEP 2396 H=2396:00 +CPU= 16.252
  11:53:22 STEP 2397 H=2397:00 +CPU= 23.881
  11:53:39 STEP 2398 H=2398:00 +CPU= 16.630
  11:53:56 STEP 2399 H=2399:00 +CPU= 16.381
  11:54:29 STEP 2400 H=2400:00 +CPU= 32.966
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002400
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002400
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002400
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002376
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002376
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002376
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002388
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002388
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002388
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_99.zip
Uploading the intermediate file: upload_file_99.zip
  11:54:47 STEP 2401 H=2401:00 +CPU= 17.542
Uploading trickle at timestep: 8640000
  11:55:05 STEP 2402 H=2402:00 +CPU= 17.795
  11:55:33 STEP 2403 H=2403:00 +CPU= 27.079
  11:55:50 STEP 2404 H=2404:00 +CPU= 16.665
  11:56:06 STEP 2405 H=2405:00 +CPU= 15.643
  11:56:32 STEP 2406 H=2406:00 +CPU= 25.907
  11:56:48 STEP 2407 H=2407:00 +CPU= 15.547
  11:57:04 STEP 2408 H=2408:00 +CPU= 16.196
  11:57:31 STEP 2409 H=2409:00 +CPU= 26.816
  11:57:48 STEP 2410 H=2410:00 +CPU= 15.939
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=            5  MAX ETADOT VERTICAL VEL.=   3.135516684927513E-008
  LON  =    138.750000000000       degrees
  LAT  =   -36.4483927977945       degrees
  11:58:05 STEP 2411 H=2411:00 +CPU= 17.047
  11:58:41 STEP 2412 H=2412:00 +CPU= 35.172
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002412
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002412
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002412
  11:58:58 STEP 2413 H=2413:00 +CPU= 16.555
  11:59:14 STEP 2414 H=2414:00 +CPU= 16.303
  11:59:39 STEP 2415 H=2415:00 +CPU= 24.673
  11:59:55 STEP 2416 H=2416:00 +CPU= 15.728
  12:00:10 STEP 2417 H=2417:00 +CPU= 14.928
  12:00:36 STEP 2418 H=2418:00 +CPU= 25.188
  12:00:52 STEP 2419 H=2419:00 +CPU= 16.142
  12:01:08 STEP 2420 H=2420:00 +CPU= 15.685
  12:01:33 STEP 2421 H=2421:00 +CPU= 25.011
  12:01:49 STEP 2422 H=2422:00 +CPU= 15.676
  12:02:06 STEP 2423 H=2423:00 +CPU= 16.242
  12:02:36 STEP 2424 H=2424:00 +CPU= 29.491
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002424
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002424
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002424
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002400
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002400
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002400
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002412
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002412
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002412
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_100.zip
Uploading the intermediate file: upload_file_100.zip
  12:02:51 STEP 2425 H=2425:00 +CPU= 15.327
  12:03:08 STEP 2426 H=2426:00 +CPU= 15.885
  12:03:33 STEP 2427 H=2427:00 +CPU= 24.556
  12:03:49 STEP 2428 H=2428:00 +CPU= 15.984
  12:04:05 STEP 2429 H=2429:00 +CPU= 16.401
  12:04:30 STEP 2430 H=2430:00 +CPU= 24.178
  12:04:46 STEP 2431 H=2431:00 +CPU= 15.372
  12:05:02 STEP 2432 H=2432:00 +CPU= 16.247
  12:05:31 STEP 2433 H=2433:00 +CPU= 28.531
  12:05:47 STEP 2434 H=2434:00 +CPU= 15.863
  12:06:07 STEP 2435 H=2435:00 +CPU= 19.258
  12:06:34 STEP 2436 H=2436:00 +CPU= 26.802
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002436
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002436
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002436
  12:06:49 STEP 2437 H=2437:00 +CPU= 14.984
  12:07:06 STEP 2438 H=2438:00 +CPU= 16.200
  12:07:31 STEP 2439 H=2439:00 +CPU= 24.288
  12:07:46 STEP 2440 H=2440:00 +CPU= 15.225
  12:08:02 STEP 2441 H=2441:00 +CPU= 15.496
  12:08:27 STEP 2442 H=2442:00 +CPU= 24.453
  12:08:42 STEP 2443 H=2443:00 +CPU= 14.965
  12:08:58 STEP 2444 H=2444:00 +CPU= 15.677
  12:09:22 STEP 2445 H=2445:00 +CPU= 23.745
  12:09:38 STEP 2446 H=2446:00 +CPU= 15.372
  12:09:53 STEP 2447 H=2447:00 +CPU= 15.316
  12:10:23 STEP 2448 H=2448:00 +CPU= 29.184
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002448
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002448
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002448
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002424
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002424
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002424
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002436
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002436
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002436
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_101.zip
Uploading the intermediate file: upload_file_101.zip
  12:10:40 STEP 2449 H=2449:00 +CPU= 16.048
  12:10:56 STEP 2450 H=2450:00 +CPU= 15.756
  12:11:21 STEP 2451 H=2451:00 +CPU= 24.996
  12:11:37 STEP 2452 H=2452:00 +CPU= 16.010
  12:11:53 STEP 2453 H=2453:00 +CPU= 15.473
  12:12:18 STEP 2454 H=2454:00 +CPU= 24.745
  12:12:37 STEP 2455 H=2455:00 +CPU= 18.645
  12:12:55 STEP 2456 H=2456:00 +CPU= 17.555
  12:13:21 STEP 2457 H=2457:00 +CPU= 26.340
  12:13:38 STEP 2458 H=2458:00 +CPU= 16.585
  12:13:55 STEP 2459 H=2459:00 +CPU= 16.138
  12:14:30 STEP 2460 H=2460:00 +CPU= 35.515
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002460
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002460
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002460
  12:14:46 STEP 2461 H=2461:00 +CPU= 15.773
  12:15:11 STEP 2462 H=2462:00 +CPU= 23.893
  12:15:35 STEP 2463 H=2463:00 +CPU= 23.539
  12:15:50 STEP 2464 H=2464:00 +CPU= 15.583
  12:16:06 STEP 2465 H=2465:00 +CPU= 15.875
  12:16:32 STEP 2466 H=2466:00 +CPU= 25.053
  12:16:47 STEP 2467 H=2467:00 +CPU= 14.658
  12:17:03 STEP 2468 H=2468:00 +CPU= 15.423
  12:17:30 STEP 2469 H=2469:00 +CPU= 27.350
  12:17:46 STEP 2470 H=2470:00 +CPU= 15.876
  12:18:02 STEP 2471 H=2471:00 +CPU= 15.021
  12:18:30 STEP 2472 H=2472:00 +CPU= 28.264
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002472
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002472
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002472
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002448
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002448
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002448
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002460
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002460
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002460
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_102.zip
Uploading the intermediate file: upload_file_102.zip
  12:18:46 STEP 2473 H=2473:00 +CPU= 15.466
  12:19:03 STEP 2474 H=2474:00 +CPU= 16.163
  12:19:28 STEP 2475 H=2475:00 +CPU= 24.504
  12:19:43 STEP 2476 H=2476:00 +CPU= 15.407
  12:19:59 STEP 2477 H=2477:00 +CPU= 15.604
  12:20:25 STEP 2478 H=2478:00 +CPU= 25.664
  12:20:40 STEP 2479 H=2479:00 +CPU= 15.050
  12:21:05 STEP 2480 H=2480:00 +CPU= 23.880
  12:21:30 STEP 2481 H=2481:00 +CPU= 24.397
  12:21:47 STEP 2482 H=2482:00 +CPU= 16.384
  12:22:03 STEP 2483 H=2483:00 +CPU= 16.533
  12:22:56 STEP 2484 H=2484:00 +CPU= 52.068
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002484
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002484
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002484
  12:23:12 STEP 2485 H=2485:00 +CPU= 16.045
  12:23:28 STEP 2486 H=2486:00 +CPU= 15.731
  12:23:53 STEP 2487 H=2487:00 +CPU= 24.777
  12:24:09 STEP 2488 H=2488:00 +CPU= 15.778
  12:24:25 STEP 2489 H=2489:00 +CPU= 15.622
  12:24:51 STEP 2490 H=2490:00 +CPU= 25.331
  12:25:06 STEP 2491 H=2491:00 +CPU= 14.761
  12:25:22 STEP 2492 H=2492:00 +CPU= 16.464
  12:25:47 STEP 2493 H=2493:00 +CPU= 23.908
  12:26:02 STEP 2494 H=2494:00 +CPU= 15.245
  12:26:18 STEP 2495 H=2495:00 +CPU= 15.503
  12:26:47 STEP 2496 H=2496:00 +CPU= 28.872
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002496
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002496
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002496
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002472
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002472
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002472
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002484
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002484
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002484
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_103.zip
Uploading the intermediate file: upload_file_103.zip
  12:27:03 STEP 2497 H=2497:00 +CPU= 15.207
  12:27:21 STEP 2498 H=2498:00 +CPU= 17.694
  12:27:46 STEP 2499 H=2499:00 +CPU= 24.365
  12:28:08 STEP 2500 H=2500:00 +CPU= 21.959
  12:28:33 STEP 2501 H=2501:00 +CPU= 24.825
  12:29:07 STEP 2502 H=2502:00 +CPU= 32.798
  12:29:23 STEP 2503 H=2503:00 +CPU= 15.678
  12:29:39 STEP 2504 H=2504:00 +CPU= 16.487
  12:30:05 STEP 2505 H=2505:00 +CPU= 24.913
  12:30:21 STEP 2506 H=2506:00 +CPU= 15.928
  12:30:37 STEP 2507 H=2507:00 +CPU= 15.711
  12:31:05 STEP 2508 H=2508:00 +CPU= 28.053
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002508
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002508
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002508
  12:31:21 STEP 2509 H=2509:00 +CPU= 15.057
  12:31:36 STEP 2510 H=2510:00 +CPU= 15.323
  12:32:02 STEP 2511 H=2511:00 +CPU= 25.406
  12:32:18 STEP 2512 H=2512:00 +CPU= 15.888
  12:32:34 STEP 2513 H=2513:00 +CPU= 15.616
  12:32:59 STEP 2514 H=2514:00 +CPU= 24.741
  12:33:15 STEP 2515 H=2515:00 +CPU= 15.532
  12:33:32 STEP 2516 H=2516:00 +CPU= 16.650
  12:33:58 STEP 2517 H=2517:00 +CPU= 25.598
  12:34:15 STEP 2518 H=2518:00 +CPU= 16.469
  12:34:30 STEP 2519 H=2519:00 +CPU= 15.598
  12:35:00 STEP 2520 H=2520:00 +CPU= 29.233
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002520
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002520
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002520
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002496
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002496
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002496
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002508
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002508
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002508
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_104.zip
Uploading the intermediate file: upload_file_104.zip
  12:35:28 STEP 2521 H=2521:00 +CPU= 26.984
  12:35:44 STEP 2522 H=2522:00 +CPU= 15.863
  12:36:09 STEP 2523 H=2523:00 +CPU= 24.609
  12:36:24 STEP 2524 H=2524:00 +CPU= 15.389
  12:36:42 STEP 2525 H=2525:00 +CPU= 14.748
  12:37:08 STEP 2526 H=2526:00 +CPU= 24.894
  12:37:25 STEP 2527 H=2527:00 +CPU= 17.505
  12:37:41 STEP 2528 H=2528:00 +CPU= 15.299
  12:38:06 STEP 2529 H=2529:00 +CPU= 24.731
  12:38:23 STEP 2530 H=2530:00 +CPU= 16.444
  12:38:38 STEP 2531 H=2531:00 +CPU= 14.429
  12:39:06 STEP 2532 H=2532:00 +CPU= 27.147
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002532
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002532
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002532
  12:39:22 STEP 2533 H=2533:00 +CPU= 16.338
  12:39:38 STEP 2534 H=2534:00 +CPU= 15.490
  12:40:10 STEP 2535 H=2535:00 +CPU= 31.332
  12:40:26 STEP 2536 H=2536:00 +CPU= 15.963
  12:40:41 STEP 2537 H=2537:00 +CPU= 15.066
  12:41:12 STEP 2538 H=2538:00 +CPU= 30.667
  12:41:29 STEP 2539 H=2539:00 +CPU= 15.925
  12:41:45 STEP 2540 H=2540:00 +CPU= 16.230
  12:42:09 STEP 2541 H=2541:00 +CPU= 23.287
  12:42:27 STEP 2542 H=2542:00 +CPU= 18.386
  12:42:43 STEP 2543 H=2543:00 +CPU= 15.967
  12:43:13 STEP 2544 H=2544:00 +CPU= 29.319
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002544
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002544
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002544
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002520
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002520
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002520
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002532
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002532
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002532
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_105.zip
Uploading the intermediate file: upload_file_105.zip
  12:43:30 STEP 2545 H=2545:00 +CPU= 15.706
  12:43:46 STEP 2546 H=2546:00 +CPU= 16.641
  12:44:12 STEP 2547 H=2547:00 +CPU= 25.526
  12:44:30 STEP 2548 H=2548:00 +CPU= 17.026
  12:44:45 STEP 2549 H=2549:00 +CPU= 14.701
  12:45:22 STEP 2550 H=2550:00 +CPU= 36.882
  12:45:38 STEP 2551 H=2551:00 +CPU= 15.595
  12:45:54 STEP 2552 H=2552:00 +CPU= 15.942
  12:46:34 STEP 2553 H=2553:00 +CPU= 39.569
  12:46:50 STEP 2554 H=2554:00 +CPU= 15.510
  12:47:07 STEP 2555 H=2555:00 +CPU= 17.358
  12:47:36 STEP 2556 H=2556:00 +CPU= 28.183
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002556
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002556
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002556
  12:47:52 STEP 2557 H=2557:00 +CPU= 15.530
  12:48:08 STEP 2558 H=2558:00 +CPU= 16.013
  12:48:33 STEP 2559 H=2559:00 +CPU= 24.058
  12:48:51 STEP 2560 H=2560:00 +CPU= 17.243
  12:49:07 STEP 2561 H=2561:00 +CPU= 16.017
  12:49:31 STEP 2562 H=2562:00 +CPU= 24.193
  12:49:47 STEP 2563 H=2563:00 +CPU= 15.705
  12:50:05 STEP 2564 H=2564:00 +CPU= 16.852
  12:50:33 STEP 2565 H=2565:00 +CPU= 27.717
  12:50:49 STEP 2566 H=2566:00 +CPU= 16.074
  12:51:05 STEP 2567 H=2567:00 +CPU= 16.026
  12:51:35 STEP 2568 H=2568:00 +CPU= 29.004
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002568
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002568
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002568
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002544
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002544
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002544
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002556
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002556
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002556
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_106.zip
Uploading the intermediate file: upload_file_106.zip
  12:51:51 STEP 2569 H=2569:00 +CPU= 16.356
  12:52:08 STEP 2570 H=2570:00 +CPU= 16.219
  12:52:32 STEP 2571 H=2571:00 +CPU= 24.407
  12:52:48 STEP 2572 H=2572:00 +CPU= 15.760
  12:53:04 STEP 2573 H=2573:00 +CPU= 15.233
  12:53:41 STEP 2574 H=2574:00 +CPU= 36.740
  12:53:57 STEP 2575 H=2575:00 +CPU= 15.837
  12:54:29 STEP 2576 H=2576:00 +CPU= 31.626
  12:55:00 STEP 2577 H=2577:00 +CPU= 30.725
  12:55:16 STEP 2578 H=2578:00 +CPU= 15.410
  12:55:32 STEP 2579 H=2579:00 +CPU= 15.365
  12:55:59 STEP 2580 H=2580:00 +CPU= 26.628
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002580
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002580
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002580
  12:56:21 STEP 2581 H=2581:00 +CPU= 21.615
  12:56:37 STEP 2582 H=2582:00 +CPU= 15.655
  12:57:02 STEP 2583 H=2583:00 +CPU= 24.965
  12:57:22 STEP 2584 H=2584:00 +CPU= 18.772
  12:57:38 STEP 2585 H=2585:00 +CPU= 16.486
  12:58:05 STEP 2586 H=2586:00 +CPU= 25.846
  12:58:21 STEP 2587 H=2587:00 +CPU= 15.845
  12:58:37 STEP 2588 H=2588:00 +CPU= 16.058
  12:59:02 STEP 2589 H=2589:00 +CPU= 24.280
  12:59:17 STEP 2590 H=2590:00 +CPU= 15.511
  12:59:33 STEP 2591 H=2591:00 +CPU= 15.773
  13:00:10 STEP 2592 H=2592:00 +CPU= 36.410
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002592
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002592
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002592
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002568
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002568
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002568
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002580
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002580
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002580
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_107.zip
Uploading the intermediate file: upload_file_107.zip
  13:00:25 STEP 2593 H=2593:00 +CPU= 14.708
  13:00:40 STEP 2594 H=2594:00 +CPU= 14.845
  13:01:05 STEP 2595 H=2595:00 +CPU= 24.304
  13:01:21 STEP 2596 H=2596:00 +CPU= 15.402
  13:01:44 STEP 2597 H=2597:00 +CPU= 23.048
  13:02:09 STEP 2598 H=2598:00 +CPU= 24.948
  13:02:24 STEP 2599 H=2599:00 +CPU= 14.951
  13:02:40 STEP 2600 H=2600:00 +CPU= 15.262
  13:03:04 STEP 2601 H=2601:00 +CPU= 24.015
  13:03:20 STEP 2602 H=2602:00 +CPU= 15.955
  13:03:43 STEP 2603 H=2603:00 +CPU= 22.309
  13:04:20 STEP 2604 H=2604:00 +CPU= 36.142
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002604
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002604
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002604
  13:04:35 STEP 2605 H=2605:00 +CPU= 15.357
  13:04:52 STEP 2606 H=2606:00 +CPU= 16.041
  13:05:16 STEP 2607 H=2607:00 +CPU= 23.711
  13:05:33 STEP 2608 H=2608:00 +CPU= 17.602
  13:05:49 STEP 2609 H=2609:00 +CPU= 15.873
  13:06:14 STEP 2610 H=2610:00 +CPU= 24.134
  13:06:29 STEP 2611 H=2611:00 +CPU= 15.334
  13:06:46 STEP 2612 H=2612:00 +CPU= 15.928
  13:07:10 STEP 2613 H=2613:00 +CPU= 24.122
  13:07:27 STEP 2614 H=2614:00 +CPU= 16.468
  13:07:43 STEP 2615 H=2615:00 +CPU= 15.772
  13:08:12 STEP 2616 H=2616:00 +CPU= 28.489
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002616
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002616
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002616
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002592
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002592
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002592
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002604
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002604
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002604
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_108.zip
Uploading the intermediate file: upload_file_108.zip
  13:08:27 STEP 2617 H=2617:00 +CPU= 15.384
  13:08:43 STEP 2618 H=2618:00 +CPU= 15.676
  13:09:07 STEP 2619 H=2619:00 +CPU= 23.938
  13:09:23 STEP 2620 H=2620:00 +CPU= 15.745
  13:09:38 STEP 2621 H=2621:00 +CPU= 14.917
  13:10:04 STEP 2622 H=2622:00 +CPU= 25.044
  13:10:19 STEP 2623 H=2623:00 +CPU= 14.915
  13:10:35 STEP 2624 H=2624:00 +CPU= 15.603
  13:10:59 STEP 2625 H=2625:00 +CPU= 23.827
  13:11:15 STEP 2626 H=2626:00 +CPU= 15.513
  13:11:37 STEP 2627 H=2627:00 +CPU= 22.233
  13:12:17 STEP 2628 H=2628:00 +CPU= 38.709
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002628
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002628
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002628
  13:12:33 STEP 2629 H=2629:00 +CPU= 15.697
  13:12:49 STEP 2630 H=2630:00 +CPU= 16.036
  13:13:14 STEP 2631 H=2631:00 +CPU= 23.778
  13:13:30 STEP 2632 H=2632:00 +CPU= 16.057
  13:13:46 STEP 2633 H=2633:00 +CPU= 15.757
  13:14:11 STEP 2634 H=2634:00 +CPU= 24.590
  13:14:26 STEP 2635 H=2635:00 +CPU= 14.853
  13:14:42 STEP 2636 H=2636:00 +CPU= 15.384
  13:15:06 STEP 2637 H=2637:00 +CPU= 24.156
  13:15:22 STEP 2638 H=2638:00 +CPU= 15.176
  13:15:38 STEP 2639 H=2639:00 +CPU= 15.845
  13:16:17 STEP 2640 H=2640:00 +CPU= 38.216
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002640
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002640
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002640
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002616
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002616
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002616
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002628
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002628
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002628
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_109.zip
Uploading the intermediate file: upload_file_109.zip
  13:16:32 STEP 2641 H=2641:00 +CPU= 14.808
Uploading trickle at timestep: 9504000
  13:16:47 STEP 2642 H=2642:00 +CPU= 15.574
  13:17:19 STEP 2643 H=2643:00 +CPU= 30.928
  13:17:34 STEP 2644 H=2644:00 +CPU= 15.471
  13:17:50 STEP 2645 H=2645:00 +CPU= 15.145
  13:18:14 STEP 2646 H=2646:00 +CPU= 23.915
  13:18:30 STEP 2647 H=2647:00 +CPU= 15.305
  13:18:45 STEP 2648 H=2648:00 +CPU= 15.238
  13:19:10 STEP 2649 H=2649:00 +CPU= 24.153
  13:19:26 STEP 2650 H=2650:00 +CPU= 15.681
  13:19:41 STEP 2651 H=2651:00 +CPU= 15.567
  13:20:09 STEP 2652 H=2652:00 +CPU= 27.278
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002652
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002652
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002652
  13:20:24 STEP 2653 H=2653:00 +CPU= 14.774
  13:20:41 STEP 2654 H=2654:00 +CPU= 16.305
  13:21:06 STEP 2655 H=2655:00 +CPU= 25.144
  13:21:22 STEP 2656 H=2656:00 +CPU= 15.060
  13:21:37 STEP 2657 H=2657:00 +CPU= 15.189
  13:22:02 STEP 2658 H=2658:00 +CPU= 24.859
  13:22:17 STEP 2659 H=2659:00 +CPU= 14.944
  13:22:33 STEP 2660 H=2660:00 +CPU= 16.010
  13:22:57 STEP 2661 H=2661:00 +CPU= 23.724
  13:23:13 STEP 2662 H=2662:00 +CPU= 15.759
  13:23:55 STEP 2663 H=2663:00 +CPU= 41.129
  13:24:39 STEP 2664 H=2664:00 +CPU= 43.824
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002664
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002664
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002664
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002640
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002640
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002640
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002652
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002652
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002652
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_110.zip
Uploading the intermediate file: upload_file_110.zip
  13:24:55 STEP 2665 H=2665:00 +CPU= 15.304
  13:25:11 STEP 2666 H=2666:00 +CPU= 15.709
  13:25:42 STEP 2667 H=2667:00 +CPU= 31.487
  13:25:58 STEP 2668 H=2668:00 +CPU= 15.369
  13:26:13 STEP 2669 H=2669:00 +CPU= 15.051
  13:26:39 STEP 2670 H=2670:00 +CPU= 25.297
  13:26:55 STEP 2671 H=2671:00 +CPU= 15.264
  13:27:11 STEP 2672 H=2672:00 +CPU= 16.090
  13:27:38 STEP 2673 H=2673:00 +CPU= 25.568
  13:27:53 STEP 2674 H=2674:00 +CPU= 15.494
  13:28:11 STEP 2675 H=2675:00 +CPU= 16.682
  13:28:38 STEP 2676 H=2676:00 +CPU= 26.901
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002676
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002676
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002676
  13:28:53 STEP 2677 H=2677:00 +CPU= 15.026
  13:29:17 STEP 2678 H=2678:00 +CPU= 23.282
  13:29:42 STEP 2679 H=2679:00 +CPU= 24.289
  13:29:58 STEP 2680 H=2680:00 +CPU= 16.418
  13:30:17 STEP 2681 H=2681:00 +CPU= 18.785
  13:30:42 STEP 2682 H=2682:00 +CPU= 24.478
  13:30:58 STEP 2683 H=2683:00 +CPU= 15.898
  13:31:14 STEP 2684 H=2684:00 +CPU= 15.107
  13:31:38 STEP 2685 H=2685:00 +CPU= 23.676
  13:31:53 STEP 2686 H=2686:00 +CPU= 15.513
  13:32:09 STEP 2687 H=2687:00 +CPU= 15.575
  13:32:38 STEP 2688 H=2688:00 +CPU= 27.762
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002688
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002688
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002688
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002664
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002664
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002664
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002676
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002676
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002676
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_111.zip
Uploading the intermediate file: upload_file_111.zip
  13:32:54 STEP 2689 H=2689:00 +CPU= 16.294
  13:33:10 STEP 2690 H=2690:00 +CPU= 15.366
  13:33:35 STEP 2691 H=2691:00 +CPU= 24.961
  13:33:57 STEP 2692 H=2692:00 +CPU= 21.735
  13:34:13 STEP 2693 H=2693:00 +CPU= 15.173
  13:34:39 STEP 2694 H=2694:00 +CPU= 25.552
  13:34:54 STEP 2695 H=2695:00 +CPU= 15.057
  13:35:10 STEP 2696 H=2696:00 +CPU= 15.593
  13:35:34 STEP 2697 H=2697:00 +CPU= 24.040
  13:35:51 STEP 2698 H=2698:00 +CPU= 15.955
  13:36:06 STEP 2699 H=2699:00 +CPU= 14.734
  13:36:33 STEP 2700 H=2700:00 +CPU= 27.030
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002700
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002700
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002700
  13:36:50 STEP 2701 H=2701:00 +CPU= 16.546
  13:37:13 STEP 2702 H=2702:00 +CPU= 22.428
  13:37:37 STEP 2703 H=2703:00 +CPU= 23.497
  13:37:52 STEP 2704 H=2704:00 +CPU= 15.241
  13:38:08 STEP 2705 H=2705:00 +CPU= 15.655
  13:38:34 STEP 2706 H=2706:00 +CPU= 25.177
  13:38:49 STEP 2707 H=2707:00 +CPU= 15.100
  13:39:05 STEP 2708 H=2708:00 +CPU= 15.946
  13:39:30 STEP 2709 H=2709:00 +CPU= 24.728
  13:39:46 STEP 2710 H=2710:00 +CPU= 15.480
  13:40:02 STEP 2711 H=2711:00 +CPU= 15.850
  13:40:30 STEP 2712 H=2712:00 +CPU= 27.226
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002712
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002712
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002712
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002688
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002688
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002688
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002700
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002700
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002700
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_112.zip
Uploading the intermediate file: upload_file_112.zip
  13:40:45 STEP 2713 H=2713:00 +CPU= 14.769
  13:41:01 STEP 2714 H=2714:00 +CPU= 15.866
  13:41:25 STEP 2715 H=2715:00 +CPU= 23.532
  13:41:40 STEP 2716 H=2716:00 +CPU= 15.309
  13:41:57 STEP 2717 H=2717:00 +CPU= 15.856
  13:42:22 STEP 2718 H=2718:00 +CPU= 24.635
  13:42:37 STEP 2719 H=2719:00 +CPU= 15.152
  13:42:53 STEP 2720 H=2720:00 +CPU= 16.004
  13:43:18 STEP 2721 H=2721:00 +CPU= 24.734
  13:43:34 STEP 2722 H=2722:00 +CPU= 15.726
  13:43:50 STEP 2723 H=2723:00 +CPU= 15.460
  13:44:18 STEP 2724 H=2724:00 +CPU= 27.050
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002724
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002724
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002724
  13:44:33 STEP 2725 H=2725:00 +CPU= 15.373
  13:44:50 STEP 2726 H=2726:00 +CPU= 16.801
  13:45:14 STEP 2727 H=2727:00 +CPU= 23.585
  13:45:30 STEP 2728 H=2728:00 +CPU= 15.338
  13:45:45 STEP 2729 H=2729:00 +CPU= 14.311
  13:46:10 STEP 2730 H=2730:00 +CPU= 24.611
  13:46:25 STEP 2731 H=2731:00 +CPU= 15.377
  13:46:43 STEP 2732 H=2732:00 +CPU= 17.745
  13:47:07 STEP 2733 H=2733:00 +CPU= 23.884
  13:47:23 STEP 2734 H=2734:00 +CPU= 15.592
  13:47:39 STEP 2735 H=2735:00 +CPU= 15.410
  13:48:08 STEP 2736 H=2736:00 +CPU= 28.729
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002736
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002736
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002736
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002712
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002712
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002712
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002724
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002724
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002724
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_113.zip
Uploading the intermediate file: upload_file_113.zip
  13:48:24 STEP 2737 H=2737:00 +CPU= 15.455
  13:48:55 STEP 2738 H=2738:00 +CPU= 30.947
  13:49:20 STEP 2739 H=2739:00 +CPU= 24.174
  13:49:36 STEP 2740 H=2740:00 +CPU= 16.090
  13:49:51 STEP 2741 H=2741:00 +CPU= 14.578
  13:50:16 STEP 2742 H=2742:00 +CPU= 24.813
  13:50:31 STEP 2743 H=2743:00 +CPU= 14.888
  13:50:46 STEP 2744 H=2744:00 +CPU= 15.169
  13:51:10 STEP 2745 H=2745:00 +CPU= 23.766
  13:51:27 STEP 2746 H=2746:00 +CPU= 15.845
  13:51:41 STEP 2747 H=2747:00 +CPU= 14.548
  13:52:12 STEP 2748 H=2748:00 +CPU= 29.998
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002748
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002748
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002748
  13:52:28 STEP 2749 H=2749:00 +CPU= 15.580
  13:52:44 STEP 2750 H=2750:00 +CPU= 16.068
  13:53:09 STEP 2751 H=2751:00 +CPU= 24.490
  13:53:25 STEP 2752 H=2752:00 +CPU= 15.792
  13:53:41 STEP 2753 H=2753:00 +CPU= 14.997
  13:54:05 STEP 2754 H=2754:00 +CPU= 24.011
  13:54:20 STEP 2755 H=2755:00 +CPU= 14.776
  13:54:35 STEP 2756 H=2756:00 +CPU= 15.292
  13:55:00 STEP 2757 H=2757:00 +CPU= 23.877
  13:55:16 STEP 2758 H=2758:00 +CPU= 16.456
  13:55:32 STEP 2759 H=2759:00 +CPU= 15.525
  13:56:01 STEP 2760 H=2760:00 +CPU= 28.201
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002760
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002760
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002760
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002736
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002736
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002736
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002748
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002748
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002748
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_114.zip
Uploading the intermediate file: upload_file_114.zip
  13:56:16 STEP 2761 H=2761:00 +CPU= 15.147
  13:56:33 STEP 2762 H=2762:00 +CPU= 16.256
  13:56:57 STEP 2763 H=2763:00 +CPU= 24.194
  13:57:14 STEP 2764 H=2764:00 +CPU= 16.716
  13:57:32 STEP 2765 H=2765:00 +CPU= 17.394
  13:57:57 STEP 2766 H=2766:00 +CPU= 25.103
  13:58:14 STEP 2767 H=2767:00 +CPU= 16.264
  13:58:30 STEP 2768 H=2768:00 +CPU= 15.784
  13:58:55 STEP 2769 H=2769:00 +CPU= 24.492
  13:59:11 STEP 2770 H=2770:00 +CPU= 15.951
  13:59:26 STEP 2771 H=2771:00 +CPU= 15.283
  13:59:58 STEP 2772 H=2772:00 +CPU= 30.773
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002772
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002772
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002772
  14:00:21 STEP 2773 H=2773:00 +CPU= 23.173
  14:00:43 STEP 2774 H=2774:00 +CPU= 20.554
  14:01:07 STEP 2775 H=2775:00 +CPU= 24.272
  14:01:24 STEP 2776 H=2776:00 +CPU= 15.924
  14:01:39 STEP 2777 H=2777:00 +CPU= 15.383
  14:02:07 STEP 2778 H=2778:00 +CPU= 26.831
  14:02:22 STEP 2779 H=2779:00 +CPU= 15.232
  14:02:38 STEP 2780 H=2780:00 +CPU= 15.491
  14:03:02 STEP 2781 H=2781:00 +CPU= 23.959
  14:03:18 STEP 2782 H=2782:00 +CPU= 15.583
  14:03:34 STEP 2783 H=2783:00 +CPU= 15.972
  14:04:07 STEP 2784 H=2784:00 +CPU= 32.315
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002784
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002784
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002784
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002760
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002760
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002760
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002772
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002772
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002772
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_115.zip
Uploading the intermediate file: upload_file_115.zip
  14:04:22 STEP 2785 H=2785:00 +CPU= 15.018
  14:04:45 STEP 2786 H=2786:00 +CPU= 23.453
  14:05:21 STEP 2787 H=2787:00 +CPU= 34.830
  14:05:36 STEP 2788 H=2788:00 +CPU= 15.529
  14:05:52 STEP 2789 H=2789:00 +CPU= 15.257
  14:06:16 STEP 2790 H=2790:00 +CPU= 24.069
  14:06:32 STEP 2791 H=2791:00 +CPU= 15.625
  14:06:49 STEP 2792 H=2792:00 +CPU= 16.276
  14:07:19 STEP 2793 H=2793:00 +CPU= 30.196
  14:07:42 STEP 2794 H=2794:00 +CPU= 22.507
  14:07:58 STEP 2795 H=2795:00 +CPU= 15.347
  14:08:26 STEP 2796 H=2796:00 +CPU= 27.928
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002796
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002796
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002796
  14:08:42 STEP 2797 H=2797:00 +CPU= 15.039
  14:08:58 STEP 2798 H=2798:00 +CPU= 15.749
  14:09:22 STEP 2799 H=2799:00 +CPU= 23.711
  14:09:37 STEP 2800 H=2800:00 +CPU= 15.151
  14:09:53 STEP 2801 H=2801:00 +CPU= 15.433
  14:10:18 STEP 2802 H=2802:00 +CPU= 24.727
  14:10:34 STEP 2803 H=2803:00 +CPU= 15.745
  14:10:50 STEP 2804 H=2804:00 +CPU= 15.857
  14:11:14 STEP 2805 H=2805:00 +CPU= 23.779
  14:11:31 STEP 2806 H=2806:00 +CPU= 16.090
  14:11:47 STEP 2807 H=2807:00 +CPU= 15.512
  14:12:18 STEP 2808 H=2808:00 +CPU= 31.454
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002808
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002808
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002808
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002784
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002784
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002784
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002796
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002796
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002796
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_116.zip
Uploading the intermediate file: upload_file_116.zip
  14:12:34 STEP 2809 H=2809:00 +CPU= 15.353
  14:12:58 STEP 2810 H=2810:00 +CPU= 23.781
  14:13:40 STEP 2811 H=2811:00 +CPU= 40.968
  14:13:56 STEP 2812 H=2812:00 +CPU= 15.850
  14:14:11 STEP 2813 H=2813:00 +CPU= 15.312
  14:14:36 STEP 2814 H=2814:00 +CPU= 24.502
  14:14:52 STEP 2815 H=2815:00 +CPU= 15.868
  14:15:09 STEP 2816 H=2816:00 +CPU= 15.944
  14:15:34 STEP 2817 H=2817:00 +CPU= 24.735
  14:15:50 STEP 2818 H=2818:00 +CPU= 15.456
  14:16:05 STEP 2819 H=2819:00 +CPU= 15.365
  14:16:32 STEP 2820 H=2820:00 +CPU= 26.514
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002820
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002820
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002820
  14:16:48 STEP 2821 H=2821:00 +CPU= 15.457
  14:17:05 STEP 2822 H=2822:00 +CPU= 16.218
  14:17:29 STEP 2823 H=2823:00 +CPU= 24.033
  14:17:45 STEP 2824 H=2824:00 +CPU= 15.250
  14:18:00 STEP 2825 H=2825:00 +CPU= 15.187
  14:18:24 STEP 2826 H=2826:00 +CPU= 24.057
  14:18:40 STEP 2827 H=2827:00 +CPU= 15.765
  14:19:00 STEP 2828 H=2828:00 +CPU= 15.107
  14:19:24 STEP 2829 H=2829:00 +CPU= 24.226
  14:19:57 STEP 2830 H=2830:00 +CPU= 32.170
  14:20:13 STEP 2831 H=2831:00 +CPU= 15.932
  14:20:42 STEP 2832 H=2832:00 +CPU= 28.386
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002832
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002832
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002832
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002808
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002808
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002808
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002820
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002820
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002820
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_117.zip
Uploading the intermediate file: upload_file_117.zip
  14:21:05 STEP 2833 H=2833:00 +CPU= 22.558
  14:21:21 STEP 2834 H=2834:00 +CPU= 15.715
  14:21:45 STEP 2835 H=2835:00 +CPU= 24.555
  14:22:02 STEP 2836 H=2836:00 +CPU= 16.312
  14:22:21 STEP 2837 H=2837:00 +CPU= 18.709
  14:22:46 STEP 2838 H=2838:00 +CPU= 24.450
  14:23:01 STEP 2839 H=2839:00 +CPU= 15.380
  14:23:17 STEP 2840 H=2840:00 +CPU= 15.306
  14:23:48 STEP 2841 H=2841:00 +CPU= 30.492
  14:24:04 STEP 2842 H=2842:00 +CPU= 15.769
  14:24:25 STEP 2843 H=2843:00 +CPU= 21.028
  14:24:51 STEP 2844 H=2844:00 +CPU= 26.275
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002844
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002844
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002844
  14:25:07 STEP 2845 H=2845:00 +CPU= 15.549
  14:25:23 STEP 2846 H=2846:00 +CPU= 15.361
  14:25:48 STEP 2847 H=2847:00 +CPU= 24.565
  14:26:05 STEP 2848 H=2848:00 +CPU= 16.403
  14:26:22 STEP 2849 H=2849:00 +CPU= 16.401
  14:26:56 STEP 2850 H=2850:00 +CPU= 34.374
  14:27:12 STEP 2851 H=2851:00 +CPU= 15.092
  14:27:28 STEP 2852 H=2852:00 +CPU= 16.345
  14:27:53 STEP 2853 H=2853:00 +CPU= 24.657
  14:28:10 STEP 2854 H=2854:00 +CPU= 16.331
  14:28:26 STEP 2855 H=2855:00 +CPU= 16.113
  14:29:01 STEP 2856 H=2856:00 +CPU= 34.625
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002856
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002856
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002856
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002832
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002832
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002832
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002844
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002844
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002844
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_118.zip
Uploading the intermediate file: upload_file_118.zip
  14:29:17 STEP 2857 H=2857:00 +CPU= 15.839
  14:29:44 STEP 2858 H=2858:00 +CPU= 26.698
  14:30:09 STEP 2859 H=2859:00 +CPU= 24.192
  14:30:24 STEP 2860 H=2860:00 +CPU= 15.547
  14:30:40 STEP 2861 H=2861:00 +CPU= 15.513
  14:31:05 STEP 2862 H=2862:00 +CPU= 24.292
  14:31:21 STEP 2863 H=2863:00 +CPU= 16.137
  14:31:37 STEP 2864 H=2864:00 +CPU= 15.772
  14:32:02 STEP 2865 H=2865:00 +CPU= 24.701
  14:32:25 STEP 2866 H=2866:00 +CPU= 22.528
  14:32:41 STEP 2867 H=2867:00 +CPU= 15.278
  14:33:08 STEP 2868 H=2868:00 +CPU= 26.817
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002868
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002868
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002868
  14:33:23 STEP 2869 H=2869:00 +CPU= 15.134
  14:33:40 STEP 2870 H=2870:00 +CPU= 15.641
  14:34:04 STEP 2871 H=2871:00 +CPU= 24.033
  14:34:20 STEP 2872 H=2872:00 +CPU= 16.020
  14:34:37 STEP 2873 H=2873:00 +CPU= 16.327
  14:35:17 STEP 2874 H=2874:00 +CPU= 39.671
  14:35:52 STEP 2875 H=2875:00 +CPU= 34.350
  14:36:08 STEP 2876 H=2876:00 +CPU= 16.489
  14:36:33 STEP 2877 H=2877:00 +CPU= 24.004
  14:36:50 STEP 2878 H=2878:00 +CPU= 17.273
  14:37:07 STEP 2879 H=2879:00 +CPU= 16.015
  14:37:36 STEP 2880 H=2880:00 +CPU= 28.821
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002880
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002880
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002880
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002856
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002856
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002856
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002868
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002868
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002868
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_119.zip
Uploading the intermediate file: upload_file_119.zip
  14:37:51 STEP 2881 H=2881:00 +CPU= 14.877
Uploading trickle at timestep: 10368000
  14:38:15 STEP 2882 H=2882:00 +CPU= 23.847
  14:38:39 STEP 2883 H=2883:00 +CPU= 23.544
  14:39:03 STEP 2884 H=2884:00 +CPU= 23.460
  14:39:21 STEP 2885 H=2885:00 +CPU= 18.005
  14:39:47 STEP 2886 H=2886:00 +CPU= 25.981
  14:40:04 STEP 2887 H=2887:00 +CPU= 16.321
  14:40:19 STEP 2888 H=2888:00 +CPU= 15.292
  14:40:51 STEP 2889 H=2889:00 +CPU= 31.403
  14:41:16 STEP 2890 H=2890:00 +CPU= 24.472
  14:41:32 STEP 2891 H=2891:00 +CPU= 15.624
  14:42:00 STEP 2892 H=2892:00 +CPU= 27.887
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002892
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002892
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002892
  14:42:18 STEP 2893 H=2893:00 +CPU= 17.242
  14:42:33 STEP 2894 H=2894:00 +CPU= 15.393
  14:43:03 STEP 2895 H=2895:00 +CPU= 29.304
  14:43:20 STEP 2896 H=2896:00 +CPU= 16.599
  14:43:36 STEP 2897 H=2897:00 +CPU= 15.584
  14:44:02 STEP 2898 H=2898:00 +CPU= 25.207
  14:44:17 STEP 2899 H=2899:00 +CPU= 14.861
  14:44:34 STEP 2900 H=2900:00 +CPU= 16.546
  14:44:59 STEP 2901 H=2901:00 +CPU= 24.843
  14:45:15 STEP 2902 H=2902:00 +CPU= 15.611
  14:45:39 STEP 2903 H=2903:00 +CPU= 24.117
  14:46:22 STEP 2904 H=2904:00 +CPU= 42.025
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002904
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002904
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002904
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002880
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002880
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002880
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002892
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002892
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002892
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_120.zip
Uploading the intermediate file: upload_file_120.zip
  14:46:37 STEP 2905 H=2905:00 +CPU= 14.874
  14:46:53 STEP 2906 H=2906:00 +CPU= 15.812
  14:47:17 STEP 2907 H=2907:00 +CPU= 24.343
  14:47:33 STEP 2908 H=2908:00 +CPU= 15.659
  14:47:50 STEP 2909 H=2909:00 +CPU= 16.221
  14:48:16 STEP 2910 H=2910:00 +CPU= 25.521
  14:48:32 STEP 2911 H=2911:00 +CPU= 16.199
  14:48:48 STEP 2912 H=2912:00 +CPU= 16.120
  14:49:13 STEP 2913 H=2913:00 +CPU= 24.583
  14:49:34 STEP 2914 H=2914:00 +CPU= 20.491
  14:49:51 STEP 2915 H=2915:00 +CPU= 16.284
  14:50:28 STEP 2916 H=2916:00 +CPU= 36.430
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002916
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002916
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002916
  14:50:43 STEP 2917 H=2917:00 +CPU= 14.815
  14:51:00 STEP 2918 H=2918:00 +CPU= 16.106
  14:51:24 STEP 2919 H=2919:00 +CPU= 24.399
  14:51:46 STEP 2920 H=2920:00 +CPU= 21.055
  14:52:10 STEP 2921 H=2921:00 +CPU= 23.851
  14:52:35 STEP 2922 H=2922:00 +CPU= 24.978
  14:52:51 STEP 2923 H=2923:00 +CPU= 15.599
  14:53:07 STEP 2924 H=2924:00 +CPU= 15.730
  14:53:31 STEP 2925 H=2925:00 +CPU= 23.401
  14:54:03 STEP 2926 H=2926:00 +CPU= 32.357
  14:54:34 STEP 2927 H=2927:00 +CPU= 30.625
  14:55:04 STEP 2928 H=2928:00 +CPU= 28.904
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002928
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002928
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002928
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002904
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002904
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002904
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002916
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002916
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002916
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_121.zip
Uploading the intermediate file: upload_file_121.zip
  14:55:26 STEP 2929 H=2929:00 +CPU= 22.519
  14:55:42 STEP 2930 H=2930:00 +CPU= 15.165
  14:56:06 STEP 2931 H=2931:00 +CPU= 23.753
  14:56:22 STEP 2932 H=2932:00 +CPU= 15.855
  14:56:39 STEP 2933 H=2933:00 +CPU= 16.350
  14:57:05 STEP 2934 H=2934:00 +CPU= 25.517
  14:57:23 STEP 2935 H=2935:00 +CPU= 17.830
  14:57:43 STEP 2936 H=2936:00 +CPU= 19.629
  14:58:09 STEP 2937 H=2937:00 +CPU= 25.818
  14:58:24 STEP 2938 H=2938:00 +CPU= 15.404
  14:58:40 STEP 2939 H=2939:00 +CPU= 15.014
  14:59:06 STEP 2940 H=2940:00 +CPU= 26.207
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002940
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002940
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002940
  14:59:21 STEP 2941 H=2941:00 +CPU= 15.221
  14:59:37 STEP 2942 H=2942:00 +CPU= 15.992
  15:00:00 STEP 2943 H=2943:00 +CPU= 23.170
  15:00:16 STEP 2944 H=2944:00 +CPU= 15.596
  15:00:31 STEP 2945 H=2945:00 +CPU= 14.817
  15:00:55 STEP 2946 H=2946:00 +CPU= 24.489
  15:01:11 STEP 2947 H=2947:00 +CPU= 15.659
  15:01:26 STEP 2948 H=2948:00 +CPU= 15.289
  15:01:51 STEP 2949 H=2949:00 +CPU= 24.062
  15:02:07 STEP 2950 H=2950:00 +CPU= 16.245
  15:02:24 STEP 2951 H=2951:00 +CPU= 16.432
  15:02:52 STEP 2952 H=2952:00 +CPU= 27.863
The child process terminated with status: 0
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002952
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002952
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002952
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002952
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002940
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002928
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002952
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002940
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMUAhq0f+002928
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMGGhq0f+002952
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002940
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12185043/ICMSHhq0f+002928
Zipping up the final file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0399_1999050100_123_968_12185043_0_r1895724334_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
15:06:18 (1455): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:50:08 1517859 22270328 oifs_43r3_ps_0399_1999050100_123_968_12185043_0 5,184,000 27,895 0.0054
09 May 2023 15:49:57 1517859 22270328 oifs_43r3_ps_0399_1999050100_123_968_12185043_0 4,320,000 23,222 0.0054
09 May 2023 15:49:51 1517859 22270328 oifs_43r3_ps_0399_1999050100_123_968_12185043_0 3,456,000 18,531 0.0054
09 May 2023 15:49:46 1517859 22270328 oifs_43r3_ps_0399_1999050100_123_968_12185043_0 2,592,000 13,886 0.0054
09 May 2023 15:49:35 1517859 22270328 oifs_43r3_ps_0399_1999050100_123_968_12185043_0 1,728,000 9,293 0.0054
09 May 2023 15:49:30 1517859 22270328 oifs_43r3_ps_0399_1999050100_123_968_12185043_0 864,000 4,790 0.0055


©2024 climateprediction.net