Initializing AMReX (26.11-56-g9b56dbc0c2e4)... MPI initialized with 1 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.11-56-g9b56dbc0c2e4) initialized MFIX git describe: 26.11-21-ge4048e5d7789 AMReX-Hydro git hash: 154572aa6cf22c35a69729c74a0276909b96d5e3 Number of levels: 2 RunType is Standard Auto-generating level-set parameters: eb_refinement = 1 levelset_pad = 2 eb_pad = 2 ================================================================================ Warning: Overwriting amr.check_file with value from mfix.check_file ================================================================================ Simulation is transient Fluid constraint not specified. Setting default: mfix.constraint = IncompressibleFluid -------------------------------------------------------------------------------- Info: Max effective viscosity factor not specified. Using default 1e+06 -------------------------------------------------------------------------------- Load balance setup: Type: SingleGrid Strategy: KnapSack Weighting: CellCount Building cylinder geometry. Internal Flow: 0 Radius: 0.2 Height: -1 Direction: 2 Rotation angle: 0 Rotation axis: 0 Center: 0.5, 0.5, 0.5 Done making the fluid eb levels ... PostProcessBaseGrids Making new level 0 from scratch On level 0 updated BC areas: ErrorErr on level 0 Making new level 1 from scratch On level 1 updated BC areas: **************************************** Coverage report: Grids Cells regular: 0 6944 covered: 0 832 cut: 1 416 **************************************** On level 0 fillpatch epf On level 0 fillpatch rho On level 0 fillpatch vel On level 1 fillpatch epf On level 1 fillpatch rho On level 1 fillpatch vel On level 0 fillpatch vel On level 0 fillpatch rho On level 1 fillpatch vel On level 1 fillpatch rho Initial projection: Before projection: max(abs(u/v/w)) = 0.70710678 0.70710678 0 max(abs(gpx/gpy/gpz)) = 0 0 0 Min and Max of epf 1 1 On level 0: set velocity bcs Before projection: max(abs(u/v/w)) = 0.70710678 0.70710678 0 max(abs(gpx/gpy/gpz)) = 0 0 0 Min and Max of epf 1 1 On level 1: set velocity bcs On level 0: set vec bcs On level 1: set vec bcs Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 29.77431549 * On lev 1 max(abs(rhs)) = 59.74347518 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 59.74347518 MLMG: Initial residual (resid0) = 59.74347518 MLMG: Iteration 1 Fine resid/bnorm = 0.03956425674 MLMG: Iteration 2 Fine resid/bnorm = 0.000421647964 MLMG: Iteration 3 Fine resid/bnorm = 7.203495227e-06 MLMG: Iteration 4 Fine resid/bnorm = 7.434409574e-08 MLMG: Iteration 5 Fine resid/bnorm = 4.76970816e-09 MLMG: Iteration 6 Fine resid/bnorm = 3.801865907e-10 MLMG: Iteration 7 Fine resid/bnorm = 3.374727358e-11 MLMG: Iteration 8 Fine resid/bnorm = 3.817709148e-12 MLMG: Iteration 8 Crse resid/bnorm = 7.918059158e-10 MLMG: Iteration 9 Fine resid/bnorm = 4.46573843e-13 MLMG: Iteration 9 Crse resid/bnorm = 1.120774937e-10 MLMG: Iteration 10 Fine resid/bnorm = 5.415292334e-14 MLMG: Iteration 10 Crse resid/bnorm = 1.601390396e-11 MLMG: Iteration 11 Fine resid/bnorm = 1.704262319e-14 MLMG: Iteration 11 Crse resid/bnorm = 2.304105009e-12 MLMG: Final Iter. 11 resid, resid/bnorm = 1.376552404e-10, 2.304105009e-12 MLMG: Timers: Solve = 0.056406042 Iter = 0.049107073 Bottom = 0.026020868 On level 0: set velocity bcs After projection: max(abs(u/v/w)) = 1.47877986 1.47877986 4.84482246e-13 On level 1: set velocity bcs After projection: max(abs(u/v/w)) = 1.526266743 1.526266743 1.345788064e-12 Doing initial pressure iterations with dt = 0.004606829071 On level 0: set velocity bcs On level 0: set density bcs On level 1: set velocity bcs On level 1: set density bcs On level 0 fillpatch vel On level 0 fillpatch rho On level 1 fillpatch vel On level 1 fillpatch rho In initial_iterations: iter = 0 On level 0: set density bcs On level 1: set density bcs MAC Projection: On level 0 fillpatch none On level 1 fillpatch none >> Before projection * On level 0 max(abs(diveu)) = 1703.813941 * On level 1 max(abs(diveu)) = 1154.780884 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 3.578916976 MLMG: Initial residual (resid0) = 3.578916976 MLMG: Iteration 1 Fine resid/bnorm = 0.08252498039 MLMG: Iteration 2 Fine resid/bnorm = 0.003377432924 MLMG: Iteration 3 Fine resid/bnorm = 0.0001784643286 MLMG: Iteration 4 Fine resid/bnorm = 9.978290809e-06 MLMG: Iteration 5 Fine resid/bnorm = 5.802593014e-07 MLMG: Iteration 6 Fine resid/bnorm = 3.455270795e-08 MLMG: Iteration 7 Fine resid/bnorm = 2.068685245e-09 MLMG: Iteration 8 Fine resid/bnorm = 1.243745387e-10 MLMG: Iteration 9 Fine resid/bnorm = 7.447362843e-12 MLMG: Iteration 9 Crse resid/bnorm = 3.584641539e-13 MLMG: Final Iter. 9 resid, resid/bnorm = 2.665349331e-11, 7.447362843e-12 MLMG: Timers: Solve = 0.059102374 Iter = 0.057220634 Bottom = 0.040881089 >> After projection * On level 0 max(abs(diveu)) = 2.60344683e-10 * On level 1 max(abs(diveu)) = 2.628228385e-10 On level 0 fillpatch none On level 1 fillpatch none On level 0: set velocity bcs On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Before projection: max(abs(u/v/w)) = 1.422205574 1.422205574 4.95904335e-11 max(abs(gpx/gpy/gpz)) = 0 0 0 Min and Max of epf 1 1 On level 0: set velocity bcs Before projection: max(abs(u/v/w)) = 1.449630343 1.449630343 8.197741949e-11 max(abs(gpx/gpy/gpz)) = 0 0 0 Min and Max of epf 1 1 On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 1.867782749 * On lev 1 max(abs(rhs)) = 4.449545874 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 4.449545874 MLMG: Initial residual (resid0) = 4.449545874 MLMG: Iteration 1 Fine resid/bnorm = 0.02638538928 MLMG: Iteration 2 Fine resid/bnorm = 0.0003114231446 MLMG: Iteration 3 Fine resid/bnorm = 5.508609671e-06 MLMG: Iteration 4 Fine resid/bnorm = 6.139801642e-08 MLMG: Iteration 5 Fine resid/bnorm = 1.893992274e-09 MLMG: Iteration 6 Fine resid/bnorm = 1.91576933e-10 MLMG: Iteration 7 Fine resid/bnorm = 2.109306014e-11 MLMG: Iteration 8 Fine resid/bnorm = 2.397524921e-12 MLMG: Iteration 8 Crse resid/bnorm = 5.560574558e-10 MLMG: Iteration 9 Fine resid/bnorm = 2.797930605e-13 MLMG: Iteration 9 Crse resid/bnorm = 7.871486251e-11 MLMG: Iteration 10 Fine resid/bnorm = 3.420911941e-14 MLMG: Iteration 10 Crse resid/bnorm = 1.12462981e-11 MLMG: Iteration 11 Fine resid/bnorm = 1.088815772e-14 MLMG: Iteration 11 Crse resid/bnorm = 1.617873101e-12 MLMG: Final Iter. 11 resid, resid/bnorm = 7.198800583e-12, 1.617873101e-12 MLMG: Timers: Solve = 0.049107865 Iter = 0.045790382 Bottom = 0.026108275 On level 0: set velocity bcs After projection: max(abs(u/v/w)) = 1.413131257 1.413131257 4.941719286e-11 On level 1: set velocity bcs After projection: max(abs(u/v/w)) = 1.444629562 1.444629562 8.185028433e-11 On level 0: set velocity bcs On level 0: set density bcs On level 1: set velocity bcs On level 1: set density bcs In initial_iterations: iter = 1 On level 0: set density bcs On level 1: set density bcs MAC Projection: On level 0 fillpatch none On level 1 fillpatch none >> Before projection * On level 0 max(abs(diveu)) = 1268.151996 * On level 1 max(abs(diveu)) = 804.0916384 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 2.70324018 MLMG: Initial residual (resid0) = 2.70324018 MLMG: Iteration 1 Fine resid/bnorm = 0.02316575942 MLMG: Iteration 2 Fine resid/bnorm = 0.001325050296 MLMG: Iteration 3 Fine resid/bnorm = 5.900485353e-05 MLMG: Iteration 4 Fine resid/bnorm = 2.949970982e-06 MLMG: Iteration 5 Fine resid/bnorm = 1.557532997e-07 MLMG: Iteration 6 Fine resid/bnorm = 8.55385241e-09 MLMG: Iteration 7 Fine resid/bnorm = 4.820339082e-10 MLMG: Iteration 8 Fine resid/bnorm = 2.760137991e-11 MLMG: Iteration 9 Fine resid/bnorm = 1.596516112e-12 MLMG: Iteration 9 Crse resid/bnorm = 7.098065688e-14 MLMG: Final Iter. 9 resid, resid/bnorm = 4.315766502e-12, 1.596516112e-12 MLMG: Timers: Solve = 0.056841769 Iter = 0.05598106 Bottom = 0.04120855 >> After projection * On level 0 max(abs(diveu)) = 4.991263467e-11 * On level 1 max(abs(diveu)) = 5.521947514e-11 On level 0 fillpatch none On level 1 fillpatch none On level 0: set velocity bcs On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Before projection: max(abs(u/v/w)) = 1.413491111 1.413491111 4.745610932e-11 max(abs(gpx/gpy/gpz)) = 14.29688751 14.29688751 3.980198975e-11 Min and Max of epf 1 1 On level 0: set velocity bcs Before projection: max(abs(u/v/w)) = 1.447074151 1.447074151 7.914877385e-11 max(abs(gpx/gpy/gpz)) = 23.00094001 23.00094001 1.177016453e-10 Min and Max of epf 1 1 On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 0.1509727113 * On lev 1 max(abs(rhs)) = 0.6927293804 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 0.6927293804 MLMG: Initial residual (resid0) = 0.6927293804 MLMG: Iteration 1 Fine resid/bnorm = 0.005130373494 MLMG: Iteration 2 Fine resid/bnorm = 0.0001111445902 MLMG: Iteration 3 Fine resid/bnorm = 2.159134709e-06 MLMG: Iteration 4 Fine resid/bnorm = 3.464097178e-08 MLMG: Iteration 5 Fine resid/bnorm = 1.216361488e-09 MLMG: Iteration 6 Fine resid/bnorm = 8.540307764e-11 MLMG: Iteration 7 Fine resid/bnorm = 6.595656292e-12 MLMG: Iteration 7 Crse resid/bnorm = 4.399390471e-10 MLMG: Iteration 8 Fine resid/bnorm = 6.162702501e-13 MLMG: Iteration 8 Crse resid/bnorm = 6.205957688e-11 MLMG: Iteration 9 Fine resid/bnorm = 5.784670526e-14 MLMG: Iteration 9 Crse resid/bnorm = 8.815504199e-12 MLMG: Final Iter. 9 resid, resid/bnorm = 6.106758762e-12, 8.815504199e-12 MLMG: Timers: Solve = 0.04165951 Iter = 0.038376968 Bottom = 0.022410372 On level 0: set velocity bcs After projection: max(abs(u/v/w)) = 1.411707841 1.411707841 4.737762193e-11 On level 1: set velocity bcs After projection: max(abs(u/v/w)) = 1.44368775 1.44368775 7.910067033e-11 On level 0: set velocity bcs On level 0: set density bcs On level 1: set velocity bcs On level 1: set density bcs In initial_iterations: iter = 2 On level 0: set density bcs On level 1: set density bcs MAC Projection: On level 0 fillpatch none On level 1 fillpatch none >> Before projection * On level 0 max(abs(diveu)) = 1241.202413 * On level 1 max(abs(diveu)) = 804.0916384 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 3.04626377 MLMG: Initial residual (resid0) = 3.04626377 MLMG: Iteration 1 Fine resid/bnorm = 0.02202726546 MLMG: Iteration 2 Fine resid/bnorm = 0.001128529574 MLMG: Iteration 3 Fine resid/bnorm = 4.922003582e-05 MLMG: Iteration 4 Fine resid/bnorm = 2.531874782e-06 MLMG: Iteration 5 Fine resid/bnorm = 1.38050988e-07 MLMG: Iteration 6 Fine resid/bnorm = 7.792644348e-09 MLMG: Iteration 7 Fine resid/bnorm = 4.500904411e-10 MLMG: Iteration 8 Fine resid/bnorm = 2.632683396e-11 MLMG: Iteration 9 Fine resid/bnorm = 1.551753512e-12 MLMG: Iteration 9 Crse resid/bnorm = 6.075078079e-14 MLMG: Final Iter. 9 resid, resid/bnorm = 4.727050502e-12, 1.551753512e-12 MLMG: Timers: Solve = 0.056731319 Iter = 0.055883843 Bottom = 0.041143117 >> After projection * On level 0 max(abs(diveu)) = 5.160543905e-11 * On level 1 max(abs(diveu)) = 5.797323855e-11 On level 0 fillpatch none On level 1 fillpatch none On level 0: set velocity bcs On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Before projection: max(abs(u/v/w)) = 1.41136956 1.41136956 3.235091122e-11 max(abs(gpx/gpy/gpz)) = 13.64992201 13.64992201 5.889104613e-11 Min and Max of epf 1 1 On level 0: set velocity bcs Before projection: max(abs(u/v/w)) = 1.44488668 1.44488668 6.416639724e-11 max(abs(gpx/gpy/gpz)) = 22.90394394 22.90394394 1.759288197e-10 Min and Max of epf 1 1 On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 0.11766771 * On lev 1 max(abs(rhs)) = 0.4624751242 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 0.4624751242 MLMG: Initial residual (resid0) = 0.4624751242 MLMG: Iteration 1 Fine resid/bnorm = 0.006958443278 MLMG: Iteration 2 Fine resid/bnorm = 0.0001285664033 MLMG: Iteration 3 Fine resid/bnorm = 2.304012747e-06 MLMG: Iteration 4 Fine resid/bnorm = 4.004948122e-08 MLMG: Iteration 5 Fine resid/bnorm = 9.73434925e-10 MLMG: Iteration 6 Fine resid/bnorm = 7.03568544e-11 MLMG: Iteration 7 Fine resid/bnorm = 5.599043338e-12 MLMG: Iteration 7 Crse resid/bnorm = 1.224053268e-10 MLMG: Iteration 8 Fine resid/bnorm = 4.99867295e-13 MLMG: Iteration 8 Crse resid/bnorm = 1.377181661e-11 MLMG: Iteration 9 Fine resid/bnorm = 4.599421277e-14 MLMG: Iteration 9 Crse resid/bnorm = 1.618167264e-12 MLMG: Final Iter. 9 resid, resid/bnorm = 7.483621063e-13, 1.618167264e-12 MLMG: Timers: Solve = 0.040240916 Iter = 0.036967464 Bottom = 0.021018734 On level 0: set velocity bcs After projection: max(abs(u/v/w)) = 1.410898546 1.410898546 3.230855792e-11 On level 1: set velocity bcs After projection: max(abs(u/v/w)) = 1.443660869 1.443660869 6.424958937e-11 On level 0: set velocity bcs On level 0: set density bcs On level 1: set velocity bcs On level 1: set density bcs Time spent in init 0.580071123 On level 0 fillpatch vel On level 1 fillpatch vel Writing plotfile BENCH09-refinement_plt00000 at time 0 ============ NEW TIME STEP ============ On level 0: set density bcs On level 1: set density bcs Step 1: from old_time 0 to new time 0.004063119654 with dt = 0.004063119654 On level 0 fillpatch vel On level 0 fillpatch rho On level 1 fillpatch vel On level 1 fillpatch rho Finished fillpatch before predictor On level 0: set density bcs On level 1: set density bcs MAC Projection: On level 0 fillpatch none On level 1 fillpatch none >> Before projection * On level 0 max(abs(diveu)) = 1212.555318 * On level 1 max(abs(diveu)) = 794.3420316 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 3.032387192 MLMG: Initial residual (resid0) = 3.032387192 MLMG: Iteration 1 Fine resid/bnorm = 0.02189094388 MLMG: Iteration 2 Fine resid/bnorm = 0.001109100122 MLMG: Iteration 3 Fine resid/bnorm = 4.946643993e-05 MLMG: Iteration 4 Fine resid/bnorm = 2.542151246e-06 MLMG: Iteration 5 Fine resid/bnorm = 1.382643613e-07 MLMG: Iteration 6 Fine resid/bnorm = 7.79490342e-09 MLMG: Iteration 7 Fine resid/bnorm = 4.491734931e-10 MLMG: Iteration 8 Fine resid/bnorm = 2.62458703e-11 MLMG: Iteration 9 Fine resid/bnorm = 1.546926319e-12 MLMG: Iteration 9 Crse resid/bnorm = 5.974263806e-14 MLMG: Final Iter. 9 resid, resid/bnorm = 4.690879555e-12, 1.546926319e-12 MLMG: Timers: Solve = 0.062124376 Iter = 0.060819082 Bottom = 0.044946114 >> After projection * On level 0 max(abs(diveu)) = 5.057241546e-11 * On level 1 max(abs(diveu)) = 5.252609963e-11 On level 0 fillpatch none On level 1 fillpatch none On level 0: set velocity bcs On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Before projection: max(abs(u/v/w)) = 1.428759962 1.428759962 2.402418042e-11 max(abs(gpx/gpy/gpz)) = 14.23448798 14.23448799 5.912824545e-11 Min and Max of epf 1 1 On level 0: set velocity bcs Before projection: max(abs(u/v/w)) = 1.454205373 1.454205373 3.872442866e-11 max(abs(gpx/gpy/gpz)) = 22.61989507 22.61989507 2.112325328e-10 Min and Max of epf 1 1 On level 1: set velocity bcs On level 0: set vec bcs On level 1: set vec bcs Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 1.736115617 * On lev 1 max(abs(rhs)) = 4.278230169 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 4.278230169 MLMG: Initial residual (resid0) = 4.278230169 MLMG: Iteration 1 Fine resid/bnorm = 0.02504966824 MLMG: Iteration 2 Fine resid/bnorm = 0.0002770456339 MLMG: Iteration 3 Fine resid/bnorm = 4.847767991e-06 MLMG: Iteration 4 Fine resid/bnorm = 5.120429789e-08 MLMG: Iteration 5 Fine resid/bnorm = 2.938109721e-09 MLMG: Iteration 6 Fine resid/bnorm = 2.611187256e-10 MLMG: Iteration 7 Fine resid/bnorm = 2.460708213e-11 MLMG: Iteration 8 Fine resid/bnorm = 2.379220008e-12 MLMG: Iteration 8 Crse resid/bnorm = 5.145451925e-10 MLMG: Iteration 9 Fine resid/bnorm = 2.620621487e-13 MLMG: Iteration 9 Crse resid/bnorm = 7.279710055e-11 MLMG: Iteration 10 Fine resid/bnorm = 3.194752439e-14 MLMG: Iteration 10 Crse resid/bnorm = 1.039804093e-11 MLMG: Iteration 11 Fine resid/bnorm = 6.983934003e-15 MLMG: Iteration 11 Crse resid/bnorm = 1.495747086e-12 MLMG: Final Iter. 11 resid, resid/bnorm = 6.399150307e-12, 1.495747086e-12 MLMG: Timers: Solve = 0.0490227 Iter = 0.045718428 Bottom = 0.02613065 On level 0: set velocity bcs After projection: max(abs(u/v/w)) = 1.424410448 1.424410448 2.393594806e-11 On level 1: set velocity bcs After projection: max(abs(u/v/w)) = 1.449945385 1.449945385 3.869275157e-11 Time per fluid step 0.167944276 Time per step 0.168768988 On level 0 fillpatch vel On level 1 fillpatch vel Writing plotfile BENCH09-refinement_plt00001 at time 0.004063119654 On level 0 fillpatch vel On level 1 fillpatch vel Time spent in main (after init) 0.191273868 Time spent in main 0.771344991 Total GPU global memory (MB) spread across MPI: [7784 ... 7784] Free GPU global memory (MB) spread across MPI: [6340 ... 6340] [The Arena] max space (MB) allocated spread across MPI: [193 ... 193] [The Arena] max space (MB) used spread across MPI: [154 ... 154] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Pinned Arena] max space (MB) used spread across MPI: [2 ... 2] AMReX (26.11-56-g9b56dbc0c2e4) finalized