Initializing AMReX (26.11-26-ga8db71f5aeb1)... MPI initialized with 1 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.11-26-ga8db71f5aeb1) initialized MFIX git describe: 26.11-21-ge4048e5d7789 AMReX-Hydro git hash: 154572aa6cf22c35a69729c74a0276909b96d5e3 Number of levels: 2 RunType is Standard Auto-generating level-set parameters: eb_refinement = 1 levelset_pad = 2 eb_pad = 2 ================================================================================ Warning: Overwriting amr.check_file with value from mfix.check_file ================================================================================ Simulation is transient Fluid constraint not specified. Setting default: mfix.constraint = IncompressibleFluid -------------------------------------------------------------------------------- Info: Max effective viscosity factor not specified. Using default 1e+06 -------------------------------------------------------------------------------- Load balance setup: Type: SingleGrid Strategy: KnapSack Weighting: CellCount Building cylinder geometry. Internal Flow: 0 Radius: 0.2 Height: -1 Direction: 2 Rotation angle: 0 Rotation axis: 0 Center: 0.5, 0.5, 0.5 Done making the fluid eb levels ... PostProcessBaseGrids Making new level 0 from scratch On level 0 updated BC areas: ErrorErr on level 0 Making new level 1 from scratch On level 1 updated BC areas: **************************************** Coverage report: Grids Cells regular: 0 6944 covered: 0 832 cut: 1 416 **************************************** On level 0 fillpatch epf On level 0 fillpatch rho On level 0 fillpatch vel On level 1 fillpatch epf On level 1 fillpatch rho On level 1 fillpatch vel On level 0 fillpatch vel On level 0 fillpatch rho On level 1 fillpatch vel On level 1 fillpatch rho Initial projection: Before projection: max(abs(u/v/w)) = 0.70710678 0.70710678 0 max(abs(gpx/gpy/gpz)) = 0 0 0 Min and Max of epf 1 1 On level 0: set velocity bcs Before projection: max(abs(u/v/w)) = 0.70710678 0.70710678 0 max(abs(gpx/gpy/gpz)) = 0 0 0 Min and Max of epf 1 1 On level 1: set velocity bcs On level 0: set vec bcs On level 1: set vec bcs Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 29.77431549 * On lev 1 max(abs(rhs)) = 59.74347518 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 59.74347518 MLMG: Initial residual (resid0) = 59.74347518 MLMG: Iteration 1 Fine resid/bnorm = 0.03956425674 MLMG: Iteration 2 Fine resid/bnorm = 0.000421647964 MLMG: Iteration 3 Fine resid/bnorm = 7.203495227e-06 MLMG: Iteration 4 Fine resid/bnorm = 7.434409574e-08 MLMG: Iteration 5 Fine resid/bnorm = 4.76970816e-09 MLMG: Iteration 6 Fine resid/bnorm = 3.801865907e-10 MLMG: Iteration 7 Fine resid/bnorm = 3.374727358e-11 MLMG: Iteration 8 Fine resid/bnorm = 3.817709148e-12 MLMG: Iteration 8 Crse resid/bnorm = 7.918059158e-10 MLMG: Iteration 9 Fine resid/bnorm = 4.46573843e-13 MLMG: Iteration 9 Crse resid/bnorm = 1.120774937e-10 MLMG: Iteration 10 Fine resid/bnorm = 5.415292334e-14 MLMG: Iteration 10 Crse resid/bnorm = 1.601390396e-11 MLMG: Iteration 11 Fine resid/bnorm = 1.704262319e-14 MLMG: Iteration 11 Crse resid/bnorm = 2.304105009e-12 MLMG: Final Iter. 11 resid, resid/bnorm = 1.376552404e-10, 2.304105009e-12 MLMG: Timers: Solve = 0.057927499 Iter = 0.050622773 Bottom = 0.027018074 On level 0: set velocity bcs After projection: max(abs(u/v/w)) = 1.47877986 1.47877986 4.84482246e-13 On level 1: set velocity bcs After projection: max(abs(u/v/w)) = 1.526266743 1.526266743 1.345788064e-12 Doing initial pressure iterations with dt = 0.004606829071 On level 0: set velocity bcs On level 0: set density bcs On level 1: set velocity bcs On level 1: set density bcs On level 0 fillpatch vel On level 0 fillpatch rho On level 1 fillpatch vel On level 1 fillpatch rho In initial_iterations: iter = 0 On level 0: set density bcs On level 1: set density bcs MAC Projection: On level 0 fillpatch none On level 1 fillpatch none >> Before projection * On level 0 max(abs(diveu)) = 1703.813941 * On level 1 max(abs(diveu)) = 1154.780884 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 3.578916976 MLMG: Initial residual (resid0) = 3.578916976 MLMG: Iteration 1 Fine resid/bnorm = 0.08236280267 MLMG: Iteration 2 Fine resid/bnorm = 0.00338245698 MLMG: Iteration 3 Fine resid/bnorm = 0.0001785526587 MLMG: Iteration 4 Fine resid/bnorm = 9.990765081e-06 MLMG: Iteration 5 Fine resid/bnorm = 5.802379906e-07 MLMG: Iteration 6 Fine resid/bnorm = 3.461423308e-08 MLMG: Iteration 7 Fine resid/bnorm = 2.069209327e-09 MLMG: Iteration 8 Fine resid/bnorm = 1.245848053e-10 MLMG: Iteration 9 Fine resid/bnorm = 7.46542329e-12 MLMG: Iteration 9 Crse resid/bnorm = 3.589779425e-13 MLMG: Final Iter. 9 resid, resid/bnorm = 2.671813015e-11, 7.46542329e-12 MLMG: Timers: Solve = 0.061654836 Iter = 0.059755564 Bottom = 0.042506514 >> After projection * On level 0 max(abs(diveu)) = 2.485564985e-10 * On level 1 max(abs(diveu)) = 2.523994578e-10 On level 0 fillpatch none On level 1 fillpatch none On level 0: set velocity bcs On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Before projection: max(abs(u/v/w)) = 1.422205574 1.422205574 3.686744111e-11 max(abs(gpx/gpy/gpz)) = 0 0 0 Min and Max of epf 1 1 On level 0: set velocity bcs Before projection: max(abs(u/v/w)) = 1.449630343 1.449630343 5.250700326e-11 max(abs(gpx/gpy/gpz)) = 0 0 0 Min and Max of epf 1 1 On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 1.867782749 * On lev 1 max(abs(rhs)) = 4.449545874 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 4.449545874 MLMG: Initial residual (resid0) = 4.449545874 MLMG: Iteration 1 Fine resid/bnorm = 0.02638538928 MLMG: Iteration 2 Fine resid/bnorm = 0.0003114231446 MLMG: Iteration 3 Fine resid/bnorm = 5.508609672e-06 MLMG: Iteration 4 Fine resid/bnorm = 6.139801543e-08 MLMG: Iteration 5 Fine resid/bnorm = 1.89399192e-09 MLMG: Iteration 6 Fine resid/bnorm = 1.915771366e-10 MLMG: Iteration 7 Fine resid/bnorm = 2.109293538e-11 MLMG: Iteration 8 Fine resid/bnorm = 2.397406402e-12 MLMG: Iteration 8 Crse resid/bnorm = 5.560574356e-10 MLMG: Iteration 9 Fine resid/bnorm = 2.79956804e-13 MLMG: Iteration 9 Crse resid/bnorm = 7.871486251e-11 MLMG: Iteration 10 Fine resid/bnorm = 3.392373802e-14 MLMG: Iteration 10 Crse resid/bnorm = 1.124630434e-11 MLMG: Iteration 11 Fine resid/bnorm = 8.814074392e-15 MLMG: Iteration 11 Crse resid/bnorm = 1.61784815e-12 MLMG: Final Iter. 11 resid, resid/bnorm = 7.198689561e-12, 1.61784815e-12 MLMG: Timers: Solve = 0.050109246 Iter = 0.046822913 Bottom = 0.026825976 On level 0: set velocity bcs After projection: max(abs(u/v/w)) = 1.413131257 1.413131257 3.673119063e-11 On level 1: set velocity bcs After projection: max(abs(u/v/w)) = 1.444629562 1.444629562 5.238789567e-11 On level 0: set velocity bcs On level 0: set density bcs On level 1: set velocity bcs On level 1: set density bcs In initial_iterations: iter = 1 On level 0: set density bcs On level 1: set density bcs MAC Projection: On level 0 fillpatch none On level 1 fillpatch none >> Before projection * On level 0 max(abs(diveu)) = 1268.151996 * On level 1 max(abs(diveu)) = 804.0916384 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 2.70324018 MLMG: Initial residual (resid0) = 2.70324018 MLMG: Iteration 1 Fine resid/bnorm = 0.02316147042 MLMG: Iteration 2 Fine resid/bnorm = 0.001321005935 MLMG: Iteration 3 Fine resid/bnorm = 5.927065027e-05 MLMG: Iteration 4 Fine resid/bnorm = 2.951067669e-06 MLMG: Iteration 5 Fine resid/bnorm = 1.55743074e-07 MLMG: Iteration 6 Fine resid/bnorm = 8.586900254e-09 MLMG: Iteration 7 Fine resid/bnorm = 4.82002154e-10 MLMG: Iteration 8 Fine resid/bnorm = 2.773105881e-11 MLMG: Iteration 9 Fine resid/bnorm = 1.604003106e-12 MLMG: Iteration 9 Crse resid/bnorm = 7.103327792e-14 MLMG: Final Iter. 9 resid, resid/bnorm = 4.336005645e-12, 1.604003106e-12 MLMG: Timers: Solve = 0.056771168 Iter = 0.05591426 Bottom = 0.040933193 >> After projection * On level 0 max(abs(diveu)) = 4.994471919e-11 * On level 1 max(abs(diveu)) = 5.271162247e-11 On level 0 fillpatch none On level 1 fillpatch none On level 0: set velocity bcs On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Before projection: max(abs(u/v/w)) = 1.413491111 1.413491111 3.63941608e-11 max(abs(gpx/gpy/gpz)) = 14.29688751 14.29688751 2.957576066e-11 Min and Max of epf 1 1 On level 0: set velocity bcs Before projection: max(abs(u/v/w)) = 1.447074151 1.447074151 5.752719226e-11 max(abs(gpx/gpy/gpz)) = 23.00094001 23.00094001 9.222953613e-11 Min and Max of epf 1 1 On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 0.1509727113 * On lev 1 max(abs(rhs)) = 0.6927293804 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 0.6927293804 MLMG: Initial residual (resid0) = 0.6927293804 MLMG: Iteration 1 Fine resid/bnorm = 0.005130373494 MLMG: Iteration 2 Fine resid/bnorm = 0.0001111445902 MLMG: Iteration 3 Fine resid/bnorm = 2.159134709e-06 MLMG: Iteration 4 Fine resid/bnorm = 3.46409717e-08 MLMG: Iteration 5 Fine resid/bnorm = 1.216361503e-09 MLMG: Iteration 6 Fine resid/bnorm = 8.540307764e-11 MLMG: Iteration 7 Fine resid/bnorm = 6.595626242e-12 MLMG: Iteration 7 Crse resid/bnorm = 4.399390455e-10 MLMG: Iteration 8 Fine resid/bnorm = 6.162902836e-13 MLMG: Iteration 8 Crse resid/bnorm = 6.205957156e-11 MLMG: Iteration 9 Fine resid/bnorm = 5.781164665e-14 MLMG: Iteration 9 Crse resid/bnorm = 8.81550013e-12 MLMG: Final Iter. 9 resid, resid/bnorm = 6.106755943e-12, 8.81550013e-12 MLMG: Timers: Solve = 0.04239414 Iter = 0.039103095 Bottom = 0.022852373 On level 0: set velocity bcs After projection: max(abs(u/v/w)) = 1.411707841 1.411707841 3.628829004e-11 On level 1: set velocity bcs After projection: max(abs(u/v/w)) = 1.44368775 1.44368775 5.749366449e-11 On level 0: set velocity bcs On level 0: set density bcs On level 1: set velocity bcs On level 1: set density bcs In initial_iterations: iter = 2 On level 0: set density bcs On level 1: set density bcs MAC Projection: On level 0 fillpatch none On level 1 fillpatch none >> Before projection * On level 0 max(abs(diveu)) = 1241.202413 * On level 1 max(abs(diveu)) = 804.0916384 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 3.04626377 MLMG: Initial residual (resid0) = 3.04626377 MLMG: Iteration 1 Fine resid/bnorm = 0.02191001781 MLMG: Iteration 2 Fine resid/bnorm = 0.001125404201 MLMG: Iteration 3 Fine resid/bnorm = 4.945345234e-05 MLMG: Iteration 4 Fine resid/bnorm = 2.538573039e-06 MLMG: Iteration 5 Fine resid/bnorm = 1.383809898e-07 MLMG: Iteration 6 Fine resid/bnorm = 7.805141774e-09 MLMG: Iteration 7 Fine resid/bnorm = 4.505108745e-10 MLMG: Iteration 8 Fine resid/bnorm = 2.635079248e-11 MLMG: Iteration 9 Fine resid/bnorm = 1.554774674e-12 MLMG: Iteration 9 Crse resid/bnorm = 6.088987124e-14 MLMG: Final Iter. 9 resid, resid/bnorm = 4.736253761e-12, 1.554774674e-12 MLMG: Timers: Solve = 0.05724433 Iter = 0.056380177 Bottom = 0.041459778 >> After projection * On level 0 max(abs(diveu)) = 5.058559201e-11 * On level 1 max(abs(diveu)) = 5.533397742e-11 On level 0 fillpatch none On level 1 fillpatch none On level 0: set velocity bcs On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Before projection: max(abs(u/v/w)) = 1.41136956 1.41136956 4.58392452e-11 max(abs(gpx/gpy/gpz)) = 13.64992201 13.64992201 4.43773429e-11 Min and Max of epf 1 1 On level 0: set velocity bcs Before projection: max(abs(u/v/w)) = 1.44488668 1.44488668 7.83067975e-11 max(abs(gpx/gpy/gpz)) = 22.90394394 22.90394394 1.400246375e-10 Min and Max of epf 1 1 On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 0.11766771 * On lev 1 max(abs(rhs)) = 0.4624751242 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 0.4624751242 MLMG: Initial residual (resid0) = 0.4624751242 MLMG: Iteration 1 Fine resid/bnorm = 0.006958443278 MLMG: Iteration 2 Fine resid/bnorm = 0.0001285664033 MLMG: Iteration 3 Fine resid/bnorm = 2.304012748e-06 MLMG: Iteration 4 Fine resid/bnorm = 4.004948062e-08 MLMG: Iteration 5 Fine resid/bnorm = 9.734349325e-10 MLMG: Iteration 6 Fine resid/bnorm = 7.03568469e-11 MLMG: Iteration 7 Fine resid/bnorm = 5.599043338e-12 MLMG: Iteration 7 Crse resid/bnorm = 1.224053277e-10 MLMG: Iteration 8 Fine resid/bnorm = 4.998522911e-13 MLMG: Iteration 8 Crse resid/bnorm = 1.377182253e-11 MLMG: Iteration 9 Fine resid/bnorm = 4.60167185e-14 MLMG: Iteration 9 Crse resid/bnorm = 1.618164773e-12 MLMG: Final Iter. 9 resid, resid/bnorm = 7.483609544e-13, 1.618164773e-12 MLMG: Timers: Solve = 0.041078031 Iter = 0.037758243 Bottom = 0.021514641 On level 0: set velocity bcs After projection: max(abs(u/v/w)) = 1.410898546 1.410898546 4.56564246e-11 On level 1: set velocity bcs After projection: max(abs(u/v/w)) = 1.443660869 1.443660869 7.815606077e-11 On level 0: set velocity bcs On level 0: set density bcs On level 1: set velocity bcs On level 1: set density bcs Time spent in init 0.591353531 On level 0 fillpatch vel On level 1 fillpatch vel Writing plotfile BENCH09-refinement_plt00000 at time 0 ============ NEW TIME STEP ============ On level 0: set density bcs On level 1: set density bcs Step 1: from old_time 0 to new time 0.004063119654 with dt = 0.004063119654 On level 0 fillpatch vel On level 0 fillpatch rho On level 1 fillpatch vel On level 1 fillpatch rho Finished fillpatch before predictor On level 0: set density bcs On level 1: set density bcs MAC Projection: On level 0 fillpatch none On level 1 fillpatch none >> Before projection * On level 0 max(abs(diveu)) = 1212.555318 * On level 1 max(abs(diveu)) = 794.3420316 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 3.032387192 MLMG: Initial residual (resid0) = 3.032387192 MLMG: Iteration 1 Fine resid/bnorm = 0.02168051579 MLMG: Iteration 2 Fine resid/bnorm = 0.001110283768 MLMG: Iteration 3 Fine resid/bnorm = 4.956726657e-05 MLMG: Iteration 4 Fine resid/bnorm = 2.545172064e-06 MLMG: Iteration 5 Fine resid/bnorm = 1.38478388e-07 MLMG: Iteration 6 Fine resid/bnorm = 7.807272832e-09 MLMG: Iteration 7 Fine resid/bnorm = 4.497982772e-10 MLMG: Iteration 8 Fine resid/bnorm = 2.627960807e-11 MLMG: Iteration 9 Fine resid/bnorm = 1.548479103e-12 MLMG: Iteration 9 Crse resid/bnorm = 5.991211241e-14 MLMG: Final Iter. 9 resid, resid/bnorm = 4.695588199e-12, 1.548479103e-12 MLMG: Timers: Solve = 0.056888672 Iter = 0.056020749 Bottom = 0.040964098 >> After projection * On level 0 max(abs(diveu)) = 5.024387395e-11 * On level 1 max(abs(diveu)) = 5.261892129e-11 On level 0 fillpatch none On level 1 fillpatch none On level 0: set velocity bcs On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Before projection: max(abs(u/v/w)) = 1.428759962 1.428759962 3.470084772e-11 max(abs(gpx/gpy/gpz)) = 14.23448798 14.23448798 4.948447655e-11 Min and Max of epf 1 1 On level 0: set velocity bcs Before projection: max(abs(u/v/w)) = 1.454205373 1.454205373 4.766761812e-11 max(abs(gpx/gpy/gpz)) = 22.61989507 22.61989507 1.671056619e-10 Min and Max of epf 1 1 On level 1: set velocity bcs On level 0: set vec bcs On level 1: set vec bcs Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 1.736115617 * On lev 1 max(abs(rhs)) = 4.278230169 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 4.278230169 MLMG: Initial residual (resid0) = 4.278230169 MLMG: Iteration 1 Fine resid/bnorm = 0.02504966824 MLMG: Iteration 2 Fine resid/bnorm = 0.0002770456339 MLMG: Iteration 3 Fine resid/bnorm = 4.84776799e-06 MLMG: Iteration 4 Fine resid/bnorm = 5.120429685e-08 MLMG: Iteration 5 Fine resid/bnorm = 2.938109599e-09 MLMG: Iteration 6 Fine resid/bnorm = 2.611186672e-10 MLMG: Iteration 7 Fine resid/bnorm = 2.460729946e-11 MLMG: Iteration 8 Fine resid/bnorm = 2.379096743e-12 MLMG: Iteration 8 Crse resid/bnorm = 5.145452343e-10 MLMG: Iteration 9 Fine resid/bnorm = 2.619680781e-13 MLMG: Iteration 9 Crse resid/bnorm = 7.27970969e-11 MLMG: Iteration 10 Fine resid/bnorm = 3.162314287e-14 MLMG: Iteration 10 Crse resid/bnorm = 1.039798538e-11 MLMG: Iteration 11 Fine resid/bnorm = 6.020520905e-15 MLMG: Iteration 11 Crse resid/bnorm = 1.495756412e-12 MLMG: Final Iter. 11 resid, resid/bnorm = 6.399190206e-12, 1.495756412e-12 MLMG: Timers: Solve = 0.050179838 Iter = 0.046869241 Bottom = 0.026804435 On level 0: set velocity bcs After projection: max(abs(u/v/w)) = 1.424410448 1.424410448 3.457107821e-11 On level 1: set velocity bcs After projection: max(abs(u/v/w)) = 1.449945385 1.449945385 4.758761354e-11 Time per fluid step 0.163646871 Time per step 0.164506391 On level 0 fillpatch vel On level 1 fillpatch vel Writing plotfile BENCH09-refinement_plt00001 at time 0.004063119654 On level 0 fillpatch vel On level 1 fillpatch vel Time spent in main (after init) 0.178139146 Time spent in main 0.769492677 Total GPU global memory (MB) spread across MPI: [7784 ... 7784] Free GPU global memory (MB) spread across MPI: [6338 ... 6338] [The Arena] max space (MB) allocated spread across MPI: [193 ... 193] [The Arena] max space (MB) used spread across MPI: [154 ... 154] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Pinned Arena] max space (MB) used spread across MPI: [2 ... 2] AMReX (26.11-26-ga8db71f5aeb1) finalized