Initializing AMReX (26.11-20-g24aedb0e61d7)... MPI initialized with 1 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.11-20-g24aedb0e61d7) initialized MFIX git describe: 26.11-21-ge4048e5d7789 AMReX-Hydro git hash: 154572aa6cf22c35a69729c74a0276909b96d5e3 Number of levels: 2 RunType is Standard Auto-generating level-set parameters: eb_refinement = 1 levelset_pad = 2 eb_pad = 2 ================================================================================ Warning: Overwriting amr.check_file with value from mfix.check_file ================================================================================ Simulation is transient Fluid constraint not specified. Setting default: mfix.constraint = IncompressibleFluid -------------------------------------------------------------------------------- Info: Max effective viscosity factor not specified. Using default 1e+06 -------------------------------------------------------------------------------- Load balance setup: Type: SingleGrid Strategy: KnapSack Weighting: CellCount Building cylinder geometry. Internal Flow: 0 Radius: 0.2 Height: -1 Direction: 2 Rotation angle: 0 Rotation axis: 0 Center: 0.5, 0.5, 0.5 Done making the fluid eb levels ... PostProcessBaseGrids Making new level 0 from scratch On level 0 updated BC areas: ErrorErr on level 0 Making new level 1 from scratch On level 1 updated BC areas: **************************************** Coverage report: Grids Cells regular: 0 6944 covered: 0 832 cut: 1 416 **************************************** On level 0 fillpatch epf On level 0 fillpatch rho On level 0 fillpatch vel On level 1 fillpatch epf On level 1 fillpatch rho On level 1 fillpatch vel On level 0 fillpatch vel On level 0 fillpatch rho On level 1 fillpatch vel On level 1 fillpatch rho Initial projection: Before projection: max(abs(u/v/w)) = 0.70710678 0.70710678 0 max(abs(gpx/gpy/gpz)) = 0 0 0 Min and Max of epf 1 1 On level 0: set velocity bcs Before projection: max(abs(u/v/w)) = 0.70710678 0.70710678 0 max(abs(gpx/gpy/gpz)) = 0 0 0 Min and Max of epf 1 1 On level 1: set velocity bcs On level 0: set vec bcs On level 1: set vec bcs Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 29.77431549 * On lev 1 max(abs(rhs)) = 59.74347518 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 59.74347518 MLMG: Initial residual (resid0) = 59.74347518 MLMG: Iteration 1 Fine resid/bnorm = 0.03956425674 MLMG: Iteration 2 Fine resid/bnorm = 0.000421647964 MLMG: Iteration 3 Fine resid/bnorm = 7.203495227e-06 MLMG: Iteration 4 Fine resid/bnorm = 7.434409574e-08 MLMG: Iteration 5 Fine resid/bnorm = 4.76970816e-09 MLMG: Iteration 6 Fine resid/bnorm = 3.801865907e-10 MLMG: Iteration 7 Fine resid/bnorm = 3.374727358e-11 MLMG: Iteration 8 Fine resid/bnorm = 3.817709148e-12 MLMG: Iteration 8 Crse resid/bnorm = 7.918059158e-10 MLMG: Iteration 9 Fine resid/bnorm = 4.46573843e-13 MLMG: Iteration 9 Crse resid/bnorm = 1.120774937e-10 MLMG: Iteration 10 Fine resid/bnorm = 5.415292334e-14 MLMG: Iteration 10 Crse resid/bnorm = 1.601390396e-11 MLMG: Iteration 11 Fine resid/bnorm = 1.704262319e-14 MLMG: Iteration 11 Crse resid/bnorm = 2.304105009e-12 MLMG: Final Iter. 11 resid, resid/bnorm = 1.376552404e-10, 2.304105009e-12 MLMG: Timers: Solve = 0.058013268 Iter = 0.050669496 Bottom = 0.027081505 On level 0: set velocity bcs After projection: max(abs(u/v/w)) = 1.47877986 1.47877986 4.84482246e-13 On level 1: set velocity bcs After projection: max(abs(u/v/w)) = 1.526266743 1.526266743 1.345788064e-12 Doing initial pressure iterations with dt = 0.004606829071 On level 0: set velocity bcs On level 0: set density bcs On level 1: set velocity bcs On level 1: set density bcs On level 0 fillpatch vel On level 0 fillpatch rho On level 1 fillpatch vel On level 1 fillpatch rho In initial_iterations: iter = 0 On level 0: set density bcs On level 1: set density bcs MAC Projection: On level 0 fillpatch none On level 1 fillpatch none >> Before projection * On level 0 max(abs(diveu)) = 1703.813941 * On level 1 max(abs(diveu)) = 1154.780884 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 3.578916976 MLMG: Initial residual (resid0) = 3.578916976 MLMG: Iteration 1 Fine resid/bnorm = 0.08232711455 MLMG: Iteration 2 Fine resid/bnorm = 0.003378796349 MLMG: Iteration 3 Fine resid/bnorm = 0.0001787772676 MLMG: Iteration 4 Fine resid/bnorm = 9.997616466e-06 MLMG: Iteration 5 Fine resid/bnorm = 5.812225467e-07 MLMG: Iteration 6 Fine resid/bnorm = 3.460584082e-08 MLMG: Iteration 7 Fine resid/bnorm = 2.070928374e-09 MLMG: Iteration 8 Fine resid/bnorm = 1.244300357e-10 MLMG: Iteration 9 Fine resid/bnorm = 7.461207192e-12 MLMG: Iteration 9 Crse resid/bnorm = 3.580705724e-13 MLMG: Final Iter. 9 resid, resid/bnorm = 2.670304108e-11, 7.461207192e-12 MLMG: Timers: Solve = 0.062709356 Iter = 0.060853275 Bottom = 0.04358853 >> After projection * On level 0 max(abs(diveu)) = 2.503193091e-10 * On level 1 max(abs(diveu)) = 2.558698762e-10 On level 0 fillpatch none On level 1 fillpatch none On level 0: set velocity bcs On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Before projection: max(abs(u/v/w)) = 1.422205574 1.422205574 4.836254156e-11 max(abs(gpx/gpy/gpz)) = 0 0 0 Min and Max of epf 1 1 On level 0: set velocity bcs Before projection: max(abs(u/v/w)) = 1.449630343 1.449630343 8.111431037e-11 max(abs(gpx/gpy/gpz)) = 0 0 0 Min and Max of epf 1 1 On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 1.867782749 * On lev 1 max(abs(rhs)) = 4.449545874 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 4.449545874 MLMG: Initial residual (resid0) = 4.449545874 MLMG: Iteration 1 Fine resid/bnorm = 0.02638538928 MLMG: Iteration 2 Fine resid/bnorm = 0.0003114231446 MLMG: Iteration 3 Fine resid/bnorm = 5.508609669e-06 MLMG: Iteration 4 Fine resid/bnorm = 6.139801613e-08 MLMG: Iteration 5 Fine resid/bnorm = 1.893991882e-09 MLMG: Iteration 6 Fine resid/bnorm = 1.915771171e-10 MLMG: Iteration 7 Fine resid/bnorm = 2.109287924e-11 MLMG: Iteration 8 Fine resid/bnorm = 2.397625507e-12 MLMG: Iteration 8 Crse resid/bnorm = 5.56057448e-10 MLMG: Iteration 9 Fine resid/bnorm = 2.801922826e-13 MLMG: Iteration 9 Crse resid/bnorm = 7.871488902e-11 MLMG: Iteration 10 Fine resid/bnorm = 3.399157458e-14 MLMG: Iteration 10 Crse resid/bnorm = 1.124633553e-11 MLMG: Iteration 11 Fine resid/bnorm = 6.013282176e-15 MLMG: Iteration 11 Crse resid/bnorm = 1.617879339e-12 MLMG: Final Iter. 11 resid, resid/bnorm = 7.198828339e-12, 1.617879339e-12 MLMG: Timers: Solve = 0.050757788 Iter = 0.047399709 Bottom = 0.027260425 On level 0: set velocity bcs After projection: max(abs(u/v/w)) = 1.413131257 1.413131257 4.819612379e-11 On level 1: set velocity bcs After projection: max(abs(u/v/w)) = 1.444629562 1.444629562 8.098879189e-11 On level 0: set velocity bcs On level 0: set density bcs On level 1: set velocity bcs On level 1: set density bcs In initial_iterations: iter = 1 On level 0: set density bcs On level 1: set density bcs MAC Projection: On level 0 fillpatch none On level 1 fillpatch none >> Before projection * On level 0 max(abs(diveu)) = 1268.151996 * On level 1 max(abs(diveu)) = 804.0916384 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 2.70324018 MLMG: Initial residual (resid0) = 2.70324018 MLMG: Iteration 1 Fine resid/bnorm = 0.02317840911 MLMG: Iteration 2 Fine resid/bnorm = 0.001319924745 MLMG: Iteration 3 Fine resid/bnorm = 5.930510889e-05 MLMG: Iteration 4 Fine resid/bnorm = 2.952071737e-06 MLMG: Iteration 5 Fine resid/bnorm = 1.561197281e-07 MLMG: Iteration 6 Fine resid/bnorm = 8.546567887e-09 MLMG: Iteration 7 Fine resid/bnorm = 4.814077817e-10 MLMG: Iteration 8 Fine resid/bnorm = 2.754314664e-11 MLMG: Iteration 9 Fine resid/bnorm = 1.593680654e-12 MLMG: Iteration 9 Crse resid/bnorm = 7.096653903e-14 MLMG: Final Iter. 9 resid, resid/bnorm = 4.308101579e-12, 1.593680654e-12 MLMG: Timers: Solve = 0.057247291 Iter = 0.056374265 Bottom = 0.04135825 >> After projection * On level 0 max(abs(diveu)) = 4.994883689e-11 * On level 1 max(abs(diveu)) = 5.254677247e-11 On level 0 fillpatch none On level 1 fillpatch none On level 0: set velocity bcs On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Before projection: max(abs(u/v/w)) = 1.413491111 1.413491111 2.645396432e-11 max(abs(gpx/gpy/gpz)) = 14.29688751 14.29688751 3.922297327e-11 Min and Max of epf 1 1 On level 0: set velocity bcs Before projection: max(abs(u/v/w)) = 1.447074151 1.447074151 5.0511287e-11 max(abs(gpx/gpy/gpz)) = 23.00094001 23.00094001 1.164431094e-10 Min and Max of epf 1 1 On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 0.1509727113 * On lev 1 max(abs(rhs)) = 0.6927293804 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 0.6927293804 MLMG: Initial residual (resid0) = 0.6927293804 MLMG: Iteration 1 Fine resid/bnorm = 0.005130373494 MLMG: Iteration 2 Fine resid/bnorm = 0.0001111445902 MLMG: Iteration 3 Fine resid/bnorm = 2.159134709e-06 MLMG: Iteration 4 Fine resid/bnorm = 3.464097162e-08 MLMG: Iteration 5 Fine resid/bnorm = 1.216361523e-09 MLMG: Iteration 6 Fine resid/bnorm = 8.540308766e-11 MLMG: Iteration 7 Fine resid/bnorm = 6.595616225e-12 MLMG: Iteration 7 Crse resid/bnorm = 4.399390505e-10 MLMG: Iteration 8 Fine resid/bnorm = 6.162902836e-13 MLMG: Iteration 8 Crse resid/bnorm = 6.205956029e-11 MLMG: Iteration 9 Fine resid/bnorm = 5.781164665e-14 MLMG: Iteration 9 Crse resid/bnorm = 8.815508268e-12 MLMG: Final Iter. 9 resid, resid/bnorm = 6.106761581e-12, 8.815508268e-12 MLMG: Timers: Solve = 0.042667268 Iter = 0.039338089 Bottom = 0.02308992 On level 0: set velocity bcs After projection: max(abs(u/v/w)) = 1.411707841 1.411707841 2.638704641e-11 On level 1: set velocity bcs After projection: max(abs(u/v/w)) = 1.44368775 1.44368775 5.059689221e-11 On level 0: set velocity bcs On level 0: set density bcs On level 1: set velocity bcs On level 1: set density bcs In initial_iterations: iter = 2 On level 0: set density bcs On level 1: set density bcs MAC Projection: On level 0 fillpatch none On level 1 fillpatch none >> Before projection * On level 0 max(abs(diveu)) = 1241.202413 * On level 1 max(abs(diveu)) = 804.0916384 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 3.04626377 MLMG: Initial residual (resid0) = 3.04626377 MLMG: Iteration 1 Fine resid/bnorm = 0.02193772167 MLMG: Iteration 2 Fine resid/bnorm = 0.001125823562 MLMG: Iteration 3 Fine resid/bnorm = 4.92685131e-05 MLMG: Iteration 4 Fine resid/bnorm = 2.534602616e-06 MLMG: Iteration 5 Fine resid/bnorm = 1.381142645e-07 MLMG: Iteration 6 Fine resid/bnorm = 7.791232833e-09 MLMG: Iteration 7 Fine resid/bnorm = 4.496383908e-10 MLMG: Iteration 8 Fine resid/bnorm = 2.628847222e-11 MLMG: Iteration 9 Fine resid/bnorm = 1.551121175e-12 MLMG: Iteration 9 Crse resid/bnorm = 6.082794253e-14 MLMG: Final Iter. 9 resid, resid/bnorm = 4.725124239e-12, 1.551121175e-12 MLMG: Timers: Solve = 0.057253503 Iter = 0.056333402 Bottom = 0.04138282 >> After projection * On level 0 max(abs(diveu)) = 5.056862248e-11 * On level 1 max(abs(diveu)) = 6.079796715e-11 On level 0 fillpatch none On level 1 fillpatch none On level 0: set velocity bcs On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Before projection: max(abs(u/v/w)) = 1.41136956 1.41136956 4.604200354e-11 max(abs(gpx/gpy/gpz)) = 13.64992201 13.64992201 4.524950728e-11 Min and Max of epf 1 1 On level 0: set velocity bcs Before projection: max(abs(u/v/w)) = 1.44488668 1.44488668 7.836645211e-11 max(abs(gpx/gpy/gpz)) = 22.90394394 22.90394394 1.734569569e-10 Min and Max of epf 1 1 On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 0.11766771 * On lev 1 max(abs(rhs)) = 0.4624751242 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 0.4624751242 MLMG: Initial residual (resid0) = 0.4624751242 MLMG: Iteration 1 Fine resid/bnorm = 0.006958443278 MLMG: Iteration 2 Fine resid/bnorm = 0.0001285664033 MLMG: Iteration 3 Fine resid/bnorm = 2.304012747e-06 MLMG: Iteration 4 Fine resid/bnorm = 4.004948098e-08 MLMG: Iteration 5 Fine resid/bnorm = 9.734349475e-10 MLMG: Iteration 6 Fine resid/bnorm = 7.03568469e-11 MLMG: Iteration 7 Fine resid/bnorm = 5.599039588e-12 MLMG: Iteration 7 Crse resid/bnorm = 1.224053268e-10 MLMG: Iteration 8 Fine resid/bnorm = 4.99867295e-13 MLMG: Iteration 8 Crse resid/bnorm = 1.377181632e-11 MLMG: Iteration 9 Fine resid/bnorm = 4.600921659e-14 MLMG: Iteration 9 Crse resid/bnorm = 1.618164421e-12 MLMG: Final Iter. 9 resid, resid/bnorm = 7.483607917e-13, 1.618164421e-12 MLMG: Timers: Solve = 0.041462282 Iter = 0.038102283 Bottom = 0.021821264 On level 0: set velocity bcs After projection: max(abs(u/v/w)) = 1.410898546 1.410898546 4.591700448e-11 On level 1: set velocity bcs After projection: max(abs(u/v/w)) = 1.443660869 1.443660869 7.82679108e-11 On level 0: set velocity bcs On level 0: set density bcs On level 1: set velocity bcs On level 1: set density bcs Time spent in init 0.593579423 On level 0 fillpatch vel On level 1 fillpatch vel Writing plotfile BENCH09-refinement_plt00000 at time 0 ============ NEW TIME STEP ============ On level 0: set density bcs On level 1: set density bcs Step 1: from old_time 0 to new time 0.004063119654 with dt = 0.004063119654 On level 0 fillpatch vel On level 0 fillpatch rho On level 1 fillpatch vel On level 1 fillpatch rho Finished fillpatch before predictor On level 0: set density bcs On level 1: set density bcs MAC Projection: On level 0 fillpatch none On level 1 fillpatch none >> Before projection * On level 0 max(abs(diveu)) = 1212.555318 * On level 1 max(abs(diveu)) = 794.3420316 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 3.032387192 MLMG: Initial residual (resid0) = 3.032387192 MLMG: Iteration 1 Fine resid/bnorm = 0.021676483 MLMG: Iteration 2 Fine resid/bnorm = 0.001119811752 MLMG: Iteration 3 Fine resid/bnorm = 4.954047377e-05 MLMG: Iteration 4 Fine resid/bnorm = 2.547345071e-06 MLMG: Iteration 5 Fine resid/bnorm = 1.383928866e-07 MLMG: Iteration 6 Fine resid/bnorm = 7.807090168e-09 MLMG: Iteration 7 Fine resid/bnorm = 4.49850366e-10 MLMG: Iteration 8 Fine resid/bnorm = 2.63105226e-11 MLMG: Iteration 9 Fine resid/bnorm = 1.549184915e-12 MLMG: Iteration 9 Crse resid/bnorm = 5.981400321e-14 MLMG: Final Iter. 9 resid, resid/bnorm = 4.697728492e-12, 1.549184915e-12 MLMG: Timers: Solve = 0.05681834 Iter = 0.055910672 Bottom = 0.040979443 >> After projection * On level 0 max(abs(diveu)) = 4.987857723e-11 * On level 1 max(abs(diveu)) = 5.25108981e-11 On level 0 fillpatch none On level 1 fillpatch none On level 0: set velocity bcs On level 1: set velocity bcs On level 0: set velocity bcs On level 1: set velocity bcs Before projection: max(abs(u/v/w)) = 1.428759962 1.428759962 1.980404098e-11 max(abs(gpx/gpy/gpz)) = 14.23448798 14.23448799 5.829226847e-11 Min and Max of epf 1 1 On level 0: set velocity bcs Before projection: max(abs(u/v/w)) = 1.454205373 1.454205373 2.423719577e-11 max(abs(gpx/gpy/gpz)) = 22.61989507 22.61989507 2.120229705e-10 Min and Max of epf 1 1 On level 1: set velocity bcs On level 0: set vec bcs On level 1: set vec bcs Nodal Projection: >> Before projection: * On lev 0 max(abs(rhs)) = 1.736115617 * On lev 1 max(abs(rhs)) = 4.278230169 MLMG: # of AMR levels: 2 # of MG levels on the coarsest AMR level: 2 MLMG: Initial rhs = 4.278230169 MLMG: Initial residual (resid0) = 4.278230169 MLMG: Iteration 1 Fine resid/bnorm = 0.02504966824 MLMG: Iteration 2 Fine resid/bnorm = 0.0002770456339 MLMG: Iteration 3 Fine resid/bnorm = 4.847767988e-06 MLMG: Iteration 4 Fine resid/bnorm = 5.12042982e-08 MLMG: Iteration 5 Fine resid/bnorm = 2.938109599e-09 MLMG: Iteration 6 Fine resid/bnorm = 2.611187208e-10 MLMG: Iteration 7 Fine resid/bnorm = 2.460707888e-11 MLMG: Iteration 8 Fine resid/bnorm = 2.379226496e-12 MLMG: Iteration 8 Crse resid/bnorm = 5.145451864e-10 MLMG: Iteration 9 Fine resid/bnorm = 2.618504898e-13 MLMG: Iteration 9 Crse resid/bnorm = 7.279710095e-11 MLMG: Iteration 10 Fine resid/bnorm = 3.15209627e-14 MLMG: Iteration 10 Crse resid/bnorm = 1.039798294e-11 MLMG: Iteration 11 Fine resid/bnorm = 6.280026117e-15 MLMG: Iteration 11 Crse resid/bnorm = 1.495734516e-12 MLMG: Final Iter. 11 resid, resid/bnorm = 6.39909653e-12, 1.495734516e-12 MLMG: Timers: Solve = 0.05229472 Iter = 0.048933512 Bottom = 0.028777224 On level 0: set velocity bcs After projection: max(abs(u/v/w)) = 1.424410448 1.424410448 1.972063256e-11 On level 1: set velocity bcs After projection: max(abs(u/v/w)) = 1.449945385 1.449945385 2.439264242e-11 Time per fluid step 0.166054266 Time per step 0.166944453 On level 0 fillpatch vel On level 1 fillpatch vel Writing plotfile BENCH09-refinement_plt00001 at time 0.004063119654 On level 0 fillpatch vel On level 1 fillpatch vel Time spent in main (after init) 0.180719667 Time spent in main 0.77429909 Total GPU global memory (MB) spread across MPI: [7784 ... 7784] Free GPU global memory (MB) spread across MPI: [6338 ... 6338] [The Arena] max space (MB) allocated spread across MPI: [193 ... 193] [The Arena] max space (MB) used spread across MPI: [154 ... 154] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Pinned Arena] max space (MB) used spread across MPI: [2 ... 2] AMReX (26.11-20-g24aedb0e61d7) finalized