Initializing AMReX (26.07-41-g87f51bf4d85a)...
MPI initialized with 1 MPI processes
MPI initialized with thread support level 0
Initializing CUDA...
CUDA initialized with 1 device.
AMReX (26.07-41-g87f51bf4d85a) initialized
   MFIX git describe: 26.07-5-g8f9f7d1af1ef
AMReX-Hydro git hash: 9446740c59feca3bf007a6e6d95a6080816c33ec
Number of levels: 2
RunType is Standard
Auto-generating level-set parameters:
  eb_refinement = 1
  levelset_pad  = 2
  eb_pad        = 2

================================================================================
Warning: Overwriting amr.check_file with value from mfix.check_file
================================================================================

Simulation is transient
Fluid constraint not specified. Setting default:
  mfix.constraint = IncompressibleFluid

--------------------------------------------------------------------------------
Info: Max effective viscosity factor not specified.
Using default 1e+06
--------------------------------------------------------------------------------


Load balance setup:
   Type:      SingleGrid
   Strategy:  KnapSack
   Weighting: CellCount


 Building cylinder geometry.
 
 Internal Flow: 0
 Radius:    0.2
 Height:    -1
 Direction: 2
 Rotation angle: 0
 Rotation axis: 0
 Center:    0.5, 0.5, 0.5
Done making the fluid eb levels ...
PostProcessBaseGrids
Making new level 0 from scratch

On level 0 updated BC areas:
ErrorErr on level 0
Making new level 1 from scratch

On level 1 updated BC areas:


****************************************
  Coverage report:  Grids        Cells
          regular:      0         6944
          covered:      0          832
              cut:      1          416
****************************************

On level 0 fillpatch epf
On level 0 fillpatch rho
On level 0 fillpatch vel
On level 1 fillpatch epf
On level 1 fillpatch rho
On level 1 fillpatch vel
On level 0 fillpatch vel
On level 0 fillpatch rho
On level 1 fillpatch vel
On level 1 fillpatch rho
Initial projection:
Before projection:
   max(abs(u/v/w))  = 0.70710678  0.70710678  0
   max(abs(gpx/gpy/gpz))  = 0  0  0
Min and Max of epf 1 1
On level 0: set velocity bcs
Before projection:
   max(abs(u/v/w))  = 0.70710678  0.70710678  0
   max(abs(gpx/gpy/gpz))  = 0  0  0
Min and Max of epf 1 1
On level 1: set velocity bcs
On level 0: set vec bcs
On level 1: set vec bcs
Nodal Projection:
 >> Before projection:
  * On lev 0 max(abs(rhs)) = 29.77431549
  * On lev 1 max(abs(rhs)) = 59.74347518

MLMG: # of AMR levels: 2
      # of MG levels on the coarsest AMR level: 2
MLMG: Initial rhs               = 59.74347518
MLMG: Initial residual (resid0) = 59.74347518
MLMG: Iteration   1 Fine resid/bnorm = 0.03957694848
MLMG: Iteration   2 Fine resid/bnorm = 0.0004626124973
MLMG: Iteration   3 Fine resid/bnorm = 8.237525502e-06
MLMG: Iteration   4 Fine resid/bnorm = 1.349267965e-07
MLMG: Iteration   5 Fine resid/bnorm = 9.501683126e-09
MLMG: Iteration   6 Fine resid/bnorm = 7.422875905e-10
MLMG: Iteration   7 Fine resid/bnorm = 6.336670193e-11
MLMG: Iteration   8 Fine resid/bnorm = 6.216690702e-12
MLMG: Iteration   8 Crse resid/bnorm = 1.262786187e-09
MLMG: Iteration   9 Fine resid/bnorm = 6.860377548e-13
MLMG: Iteration   9 Crse resid/bnorm = 1.781843235e-10
MLMG: Iteration  10 Fine resid/bnorm = 7.794694908e-14
MLMG: Iteration  10 Crse resid/bnorm = 2.540376411e-11
MLMG: Iteration  11 Fine resid/bnorm = 1.620452233e-14
MLMG: Iteration  11 Crse resid/bnorm = 3.641251373e-12
MLMG: Final Iter. 11 resid, resid/bnorm = 2.17541011e-10, 3.641251373e-12
MLMG: Timers: Solve = 0.055493525 Iter = 0.048696251 Bottom = 0.026226658
On level 0: set velocity bcs
After  projection:
   max(abs(u/v/w))  = 1.47855403  1.47855403  8.165594936e-13
On level 1: set velocity bcs
After  projection:
   max(abs(u/v/w))  = 1.525952535  1.525952535  2.167571678e-12
Doing initial pressure iterations with dt = 0.004607777661
On level 0: set velocity bcs
On level 0: set density bcs
On level 1: set velocity bcs
On level 1: set density bcs
On level 0 fillpatch vel
On level 0 fillpatch rho
On level 1 fillpatch vel
On level 1 fillpatch rho

In initial_iterations: iter = 0
On level 0: set density bcs
On level 1: set density bcs
MAC Projection:
On level 0 fillpatch none
On level 1 fillpatch none
 >> Before projection
  * On level 0 max(abs(diveu)) = 1569.025929
  * On level 1 max(abs(diveu)) = 862.6467999
MLMG: # of AMR levels: 2
      # of MG levels on the coarsest AMR level: 2
MLMG: Initial rhs               = 4.691225192
MLMG: Initial residual (resid0) = 4.691225192
MLMG: Iteration   1 Fine resid/bnorm = 0.06790188225
MLMG: Iteration   2 Fine resid/bnorm = 0.002694811672
MLMG: Iteration   3 Fine resid/bnorm = 0.0001251718248
MLMG: Iteration   4 Fine resid/bnorm = 6.233876314e-06
MLMG: Iteration   5 Fine resid/bnorm = 2.976583319e-07
MLMG: Iteration   6 Fine resid/bnorm = 1.444614898e-08
MLMG: Iteration   7 Fine resid/bnorm = 6.670580201e-10
MLMG: Iteration   8 Fine resid/bnorm = 2.955124715e-11
MLMG: Iteration   9 Fine resid/bnorm = 1.360132457e-12
MLMG: Iteration   9 Crse resid/bnorm = 1.776580706e-13
MLMG: Final Iter. 9 resid, resid/bnorm = 6.380687645e-12, 1.360132457e-12
MLMG: Timers: Solve = 0.068460571 Iter = 0.066547846 Bottom = 0.046332471
 >> After projection
  * On level 0 max(abs(diveu)) = 1.220125909e-10
  * On level 1 max(abs(diveu)) = 1.692562235e-10
On level 0 fillpatch none
On level 1 fillpatch none
On level 0: set velocity bcs
On level 1: set velocity bcs
On level 0: set velocity bcs
On level 1: set velocity bcs
Before projection:
   max(abs(u/v/w))  = 1.427356918  1.427356918  2.508860171e-11
   max(abs(gpx/gpy/gpz))  = 0  0  0
Min and Max of epf 1 1
On level 0: set velocity bcs
Before projection:
   max(abs(u/v/w))  = 1.444743527  1.444743527  2.808977583e-11
   max(abs(gpx/gpy/gpz))  = 0  0  0
Min and Max of epf 1 1
On level 1: set velocity bcs
On level 0: set velocity bcs
On level 1: set velocity bcs
Nodal Projection:
 >> Before projection:
  * On lev 0 max(abs(rhs)) = 1.896607469
  * On lev 1 max(abs(rhs)) = 4.429769399

MLMG: # of AMR levels: 2
      # of MG levels on the coarsest AMR level: 2
MLMG: Initial rhs               = 4.429769399
MLMG: Initial residual (resid0) = 4.429769399
MLMG: Iteration   1 Fine resid/bnorm = 0.0265321453
MLMG: Iteration   2 Fine resid/bnorm = 0.0003417819947
MLMG: Iteration   3 Fine resid/bnorm = 6.118301295e-06
MLMG: Iteration   4 Fine resid/bnorm = 6.667032561e-08
MLMG: Iteration   5 Fine resid/bnorm = 3.680694125e-09
MLMG: Iteration   6 Fine resid/bnorm = 3.020847543e-10
MLMG: Iteration   7 Fine resid/bnorm = 2.933198391e-11
MLMG: Iteration   8 Fine resid/bnorm = 2.930295495e-12
MLMG: Iteration   8 Crse resid/bnorm = 5.435258323e-10
MLMG: Iteration   9 Fine resid/bnorm = 3.009788344e-13
MLMG: Iteration   9 Crse resid/bnorm = 7.53244458e-11
MLMG: Iteration  10 Fine resid/bnorm = 3.37940158e-14
MLMG: Iteration  10 Crse resid/bnorm = 1.057537155e-11
MLMG: Iteration  11 Fine resid/bnorm = 8.070212729e-15
MLMG: Iteration  11 Crse resid/bnorm = 1.499677982e-12
MLMG: Final Iter. 11 resid, resid/bnorm = 6.643227635e-12, 1.499677982e-12
MLMG: Timers: Solve = 0.052496613 Iter = 0.048746531 Bottom = 0.026938098
On level 0: set velocity bcs
After  projection:
   max(abs(u/v/w))  = 1.416036634  1.416036634  2.499378417e-11
On level 1: set velocity bcs
After  projection:
   max(abs(u/v/w))  = 1.441392518  1.441392518  2.798716895e-11
On level 0: set velocity bcs
On level 0: set density bcs
On level 1: set velocity bcs
On level 1: set density bcs

In initial_iterations: iter = 1
On level 0: set density bcs
On level 1: set density bcs
MAC Projection:
On level 0 fillpatch none
On level 1 fillpatch none
 >> Before projection
  * On level 0 max(abs(diveu)) = 1128.211073
  * On level 1 max(abs(diveu)) = 804.1010569
MLMG: # of AMR levels: 2
      # of MG levels on the coarsest AMR level: 2
MLMG: Initial rhs               = 2.815118191
MLMG: Initial residual (resid0) = 2.815118191
MLMG: Iteration   1 Fine resid/bnorm = 0.02539536307
MLMG: Iteration   2 Fine resid/bnorm = 0.00152063789
MLMG: Iteration   3 Fine resid/bnorm = 6.656386134e-05
MLMG: Iteration   4 Fine resid/bnorm = 3.185089222e-06
MLMG: Iteration   5 Fine resid/bnorm = 1.548329181e-07
MLMG: Iteration   6 Fine resid/bnorm = 7.537408633e-09
MLMG: Iteration   7 Fine resid/bnorm = 3.620782656e-10
MLMG: Iteration   8 Fine resid/bnorm = 1.846531295e-11
MLMG: Iteration   9 Fine resid/bnorm = 9.223780147e-13
MLMG: Iteration   9 Crse resid/bnorm = 7.713556983e-14
MLMG: Final Iter. 9 resid, resid/bnorm = 2.596603128e-12, 9.223780147e-13
MLMG: Timers: Solve = 0.064118661 Iter = 0.063236399 Bottom = 0.045895188
 >> After projection
  * On level 0 max(abs(diveu)) = 2.869093621e-11
  * On level 1 max(abs(diveu)) = 3.401215743e-11
On level 0 fillpatch none
On level 1 fillpatch none
On level 0: set velocity bcs
On level 1: set velocity bcs
On level 0: set velocity bcs
On level 1: set velocity bcs
Before projection:
   max(abs(u/v/w))  = 1.415907433  1.415907433  2.672077064e-11
   max(abs(gpx/gpy/gpz))  = 14.04293815  14.04293815  2.544252099e-11
Min and Max of epf 1 1
On level 0: set velocity bcs
Before projection:
   max(abs(u/v/w))  = 1.443617484  1.443617484  3.570785997e-11
   max(abs(gpx/gpy/gpz))  = 21.464453  21.464453  5.962722028e-11
Min and Max of epf 1 1
On level 1: set velocity bcs
On level 0: set velocity bcs
On level 1: set velocity bcs
Nodal Projection:
 >> Before projection:
  * On lev 0 max(abs(rhs)) = 0.1644546147
  * On lev 1 max(abs(rhs)) = 0.7043483107

MLMG: # of AMR levels: 2
      # of MG levels on the coarsest AMR level: 2
MLMG: Initial rhs               = 0.7043483107
MLMG: Initial residual (resid0) = 0.7043483107
MLMG: Iteration   1 Fine resid/bnorm = 0.004639092835
MLMG: Iteration   2 Fine resid/bnorm = 0.0001025124452
MLMG: Iteration   3 Fine resid/bnorm = 2.04956731e-06
MLMG: Iteration   4 Fine resid/bnorm = 3.658830119e-08
MLMG: Iteration   5 Fine resid/bnorm = 2.6695891e-09
MLMG: Iteration   6 Fine resid/bnorm = 2.399098249e-10
MLMG: Iteration   7 Fine resid/bnorm = 2.163593389e-11
MLMG: Iteration   8 Fine resid/bnorm = 1.997088105e-12
MLMG: Iteration   8 Crse resid/bnorm = 1.028788778e-10
MLMG: Iteration   9 Fine resid/bnorm = 2.057586224e-13
MLMG: Iteration   9 Crse resid/bnorm = 1.350576668e-11
MLMG: Iteration  10 Fine resid/bnorm = 2.136792358e-14
MLMG: Iteration  10 Crse resid/bnorm = 1.838587327e-12
MLMG: Final Iter. 10 resid, resid/bnorm = 1.295005878e-12, 1.838587327e-12
MLMG: Timers: Solve = 0.046152857 Iter = 0.042898006 Bottom = 0.025768933
On level 0: set velocity bcs
After  projection:
   max(abs(u/v/w))  = 1.413408114  1.413408114  2.665100855e-11
On level 1: set velocity bcs
After  projection:
   max(abs(u/v/w))  = 1.441345994  1.441345994  3.565282681e-11
On level 0: set velocity bcs
On level 0: set density bcs
On level 1: set velocity bcs
On level 1: set density bcs

In initial_iterations: iter = 2
On level 0: set density bcs
On level 1: set density bcs
MAC Projection:
On level 0 fillpatch none
On level 1 fillpatch none
 >> Before projection
  * On level 0 max(abs(diveu)) = 1099.736024
  * On level 1 max(abs(diveu)) = 804.1010569
MLMG: # of AMR levels: 2
      # of MG levels on the coarsest AMR level: 2
MLMG: Initial rhs               = 2.899343362
MLMG: Initial residual (resid0) = 2.899343362
MLMG: Iteration   1 Fine resid/bnorm = 0.02429658865
MLMG: Iteration   2 Fine resid/bnorm = 0.001453731382
MLMG: Iteration   3 Fine resid/bnorm = 6.269950589e-05
MLMG: Iteration   4 Fine resid/bnorm = 2.999409607e-06
MLMG: Iteration   5 Fine resid/bnorm = 1.47659769e-07
MLMG: Iteration   6 Fine resid/bnorm = 7.309342523e-09
MLMG: Iteration   7 Fine resid/bnorm = 3.575321278e-10
MLMG: Iteration   8 Fine resid/bnorm = 1.751048916e-11
MLMG: Iteration   9 Fine resid/bnorm = 8.534320036e-13
MLMG: Iteration   9 Crse resid/bnorm = 6.890386365e-14
MLMG: Final Iter. 9 resid, resid/bnorm = 2.474392414e-12, 8.534320036e-13
MLMG: Timers: Solve = 0.064593326 Iter = 0.063722985 Bottom = 0.046374556
 >> After projection
  * On level 0 max(abs(diveu)) = 3.040342425e-11
  * On level 1 max(abs(diveu)) = 3.195180202e-11
On level 0 fillpatch none
On level 1 fillpatch none
On level 0: set velocity bcs
On level 1: set velocity bcs
On level 0: set velocity bcs
On level 1: set velocity bcs
Before projection:
   max(abs(u/v/w))  = 1.412791142  1.412791142  2.518587659e-11
   max(abs(gpx/gpy/gpz))  = 13.48981927  13.48981927  3.617381391e-11
Min and Max of epf 1 1
On level 0: set velocity bcs
Before projection:
   max(abs(u/v/w))  = 1.44212426  1.44212426  2.670774065e-11
   max(abs(gpx/gpy/gpz))  = 21.77959645  21.77959645  1.011938326e-10
Min and Max of epf 1 1
On level 1: set velocity bcs
On level 0: set velocity bcs
On level 1: set velocity bcs
Nodal Projection:
 >> Before projection:
  * On lev 0 max(abs(rhs)) = 0.125527158
  * On lev 1 max(abs(rhs)) = 0.5030442912

MLMG: # of AMR levels: 2
      # of MG levels on the coarsest AMR level: 2
MLMG: Initial rhs               = 0.5030442912
MLMG: Initial residual (resid0) = 0.5030442912
MLMG: Iteration   1 Fine resid/bnorm = 0.005443865024
MLMG: Iteration   2 Fine resid/bnorm = 9.668175276e-05
MLMG: Iteration   3 Fine resid/bnorm = 1.779596053e-06
MLMG: Iteration   4 Fine resid/bnorm = 3.192166173e-08
MLMG: Iteration   5 Fine resid/bnorm = 1.659841381e-09
MLMG: Iteration   6 Fine resid/bnorm = 1.409726229e-10
MLMG: Iteration   7 Fine resid/bnorm = 1.263488915e-11
MLMG: Iteration   8 Fine resid/bnorm = 1.236117871e-12
MLMG: Iteration   8 Crse resid/bnorm = 3.543358316e-11
MLMG: Iteration   9 Fine resid/bnorm = 1.201578188e-13
MLMG: Iteration   9 Crse resid/bnorm = 4.069586699e-12
MLMG: Final Iter. 9 resid, resid/bnorm = 2.047182356e-12, 4.069586699e-12
MLMG: Timers: Solve = 0.040377657 Iter = 0.037108451 Bottom = 0.022053112
On level 0: set velocity bcs
After  projection:
   max(abs(u/v/w))  = 1.412335351  1.412335351  2.516162186e-11
On level 1: set velocity bcs
After  projection:
   max(abs(u/v/w))  = 1.4408626  1.4408626  2.66725071e-11
On level 0: set velocity bcs
On level 0: set density bcs
On level 1: set velocity bcs
On level 1: set density bcs
Time spent in init      0.606287153

On level 0 fillpatch vel
On level 1 fillpatch vel
  Writing plotfile BENCH09-refinement_plt00000 at time 0

 ============   NEW TIME STEP   ============ 
On level 0: set density bcs
On level 1: set density bcs

   Step 1: from old_time 0 to new time 0.004079710974 with dt = 0.004079710974

On level 0 fillpatch vel
On level 0 fillpatch rho
On level 1 fillpatch vel
On level 1 fillpatch rho
 Finished fillpatch before predictor
On level 0: set density bcs
On level 1: set density bcs
MAC Projection:
On level 0 fillpatch none
On level 1 fillpatch none
 >> Before projection
  * On level 0 max(abs(diveu)) = 1087.347097
  * On level 1 max(abs(diveu)) = 794.6877816
MLMG: # of AMR levels: 2
      # of MG levels on the coarsest AMR level: 2
MLMG: Initial rhs               = 3.19762189
MLMG: Initial residual (resid0) = 3.19762189
MLMG: Iteration   1 Fine resid/bnorm = 0.02177450594
MLMG: Iteration   2 Fine resid/bnorm = 0.001281788831
MLMG: Iteration   3 Fine resid/bnorm = 5.553196855e-05
MLMG: Iteration   4 Fine resid/bnorm = 2.673791831e-06
MLMG: Iteration   5 Fine resid/bnorm = 1.319573116e-07
MLMG: Iteration   6 Fine resid/bnorm = 6.573585508e-09
MLMG: Iteration   7 Fine resid/bnorm = 3.241329181e-10
MLMG: Iteration   8 Fine resid/bnorm = 1.553629296e-11
MLMG: Iteration   9 Fine resid/bnorm = 7.581936396e-13
MLMG: Iteration   9 Crse resid/bnorm = 6.543794804e-14
MLMG: Final Iter. 9 resid, resid/bnorm = 2.424416579e-12, 7.581936396e-13
MLMG: Timers: Solve = 0.065214803 Iter = 0.064328373 Bottom = 0.04696374
 >> After projection
  * On level 0 max(abs(diveu)) = 3.036887285e-11
  * On level 1 max(abs(diveu)) = 4.359820339e-11
On level 0 fillpatch none
On level 1 fillpatch none
On level 0: set velocity bcs
On level 1: set velocity bcs
On level 0: set velocity bcs
On level 1: set velocity bcs
Before projection:
   max(abs(u/v/w))  = 1.432518559  1.432518559  1.433831242e-11
   max(abs(gpx/gpy/gpz))  = 14.96246885  14.96246886  4.350564223e-11
Min and Max of epf 1 1
On level 0: set velocity bcs
Before projection:
   max(abs(u/v/w))  = 1.44886438  1.44886438  1.682397934e-11
   max(abs(gpx/gpy/gpz))  = 21.78630181  21.78630181  1.322468941e-10
Min and Max of epf 1 1
On level 1: set velocity bcs
On level 0: set vec bcs
On level 1: set vec bcs
Nodal Projection:
 >> Before projection:
  * On lev 0 max(abs(rhs)) = 1.667871118
  * On lev 1 max(abs(rhs)) = 4.111090796

MLMG: # of AMR levels: 2
      # of MG levels on the coarsest AMR level: 2
MLMG: Initial rhs               = 4.111090796
MLMG: Initial residual (resid0) = 4.111090796
MLMG: Iteration   1 Fine resid/bnorm = 0.02596568265
MLMG: Iteration   2 Fine resid/bnorm = 0.0003178364795
MLMG: Iteration   3 Fine resid/bnorm = 5.62944748e-06
MLMG: Iteration   4 Fine resid/bnorm = 7.937601591e-08
MLMG: Iteration   5 Fine resid/bnorm = 5.536576868e-09
MLMG: Iteration   6 Fine resid/bnorm = 4.560212564e-10
MLMG: Iteration   7 Fine resid/bnorm = 4.121081964e-11
MLMG: Iteration   8 Fine resid/bnorm = 4.053399284e-12
MLMG: Iteration   8 Crse resid/bnorm = 5.190821163e-10
MLMG: Iteration   9 Fine resid/bnorm = 4.080337328e-13
MLMG: Iteration   9 Crse resid/bnorm = 7.181188624e-11
MLMG: Iteration  10 Fine resid/bnorm = 4.178097449e-14
MLMG: Iteration  10 Crse resid/bnorm = 1.007469529e-11
MLMG: Iteration  11 Fine resid/bnorm = 7.710086915e-15
MLMG: Iteration  11 Crse resid/bnorm = 1.428433692e-12
MLMG: Final Iter. 11 resid, resid/bnorm = 5.872420605e-12, 1.428433692e-12
MLMG: Timers: Solve = 0.047874407 Iter = 0.044556838 Bottom = 0.025446591
On level 0: set velocity bcs
After  projection:
   max(abs(u/v/w))  = 1.425910545  1.425910545  1.425591606e-11
On level 1: set velocity bcs
After  projection:
   max(abs(u/v/w))  = 1.44797821  1.44797821  1.672128541e-11
   Time per fluid step      0.166590336
   Time per step        0.167494962
On level 0 fillpatch vel
On level 1 fillpatch vel
  Writing plotfile BENCH09-refinement_plt00001 at time 0.004079710974
On level 0 fillpatch vel
On level 1 fillpatch vel
Time spent in main (after init) 0.182155259
Time spent in main      0.788442412

Total GPU global memory (MB) spread across MPI: [7965 ... 7965]
Free  GPU global memory (MB) spread across MPI: [6579 ... 6579]
[The         Arena] max space (MB) allocated spread across MPI: [137 ... 137]
[The         Arena] max space (MB) used      spread across MPI: [126 ... 126]
[The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8]
[The Managed Arena] max space (MB) used      spread across MPI: [0 ... 0]
[The  Pinned Arena] max space (MB) allocated spread across MPI: [8 ... 8]
[The  Pinned Arena] max space (MB) used      spread across MPI: [2 ... 2]
AMReX (26.07-41-g87f51bf4d85a) finalized