Initializing AMReX (26.04-83-g515ed5e0f061)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.04-83-g515ed5e0f061) initialized Successfully read inputs file ... ERF git hash: 44ef8a40 AMReX git hash: 26.04-83-g515ed5e0f0 Problem name (from inputs file) is ABL Thermal stratification based on gradient of potential temperature Null land surface model! SOLVER CHOICE: At level 0 : compressible with implicit substepping vert_implicit_fac : 0 0 0 use_coriolis : 0 use_gravity : 0 Moisture Model: None Terrain Type: None Buildings Type: None Mesh Type: ConstantDz ABL Driver Type: None Radiation Model: None Gradp_type : 0 Buoyancy_type at level 0 : 1 Advection Choices: dycore_horiz_adv_type : Upwind_3rd dycore_vert_adv_type : Upwind_3rd dryscal_horiz_adv_type : Upwind_3rd dryscal_vert_adv_type : Upwind_3rd moistscal_horiz_adv_type : WENO3 moistscal_vert_adv_type : WENO3 Diffusion choices: rho0_trans : 1 alpha_T : 0 alpha_C : 0 dynamic_viscosity : 0 Not using any molecular diffusivity, i.e. using the modeled turbulent diffusivity Sponge choices: Rayleigh damping : None w damping : 0 (coeff = -1 * dz/dt**2) Turbulence Settings at level 0 Using Smagorinsky LES model at level 0 Smagorinsky uses Richardson number correction with Ri_crit = 0.25 Cs : 0.1 isotropic mixing : 1 Pr_t : 0.3333333333 Sc_t : 1 Restart from native checkpoint chk00000_for_restart BA FROM SCRATCH AT LEVEL 0 (BoxArray maxbox(4) m_ref->m_hash_sig(0) ((0,0,0) (31,31,63) (0,0,0)) ((32,0,0) (63,31,63) (0,0,0)) ((0,32,0) (31,63,63) (0,0,0)) ((32,32,0) (63,63,63) (0,0,0)) ) Reading LMASK data Restart time = 0.105687531 seconds. If there is water, Charnock relation with C_a=0.0185 will be used Reading MOST variables Writing native 3D plotfile ABL-MOST_plt00000 3DPlotfile write time = 0.025973838 seconds. TIME= 0 MASS = 1073741824 RHO THETA = 3.221225472e+11 RHO SCALAR = 0 Coarse STEP 1 starts ... Based on cfl of one Compressible dt at level 0 would be: 0.0461055476 Anelastic dt at level 0 would be: 1.58742278 Fixed dt at level 0 is: 0.1 smallest even ratio is: 4 [Level 0 step 1] ADVANCE from elapsed time = 0 to 0.1 with dt = 0.1 Making slow rhs at time 0 for fast variables advancing from 0 to 0.03333333333333 Fast time integration at level 0 from 0 to 0.03333333333333 with dt = 0.03333333333333 Time integration of scalars at level 0 from 0 to 0.03333333333333 with dt = 0.03333333333333 using RHS created at 0 Making slow rhs at time 0.03333333333333 for fast variables advancing from 0 to 0.05 Fast time integration at level 0 from 0 to 0.025 with dt = 0.025 Fast time integration at level 0 from 0.025 to 0.05 with dt = 0.025 Time integration of scalars at level 0 from 0 to 0.05 with dt = 0.05 using RHS created at 0.03333333333333 Making slow rhs at time 0.05 for fast variables advancing from 0 to 0.1 Fast time integration at level 0 from 0 to 0.025 with dt = 0.025 Fast time integration at level 0 from 0.025 to 0.05 with dt = 0.025 Fast time integration at level 0 from 0.05 to 0.075 with dt = 0.025 Fast time integration at level 0 from 0.075 to 0.1 with dt = 0.025 Time integration of scalars at level 0 from 0 to 0.1 with dt = 0.1 using RHS created at 0.05 Done with advance_dycore at level 0 [Level 0 step 1] Advanced 262144 cells Coarse STEP 1 ends. TIME = 0.1 DT = 0.1 Timestep time = 0.26332438 seconds. TIME= 0.1 MASS = 1073741824 RHO THETA = 3.221225472e+11 RHO SCALAR = 0 Coarse STEP 2 starts ... Based on cfl of one Compressible dt at level 0 would be: 0.0461021971 Anelastic dt at level 0 would be: 1.584988778 Fixed dt at level 0 is: 0.1 smallest even ratio is: 4 [Level 0 step 2] ADVANCE from elapsed time = 0.1 to 0.2 with dt = 0.1 Making slow rhs at time 0.1 for fast variables advancing from 0.1 to 0.1333333333333 Fast time integration at level 0 from 0.1 to 0.1333333333333 with dt = 0.03333333333333 Time integration of scalars at level 0 from 0.1 to 0.1333333333333 with dt = 0.03333333333333 using RHS created at 0.1 Making slow rhs at time 0.1333333333333 for fast variables advancing from 0.1 to 0.15 Fast time integration at level 0 from 0.1 to 0.125 with dt = 0.025 Fast time integration at level 0 from 0.125 to 0.15 with dt = 0.025 Time integration of scalars at level 0 from 0.1 to 0.15 with dt = 0.05 using RHS created at 0.1333333333333 Making slow rhs at time 0.15 for fast variables advancing from 0.1 to 0.2 Fast time integration at level 0 from 0.1 to 0.125 with dt = 0.025 Fast time integration at level 0 from 0.125 to 0.15 with dt = 0.025 Fast time integration at level 0 from 0.15 to 0.175 with dt = 0.025 Fast time integration at level 0 from 0.175 to 0.2 with dt = 0.025 Time integration of scalars at level 0 from 0.1 to 0.2 with dt = 0.1 using RHS created at 0.15 Done with advance_dycore at level 0 [Level 0 step 2] Advanced 262144 cells Coarse STEP 2 ends. TIME = 0.2 DT = 0.1 Timestep time = 0.255976814 seconds. TIME= 0.2 MASS = 1073741824 RHO THETA = 3.221225472e+11 RHO SCALAR = 0 Coarse STEP 3 starts ... Based on cfl of one Compressible dt at level 0 would be: 0.04610395897 Anelastic dt at level 0 would be: 1.586478575 Fixed dt at level 0 is: 0.1 smallest even ratio is: 4 [Level 0 step 3] ADVANCE from elapsed time = 0.2 to 0.3 with dt = 0.1 Making slow rhs at time 0.2 for fast variables advancing from 0.2 to 0.2333333333333 Fast time integration at level 0 from 0.2 to 0.2333333333333 with dt = 0.03333333333333 Time integration of scalars at level 0 from 0.2 to 0.2333333333333 with dt = 0.03333333333333 using RHS created at 0.2 Making slow rhs at time 0.2333333333333 for fast variables advancing from 0.2 to 0.25 Fast time integration at level 0 from 0.2 to 0.225 with dt = 0.025 Fast time integration at level 0 from 0.225 to 0.25 with dt = 0.025 Time integration of scalars at level 0 from 0.2 to 0.25 with dt = 0.05 using RHS created at 0.2333333333333 Making slow rhs at time 0.25 for fast variables advancing from 0.2 to 0.3 Fast time integration at level 0 from 0.2 to 0.225 with dt = 0.025 Fast time integration at level 0 from 0.225 to 0.25 with dt = 0.025 Fast time integration at level 0 from 0.25 to 0.275 with dt = 0.025 Fast time integration at level 0 from 0.275 to 0.3 with dt = 0.025 Time integration of scalars at level 0 from 0.2 to 0.3 with dt = 0.1 using RHS created at 0.25 Done with advance_dycore at level 0 [Level 0 step 3] Advanced 262144 cells Coarse STEP 3 ends. TIME = 0.3 DT = 0.1 Timestep time = 0.255795524 seconds. TIME= 0.3 MASS = 1073741824 RHO THETA = 3.221225472e+11 RHO SCALAR = 0 Coarse STEP 4 starts ... Based on cfl of one Compressible dt at level 0 would be: 0.04610371313 Anelastic dt at level 0 would be: 1.58610933 Fixed dt at level 0 is: 0.1 smallest even ratio is: 4 [Level 0 step 4] ADVANCE from elapsed time = 0.3 to 0.4 with dt = 0.1 Making slow rhs at time 0.3 for fast variables advancing from 0.3 to 0.3333333333333 Fast time integration at level 0 from 0.3 to 0.3333333333333 with dt = 0.03333333333333 Time integration of scalars at level 0 from 0.3 to 0.3333333333333 with dt = 0.03333333333333 using RHS created at 0.3 Making slow rhs at time 0.3333333333333 for fast variables advancing from 0.3 to 0.35 Fast time integration at level 0 from 0.3 to 0.325 with dt = 0.025 Fast time integration at level 0 from 0.325 to 0.35 with dt = 0.025 Time integration of scalars at level 0 from 0.3 to 0.35 with dt = 0.05 using RHS created at 0.3333333333333 Making slow rhs at time 0.35 for fast variables advancing from 0.3 to 0.4 Fast time integration at level 0 from 0.3 to 0.325 with dt = 0.025 Fast time integration at level 0 from 0.325 to 0.35 with dt = 0.025 Fast time integration at level 0 from 0.35 to 0.375 with dt = 0.025 Fast time integration at level 0 from 0.375 to 0.4 with dt = 0.025 Time integration of scalars at level 0 from 0.3 to 0.4 with dt = 0.1 using RHS created at 0.35 Done with advance_dycore at level 0 [Level 0 step 4] Advanced 262144 cells Coarse STEP 4 ends. TIME = 0.4 DT = 0.1 Timestep time = 0.255789766 seconds. TIME= 0.4 MASS = 1073741824 RHO THETA = 3.221225472e+11 RHO SCALAR = 0 Coarse STEP 5 starts ... Based on cfl of one Compressible dt at level 0 would be: 0.04610456755 Anelastic dt at level 0 would be: 1.586119519 Fixed dt at level 0 is: 0.1 smallest even ratio is: 4 [Level 0 step 5] ADVANCE from elapsed time = 0.4 to 0.5 with dt = 0.1 Making slow rhs at time 0.4 for fast variables advancing from 0.4 to 0.4333333333333 Fast time integration at level 0 from 0.4 to 0.4333333333333 with dt = 0.03333333333333 Time integration of scalars at level 0 from 0.4 to 0.4333333333333 with dt = 0.03333333333333 using RHS created at 0.4 Making slow rhs at time 0.4333333333333 for fast variables advancing from 0.4 to 0.45 Fast time integration at level 0 from 0.4 to 0.425 with dt = 0.025 Fast time integration at level 0 from 0.425 to 0.45 with dt = 0.025 Time integration of scalars at level 0 from 0.4 to 0.45 with dt = 0.05 using RHS created at 0.4333333333333 Making slow rhs at time 0.45 for fast variables advancing from 0.4 to 0.5 Fast time integration at level 0 from 0.4 to 0.425 with dt = 0.025 Fast time integration at level 0 from 0.425 to 0.45 with dt = 0.025 Fast time integration at level 0 from 0.45 to 0.475 with dt = 0.025 Fast time integration at level 0 from 0.475 to 0.5 with dt = 0.025 Time integration of scalars at level 0 from 0.4 to 0.5 with dt = 0.1 using RHS created at 0.45 Done with advance_dycore at level 0 [Level 0 step 5] Advanced 262144 cells Coarse STEP 5 ends. TIME = 0.5 DT = 0.1 Timestep time = 0.255858318 seconds. TIME= 0.5 MASS = 1073741824 RHO THETA = 3.221225472e+11 RHO SCALAR = 0 Coarse STEP 6 starts ... Based on cfl of one Compressible dt at level 0 would be: 0.04610382307 Anelastic dt at level 0 would be: 1.585920129 Fixed dt at level 0 is: 0.1 smallest even ratio is: 4 [Level 0 step 6] ADVANCE from elapsed time = 0.5 to 0.6 with dt = 0.1 Making slow rhs at time 0.5 for fast variables advancing from 0.5 to 0.5333333333333 Fast time integration at level 0 from 0.5 to 0.5333333333333 with dt = 0.03333333333333 Time integration of scalars at level 0 from 0.5 to 0.5333333333333 with dt = 0.03333333333333 using RHS created at 0.5 Making slow rhs at time 0.5333333333333 for fast variables advancing from 0.5 to 0.55 Fast time integration at level 0 from 0.5 to 0.525 with dt = 0.025 Fast time integration at level 0 from 0.525 to 0.55 with dt = 0.025 Time integration of scalars at level 0 from 0.5 to 0.55 with dt = 0.05 using RHS created at 0.5333333333333 Making slow rhs at time 0.55 for fast variables advancing from 0.5 to 0.6 Fast time integration at level 0 from 0.5 to 0.525 with dt = 0.025 Fast time integration at level 0 from 0.525 to 0.55 with dt = 0.025 Fast time integration at level 0 from 0.55 to 0.575 with dt = 0.025 Fast time integration at level 0 from 0.575 to 0.6 with dt = 0.025 Time integration of scalars at level 0 from 0.5 to 0.6 with dt = 0.1 using RHS created at 0.55 Done with advance_dycore at level 0 [Level 0 step 6] Advanced 262144 cells Coarse STEP 6 ends. TIME = 0.6 DT = 0.1 Timestep time = 0.25576115 seconds. TIME= 0.6 MASS = 1073741824 RHO THETA = 3.221225472e+11 RHO SCALAR = 0 Coarse STEP 7 starts ... Based on cfl of one Compressible dt at level 0 would be: 0.04610490538 Anelastic dt at level 0 would be: 1.586594338 Fixed dt at level 0 is: 0.1 smallest even ratio is: 4 [Level 0 step 7] ADVANCE from elapsed time = 0.6 to 0.7 with dt = 0.1 Making slow rhs at time 0.6 for fast variables advancing from 0.6 to 0.6333333333333 Fast time integration at level 0 from 0.6 to 0.6333333333333 with dt = 0.03333333333333 Time integration of scalars at level 0 from 0.6 to 0.6333333333333 with dt = 0.03333333333333 using RHS created at 0.6 Making slow rhs at time 0.6333333333333 for fast variables advancing from 0.6 to 0.65 Fast time integration at level 0 from 0.6 to 0.625 with dt = 0.025 Fast time integration at level 0 from 0.625 to 0.65 with dt = 0.025 Time integration of scalars at level 0 from 0.6 to 0.65 with dt = 0.05 using RHS created at 0.6333333333333 Making slow rhs at time 0.65 for fast variables advancing from 0.6 to 0.7 Fast time integration at level 0 from 0.6 to 0.625 with dt = 0.025 Fast time integration at level 0 from 0.625 to 0.65 with dt = 0.025 Fast time integration at level 0 from 0.65 to 0.675 with dt = 0.025 Fast time integration at level 0 from 0.675 to 0.7 with dt = 0.025 Time integration of scalars at level 0 from 0.6 to 0.7 with dt = 0.1 using RHS created at 0.65 Done with advance_dycore at level 0 [Level 0 step 7] Advanced 262144 cells Coarse STEP 7 ends. TIME = 0.7 DT = 0.1 Timestep time = 0.255784204 seconds. TIME= 0.7 MASS = 1073741824 RHO THETA = 3.221225472e+11 RHO SCALAR = 0 Coarse STEP 8 starts ... Based on cfl of one Compressible dt at level 0 would be: 0.04610506346 Anelastic dt at level 0 would be: 1.586930557 Fixed dt at level 0 is: 0.1 smallest even ratio is: 4 [Level 0 step 8] ADVANCE from elapsed time = 0.7 to 0.8 with dt = 0.1 Making slow rhs at time 0.7 for fast variables advancing from 0.7 to 0.7333333333333 Fast time integration at level 0 from 0.7 to 0.7333333333333 with dt = 0.03333333333333 Time integration of scalars at level 0 from 0.7 to 0.7333333333333 with dt = 0.03333333333333 using RHS created at 0.7 Making slow rhs at time 0.7333333333333 for fast variables advancing from 0.7 to 0.75 Fast time integration at level 0 from 0.7 to 0.725 with dt = 0.025 Fast time integration at level 0 from 0.725 to 0.75 with dt = 0.025 Time integration of scalars at level 0 from 0.7 to 0.75 with dt = 0.05 using RHS created at 0.7333333333333 Making slow rhs at time 0.75 for fast variables advancing from 0.7 to 0.8 Fast time integration at level 0 from 0.7 to 0.725 with dt = 0.025 Fast time integration at level 0 from 0.725 to 0.75 with dt = 0.025 Fast time integration at level 0 from 0.75 to 0.775 with dt = 0.025 Fast time integration at level 0 from 0.775 to 0.8 with dt = 0.025 Time integration of scalars at level 0 from 0.7 to 0.8 with dt = 0.1 using RHS created at 0.75 Done with advance_dycore at level 0 [Level 0 step 8] Advanced 262144 cells Coarse STEP 8 ends. TIME = 0.8 DT = 0.1 Timestep time = 0.255870949 seconds. TIME= 0.8 MASS = 1073741824 RHO THETA = 3.221225472e+11 RHO SCALAR = 0 Coarse STEP 9 starts ... Based on cfl of one Compressible dt at level 0 would be: 0.04610509547 Anelastic dt at level 0 would be: 1.586809621 Fixed dt at level 0 is: 0.1 smallest even ratio is: 4 [Level 0 step 9] ADVANCE from elapsed time = 0.8 to 0.9 with dt = 0.1 Making slow rhs at time 0.8 for fast variables advancing from 0.8 to 0.8333333333333 Fast time integration at level 0 from 0.8 to 0.8333333333333 with dt = 0.03333333333333 Time integration of scalars at level 0 from 0.8 to 0.8333333333333 with dt = 0.03333333333333 using RHS created at 0.8 Making slow rhs at time 0.8333333333333 for fast variables advancing from 0.8 to 0.85 Fast time integration at level 0 from 0.8 to 0.825 with dt = 0.025 Fast time integration at level 0 from 0.825 to 0.85 with dt = 0.025 Time integration of scalars at level 0 from 0.8 to 0.85 with dt = 0.05 using RHS created at 0.8333333333333 Making slow rhs at time 0.85 for fast variables advancing from 0.8 to 0.9 Fast time integration at level 0 from 0.8 to 0.825 with dt = 0.025 Fast time integration at level 0 from 0.825 to 0.85 with dt = 0.025 Fast time integration at level 0 from 0.85 to 0.875 with dt = 0.025 Fast time integration at level 0 from 0.875 to 0.9 with dt = 0.025 Time integration of scalars at level 0 from 0.8 to 0.9 with dt = 0.1 using RHS created at 0.85 Done with advance_dycore at level 0 [Level 0 step 9] Advanced 262144 cells Coarse STEP 9 ends. TIME = 0.9 DT = 0.1 Timestep time = 0.255886041 seconds. TIME= 0.9 MASS = 1073741824 RHO THETA = 3.221225472e+11 RHO SCALAR = 0 Coarse STEP 10 starts ... Based on cfl of one Compressible dt at level 0 would be: 0.0461047708 Anelastic dt at level 0 would be: 1.586965761 Fixed dt at level 0 is: 0.1 smallest even ratio is: 4 [Level 0 step 10] ADVANCE from elapsed time = 0.9 to 1 with dt = 0.1 Making slow rhs at time 0.9 for fast variables advancing from 0.9 to 0.9333333333333 Fast time integration at level 0 from 0.9 to 0.9333333333333 with dt = 0.03333333333333 Time integration of scalars at level 0 from 0.9 to 0.9333333333333 with dt = 0.03333333333333 using RHS created at 0.9 Making slow rhs at time 0.9333333333333 for fast variables advancing from 0.9 to 0.95 Fast time integration at level 0 from 0.9 to 0.925 with dt = 0.025 Fast time integration at level 0 from 0.925 to 0.95 with dt = 0.025 Time integration of scalars at level 0 from 0.9 to 0.95 with dt = 0.05 using RHS created at 0.9333333333333 Making slow rhs at time 0.95 for fast variables advancing from 0.9 to 1 Fast time integration at level 0 from 0.9 to 0.925 with dt = 0.025 Fast time integration at level 0 from 0.925 to 0.95 with dt = 0.025 Fast time integration at level 0 from 0.95 to 0.975 with dt = 0.025 Fast time integration at level 0 from 0.975 to 1 with dt = 0.025 Time integration of scalars at level 0 from 0.9 to 1 with dt = 0.1 using RHS created at 0.95 Done with advance_dycore at level 0 [Level 0 step 10] Advanced 262144 cells Coarse STEP 10 ends. TIME = 1 DT = 0.1 Timestep time = 0.25597427 seconds. TIME= 1 MASS = 1073741824 RHO THETA = 3.221225472e+11 RHO SCALAR = 0 Writing native 3D plotfile ABL-MOST_plt00010 3DPlotfile write time = 0.025758715 seconds. Writing native checkpoint ABL-MOST_chk00010 Writing SurfaceLayer variables at level 0 Checkpoint write time = 0.04014127 seconds. Total Time: 2.977932621 Unused ParmParse Variables: [TOP]::amr.checkpoint_files_output(nvals = 1) :: [0] [TOP]::prob.A_0(nvals = 1) :: [1.0] [TOP]::prob.U_0(nvals = 1) :: [10.0] [TOP]::prob.U_0_Pert_Mag(nvals = 1) :: [0.08] [TOP]::prob.V_0(nvals = 1) :: [0.0] [TOP]::prob.V_0_Pert_Mag(nvals = 1) :: [0.08] [TOP]::prob.W_0(nvals = 1) :: [0.0] [TOP]::prob.W_0_Pert_Mag(nvals = 1) :: [0.0] Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [6987 ... 7193] [The Arena] max space (MB) allocated spread across MPI: [89 ... 89] [The Arena] max space (MB) used spread across MPI: [81 ... 81] [The Device Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Device Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Pinned Arena] max space (MB) used spread across MPI: [4 ... 4] AMReX (26.04-83-g515ed5e0f061) finalized