Initializing AMReX (26.07-41-g5ca2c5e23267)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.07-41-g5ca2c5e23267) initialized Successfully read inputs file ... ERF git hash: bbf37e8c AMReX git hash: 26.07-41-g5ca2c5e232 Problem name (from inputs file) is: "WPS" Reading NC init file names at level 0 and index 0 : wrfinput_chisholmview_d01 Reading NC bdy file name wrfbdy_chisholmview_d01_width1 Level 0 is land Coriolis frequency, f = 0.000145842318 1/s Thermal stratification based on gradient of potential temperature Reading Times wrfinput datetime 0 : 2013-11-21_06:00:00 1385013600 Have read start_time string at level 0 is 2013-11-21_06:00:00 Have read start_time number at level 0 is 1385013600 Null land surface model! SOLVER CHOICE: At level 0 : compressible with implicit substepping vert_implicit_fac at level 0 : 1 1 0 (theta = 1, moisture = 1, tke = 1, momenta = 1) use_coriolis : 1 use_gravity : 1 Moisture Model: Kessler Terrain Type: StaticFittedMesh Buildings Type: None Mesh Type: VariableDz ABL Driver Type: Geostrophic Wind (-0,0.00218763477,-1.339539959e-19) Radiation Model: None Gradp_type : 0 Buoyancy_type at level 0 : 2 Advection Choices: dycore_horiz_adv_type : Upwind_3rd dycore_vert_adv_type : Upwind_3rd dryscal_horiz_adv_type : Upwind_3rd dryscal_vert_adv_type : Upwind_3rd moistscal_horiz_adv_type : WENO3 moistscal_vert_adv_type : WENO3 Diffusion choices: rho0_trans : 1 alpha_T : 0 alpha_C : 0 dynamic_viscosity : 0 Not using any molecular diffusivity, i.e. using the modeled turbulent diffusivity Sponge choices: Rayleigh damping : None w damping : 0 (coeff = -1 * dz/dt**2) Turbulence Settings at level 0 Using Smagorinsky LES model at level 0 Smagorinsky uses Richardson number correction with Ri_crit = 0.25 Cs : 0.005 isotropic mixing : 1 Pr_t : 0.3333333333 Sc_t : 1 Creating new distribution map on level: 0 BA FROM SCRATCH AT LEVEL 0 (BoxArray maxbox(4) m_ref->m_hash_sig(0) ((0,0,0) (99,99,175) (0,0,0)) ((100,0,0) (199,99,175) (0,0,0)) ((0,100,0) (99,199,175) (0,0,0)) ((100,100,0) (199,199,175) (0,0,0)) ) Reading T00 from wrfinput Reading P00 from wrfinput Reading TLP from wrfinput Reading TISO from wrfinput Reading TLP_STRAT from wrfinput Reading P_STRAT from wrfinput WRF base state parameters (T00, P00, TLP, TISO, TLP_STRAT, P_STRAT) are: (290, 100000, 50, 200, -11, 0) Loading initial data from NetCDF file at level 0 Reading from file wrfinput_chisholmview_d01 Will read density from ALB + AL variables Have read (nx,ny,nz) = 200 200 176 Have read (parent_ilo,parent_jlo) = 0 0 Have read refinement ratio = 1 Box in file ((0,0,0) (199,199,175) (0,0,0)) Box to fill ((0,0,0) (199,199,175) (0,0,0)) Checking for ALB ... Reading ALB Checking for AL ... Reading AL Checking for U ... Reading U Checking for V ... Reading V Checking for W ... Reading W Checking for THM ... Reading THM Checking for PH ... Reading PH Checking for PHB ... Reading PHB Checking for PB ... Reading PB Checking for P ... Reading P Checking for PSFC ... Reading PSFC Checking for MUB ... Reading MUB Checking for MAPFAC_U ... Reading MAPFAC_U Checking for MAPFAC_V ... Reading MAPFAC_V Checking for MAPFAC_M ... Reading MAPFAC_M Checking for SST ... Reading SST Checking for TSK ... Reading TSK Checking for LANDMASK ... Reading LANDMASK Checking for C1H ... Reading C1H Checking for C2H ... Reading C2H Checking for RDNW ... Reading RDNW Checking for XLAT_V ... Reading XLAT_V Checking for XLONG_U ... Reading XLONG_U Checking for QVAPOR ... Reading QVAPOR Checking for QCLOUD ... Reading QCLOUD Checking for QRAIN ... Reading QRAIN Checking for IVGTYP ... Reading IVGTYP Checking for ISLTYP ... Reading ISLTYP Terrain has min value = -50.12749566 and max value = 3509.155697 Top of mesh has min value = 15684.13091 and max value = 16560.73164 Max of second-to-highest row = 16382.9893 which is greater than average of top row so defaulting to alternate approach Warning: ProbHi(2) will be ignored; we are setting top of domain to 16471.86047 Verifying grid integrity The state read from WRF is being rebalanced! Loading boundary data from NetCDF file Reading Times wrfbdy datetime 0 : 2013-11-21_06:00:00 1385013600 wrfbdy datetime 1 : 2013-11-21_09:00:00 1385024400 wrfbdy datetime 2 : 2013-11-21_12:00:00 1385035200 wrfbdy datetime 3 : 2013-11-21_15:00:00 1385046000 wrfbdy datetime 4 : 2013-11-21_18:00:00 1385056800 wrfbdy datetime 5 : 2013-11-21_21:00:00 1385067600 wrfbdy datetime 6 : 2013-11-22_00:00:00 1385078400 wrfbdy datetime 7 : 2013-11-22_03:00:00 1385089200 start_bdy_time 1385013600 final_bdy_time 1385100000 Initialized erfbdy file: erfbdy Reading time slice 0 from wrfbdy_chisholmview_d01_width1 Wrote erfbdy time index 0 of 8 Reading time slice 1 from wrfbdy_chisholmview_d01_width1 Wrote erfbdy time index 1 of 8 Reading time slice 2 from wrfbdy_chisholmview_d01_width1 Wrote erfbdy time index 2 of 8 Processing remaining 6 boundary times... Reading time slice 3 from wrfbdy_chisholmview_d01_width1 Wrote erfbdy time index 3 of 8 Reading time slice 4 from wrfbdy_chisholmview_d01_width1 Wrote erfbdy time index 4 of 8 Reading time slice 5 from wrfbdy_chisholmview_d01_width1 Wrote erfbdy time index 5 of 8 Reading time slice 6 from wrfbdy_chisholmview_d01_width1 Wrote erfbdy time index 6 of 8 Reading time slice 7 from wrfbdy_chisholmview_d01_width1 Wrote erfbdy time index 7 of 8 Reading time slice 7 from wrfbdy_chisholmview_d01_width1 Reading time slice 7 from wrfbdy_chisholmview_d01_width1 Wrote erfbdy time index 8 of 8 Completed writing erfbdy times start_bdy_time is 1385013600 from wrfbdy but note that time variable in simulation is elapsed time Problem name "wps" does not add any state perturbations. Problem name "wps" does not add any velocity perturbations. Based on cfl of one Compressible dt at level 0 would be: 52.0393795 Anelastic dt at level 0 would be: 345.9945654 Fixed dt at level 0 is: 0.5 smallest even ratio is: 4 If there is water, Charnock relation with C_a=0.0185 will be used Variable sea roughness (type charnock) Using MOST with specified surface temperature (land: TSK, sea: SST) Max value of |p_hse - p_eos| is less than 1e-8 Min/max value of dp0/dx are -0.005554718214 0.00722119235 Min/max value of dp0/dy are -0.005720939131 0.005561459673 Min/max value of dp0/dz + rho0*|g| are less than 1e-08 Min/Max value of x-gradient of full (moist) pressure are -0.3080329595 0.1947626466 with min at face (1,110,0) with max at face (2,115,0) Min/Max value of y-gradient of full (moist) pressure are -0.1928760573 0.176227854 with min at face (7,94,0) with max at face (101,1,0) Min/max value of moist dp/dz + rho_m*|g| are -1.446720842e-11 2.440029171e-11 with min at face (121,137,3) with max at face (10,150,3) Writing native checkpoint WPS_Test_restart_chk00000 Writing SurfaceLayer variables at level 0 Writing Lat/Lon variables at level 0 Writing C1H/C2H/RDNW/MUB/PHB variables at level 0 Checkpoint write time = 0.447769749 seconds. Writing native 3D plotfile WPS_Test_restart_plt00000 3DPlotfile write time = 0.743479267 seconds. TIME= 0 MASS = 1.184547956e+17 RHO THETA = 3.765686971e+19 RHO SCALAR = 0 RHO QTOTAL = 2.53594267e+14 2013-11-21 06:00:00.000000 (0 s elapsed) Coarse STEP 1 starts ... Based on cfl of one Compressible dt at level 0 would be: 52.0393795 Anelastic dt at level 0 would be: 345.9945654 Fixed dt at level 0 is: 0.5 smallest even ratio is: 4 [Level 0 step 1] ADVANCE from elapsed time = 0 to 0.5 with dt = 0.5 Making slow rhs at time 0 for fast variables advancing from 0 to 0.1666666666667 Fast time integration at level 0 from 0 to 0.1666666666667 with dt = 0.1666666666667 Time integration of scalars at level 0 from 0 to 0.1666666666667 with dt = 0.1666666666667 using RHS created at 0 Making slow rhs at time 0.1666666666667 for fast variables advancing from 0 to 0.25 Fast time integration at level 0 from 0 to 0.125 with dt = 0.125 Fast time integration at level 0 from 0.125 to 0.25 with dt = 0.125 Time integration of scalars at level 0 from 0 to 0.25 with dt = 0.25 using RHS created at 0.1666666666667 Making slow rhs at time 0.25 for fast variables advancing from 0 to 0.5 Fast time integration at level 0 from 0 to 0.125 with dt = 0.125 Fast time integration at level 0 from 0.125 to 0.25 with dt = 0.125 Fast time integration at level 0 from 0.25 to 0.375 with dt = 0.125 Fast time integration at level 0 from 0.375 to 0.5 with dt = 0.125 Time integration of scalars at level 0 from 0 to 0.5 with dt = 0.5 using RHS created at 0.25 Done with advance_dycore at level 0 [Level 0 step 1] Advanced 7040000 cells Cloud fraction 0 0.00995 Coarse STEP 1 ends. TIME = 0.5 DT = 0.5 Timestep time = 24.21644369 seconds. TIME= 0.5 MASS = 1.184548002e+17 RHO THETA = 3.765687146e+19 RHO SCALAR = 0 RHO QTOTAL = 2.535939229e+14 Writing native 3D plotfile WPS_Test_restart_plt00001 3DPlotfile write time = 1.199344787 seconds. Writing native checkpoint WPS_Test_restart_chk00001 Writing SurfaceLayer variables at level 0 Writing Lat/Lon variables at level 0 Writing C1H/C2H/RDNW/MUB/PHB variables at level 0 Checkpoint write time = 0.688986113 seconds. 2013-11-21 06:00:00.500000 (0.5 s elapsed) Coarse STEP 2 starts ... Based on cfl of one Compressible dt at level 0 would be: 42.55935375 Anelastic dt at level 0 would be: 104.9845597 Fixed dt at level 0 is: 0.5 smallest even ratio is: 4 [Level 0 step 2] ADVANCE from elapsed time = 0.5 to 1 with dt = 0.5 Making slow rhs at time 0.5 for fast variables advancing from 0.5 to 0.6666666666667 Fast time integration at level 0 from 0.5 to 0.6666666666667 with dt = 0.1666666666667 Time integration of scalars at level 0 from 0.5 to 0.6666666666667 with dt = 0.1666666666667 using RHS created at 0.5 Making slow rhs at time 0.6666666666667 for fast variables advancing from 0.5 to 0.75 Fast time integration at level 0 from 0.5 to 0.625 with dt = 0.125 Fast time integration at level 0 from 0.625 to 0.75 with dt = 0.125 Time integration of scalars at level 0 from 0.5 to 0.75 with dt = 0.25 using RHS created at 0.6666666666667 Making slow rhs at time 0.75 for fast variables advancing from 0.5 to 1 Fast time integration at level 0 from 0.5 to 0.625 with dt = 0.125 Fast time integration at level 0 from 0.625 to 0.75 with dt = 0.125 Fast time integration at level 0 from 0.75 to 0.875 with dt = 0.125 Fast time integration at level 0 from 0.875 to 1 with dt = 0.125 Time integration of scalars at level 0 from 0.5 to 1 with dt = 0.5 using RHS created at 0.75 Done with advance_dycore at level 0 [Level 0 step 2] Advanced 7040000 cells Cloud fraction 0.5 0.012975 Coarse STEP 2 ends. TIME = 1 DT = 0.5 Timestep time = 9.31220694 seconds. TIME= 1 MASS = 1.184548048e+17 RHO THETA = 3.765687299e+19 RHO SCALAR = 0 RHO QTOTAL = 2.535935773e+14 Writing native 3D plotfile WPS_Test_restart_plt00002 3DPlotfile write time = 0.91124311 seconds. Writing native checkpoint WPS_Test_restart_chk00002 Writing SurfaceLayer variables at level 0 Writing Lat/Lon variables at level 0 Writing C1H/C2H/RDNW/MUB/PHB variables at level 0 Checkpoint write time = 4.064228237 seconds. Total Time: 49.88861469 Unused ParmParse Variables: [TOP]::amr.checkpoint_files_output(nvals = 1) :: [1] Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [3 ... 205] [The Arena] max space (MB) allocated spread across MPI: [2520 ... 2520] [The Arena] max space (MB) used spread across MPI: [2351 ... 2351] [The Device Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Device Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [787 ... 787] [The Pinned Arena] max space (MB) used spread across MPI: [213 ... 213] AMReX (26.07-41-g5ca2c5e23267) finalized Initializing AMReX (26.07-41-g5ca2c5e23267)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.07-41-g5ca2c5e23267) initialized Successfully read inputs file ... ERF git hash: bbf37e8c AMReX git hash: 26.07-41-g5ca2c5e232 Problem name (from inputs file) is: "WPS" Reading NC init file names at level 0 and index 0 : wrfinput_chisholmview_d01 Reading NC bdy file name wrfbdy_chisholmview_d01_width1 Level 0 is land Coriolis frequency, f = 0.000145842318 1/s Thermal stratification based on gradient of potential temperature Reading Times wrfinput datetime 0 : 2013-11-21_06:00:00 1385013600 Have read start_time string at level 0 is 2013-11-21_06:00:00 Have read start_time number at level 0 is 1385013600 Null land surface model! SOLVER CHOICE: At level 0 : compressible with implicit substepping vert_implicit_fac at level 0 : 1 1 0 (theta = 1, moisture = 1, tke = 1, momenta = 1) use_coriolis : 1 use_gravity : 1 Moisture Model: Kessler Terrain Type: StaticFittedMesh Buildings Type: None Mesh Type: VariableDz ABL Driver Type: Geostrophic Wind (-0,0.00218763477,-1.339539959e-19) Radiation Model: None Gradp_type : 0 Buoyancy_type at level 0 : 2 Advection Choices: dycore_horiz_adv_type : Upwind_3rd dycore_vert_adv_type : Upwind_3rd dryscal_horiz_adv_type : Upwind_3rd dryscal_vert_adv_type : Upwind_3rd moistscal_horiz_adv_type : WENO3 moistscal_vert_adv_type : WENO3 Diffusion choices: rho0_trans : 1 alpha_T : 0 alpha_C : 0 dynamic_viscosity : 0 Not using any molecular diffusivity, i.e. using the modeled turbulent diffusivity Sponge choices: Rayleigh damping : None w damping : 0 (coeff = -1 * dz/dt**2) Turbulence Settings at level 0 Using Smagorinsky LES model at level 0 Smagorinsky uses Richardson number correction with Ri_crit = 0.25 Cs : 0.005 isotropic mixing : 1 Pr_t : 0.3333333333 Sc_t : 1 Restart from native checkpoint WPS_Test_restart_chk00001 BA FROM SCRATCH AT LEVEL 0 (BoxArray maxbox(4) m_ref->m_hash_sig(0) ((0,0,0) (99,99,175) (0,0,0)) ((100,0,0) (199,99,175) (0,0,0)) ((0,100,0) (99,199,175) (0,0,0)) ((100,100,0) (199,199,175) (0,0,0)) ) Reading SST data Reading TSK data Reading LMASK data Reading Lat/Lon variables Reading C1H/C2H/RDNW/MUB/PHB variables at level 0 Restart time = 1.126416609 seconds. Loading boundary data from NetCDF file Reading Times wrfbdy datetime 0 : 2013-11-21_06:00:00 1385013600 wrfbdy datetime 1 : 2013-11-21_09:00:00 1385024400 wrfbdy datetime 2 : 2013-11-21_12:00:00 1385035200 wrfbdy datetime 3 : 2013-11-21_15:00:00 1385046000 wrfbdy datetime 4 : 2013-11-21_18:00:00 1385056800 wrfbdy datetime 5 : 2013-11-21_21:00:00 1385067600 wrfbdy datetime 6 : 2013-11-22_00:00:00 1385078400 wrfbdy datetime 7 : 2013-11-22_03:00:00 1385089200 start_bdy_time 1385013600 final_bdy_time 1385100000 Reading time slice 0 from wrfbdy_chisholmview_d01_width1 Reading time slice 1 from wrfbdy_chisholmview_d01_width1 Reading time slice 2 from wrfbdy_chisholmview_d01_width1 If there is water, Charnock relation with C_a=0.0185 will be used Variable sea roughness (type charnock) Using MOST with specified surface temperature (land: TSK, sea: SST) Reading MOST variables Max value of |p_hse - p_eos| is less than 1e-8 Min/max value of dp0/dx are -0.005554718214 0.00722119235 Min/max value of dp0/dy are -0.005720939131 0.005561459673 Min/max value of dp0/dz + rho0*|g| are less than 1e-08 Min/Max value of x-gradient of full (moist) pressure are -0.3113391985 0.1928769608 with min at face (1,110,5) with max at face (2,115,12) Min/Max value of y-gradient of full (moist) pressure are -0.1914611284 0.1770856046 with min at face (7,94,12) with max at face (101,1,7) Min/max value of moist dp/dz + rho_m*|g| are -0.5448044426 0.9223306193 with min at face (9,82,5) with max at face (9,85,11) Writing native 3D plotfile WPS_Test_restart_plt00001 3DPlotfile write time = 1.572242561 seconds. TIME= 0.5 MASS = 1.184548002e+17 RHO THETA = 3.765687146e+19 RHO SCALAR = 0 RHO QTOTAL = 2.535939229e+14 2013-11-21 06:00:00.500000 (0.5 s elapsed) Coarse STEP 2 starts ... Based on cfl of one Compressible dt at level 0 would be: 42.55935375 Anelastic dt at level 0 would be: 104.9845597 Fixed dt at level 0 is: 0.5 smallest even ratio is: 4 [Level 0 step 2] ADVANCE from elapsed time = 0.5 to 1 with dt = 0.5 Making slow rhs at time 0.5 for fast variables advancing from 0.5 to 0.6666666666667 Fast time integration at level 0 from 0.5 to 0.6666666666667 with dt = 0.1666666666667 Time integration of scalars at level 0 from 0.5 to 0.6666666666667 with dt = 0.1666666666667 using RHS created at 0.5 Making slow rhs at time 0.6666666666667 for fast variables advancing from 0.5 to 0.75 Fast time integration at level 0 from 0.5 to 0.625 with dt = 0.125 Fast time integration at level 0 from 0.625 to 0.75 with dt = 0.125 Time integration of scalars at level 0 from 0.5 to 0.75 with dt = 0.25 using RHS created at 0.6666666666667 Making slow rhs at time 0.75 for fast variables advancing from 0.5 to 1 Fast time integration at level 0 from 0.5 to 0.625 with dt = 0.125 Fast time integration at level 0 from 0.625 to 0.75 with dt = 0.125 Fast time integration at level 0 from 0.75 to 0.875 with dt = 0.125 Fast time integration at level 0 from 0.875 to 1 with dt = 0.125 Time integration of scalars at level 0 from 0.5 to 1 with dt = 0.5 using RHS created at 0.75 Done with advance_dycore at level 0 [Level 0 step 2] Advanced 7040000 cells Cloud fraction 0.5 0.012975 Coarse STEP 2 ends. TIME = 1 DT = 0.5 Timestep time = 17.7405946 seconds. TIME= 1 MASS = 1.184548048e+17 RHO THETA = 3.765687299e+19 RHO SCALAR = 0 RHO QTOTAL = 2.535935773e+14 Writing native 3D plotfile WPS_Test_restart_plt00002 3DPlotfile write time = 1.526846782 seconds. Total Time: 22.91099545 Unused ParmParse Variables: [TOP]::amr.checkpoint_files_output(nvals = 1) :: [0] Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [3 ... 225] [The Arena] max space (MB) allocated spread across MPI: [2642 ... 2642] [The Arena] max space (MB) used spread across MPI: [2351 ... 2351] [The Device Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Device Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [477 ... 477] [The Pinned Arena] max space (MB) used spread across MPI: [207 ... 207] AMReX (26.07-41-g5ca2c5e23267) finalized