Initializing AMReX (26.05-3-g4ceb814db7d7)... MPI initialized with 1 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.05-3-g4ceb814db7d7) initialized REMORA git hash: v1.5-98-g402a0a3 AMReX git hash: 26.05-3-g4ceb814db7 Making level 0 from scratch GRIDS AT LEVEL 0 ARE (BoxArray maxbox(1) m_ref->m_hash_sig(0) ((0,0,0) (9,15,19) (0,0,0)) ) Calling init_masks_from_netcdf level 0 Loading masks from NetCDF file idmini_grd_v1_classic64.nc Masks loaded from netcdf file Calling init_bdry_from_netcdf at level 0 DOING INIT AT LEVEL 0 Setting up boundary data for u coming from NetCDF file Setting up boundary data for v coming from NetCDF file Setting up boundary data for temp coming from NetCDF file Setting up boundary data for salt coming from NetCDF file Setting up boundary data for ubar coming from NetCDF file Setting up boundary data for vbar coming from NetCDF file Setting up boundary data for zeta coming from NetCDF file Boundary data loaded from netcdf file Loading Uwind from NetCDF file(s) Loading Vwind from NetCDF file(s) Calling init_bathymetry_from_netcdf Loading initial bathymetry from NetCDF file idmini_grd_v1_classic64.nc Bathymetry loaded from netcdf file Calling init_zeta_from_netcdf on level 0 Loading initial sea surface height from NetCDF file idmini_ini_v1_classic64.nc Reading in zeta at (before) time 0 Actually reading in zeta at time 0 DONE reading in zeta at time 0 Reading in zeta at (after) time 1 Actually reading in zeta at time 86400 DONE reading in zeta at time 86400 Sea surface height loaded from netcdf file Calling init_data_from_netcdf Loading initial solution data from NetCDF file idmini_ini_v1_classic64.nc Initial data loaded from netcdf file Reading in ubar at (before) time 0 Actually reading in ubar at time 0 DONE reading in ubar at time 0 Reading in ubar at (after) time 1 Actually reading in ubar at time 86400 DONE reading in ubar at time 86400 Reading in vbar at (before) time 0 Actually reading in vbar at time 0 DONE reading in vbar at time 0 Reading in vbar at (after) time 1 Actually reading in vbar at time 86400 DONE reading in vbar at time 86400 Calling init_coriolis_from_netcdf Loading initial coriolis from NetCDF file idmini_grd_v1_classic64.nc Coriolis loaded from netcdf file Reading in temp at (before) time 0 Actually reading in temp at time 0 DONE reading in temp at time 0 Reading in temp at (after) time 1 Actually reading in temp at time 86400 DONE reading in temp at time 86400 Reading in salt at (before) time 0 Actually reading in salt at time 0 DONE reading in salt at time 0 Reading in salt at (after) time 1 Actually reading in salt at time 86400 DONE reading in salt at time 86400 Reading in u at (before) time 0 Actually reading in u at time 0 DONE reading in u at time 0 Reading in u at (after) time 1 Actually reading in u at time 86400 DONE reading in u at time 86400 Reading in v at (before) time 0 Actually reading in v at time 0 DONE reading in v at time 0 Reading in v at (after) time 1 Actually reading in v at time 86400 DONE reading in v at time 86400 Writing plotfile IdealMiniGridWind-1grid_plt00000 Coarse STEP 1 starts ... Reading in Uwind at time index 1 from idmini_frc_wind_classic64.nc Reading in Uwind at time index 0 from idmini_frc_wind_classic64.nc Reading in Vwind at time index 1 from idmini_frc_wind_classic64.nc Reading in Vwind at time index 0 from idmini_frc_wind_classic64.nc Coarse STEP 1 ends. TIME = 200 DT = 200 Coarse STEP 2 starts ... Coarse STEP 2 ends. TIME = 400 DT = 200 Coarse STEP 3 starts ... Coarse STEP 3 ends. TIME = 600 DT = 200 Coarse STEP 4 starts ... Coarse STEP 4 ends. TIME = 800 DT = 200 Coarse STEP 5 starts ... Coarse STEP 5 ends. TIME = 1000 DT = 200 Coarse STEP 6 starts ... Coarse STEP 6 ends. TIME = 1200 DT = 200 Coarse STEP 7 starts ... Coarse STEP 7 ends. TIME = 1400 DT = 200 Coarse STEP 8 starts ... Coarse STEP 8 ends. TIME = 1600 DT = 200 Coarse STEP 9 starts ... Coarse STEP 9 ends. TIME = 1800 DT = 200 Coarse STEP 10 starts ... Coarse STEP 10 ends. TIME = 2000 DT = 200 Writing plotfile IdealMiniGridWind-1grid_plt00010 TinyProfiler total time across processes [min...avg...max]: 0.08714 ... 0.08714 ... 0.08714 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- REMORA::init_only() 1 0.01348 0.01348 0.01348 15.47% REMORA::FillPatch() 208 0.01237 0.01237 0.01237 14.19% VisMF::Write(FabArray) 4 0.00721 0.00721 0.00721 8.27% REMORA::vert_visc_3d() 50 0.005163 0.005163 0.005163 5.92% REMORA::advance_3d() 10 0.004309 0.004309 0.004309 4.94% REMORA::advance2d() 30 0.003633 0.003633 0.003633 4.17% impose_cons_bcs() 80 0.003169 0.003169 0.003169 3.64% REMORA::bulk_fluxes() 10 0.003107 0.003107 0.003107 3.57% REMORA::prestep_t_advection() 30 0.002293 0.002293 0.002293 2.63% REMORA::t3dmix2_s() 20 0.002044 0.002044 0.002044 2.35% REMORA::setup_step() 10 0.001934 0.001934 0.001934 2.22% WriteMultiLevelPlotfileWithBathymetry() 2 0.001822 0.001822 0.001822 2.09% REMORA::set_zeta() 1 0.001806 0.001806 0.001806 2.07% REMORAPhysBCFunct::() 239 0.00171 0.00171 0.00171 1.96% REMORA::set_wind() 10 0.001692 0.001692 0.001692 1.94% REMORA::InitData() 2 0.001555 0.001555 0.001555 1.78% REMORA::ReadParameters() 1 0.001535 0.001535 0.001535 1.76% REMORA::rhs_uv_3d() 10 0.001403 0.001403 0.001403 1.61% impose_xvel_bcs() 42 0.001278 0.001278 0.001278 1.47% impose_yvel_bcs() 42 0.001265 0.001265 0.001265 1.45% REMORA::Evolve() 1 0.001249 0.001249 0.001249 1.43% REMORA::rhs_t_3d() 30 0.001182 0.001182 0.001182 1.36% FabArray::setVal() 159 0.001156 0.001156 0.001156 1.33% REMORA::uv3dmix() 30 0.001065 0.001065 0.001065 1.22% REMORA::rhs_t_3d()::hadv 30 0.000779 0.000779 0.000779 0.89% REMORA::update_massflux_3d() 20 0.0007743 0.0007743 0.0007743 0.89% REMORA::prestep() 10 0.0007361 0.0007361 0.0007361 0.84% REMORA::bathymetry() 1 0.0007354 0.0007354 0.0007354 0.84% REMORA::rhs_uv_2d() 20 0.0006643 0.0006643 0.0006643 0.76% REMORA::estTimeStep() 11 0.0006595 0.0006595 0.0006595 0.76% amrex::Copy() 72 0.0006008 0.0006008 0.0006008 0.69% REMORA::prsgrd() 10 0.0005225 0.0005225 0.0005225 0.60% REMORA::stretch_transform() 11 0.0005124 0.0005124 0.0005124 0.59% REMORA::prestep_diffusion() 50 0.000458 0.000458 0.000458 0.53% REMORA::set_coriolis() 1 0.0003462 0.0003462 0.0003462 0.40% MultiFab::contains_nan() 10 0.0002533 0.0002533 0.0002533 0.29% FillPatchSingleLevel 380 0.000227 0.000227 0.000227 0.26% FabArray::FillBoundary() 596 0.0002261 0.0002261 0.0002261 0.26% MultiFab::contains_inf() 10 0.0002084 0.0002084 0.0002084 0.24% REMORA::vert_mean_3d() 20 0.0001939 0.0001939 0.0001939 0.22% REMORA::rhs_t_3d()::vadv 30 0.000171 0.000171 0.000171 0.20% REMORA::set_analytic_vmix() 11 0.0001635 0.0001635 0.0001635 0.19% FabArrayBase::getFB() 599 0.0001543 0.0001543 0.0001543 0.18% REMORA::advance_3d_ml() 10 0.0001536 0.0001536 0.0001536 0.18% FillBoundary_nowait() 599 0.00013 0.00013 0.00013 0.15% main() 1 7.55e-05 7.55e-05 7.55e-05 0.09% REMORA::FillPatchNoBC() 172 7.315e-05 7.315e-05 7.315e-05 0.08% REMORA::set_hmixcoef() 1 3.819e-05 3.819e-05 3.819e-05 0.04% REMORA::REMORA() 1 3.023e-05 3.023e-05 3.023e-05 0.03% REMORA::Advance() 10 2.4e-05 2.4e-05 2.4e-05 0.03% MultiFab::Copy() 59 1.477e-05 1.477e-05 1.477e-05 0.02% REMORA::init_set_vmix() 1 9.68e-07 9.68e-07 9.68e-07 0.00% Other 799 0.0007843 0.0007843 0.0007843 0.90% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 0.08713 0.08713 0.08713 99.99% REMORA::Evolve() 1 0.05495 0.05495 0.05495 63.06% REMORA::Advance() 10 0.04842 0.04842 0.04842 55.56% REMORA::InitData() 2 0.03211 0.03211 0.03211 36.85% REMORA::setup_step() 10 0.02064 0.02064 0.02064 23.68% REMORA::init_only() 1 0.01984 0.01984 0.01984 22.77% REMORA::FillPatch() 208 0.01949 0.01949 0.01949 22.36% REMORA::advance_3d_ml() 10 0.01753 0.01753 0.01753 20.11% REMORA::advance_3d() 10 0.01706 0.01706 0.01706 19.57% REMORA::advance2d() 30 0.01023 0.01023 0.01023 11.74% WriteMultiLevelPlotfileWithBathymetry() 2 0.009186 0.009186 0.009186 10.54% REMORAPhysBCFunct::() 239 0.007422 0.007422 0.007422 8.52% VisMF::Write(FabArray) 4 0.00721 0.00721 0.00721 8.27% REMORA::vert_visc_3d() 50 0.005163 0.005163 0.005163 5.92% REMORA::prestep() 10 0.003728 0.003728 0.003728 4.28% impose_cons_bcs() 80 0.003169 0.003169 0.003169 3.64% REMORA::bulk_fluxes() 10 0.003107 0.003107 0.003107 3.57% REMORA::set_wind() 10 0.002605 0.002605 0.002605 2.99% REMORA::prestep_t_advection() 30 0.002293 0.002293 0.002293 2.63% REMORA::rhs_t_3d() 30 0.002132 0.002132 0.002132 2.45% REMORA::t3dmix2_s() 20 0.002044 0.002044 0.002044 2.35% REMORA::set_zeta() 1 0.001844 0.001844 0.001844 2.12% REMORA::REMORA() 1 0.001565 0.001565 0.001565 1.80% REMORA::ReadParameters() 1 0.001535 0.001535 0.001535 1.76% REMORA::rhs_uv_3d() 10 0.001403 0.001403 0.001403 1.61% impose_xvel_bcs() 42 0.001278 0.001278 0.001278 1.47% impose_yvel_bcs() 42 0.001265 0.001265 0.001265 1.45% FabArray::setVal() 159 0.001156 0.001156 0.001156 1.33% REMORA::set_analytic_vmix() 11 0.001122 0.001122 0.001122 1.29% REMORA::bathymetry() 1 0.001118 0.001118 0.001118 1.28% REMORA::uv3dmix() 30 0.001065 0.001065 0.001065 1.22% REMORA::rhs_t_3d()::hadv 30 0.000779 0.000779 0.000779 0.89% REMORA::update_massflux_3d() 20 0.0007743 0.0007743 0.0007743 0.89% REMORA::rhs_uv_2d() 20 0.0006643 0.0006643 0.0006643 0.76% REMORA::estTimeStep() 11 0.0006595 0.0006595 0.0006595 0.76% FillPatchSingleLevel 380 0.000625 0.000625 0.000625 0.72% FabArray::FillBoundary() 596 0.0006009 0.0006009 0.0006009 0.69% amrex::Copy() 72 0.0006008 0.0006008 0.0006008 0.69% REMORA::stretch_transform() 11 0.0005505 0.0005505 0.0005505 0.63% REMORA::prsgrd() 10 0.0005225 0.0005225 0.0005225 0.60% MultiFab::Copy() 59 0.000465 0.000465 0.000465 0.53% REMORA::prestep_diffusion() 50 0.000458 0.000458 0.000458 0.53% REMORA::set_coriolis() 1 0.0004038 0.0004038 0.0004038 0.46% FillBoundary_nowait() 599 0.000338 0.000338 0.000338 0.39% REMORA::FillPatchNoBC() 172 0.0003003 0.0003003 0.0003003 0.34% MultiFab::contains_nan() 10 0.0002533 0.0002533 0.0002533 0.29% REMORA::set_hmixcoef() 1 0.0002368 0.0002368 0.0002368 0.27% REMORA::init_set_vmix() 1 0.000224 0.000224 0.000224 0.26% MultiFab::contains_inf() 10 0.0002084 0.0002084 0.0002084 0.24% FabArrayBase::getFB() 599 0.000208 0.000208 0.000208 0.24% REMORA::vert_mean_3d() 20 0.0001939 0.0001939 0.0001939 0.22% REMORA::rhs_t_3d()::vadv 30 0.000171 0.000171 0.000171 0.20% Other 799 0.0008483 0.0008483 0.0008483 0.97% ------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::amr.checkpoint_files_output(nvals = 1) :: [0] Device Memory Usage: -------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem MaxMem CurrentMem -------------------------------------------------------------------------------------- The_Arena::Initialize() 1 1 3527 B 1024 MiB 0 B ResizeRandomSeed 1 1 9020 KiB 9216 KiB 0 B REMORA::InitData() 104 96 722 KiB 3092 KiB 846 KiB REMORA::advance_3d() 190 190 14 KiB 379 KiB 0 B REMORA::Evolve() 3 3 5718 B 362 KiB 0 B REMORA::prestep_t_advection() 30 30 2091 B 350 KiB 0 B REMORA::prestep() 40 40 2959 B 264 KiB 0 B REMORA::prsgrd() 10 10 1703 B 262 KiB 0 B REMORA::setup_step() 60 60 11 KiB 241 KiB 0 B REMORA::rhs_t_3d() 120 120 1208 B 218 KiB 0 B REMORA::uv3dmix() 30 30 610 B 175 KiB 0 B REMORA::init_only() 15 9 3167 B 147 KiB 13 KiB REMORA::t3dmix2_s() 40 40 470 B 135 KiB 0 B WriteMultiLevelPlotfileWithBathymetry() 2 2 2867 B 125 KiB 0 B REMORA::estTimeStep() 33 33 194 B 123 KiB 0 B REMORA::rhs_uv_3d() 40 40 526 B 116 KiB 0 B VisMF::Write(FabArray) 16 16 90 B 96 KiB 0 B REMORA::bulk_fluxes() 20 20 840 B 87 KiB 0 B REMORA::advance2d() 120 120 1174 B 40 KiB 0 B REMORA::bathymetry() 11 11 39 B 18 KiB 0 B REMORA::rhs_uv_2d() 20 20 28 B 8960 B 0 B MultiFab::contains_nan() 13 12 60 B 6656 B 256 B MultiFab::contains_inf() 10 10 3 B 6144 B 0 B REMORA::update_massflux_3d() 20 20 11 B 4608 B 0 B REMORA::set_zeta() 1 1 7 B 1792 B 0 B REMORA::set_wind() 4 4 0 B 1792 B 0 B REMORA::set_coriolis() 1 1 0 B 1792 B 0 B REMORAPhysBCFunct::() 75 75 0 B 256 B 0 B impose_cons_bcs() 80 80 2 B 256 B 0 B impose_xvel_bcs() 42 42 0 B 256 B 0 B impose_yvel_bcs() 42 42 0 B 256 B 0 B -------------------------------------------------------------------------------------- Async Memory Usage: --------------------------------------------------------- Name Nalloc AvgMem MaxMem --------------------------------------------------------- REMORA::prsgrd() 10 423 B 262 KiB REMORA::advance_3d() 150 4633 B 183 KiB REMORA::prestep_t_advection() 30 1249 B 175 KiB REMORA::rhs_t_3d() 120 918 B 175 KiB REMORA::uv3dmix() 30 287 B 175 KiB REMORA::rhs_uv_3d() 40 446 B 116 KiB REMORA::t3dmix2_s() 40 373 B 108 KiB REMORA::setup_step() 20 729 B 48 KiB REMORA::prestep() 10 411 B 45 KiB REMORA::rhs_uv_2d() 20 17 B 8960 B REMORA::advance2d() 30 3 B 2816 B REMORA::update_massflux_3d() 20 5 B 2240 B --------------------------------------------------------- Managed Memory Usage: --------------------------------------------------------- Name Nalloc AvgMem MaxMem --------------------------------------------------------- The_Managed_Arena::Initialize() 1 11 B 8192 KiB --------------------------------------------------------- Pinned Memory Usage: --------------------------------------------------------------------------- Name Nalloc Nfree AvgMem MaxMem CurrentMem --------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 1 1 26 B 8192 KiB 0 B REMORA::init_only() 93 93 29 KiB 159 KiB 0 B VisMF::Write(FabArray) 20 20 60 B 125 KiB 0 B REMORA::bathymetry() 11 11 1 B 1792 B 0 B REMORA::set_coriolis() 1 1 0 B 1792 B 0 B REMORA::set_wind() 4 4 0 B 1792 B 0 B REMORA::set_zeta() 1 1 0 B 1792 B 0 B MultiFab::contains_nan() 13 12 56 B 768 B 256 B REMORA::estTimeStep() 22 22 0 B 512 B 0 B MultiFab::contains_inf() 10 10 0 B 256 B 0 B --------------------------------------------------------------------------- Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [6789 ... 6789] [The Arena] max space (MB) allocated spread across MPI: [1024 ... 1024] [The Arena] max space (MB) used spread across MPI: [12 ... 12] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Pinned Arena] max space (MB) used spread across MPI: [0 ... 0] AMReX (26.05-3-g4ceb814db7d7) finalized