Initializing AMReX (26.06-26-ge95d50b2b424)... MPI initialized with 1 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.06-26-ge95d50b2b424) initialized REMORA git hash: v1.6-6-g21efc25 AMReX git hash: 26.06-26-ge95d50b2b4 Making level 0 from scratch GRIDS AT LEVEL 0 ARE (BoxArray maxbox(1) m_ref->m_hash_sig(0) ((0,0,0) (9,15,19) (0,0,0)) ) Calling init_masks_from_netcdf level 0 Loading masks from NetCDF file idRiv_grd_masked_v0.2_classic64.nc Masks loaded from netcdf file Calling init_bdry_from_netcdf at level 0 DOING INIT AT LEVEL 0 Setting up boundary data for u coming from NetCDF file Setting up boundary data for v coming from NetCDF file Setting up boundary data for temp coming from NetCDF file Setting up boundary data for salt coming from NetCDF file Setting up boundary data for ubar coming from NetCDF file Setting up boundary data for vbar coming from NetCDF file Setting up boundary data for zeta coming from NetCDF file Boundary data loaded from netcdf file Loading river_salt from rivers NetCDF file(s) Loading river_temp from rivers NetCDF file(s) Loading river_transport from rivers NetCDF file(s) Reading in river_Vshape from idRiv_riv_v0.2_classic64.nc Loading river_transport from rivers NetCDF file(s) Reading river_Xposition from NetCDF file Reading river_Eposition from NetCDF file Reading river_direction from NetCDF file Calling init_bathymetry_from_netcdf Loading initial bathymetry from NetCDF file idRiv_grd_masked_v0.2_classic64.nc Bathymetry loaded from netcdf file Calling init_grid_vars_from_netcdf Loading grid variables from NetCDF file idRiv_grd_masked_v0.2_classic64.nc Grid variables loaded from netcdf file Loading grid variables from NetCDF file idRiv_grd_masked_v0.2_classic64.nc Calling init_zeta_from_netcdf on level 0 Loading initial sea surface height from NetCDF file idRiv_ini_consTS_v0.2_classic64.nc Reading in zeta at (before) time 0 Actually reading in zeta at time 0 DONE reading in zeta at time 0 Reading in zeta at (after) time 1 Actually reading in zeta at time 86400 DONE reading in zeta at time 86400 Sea surface height loaded from netcdf file Calling init_data_from_netcdf Loading initial solution data from NetCDF file idRiv_ini_consTS_v0.2_classic64.nc Initial data loaded from netcdf file Reading in ubar at (before) time 0 Actually reading in ubar at time 0 DONE reading in ubar at time 0 Reading in ubar at (after) time 1 Actually reading in ubar at time 86400 DONE reading in ubar at time 86400 Reading in vbar at (before) time 0 Actually reading in vbar at time 0 DONE reading in vbar at time 0 Reading in vbar at (after) time 1 Actually reading in vbar at time 86400 DONE reading in vbar at time 86400 Calling init_coriolis_from_netcdf Loading initial coriolis from NetCDF file idRiv_grd_masked_v0.2_classic64.nc Coriolis loaded from netcdf file Reading in temp at (before) time 0 Actually reading in temp at time 0 DONE reading in temp at time 0 Reading in temp at (after) time 1 Actually reading in temp at time 86400 DONE reading in temp at time 86400 Reading in salt at (before) time 0 Actually reading in salt at time 0 DONE reading in salt at time 0 Reading in salt at (after) time 1 Actually reading in salt at time 86400 DONE reading in salt at time 86400 Reading in u at (before) time 0 Actually reading in u at time 0 DONE reading in u at time 0 Reading in u at (after) time 1 Actually reading in u at time 86400 DONE reading in u at time 86400 Reading in v at (before) time 0 Actually reading in v at time 0 DONE reading in v at time 0 Reading in v at (after) time 1 Actually reading in v at time 86400 DONE reading in v at time 86400 Writing plotfile IdealRivGrid-1grid_plt00000 Coarse STEP 1 starts ... Reading in river_temp at time index 1 from idRiv_riv_v0.2_classic64.nc Reading in river_temp at time index 0 from idRiv_riv_v0.2_classic64.nc Reading in river_salt at time index 1 from idRiv_riv_v0.2_classic64.nc Reading in river_salt at time index 0 from idRiv_riv_v0.2_classic64.nc Reading in river_transport at time index 1 from idRiv_riv_v0.2_classic64.nc Reading in river_transport at time index 0 from idRiv_riv_v0.2_classic64.nc Reading in river_transport at time index 1 from idRiv_riv_v0.2_classic64.nc Reading in river_transport at time index 0 from idRiv_riv_v0.2_classic64.nc Coarse STEP 1 ends. TIME = 4000 DT = 4000 Coarse STEP 2 starts ... Coarse STEP 2 ends. TIME = 8000 DT = 4000 Coarse STEP 3 starts ... Coarse STEP 3 ends. TIME = 12000 DT = 4000 Coarse STEP 4 starts ... Coarse STEP 4 ends. TIME = 16000 DT = 4000 Coarse STEP 5 starts ... Coarse STEP 5 ends. TIME = 20000 DT = 4000 Coarse STEP 6 starts ... Coarse STEP 6 ends. TIME = 24000 DT = 4000 Coarse STEP 7 starts ... Coarse STEP 7 ends. TIME = 28000 DT = 4000 Coarse STEP 8 starts ... Coarse STEP 8 ends. TIME = 32000 DT = 4000 Coarse STEP 9 starts ... Coarse STEP 9 ends. TIME = 36000 DT = 4000 Coarse STEP 10 starts ... Coarse STEP 10 ends. TIME = 40000 DT = 4000 Writing plotfile IdealRivGrid-1grid_plt00010 TinyProfiler total time across processes [min...avg...max]: 0.2076 ... 0.2076 ... 0.2076 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- REMORA::FillPatch() 1028 0.06013 0.06013 0.06013 28.97% REMORA::advance2d() 310 0.03823 0.03823 0.03823 18.42% REMORA::init_only() 1 0.01336 0.01336 0.01336 6.44% impose_cons_bcs() 340 0.01091 0.01091 0.01091 5.25% REMORA::uv3dmix() 310 0.00885 0.00885 0.00885 4.26% REMORA::rhs_uv_2d() 300 0.007889 0.007889 0.007889 3.80% VisMF::Write(FabArray) 4 0.006901 0.006901 0.006901 3.32% impose_xvel_bcs() 322 0.006888 0.006888 0.006888 3.32% impose_yvel_bcs() 322 0.006833 0.006833 0.006833 3.29% REMORA::vert_visc_3d() 50 0.005051 0.005051 0.005051 2.43% REMORA::advance_3d() 10 0.00501 0.00501 0.00501 2.41% REMORAPhysBCFunct::() 1059 0.004347 0.004347 0.004347 2.09% REMORA::prestep_t_advection() 30 0.002526 0.002526 0.002526 1.22% REMORA::prestep() 10 0.002304 0.002304 0.002304 1.11% REMORA::t3dmix2_s() 20 0.002081 0.002081 0.002081 1.00% REMORA::setup_step() 10 0.001684 0.001684 0.001684 0.81% WriteMultiLevelPlotfileWithBathymetry() 2 0.001678 0.001678 0.001678 0.81% REMORA::set_zeta() 1 0.001645 0.001645 0.001645 0.79% REMORA::bathymetry() 1 0.001618 0.001618 0.001618 0.78% REMORA::rhs_uv_3d() 10 0.001514 0.001514 0.001514 0.73% REMORA::ReadParameters() 1 0.001486 0.001486 0.001486 0.72% REMORA::InitData() 2 0.001461 0.001461 0.001461 0.70% FabArray::setVal() 171 0.001246 0.001246 0.001246 0.60% REMORA::rhs_t_3d() 30 0.001245 0.001245 0.001245 0.60% REMORA::coriolis() 310 0.001165 0.001165 0.001165 0.56% FillPatchSingleLevel 1760 0.00109 0.00109 0.00109 0.53% REMORA::Evolve() 1 0.001002 0.001002 0.001002 0.48% REMORA::rhs_t_3d()::hadv 30 0.0008559 0.0008559 0.0008559 0.41% FabArray::FillBoundary() 2536 0.0008463 0.0008463 0.0008463 0.41% REMORA::update_massflux_3d() 20 0.0007648 0.0007648 0.0007648 0.37% REMORA::estTimeStep() 11 0.0007431 0.0007431 0.0007431 0.36% REMORA::prsgrd() 10 0.0006107 0.0006107 0.0006107 0.29% FabArrayBase::getFB() 2539 0.0005468 0.0005468 0.0005468 0.26% REMORA::stretch_transform() 11 0.0004954 0.0004954 0.0004954 0.24% REMORA::prestep_diffusion() 50 0.0004764 0.0004764 0.0004764 0.23% FillBoundary_nowait() 2539 0.0004653 0.0004653 0.0004653 0.22% amrex::Copy() 52 0.0004381 0.0004381 0.0004381 0.21% main() 1 0.0003798 0.0003798 0.0003798 0.18% REMORA::set_coriolis() 1 0.0003139 0.0003139 0.0003139 0.15% REMORA::FillPatchNoBC() 732 0.0002757 0.0002757 0.0002757 0.13% FillBoundary_finish() 2539 0.0002705 0.0002705 0.0002705 0.13% REMORA::set_analytic_vmix() 11 0.0001663 0.0001663 0.0001663 0.08% REMORA::advance_3d_ml() 10 0.0001307 0.0001307 0.0001307 0.06% REMORA::Advance() 10 0.0001154 0.0001154 0.0001154 0.06% REMORA::REMORA() 1 3.624e-05 3.624e-05 3.624e-05 0.02% MultiFab::Copy() 39 1.053e-05 1.053e-05 1.053e-05 0.01% REMORA::init_set_vmix() 1 9.33e-07 9.33e-07 9.33e-07 0.00% Other 511 0.001491 0.001491 0.001491 0.72% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 0.2076 0.2076 0.2076 100.00% REMORA::Evolve() 1 0.1702 0.1702 0.1702 82.01% REMORA::Advance() 10 0.1686 0.1686 0.1686 81.23% REMORA::advance2d() 310 0.1332 0.1332 0.1332 64.17% REMORA::FillPatch() 1028 0.09016 0.09016 0.09016 43.43% REMORA::InitData() 2 0.03238 0.03238 0.03238 15.60% REMORAPhysBCFunct::() 1059 0.02898 0.02898 0.02898 13.96% REMORA::init_only() 1 0.02038 0.02038 0.02038 9.82% REMORA::advance_3d_ml() 10 0.01817 0.01817 0.01817 8.75% REMORA::advance_3d() 10 0.01773 0.01773 0.01773 8.54% REMORA::setup_step() 10 0.01713 0.01713 0.01713 8.25% impose_cons_bcs() 340 0.01091 0.01091 0.01091 5.25% REMORA::uv3dmix() 310 0.00885 0.00885 0.00885 4.26% WriteMultiLevelPlotfileWithBathymetry() 2 0.008741 0.008741 0.008741 4.21% REMORA::rhs_uv_2d() 300 0.007889 0.007889 0.007889 3.80% VisMF::Write(FabArray) 4 0.006901 0.006901 0.006901 3.32% impose_xvel_bcs() 322 0.006888 0.006888 0.006888 3.32% impose_yvel_bcs() 322 0.006833 0.006833 0.006833 3.29% REMORA::prestep() 10 0.005546 0.005546 0.005546 2.67% REMORA::vert_visc_3d() 50 0.005051 0.005051 0.005051 2.43% FillPatchSingleLevel 1760 0.002655 0.002655 0.002655 1.28% REMORA::prestep_t_advection() 30 0.002526 0.002526 0.002526 1.22% REMORA::rhs_t_3d() 30 0.002262 0.002262 0.002262 1.09% FabArray::FillBoundary() 2536 0.002164 0.002164 0.002164 1.04% REMORA::t3dmix2_s() 20 0.002081 0.002081 0.002081 1.00% REMORA::bathymetry() 1 0.001996 0.001996 0.001996 0.96% REMORA::set_zeta() 1 0.001684 0.001684 0.001684 0.81% REMORA::REMORA() 1 0.001522 0.001522 0.001522 0.73% REMORA::rhs_uv_3d() 10 0.001514 0.001514 0.001514 0.73% REMORA::ReadParameters() 1 0.001486 0.001486 0.001486 0.72% FabArray::setVal() 171 0.001246 0.001246 0.001246 0.60% REMORA::FillPatchNoBC() 732 0.001227 0.001227 0.001227 0.59% REMORA::set_analytic_vmix() 11 0.001189 0.001189 0.001189 0.57% REMORA::coriolis() 310 0.001165 0.001165 0.001165 0.56% FillBoundary_nowait() 2539 0.001066 0.001066 0.001066 0.51% REMORA::rhs_t_3d()::hadv 30 0.0008559 0.0008559 0.0008559 0.41% REMORA::update_massflux_3d() 20 0.0007648 0.0007648 0.0007648 0.37% REMORA::estTimeStep() 11 0.0007431 0.0007431 0.0007431 0.36% REMORA::prsgrd() 10 0.0006107 0.0006107 0.0006107 0.29% FabArrayBase::getFB() 2539 0.000601 0.000601 0.000601 0.29% REMORA::stretch_transform() 11 0.0005335 0.0005335 0.0005335 0.26% REMORA::prestep_diffusion() 50 0.0004764 0.0004764 0.0004764 0.23% amrex::Copy() 52 0.0004381 0.0004381 0.0004381 0.21% REMORA::set_coriolis() 1 0.0003714 0.0003714 0.0003714 0.18% MultiFab::Copy() 39 0.0003122 0.0003122 0.0003122 0.15% FillBoundary_finish() 2539 0.0002705 0.0002705 0.0002705 0.13% REMORA::init_set_vmix() 1 0.0002567 0.0002567 0.0002567 0.12% Other 511 0.00187 0.00187 0.00187 0.90% ------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::amr.checkpoint_files_output(nvals = 1) :: [0] Device Memory Usage: -------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem MaxMem CurrentMem -------------------------------------------------------------------------------------- The_Arena::Initialize() 1 1 3003 B 1024 MiB 0 B ResizeRandomSeed 1 1 9078 KiB 9216 KiB 0 B REMORA::InitData() 89 81 1254 KiB 3057 KiB 846 KiB REMORA::advance_3d() 192 192 10 KiB 379 KiB 0 B main() 3 3 4039 B 362 KiB 0 B REMORA::prestep_t_advection() 30 30 1788 B 350 KiB 0 B REMORA::prestep() 44 44 3273 B 265 KiB 0 B REMORA::prsgrd() 10 10 1308 B 262 KiB 0 B REMORA::setup_step() 60 60 7664 B 241 KiB 0 B REMORA::rhs_t_3d() 120 120 949 B 218 KiB 0 B REMORA::uv3dmix() 310 310 813 B 175 KiB 0 B REMORA::t3dmix2_s() 40 40 354 B 135 KiB 0 B REMORA::init_only() 11 10 601 B 131 KiB 1024 B WriteMultiLevelPlotfileWithBathymetry() 2 2 2008 B 125 KiB 0 B REMORA::estTimeStep() 33 33 158 B 123 KiB 0 B REMORA::rhs_uv_3d() 40 40 411 B 116 KiB 0 B VisMF::Write(FabArray) 16 16 66 B 96 KiB 0 B REMORA::advance2d() 1242 1242 11 KiB 40 KiB 0 B REMORA::bathymetry() 21 21 19 B 17 KiB 0 B REMORA::rhs_uv_2d() 300 300 272 B 8960 B 0 B MultiFab::contains_nan() 13 12 113 B 6656 B 256 B MultiFab::contains_inf() 10 10 2 B 6144 B 0 B REMORA::update_massflux_3d() 20 20 8 B 4608 B 0 B REMORA::set_zeta() 1 1 5 B 1792 B 0 B REMORA::set_coriolis() 1 1 0 B 1792 B 0 B REMORAPhysBCFunct::() 75 75 0 B 256 B 0 B impose_cons_bcs() 340 340 6 B 256 B 0 B impose_xvel_bcs() 322 322 3 B 256 B 0 B impose_yvel_bcs() 322 322 3 B 256 B 0 B -------------------------------------------------------------------------------------- Async Memory Usage: --------------------------------------------------------- Name Nalloc AvgMem MaxMem --------------------------------------------------------- REMORA::prsgrd() 10 362 B 262 KiB REMORA::advance_3d() 150 3383 B 183 KiB REMORA::prestep_t_advection() 30 1005 B 175 KiB REMORA::rhs_t_3d() 120 702 B 175 KiB REMORA::uv3dmix() 310 379 B 175 KiB REMORA::rhs_uv_3d() 40 354 B 116 KiB REMORA::t3dmix2_s() 40 279 B 108 KiB REMORA::setup_step() 20 567 B 48 KiB REMORA::prestep() 10 493 B 45 KiB REMORA::rhs_uv_2d() 300 152 B 8960 B REMORA::advance2d() 310 22 B 2816 B REMORA::update_massflux_3d() 20 3 B 2240 B --------------------------------------------------------- Managed Memory Usage: --------------------------------------------------------- Name Nalloc AvgMem MaxMem --------------------------------------------------------- The_Managed_Arena::Initialize() 1 8 B 8192 KiB --------------------------------------------------------- Pinned Memory Usage: ---------------------------------------------------------------------------- Name Nalloc Nfree AvgMem MaxMem CurrentMem ---------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 1 1 16 B 8192 KiB 0 B REMORA::init_only() 104 92 61 KiB 169 KiB 9984 B VisMF::Write(FabArray) 20 20 43 B 125 KiB 0 B REMORA::bathymetry() 21 21 1 B 1792 B 0 B REMORA::set_coriolis() 1 1 0 B 1792 B 0 B REMORA::set_zeta() 1 1 0 B 1792 B 0 B REMORA::prestep() 4 4 0 B 1024 B 0 B MultiFab::contains_nan() 13 12 110 B 768 B 256 B REMORA::estTimeStep() 22 22 0 B 512 B 0 B MultiFab::contains_inf() 10 10 0 B 256 B 0 B REMORA::advance2d() 2 2 0 B 256 B 0 B REMORA::advance_3d() 2 2 0 B 256 B 0 B ---------------------------------------------------------------------------- Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [6789 ... 6789] [The Arena] max space (MB) allocated spread across MPI: [1024 ... 1024] [The Arena] max space (MB) used spread across MPI: [12 ... 12] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Pinned Arena] max space (MB) used spread across MPI: [0 ... 0] AMReX (26.06-26-ge95d50b2b424) finalized