Initializing AMReX (26.06-11-g9ed35ab0e584)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.06-11-g9ed35ab0e584) initialized REMORA git hash: v1.5-149-g3b71c66 AMReX git hash: 26.06-11-g9ed35ab0e5 Making level 0 from scratch GRIDS AT LEVEL 0 ARE (BoxArray maxbox(4) m_ref->m_hash_sig(0) ((0,0,0) (159,159,63) (0,0,0)) ((160,0,0) (319,159,63) (0,0,0)) ((0,160,0) (159,319,63) (0,0,0)) ((160,160,0) (319,319,63) (0,0,0)) ) Writing plotfile Seamount64-xy_plt00000 Coarse STEP 1 starts ... Coarse STEP 1 ends. TIME = 60 DT = 60 Coarse STEP 2 starts ... Coarse STEP 2 ends. TIME = 120 DT = 60 Coarse STEP 3 starts ... Coarse STEP 3 ends. TIME = 180 DT = 60 Coarse STEP 4 starts ... Coarse STEP 4 ends. TIME = 240 DT = 60 Coarse STEP 5 starts ... Coarse STEP 5 ends. TIME = 300 DT = 60 Coarse STEP 6 starts ... Coarse STEP 6 ends. TIME = 360 DT = 60 Coarse STEP 7 starts ... Coarse STEP 7 ends. TIME = 420 DT = 60 Coarse STEP 8 starts ... Coarse STEP 8 ends. TIME = 480 DT = 60 Coarse STEP 9 starts ... Coarse STEP 9 ends. TIME = 540 DT = 60 Coarse STEP 10 starts ... Coarse STEP 10 ends. TIME = 600 DT = 60 Writing plotfile Seamount64-xy_plt00010 TinyProfiler total time across processes [min...avg...max]: 7.661 ... 7.681 ... 7.695 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- FillBoundary_finish() 3935 1.547 1.622 1.706 22.17% FillBoundary_nowait() 3935 1.427 1.507 1.613 20.97% REMORA::advance2d() 590 1.447 1.461 1.475 19.17% VisMF::Write(FabArray) 4 0.4059 0.4377 0.4516 5.87% REMORAPhysBCFunct::() 1896 0.2477 0.3066 0.3822 4.97% REMORA::uv3dmix() 590 0.3263 0.3341 0.3383 4.40% REMORA::rhs_uv_2d() 580 0.2596 0.2732 0.2873 3.73% REMORA::vert_visc_3d() 50 0.268 0.2791 0.2839 3.69% REMORA::advance_3d() 10 0.1916 0.1995 0.2096 2.72% REMORA::prestep() 10 0.1507 0.1578 0.1629 2.12% REMORA::t3dmix2_s() 20 0.1493 0.1535 0.1614 2.10% REMORA::setup_step() 10 0.13 0.1439 0.1562 2.03% REMORA::rhs_t_3d() 30 0.1364 0.1364 0.1364 1.77% REMORA::prestep_t_advection() 30 0.1054 0.1177 0.136 1.77% REMORA::rhs_uv_3d() 10 0.1261 0.1287 0.1315 1.71% FabArray::setVal() 171 0.07968 0.08176 0.08299 1.08% REMORA::estTimeStep() 11 0.03756 0.05045 0.07128 0.93% REMORA::InitData() 2 0.04538 0.05146 0.06035 0.78% REMORA::Evolve() 1 0.03112 0.03861 0.04459 0.58% amrex::Copy() 52 0.03542 0.03686 0.0389 0.51% REMORA::stretch_transform() 11 0.03303 0.03358 0.03423 0.44% REMORA::update_massflux_3d() 20 0.01941 0.02275 0.02606 0.34% REMORA::prsgrd() 10 0.008026 0.01565 0.02406 0.31% MultiFab::contains_nan() 10 0.01184 0.01322 0.0143 0.19% DistributionMapping::LeastUsedCPUs() 1 1.401e-05 0.008012 0.01234 0.16% MultiFab::contains_inf() 10 0.009827 0.009828 0.009829 0.13% REMORA::set_analytic_vmix() 11 0.008586 0.00878 0.009009 0.12% REMORA::advance_3d_ml() 10 0.005509 0.007195 0.008879 0.12% main() 1 0.0001316 0.002133 0.008136 0.11% WriteMultiLevelPlotfileWithBathymetry() 2 0.001183 0.002236 0.004622 0.06% REMORA::init_only() 1 0.00344 0.003892 0.004444 0.06% REMORA::FillPatch() 1866 0.003545 0.003643 0.003837 0.05% REMORA::set_zeta_to_Ztavg() 10 0.003631 0.003665 0.003688 0.05% FillPatchSingleLevel 1998 0.002117 0.00228 0.002482 0.03% FabArray::FillBoundary() 3313 0.001873 0.001942 0.002045 0.03% FabArray::EnforcePeriodicity 619 0.0003284 0.0003447 0.0003644 0.00% REMORA::Advance() 10 0.0003082 0.000317 0.0003297 0.00% REMORA::FillPatchNoBC() 132 0.0001235 0.0001254 0.0001277 0.00% MultiFab::Copy() 39 1.36e-05 1.479e-05 1.619e-05 0.00% AmrMesh::MakeDistributionMap() 1 1.686e-06 2.913e-06 4.326e-06 0.00% DistributionMapping::SFCProcessorMapDoIt() 1 2.144e-06 2.7e-06 3.269e-06 0.00% Other 4791 0.01967 0.02366 0.0305 0.40% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 7.661 7.681 7.695 100.00% REMORA::Evolve() 1 7.19 7.208 7.217 93.78% REMORA::Advance() 10 6.942 6.944 6.948 90.29% REMORA::advance2d() 590 4.775 4.776 4.776 62.07% FabArray::FillBoundary() 3313 2.681 2.708 2.737 35.57% REMORA::FillPatch() 1866 2.133 2.157 2.193 28.49% FillBoundary_finish() 3935 1.547 1.622 1.706 22.17% FillPatchSingleLevel 1998 1.589 1.628 1.648 21.42% FillBoundary_nowait() 3935 1.429 1.509 1.615 20.99% REMORA::setup_step() 10 1.149 1.153 1.158 15.05% REMORA::advance_3d_ml() 10 1.012 1.014 1.016 13.20% REMORA::advance_3d() 10 0.8846 0.8856 0.888 11.54% REMORA::InitData() 2 0.4634 0.4703 0.4782 6.21% FabArray::EnforcePeriodicity 619 0.3882 0.4246 0.457 5.94% WriteMultiLevelPlotfileWithBathymetry() 2 0.4097 0.4418 0.4557 5.92% VisMF::Write(FabArray) 4 0.4059 0.4377 0.4516 5.87% REMORAPhysBCFunct::() 1896 0.26 0.3188 0.3961 5.15% REMORA::uv3dmix() 590 0.3263 0.3341 0.3383 4.40% REMORA::prestep() 10 0.293 0.3 0.3108 4.04% REMORA::rhs_uv_2d() 580 0.2596 0.2732 0.2873 3.73% REMORA::vert_visc_3d() 50 0.268 0.2791 0.2839 3.69% REMORA::FillPatchNoBC() 132 0.1939 0.1994 0.2066 2.69% REMORA::t3dmix2_s() 20 0.1493 0.1535 0.1614 2.10% REMORA::rhs_t_3d() 30 0.1371 0.1372 0.1372 1.78% REMORA::prestep_t_advection() 30 0.1054 0.1177 0.136 1.77% REMORA::rhs_uv_3d() 10 0.1261 0.1287 0.1315 1.71% REMORA::stretch_transform() 11 0.09574 0.09709 0.09807 1.27% FabArray::setVal() 171 0.07968 0.08176 0.08299 1.08% REMORA::set_analytic_vmix() 11 0.07787 0.07787 0.07787 1.01% REMORA::estTimeStep() 11 0.03756 0.05045 0.07128 0.93% REMORA::init_only() 1 0.05926 0.05926 0.05927 0.77% amrex::Copy() 52 0.03542 0.03686 0.0389 0.51% MultiFab::Copy() 39 0.02839 0.02904 0.02964 0.39% REMORA::update_massflux_3d() 20 0.01941 0.02275 0.02606 0.34% REMORA::prsgrd() 10 0.008026 0.01565 0.02406 0.31% MultiFab::contains_nan() 10 0.01184 0.01322 0.0143 0.19% AmrMesh::MakeDistributionMap() 1 2.02e-05 0.008019 0.01235 0.16% DistributionMapping::SFCProcessorMapDoIt() 1 1.83e-05 0.008016 0.01234 0.16% DistributionMapping::LeastUsedCPUs() 1 1.401e-05 0.008012 0.01234 0.16% REMORA::set_zeta_to_Ztavg() 10 0.009144 0.009873 0.01096 0.14% MultiFab::contains_inf() 10 0.009827 0.009828 0.009829 0.13% Other 4791 0.05662 0.0637 0.07649 0.99% ------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::amr.checkpoint_files_output(nvals = 1) :: [0] Device Memory Usage: ---------------------------------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max CurrentMem max ---------------------------------------------------------------------------------------------------------------------------------------------- The_Arena::Initialize() 4 4 78 B 98 B 110 B 1024 MiB 1024 MiB 1024 MiB 0 B REMORA::InitData() 352 320 625 MiB 627 MiB 628 MiB 786 MiB 786 MiB 786 MiB 174 MiB REMORA::Evolve() 12 12 2984 KiB 3314 KiB 3480 KiB 141 MiB 141 MiB 141 MiB 0 B REMORA::advance_3d() 760 760 8512 KiB 8554 KiB 8597 KiB 107 MiB 107 MiB 107 MiB 0 B REMORA::prestep_t_advection() 120 120 2342 KiB 2469 KiB 2660 KiB 105 MiB 105 MiB 105 MiB 0 B REMORA::prestep() 160 160 2861 KiB 2933 KiB 3043 KiB 79 MiB 79 MiB 79 MiB 0 B REMORA::prsgrd() 40 40 846 KiB 1055 KiB 1149 KiB 78 MiB 78 MiB 78 MiB 0 B REMORA::setup_step() 240 240 7589 KiB 7626 KiB 7648 KiB 67 MiB 67 MiB 67 MiB 0 B REMORA::rhs_t_3d() 480 480 922 KiB 1044 KiB 1166 KiB 65 MiB 65 MiB 65 MiB 0 B REMORA::t3dmix2_s() 160 160 749 KiB 773 KiB 797 KiB 62 MiB 62 MiB 62 MiB 0 B WriteMultiLevelPlotfileWithBathymetry() 8 8 3297 KiB 3554 KiB 3671 KiB 62 MiB 62 MiB 62 MiB 0 B REMORA::uv3dmix() 2360 2360 712 KiB 797 KiB 832 KiB 52 MiB 52 MiB 52 MiB 0 B REMORA::rhs_uv_3d() 160 160 737 KiB 784 KiB 822 KiB 50 MiB 50 MiB 50 MiB 0 B REMORA::estTimeStep() 132 132 181 KiB 243 KiB 344 KiB 37 MiB 37 MiB 37 MiB 0 B ResizeRandomSeed 4 4 9203 KiB 9205 KiB 9207 KiB 9216 KiB 9216 KiB 9216 KiB 0 B REMORA::advance2d() 9440 9440 1677 KiB 1678 KiB 1678 KiB 3222 KiB 3222 KiB 3222 KiB 0 B REMORA::rhs_uv_2d() 2320 2320 60 KiB 61 KiB 62 KiB 840 KiB 840 KiB 840 KiB 0 B REMORA::update_massflux_3d() 80 80 1508 B 1669 B 1828 B 420 KiB 420 KiB 420 KiB 0 B VisMF::Write(FabArray) 64 64 19 B 92 B 136 B 96 KiB 96 KiB 96 KiB 0 B FillBoundary_nowait() 4936 4892 47 KiB 47 KiB 48 KiB 51 KiB 52 KiB 52 KiB 13 KiB FillBoundary_finish() 4936 4892 47 KiB 47 KiB 48 KiB 51 KiB 52 KiB 52 KiB 13 KiB MultiFab::contains_nan() 52 48 265 B 265 B 266 B 6656 B 6656 B 6656 B 256 B MultiFab::contains_inf() 40 40 6 B 6 B 7 B 6144 B 6144 B 6144 B 0 B impose_cons_bcs() 88 88 0 B 0 B 0 B 256 B 256 B 256 B 0 B impose_xvel_bcs() 44 44 0 B 0 B 0 B 256 B 256 B 256 B 0 B impose_yvel_bcs() 44 44 0 B 0 B 0 B 256 B 256 B 256 B 0 B ---------------------------------------------------------------------------------------------------------------------------------------------- Async Memory Usage: ------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ------------------------------------------------------------------------------------------------------------- REMORA::prsgrd() 40 81 KiB 158 KiB 244 KiB 78 MiB 78 MiB 78 MiB REMORA::advance_3d() 600 2871 KiB 2946 KiB 2988 KiB 53 MiB 53 MiB 53 MiB REMORA::prestep_t_advection() 120 714 KiB 797 KiB 921 KiB 52 MiB 52 MiB 52 MiB REMORA::rhs_t_3d() 480 409 KiB 432 KiB 456 KiB 52 MiB 52 MiB 52 MiB REMORA::uv3dmix() 2360 109 KiB 167 KiB 206 KiB 52 MiB 52 MiB 52 MiB REMORA::rhs_uv_3d() 160 649 KiB 656 KiB 667 KiB 50 MiB 50 MiB 50 MiB REMORA::t3dmix2_s() 160 477 KiB 513 KiB 561 KiB 50 MiB 50 MiB 50 MiB REMORA::setup_step() 80 559 KiB 572 KiB 589 KiB 13 MiB 13 MiB 13 MiB REMORA::prestep() 40 191 KiB 212 KiB 243 KiB 13 MiB 13 MiB 13 MiB REMORA::rhs_uv_2d() 2320 27 KiB 28 KiB 30 KiB 840 KiB 840 KiB 840 KiB REMORA::advance2d() 2360 5986 B 6032 B 6080 B 215 KiB 215 KiB 215 KiB REMORA::update_massflux_3d() 80 525 B 615 B 705 B 210 KiB 210 KiB 210 KiB ------------------------------------------------------------------------------------------------------------- Managed Memory Usage: --------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------- The_Managed_Arena::Initialize() 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------- Pinned Memory Usage: ------------------------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max CurrentMem max ------------------------------------------------------------------------------------------------------------------------------------- VisMF::Write(FabArray) 80 80 1600 KiB 1734 KiB 1790 KiB 62 MiB 62 MiB 62 MiB 0 B The_Pinned_Arena::Initialize() 4 4 1 B 1 B 1 B 8192 KiB 8192 KiB 8192 KiB 0 B FillBoundary_nowait() 36400 36356 271 KiB 283 KiB 308 KiB 5917 KiB 5917 KiB 5918 KiB 13 KiB FillBoundary_finish() 4936 4892 47 KiB 47 KiB 48 KiB 51 KiB 52 KiB 52 KiB 13 KiB MultiFab::contains_nan() 52 48 257 B 257 B 258 B 768 B 768 B 768 B 256 B REMORA::estTimeStep() 88 88 1 B 1 B 2 B 512 B 512 B 512 B 0 B MultiFab::contains_inf() 40 40 0 B 0 B 0 B 256 B 256 B 256 B 0 B ------------------------------------------------------------------------------------------------------------------------------------- Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [3315 ... 3315] [The Arena] max space (MB) allocated spread across MPI: [1024 ... 1024] [The Arena] max space (MB) used spread across MPI: [891 ... 891] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [133 ... 133] [The Pinned Arena] max space (MB) used spread across MPI: [62 ... 62] AMReX (26.06-11-g9ed35ab0e584) finalized