Initializing AMReX (26.06-13-g87eac2327e9f)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.06-13-g87eac2327e9f) initialized REMORA git hash: v1.5-149-g3b71c66 AMReX git hash: 26.06-13-g87eac2327e Making level 0 from scratch GRIDS AT LEVEL 0 ARE (BoxArray maxbox(4) m_ref->m_hash_sig(0) ((0,0,0) (159,159,63) (0,0,0)) ((160,0,0) (319,159,63) (0,0,0)) ((0,160,0) (159,319,63) (0,0,0)) ((160,160,0) (319,319,63) (0,0,0)) ) Writing plotfile Seamount64-xy_plt00000 Coarse STEP 1 starts ... Coarse STEP 1 ends. TIME = 60 DT = 60 Coarse STEP 2 starts ... Coarse STEP 2 ends. TIME = 120 DT = 60 Coarse STEP 3 starts ... Coarse STEP 3 ends. TIME = 180 DT = 60 Coarse STEP 4 starts ... Coarse STEP 4 ends. TIME = 240 DT = 60 Coarse STEP 5 starts ... Coarse STEP 5 ends. TIME = 300 DT = 60 Coarse STEP 6 starts ... Coarse STEP 6 ends. TIME = 360 DT = 60 Coarse STEP 7 starts ... Coarse STEP 7 ends. TIME = 420 DT = 60 Coarse STEP 8 starts ... Coarse STEP 8 ends. TIME = 480 DT = 60 Coarse STEP 9 starts ... Coarse STEP 9 ends. TIME = 540 DT = 60 Coarse STEP 10 starts ... Coarse STEP 10 ends. TIME = 600 DT = 60 Writing plotfile Seamount64-xy_plt00010 TinyProfiler total time across processes [min...avg...max]: 7.575 ... 7.598 ... 7.622 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- FillBoundary_finish() 3935 1.568 1.623 1.682 22.07% FillBoundary_nowait() 3935 1.439 1.504 1.554 20.39% REMORA::advance2d() 590 1.451 1.46 1.471 19.30% REMORAPhysBCFunct::() 1896 0.2185 0.3051 0.3693 4.84% VisMF::Write(FabArray) 4 0.3262 0.3479 0.3652 4.79% REMORA::uv3dmix() 590 0.3243 0.3311 0.3368 4.42% REMORA::rhs_uv_2d() 580 0.2648 0.2739 0.285 3.74% REMORA::vert_visc_3d() 50 0.2688 0.2785 0.283 3.71% REMORA::advance_3d() 10 0.1959 0.1995 0.2035 2.67% REMORA::prestep() 10 0.1508 0.1578 0.1633 2.14% REMORA::t3dmix2_s() 20 0.1516 0.1564 0.1622 2.13% REMORA::setup_step() 10 0.1408 0.1485 0.1563 2.05% REMORA::rhs_t_3d() 30 0.1361 0.1362 0.1363 1.79% REMORA::prestep_t_advection() 30 0.1039 0.1177 0.1346 1.77% REMORA::rhs_uv_3d() 10 0.1267 0.1289 0.1308 1.72% FabArray::setVal() 171 0.0779 0.08195 0.08426 1.11% REMORA::estTimeStep() 11 0.03812 0.05123 0.07385 0.97% REMORA::Evolve() 1 0.03104 0.05327 0.07334 0.96% REMORA::InitData() 2 0.04464 0.0508 0.05783 0.76% amrex::Copy() 52 0.03691 0.0372 0.03766 0.49% REMORA::stretch_transform() 11 0.03213 0.0335 0.03419 0.45% REMORA::update_massflux_3d() 20 0.02132 0.02273 0.02417 0.32% REMORA::prsgrd() 10 0.007172 0.01322 0.02272 0.30% MultiFab::contains_nan() 10 0.01353 0.01409 0.01511 0.20% main() 1 0.0001344 0.00354 0.01358 0.18% MultiFab::contains_inf() 10 0.01048 0.01049 0.01049 0.14% REMORA::set_analytic_vmix() 11 0.008343 0.008781 0.009098 0.12% REMORA::advance_3d_ml() 10 0.005795 0.0072 0.00861 0.11% WriteMultiLevelPlotfileWithBathymetry() 2 0.001192 0.00219 0.004466 0.06% REMORA::init_only() 1 0.003588 0.004007 0.004434 0.06% REMORA::set_zeta_to_Ztavg() 10 0.003633 0.003661 0.003691 0.05% REMORA::FillPatch() 1866 0.0035 0.003552 0.003597 0.05% FillPatchSingleLevel 1998 0.001993 0.002083 0.002201 0.03% FabArray::FillBoundary() 3313 0.001821 0.001871 0.001914 0.03% FabArray::EnforcePeriodicity 619 0.0003328 0.0003405 0.0003505 0.00% REMORA::Advance() 10 0.0002828 0.0002998 0.0003097 0.00% REMORA::FillPatchNoBC() 132 0.0001316 0.0001327 0.000135 0.00% MultiFab::Copy() 39 1.443e-05 1.649e-05 1.907e-05 0.00% Other 4794 0.02017 0.02306 0.02762 0.36% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 7.575 7.598 7.622 100.00% REMORA::Evolve() 1 7.189 7.219 7.243 95.03% REMORA::Advance() 10 6.936 6.94 6.944 91.11% REMORA::advance2d() 590 4.769 4.77 4.772 62.61% FabArray::FillBoundary() 3313 2.676 2.705 2.733 35.85% REMORA::FillPatch() 1866 2.128 2.151 2.182 28.63% FillBoundary_finish() 3935 1.568 1.623 1.682 22.07% FillPatchSingleLevel 1998 1.587 1.625 1.666 21.86% FillBoundary_nowait() 3935 1.441 1.506 1.556 20.42% REMORA::setup_step() 10 1.147 1.154 1.158 15.20% REMORA::advance_3d_ml() 10 1.014 1.016 1.017 13.34% REMORA::advance_3d() 10 0.8852 0.8864 0.8877 11.65% FabArray::EnforcePeriodicity 619 0.405 0.425 0.449 5.89% REMORAPhysBCFunct::() 1896 0.2298 0.3173 0.3809 5.00% REMORA::InitData() 2 0.3719 0.3748 0.3787 4.97% WriteMultiLevelPlotfileWithBathymetry() 2 0.3318 0.3521 0.3695 4.85% VisMF::Write(FabArray) 4 0.3262 0.3479 0.3652 4.79% REMORA::uv3dmix() 590 0.3243 0.3311 0.3368 4.42% REMORA::prestep() 10 0.2919 0.3 0.3098 4.06% REMORA::rhs_uv_2d() 580 0.2648 0.2739 0.285 3.74% REMORA::vert_visc_3d() 50 0.2688 0.2785 0.283 3.71% REMORA::FillPatchNoBC() 132 0.1936 0.2005 0.2074 2.72% REMORA::t3dmix2_s() 20 0.1516 0.1564 0.1622 2.13% REMORA::rhs_t_3d() 30 0.1371 0.1372 0.1372 1.80% REMORA::prestep_t_advection() 30 0.1039 0.1177 0.1346 1.77% REMORA::rhs_uv_3d() 10 0.1267 0.1289 0.1308 1.72% REMORA::stretch_transform() 11 0.09672 0.0973 0.09794 1.29% FabArray::setVal() 171 0.0779 0.08195 0.08426 1.11% REMORA::set_analytic_vmix() 11 0.07781 0.07781 0.07782 1.02% REMORA::estTimeStep() 11 0.03812 0.05123 0.07385 0.97% REMORA::init_only() 1 0.05902 0.05934 0.05945 0.78% amrex::Copy() 52 0.03691 0.0372 0.03766 0.49% MultiFab::Copy() 39 0.02889 0.02916 0.02936 0.39% REMORA::update_massflux_3d() 20 0.02132 0.02273 0.02417 0.32% REMORA::prsgrd() 10 0.007172 0.01322 0.02272 0.30% MultiFab::contains_nan() 10 0.01353 0.01409 0.01511 0.20% REMORA::set_zeta_to_Ztavg() 10 0.009058 0.009873 0.01087 0.14% MultiFab::contains_inf() 10 0.01048 0.01049 0.01049 0.14% Other 4794 0.05677 0.0628 0.06989 0.92% ------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::amr.checkpoint_files_output(nvals = 1) :: [0] Device Memory Usage: ---------------------------------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max CurrentMem max ---------------------------------------------------------------------------------------------------------------------------------------------- The_Arena::Initialize() 4 4 97 B 227 B 571 B 1024 MiB 1024 MiB 1024 MiB 0 B REMORA::InitData() 352 320 624 MiB 625 MiB 627 MiB 786 MiB 786 MiB 786 MiB 174 MiB REMORA::Evolve() 12 12 3063 KiB 3612 KiB 4043 KiB 141 MiB 141 MiB 141 MiB 0 B REMORA::advance_3d() 760 760 8593 KiB 8631 KiB 8658 KiB 107 MiB 107 MiB 107 MiB 0 B REMORA::prestep_t_advection() 120 120 2347 KiB 2491 KiB 2666 KiB 105 MiB 105 MiB 105 MiB 0 B REMORA::prestep() 160 160 2876 KiB 2959 KiB 3059 KiB 79 MiB 79 MiB 79 MiB 0 B REMORA::prsgrd() 40 40 935 KiB 1092 KiB 1155 KiB 78 MiB 78 MiB 78 MiB 0 B REMORA::setup_step() 240 240 7668 KiB 7699 KiB 7720 KiB 67 MiB 67 MiB 67 MiB 0 B REMORA::rhs_t_3d() 480 480 1000 KiB 1053 KiB 1107 KiB 65 MiB 65 MiB 65 MiB 0 B REMORA::t3dmix2_s() 160 160 755 KiB 782 KiB 810 KiB 62 MiB 62 MiB 62 MiB 0 B WriteMultiLevelPlotfileWithBathymetry() 8 8 2679 KiB 2856 KiB 2999 KiB 62 MiB 62 MiB 62 MiB 0 B REMORA::uv3dmix() 2360 2360 754 KiB 815 KiB 838 KiB 52 MiB 52 MiB 52 MiB 0 B REMORA::rhs_uv_3d() 160 160 741 KiB 771 KiB 817 KiB 50 MiB 50 MiB 50 MiB 0 B REMORA::estTimeStep() 132 132 185 KiB 249 KiB 360 KiB 37 MiB 37 MiB 37 MiB 0 B ResizeRandomSeed 4 4 9202 KiB 9204 KiB 9206 KiB 9216 KiB 9216 KiB 9216 KiB 0 B REMORA::advance2d() 9440 9440 1690 KiB 1691 KiB 1691 KiB 3222 KiB 3222 KiB 3222 KiB 0 B REMORA::rhs_uv_2d() 2320 2320 61 KiB 62 KiB 63 KiB 840 KiB 840 KiB 840 KiB 0 B REMORA::update_massflux_3d() 80 80 1611 B 1678 B 1750 B 420 KiB 420 KiB 420 KiB 0 B VisMF::Write(FabArray) 64 64 17 B 95 B 191 B 96 KiB 96 KiB 96 KiB 0 B FillBoundary_nowait() 4936 4892 47 KiB 47 KiB 48 KiB 51 KiB 52 KiB 52 KiB 13 KiB FillBoundary_finish() 4936 4892 47 KiB 47 KiB 48 KiB 51 KiB 52 KiB 52 KiB 13 KiB MultiFab::contains_nan() 52 48 262 B 263 B 265 B 6656 B 6656 B 6656 B 256 B MultiFab::contains_inf() 40 40 6 B 6 B 7 B 6144 B 6144 B 6144 B 0 B impose_cons_bcs() 88 88 0 B 0 B 0 B 256 B 256 B 256 B 0 B impose_xvel_bcs() 44 44 0 B 0 B 0 B 256 B 256 B 256 B 0 B impose_yvel_bcs() 44 44 0 B 0 B 0 B 256 B 256 B 256 B 0 B ---------------------------------------------------------------------------------------------------------------------------------------------- Async Memory Usage: ------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ------------------------------------------------------------------------------------------------------------- REMORA::prsgrd() 40 73 KiB 135 KiB 232 KiB 78 MiB 78 MiB 78 MiB REMORA::advance_3d() 600 2898 KiB 2973 KiB 3013 KiB 53 MiB 53 MiB 53 MiB REMORA::prestep_t_advection() 120 710 KiB 804 KiB 919 KiB 52 MiB 52 MiB 52 MiB REMORA::rhs_t_3d() 480 426 KiB 437 KiB 448 KiB 52 MiB 52 MiB 52 MiB REMORA::uv3dmix() 2360 106 KiB 146 KiB 198 KiB 52 MiB 52 MiB 52 MiB REMORA::rhs_uv_3d() 160 660 KiB 664 KiB 673 KiB 50 MiB 50 MiB 50 MiB REMORA::t3dmix2_s() 160 485 KiB 513 KiB 559 KiB 50 MiB 50 MiB 50 MiB REMORA::setup_step() 80 561 KiB 573 KiB 591 KiB 13 MiB 13 MiB 13 MiB REMORA::prestep() 40 191 KiB 214 KiB 243 KiB 13 MiB 13 MiB 13 MiB REMORA::rhs_uv_2d() 2320 28 KiB 29 KiB 30 KiB 840 KiB 840 KiB 840 KiB REMORA::advance2d() 2360 6044 B 6075 B 6105 B 215 KiB 215 KiB 215 KiB REMORA::update_massflux_3d() 80 582 B 620 B 660 B 210 KiB 210 KiB 210 KiB ------------------------------------------------------------------------------------------------------------- Managed Memory Usage: --------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------- The_Managed_Arena::Initialize() 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------- Pinned Memory Usage: ------------------------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max CurrentMem max ------------------------------------------------------------------------------------------------------------------------------------- VisMF::Write(FabArray) 80 80 1743 KiB 1815 KiB 1941 KiB 62 MiB 62 MiB 62 MiB 0 B The_Pinned_Arena::Initialize() 4 4 0 B 1 B 2 B 8192 KiB 8192 KiB 8192 KiB 0 B FillBoundary_nowait() 36400 36356 279 KiB 285 KiB 299 KiB 5917 KiB 5917 KiB 5918 KiB 13 KiB FillBoundary_finish() 4936 4892 47 KiB 47 KiB 48 KiB 51 KiB 52 KiB 52 KiB 13 KiB MultiFab::contains_nan() 52 48 254 B 255 B 257 B 768 B 768 B 768 B 256 B REMORA::estTimeStep() 88 88 1 B 1 B 2 B 512 B 512 B 512 B 0 B MultiFab::contains_inf() 40 40 0 B 0 B 0 B 256 B 256 B 256 B 0 B ------------------------------------------------------------------------------------------------------------------------------------- Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [3315 ... 3315] [The Arena] max space (MB) allocated spread across MPI: [1024 ... 1024] [The Arena] max space (MB) used spread across MPI: [891 ... 891] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [133 ... 133] [The Pinned Arena] max space (MB) used spread across MPI: [62 ... 62] AMReX (26.06-13-g87eac2327e9f) finalized