Initializing AMReX (26.06-11-g9ed35ab0e584)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.06-11-g9ed35ab0e584) initialized REMORA git hash: v1.5-149-g3b71c66 AMReX git hash: 26.06-11-g9ed35ab0e5 Making level 0 from scratch GRIDS AT LEVEL 0 ARE (BoxArray maxbox(4) m_ref->m_hash_sig(0) ((0,0,0) (159,159,63) (0,0,0)) ((160,0,0) (319,159,63) (0,0,0)) ((0,160,0) (159,319,63) (0,0,0)) ((160,160,0) (319,319,63) (0,0,0)) ) Writing plotfile Seamount64-xy_plt00000 Coarse STEP 1 starts ... Coarse STEP 1 ends. TIME = 60 DT = 60 Coarse STEP 2 starts ... Coarse STEP 2 ends. TIME = 120 DT = 60 Coarse STEP 3 starts ... Coarse STEP 3 ends. TIME = 180 DT = 60 Coarse STEP 4 starts ... Coarse STEP 4 ends. TIME = 240 DT = 60 Coarse STEP 5 starts ... Coarse STEP 5 ends. TIME = 300 DT = 60 Coarse STEP 6 starts ... Coarse STEP 6 ends. TIME = 360 DT = 60 Coarse STEP 7 starts ... Coarse STEP 7 ends. TIME = 420 DT = 60 Coarse STEP 8 starts ... Coarse STEP 8 ends. TIME = 480 DT = 60 Coarse STEP 9 starts ... Coarse STEP 9 ends. TIME = 540 DT = 60 Coarse STEP 10 starts ... Coarse STEP 10 ends. TIME = 600 DT = 60 Writing plotfile Seamount64-xy_plt00010 TinyProfiler total time across processes [min...avg...max]: 7.637 ... 7.642 ... 7.647 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- FillBoundary_finish() 3935 1.578 1.617 1.684 22.02% FillBoundary_nowait() 3935 1.431 1.505 1.549 20.25% REMORA::advance2d() 590 1.456 1.462 1.467 19.18% VisMF::Write(FabArray) 4 0.3747 0.4062 0.4272 5.59% REMORA::uv3dmix() 590 0.3256 0.3296 0.3325 4.35% REMORAPhysBCFunct::() 1896 0.2906 0.3052 0.3178 4.16% REMORA::vert_visc_3d() 50 0.2774 0.2819 0.2855 3.73% REMORA::rhs_uv_2d() 580 0.2638 0.273 0.2803 3.66% REMORA::advance_3d() 10 0.1953 0.1989 0.2023 2.64% REMORA::prestep() 10 0.1527 0.1577 0.1629 2.13% REMORA::t3dmix2_s() 20 0.1551 0.1576 0.1619 2.12% REMORA::setup_step() 10 0.1442 0.1498 0.1571 2.05% REMORA::rhs_t_3d() 30 0.136 0.1362 0.1363 1.78% REMORA::rhs_uv_3d() 10 0.1279 0.1291 0.1306 1.71% REMORA::prestep_t_advection() 30 0.1054 0.1177 0.13 1.70% FabArray::setVal() 171 0.0801 0.08228 0.0841 1.10% REMORA::estTimeStep() 11 0.04004 0.04902 0.06139 0.80% REMORA::InitData() 2 0.04493 0.0479 0.05289 0.69% REMORA::Evolve() 1 0.03388 0.03842 0.04198 0.55% amrex::Copy() 52 0.03595 0.03687 0.0381 0.50% REMORA::stretch_transform() 11 0.0332 0.03358 0.03413 0.45% main() 1 0.0001379 0.007348 0.02881 0.38% REMORA::update_massflux_3d() 20 0.01944 0.02274 0.02508 0.33% REMORA::prsgrd() 10 0.007769 0.01224 0.01742 0.23% MultiFab::contains_nan() 10 0.01333 0.01413 0.01499 0.20% DistributionMapping::LeastUsedCPUs() 1 1.007e-05 0.003786 0.01145 0.15% MultiFab::contains_inf() 10 0.0104 0.0104 0.0104 0.14% REMORA::advance_3d_ml() 10 0.005885 0.007292 0.009259 0.12% REMORA::set_analytic_vmix() 11 0.008678 0.008782 0.008887 0.12% WriteMultiLevelPlotfileWithBathymetry() 2 0.0003518 0.002108 0.004538 0.06% REMORA::init_only() 1 0.003479 0.003953 0.004279 0.06% REMORA::FillPatch() 1866 0.003488 0.00363 0.00392 0.05% REMORA::set_zeta_to_Ztavg() 10 0.003647 0.003663 0.003675 0.05% FillPatchSingleLevel 1998 0.00203 0.002171 0.002371 0.03% FabArray::FillBoundary() 3313 0.001865 0.002024 0.002254 0.03% FabArray::EnforcePeriodicity 619 0.0003467 0.0003705 0.0004126 0.01% REMORA::Advance() 10 0.0002871 0.0003032 0.0003291 0.00% REMORA::FillPatchNoBC() 132 0.0001319 0.0001394 0.0001513 0.00% MultiFab::Copy() 39 1.581e-05 1.66e-05 1.723e-05 0.00% AmrMesh::MakeDistributionMap() 1 1.128e-06 2.246e-06 3.664e-06 0.00% DistributionMapping::SFCProcessorMapDoIt() 1 1.853e-06 2.553e-06 3.441e-06 0.00% Other 4791 0.0195 0.02271 0.02634 0.34% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 7.637 7.642 7.647 100.00% REMORA::Evolve() 1 7.188 7.208 7.224 94.46% REMORA::Advance() 10 6.937 6.94 6.942 90.78% REMORA::advance2d() 590 4.768 4.77 4.771 62.39% FabArray::FillBoundary() 3313 2.693 2.702 2.715 35.50% REMORA::FillPatch() 1866 2.111 2.147 2.181 28.52% FillBoundary_finish() 3935 1.578 1.617 1.684 22.02% FillPatchSingleLevel 1998 1.601 1.623 1.652 21.61% FillBoundary_nowait() 3935 1.433 1.506 1.551 20.28% REMORA::setup_step() 10 1.147 1.153 1.157 15.14% REMORA::advance_3d_ml() 10 1.016 1.017 1.018 13.32% REMORA::advance_3d() 10 0.8864 0.8884 0.8892 11.63% FabArray::EnforcePeriodicity 619 0.4008 0.4225 0.4366 5.71% REMORA::InitData() 2 0.4209 0.4273 0.4353 5.69% WriteMultiLevelPlotfileWithBathymetry() 2 0.3768 0.4101 0.4311 5.64% VisMF::Write(FabArray) 4 0.3747 0.4062 0.4272 5.59% REMORA::uv3dmix() 590 0.3256 0.3296 0.3325 4.35% REMORAPhysBCFunct::() 1896 0.3037 0.3175 0.3287 4.30% REMORA::prestep() 10 0.2929 0.2999 0.3071 4.02% REMORA::vert_visc_3d() 50 0.2774 0.2819 0.2855 3.73% REMORA::rhs_uv_2d() 580 0.2638 0.273 0.2803 3.66% REMORA::FillPatchNoBC() 132 0.1959 0.2004 0.2044 2.67% REMORA::t3dmix2_s() 20 0.1551 0.1576 0.1619 2.12% REMORA::rhs_t_3d() 30 0.1371 0.1372 0.1372 1.79% REMORA::rhs_uv_3d() 10 0.1279 0.1291 0.1306 1.71% REMORA::prestep_t_advection() 30 0.1054 0.1177 0.13 1.70% REMORA::stretch_transform() 11 0.09604 0.09698 0.09768 1.28% FabArray::setVal() 171 0.0801 0.08228 0.0841 1.10% REMORA::set_analytic_vmix() 11 0.07732 0.07732 0.07733 1.01% REMORA::estTimeStep() 11 0.04004 0.04902 0.06139 0.80% REMORA::init_only() 1 0.05922 0.05937 0.05973 0.78% amrex::Copy() 52 0.03595 0.03687 0.0381 0.50% MultiFab::Copy() 39 0.0284 0.02902 0.02951 0.39% REMORA::update_massflux_3d() 20 0.01944 0.02274 0.02508 0.33% REMORA::prsgrd() 10 0.007769 0.01224 0.01742 0.23% MultiFab::contains_nan() 10 0.01333 0.01413 0.01499 0.20% AmrMesh::MakeDistributionMap() 1 1.381e-05 0.003792 0.01145 0.15% DistributionMapping::SFCProcessorMapDoIt() 1 1.269e-05 0.003789 0.01145 0.15% DistributionMapping::LeastUsedCPUs() 1 1.007e-05 0.003786 0.01145 0.15% REMORA::set_zeta_to_Ztavg() 10 0.009057 0.009786 0.01051 0.14% MultiFab::contains_inf() 10 0.0104 0.0104 0.0104 0.14% Other 4791 0.05625 0.06195 0.06816 0.89% ------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::amr.checkpoint_files_output(nvals = 1) :: [0] Device Memory Usage: ---------------------------------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max CurrentMem max ---------------------------------------------------------------------------------------------------------------------------------------------- The_Arena::Initialize() 4 4 71 B 99 B 129 B 1024 MiB 1024 MiB 1024 MiB 0 B REMORA::InitData() 352 320 626 MiB 626 MiB 627 MiB 786 MiB 786 MiB 786 MiB 174 MiB REMORA::Evolve() 12 12 3033 KiB 3390 KiB 3686 KiB 141 MiB 141 MiB 141 MiB 0 B REMORA::advance_3d() 760 760 8580 KiB 8623 KiB 8642 KiB 107 MiB 107 MiB 107 MiB 0 B REMORA::prestep_t_advection() 120 120 2353 KiB 2479 KiB 2606 KiB 105 MiB 105 MiB 105 MiB 0 B REMORA::prestep() 160 160 2873 KiB 2945 KiB 3018 KiB 79 MiB 79 MiB 79 MiB 0 B REMORA::prsgrd() 40 40 982 KiB 1099 KiB 1158 KiB 78 MiB 78 MiB 78 MiB 0 B REMORA::setup_step() 240 240 7631 KiB 7659 KiB 7675 KiB 67 MiB 67 MiB 67 MiB 0 B REMORA::rhs_t_3d() 480 480 926 KiB 1048 KiB 1136 KiB 65 MiB 65 MiB 65 MiB 0 B REMORA::t3dmix2_s() 160 160 757 KiB 779 KiB 823 KiB 62 MiB 62 MiB 62 MiB 0 B WriteMultiLevelPlotfileWithBathymetry() 8 8 3051 KiB 3313 KiB 3483 KiB 62 MiB 62 MiB 62 MiB 0 B REMORA::uv3dmix() 2360 2360 771 KiB 817 KiB 836 KiB 52 MiB 52 MiB 52 MiB 0 B REMORA::rhs_uv_3d() 160 160 738 KiB 760 KiB 776 KiB 50 MiB 50 MiB 50 MiB 0 B REMORA::estTimeStep() 132 132 194 KiB 238 KiB 298 KiB 37 MiB 37 MiB 37 MiB 0 B ResizeRandomSeed 4 4 9206 KiB 9206 KiB 9206 KiB 9216 KiB 9216 KiB 9216 KiB 0 B REMORA::advance2d() 9440 9440 1682 KiB 1683 KiB 1684 KiB 3222 KiB 3222 KiB 3222 KiB 0 B REMORA::rhs_uv_2d() 2320 2320 60 KiB 61 KiB 62 KiB 840 KiB 840 KiB 840 KiB 0 B REMORA::update_massflux_3d() 80 80 1514 B 1673 B 1790 B 420 KiB 420 KiB 420 KiB 0 B VisMF::Write(FabArray) 64 64 17 B 117 B 190 B 96 KiB 96 KiB 96 KiB 0 B FillBoundary_nowait() 4936 4892 47 KiB 47 KiB 48 KiB 51 KiB 52 KiB 52 KiB 13 KiB FillBoundary_finish() 4936 4892 47 KiB 47 KiB 48 KiB 51 KiB 52 KiB 52 KiB 13 KiB MultiFab::contains_nan() 52 48 264 B 265 B 266 B 6656 B 6656 B 6656 B 256 B MultiFab::contains_inf() 40 40 6 B 6 B 7 B 6144 B 6144 B 6144 B 0 B impose_cons_bcs() 88 88 0 B 0 B 0 B 256 B 256 B 256 B 0 B impose_xvel_bcs() 44 44 0 B 0 B 0 B 256 B 256 B 256 B 0 B impose_yvel_bcs() 44 44 0 B 0 B 0 B 256 B 256 B 256 B 0 B ---------------------------------------------------------------------------------------------------------------------------------------------- Async Memory Usage: ------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ------------------------------------------------------------------------------------------------------------- REMORA::prsgrd() 40 79 KiB 124 KiB 177 KiB 78 MiB 78 MiB 78 MiB REMORA::advance_3d() 600 2948 KiB 2977 KiB 3003 KiB 53 MiB 53 MiB 53 MiB REMORA::prestep_t_advection() 120 717 KiB 800 KiB 884 KiB 52 MiB 52 MiB 52 MiB REMORA::rhs_t_3d() 480 418 KiB 435 KiB 459 KiB 52 MiB 52 MiB 52 MiB REMORA::uv3dmix() 2360 106 KiB 137 KiB 161 KiB 52 MiB 52 MiB 52 MiB REMORA::rhs_uv_3d() 160 655 KiB 662 KiB 669 KiB 50 MiB 50 MiB 50 MiB REMORA::t3dmix2_s() 160 475 KiB 509 KiB 543 KiB 50 MiB 50 MiB 50 MiB REMORA::setup_step() 80 559 KiB 568 KiB 577 KiB 13 MiB 13 MiB 13 MiB REMORA::prestep() 40 192 KiB 213 KiB 234 KiB 13 MiB 13 MiB 13 MiB REMORA::rhs_uv_2d() 2320 27 KiB 28 KiB 29 KiB 840 KiB 840 KiB 840 KiB REMORA::advance2d() 2360 6042 B 6075 B 6124 B 215 KiB 215 KiB 215 KiB REMORA::update_massflux_3d() 80 528 B 617 B 682 B 210 KiB 210 KiB 210 KiB ------------------------------------------------------------------------------------------------------------- Managed Memory Usage: --------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------- The_Managed_Arena::Initialize() 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------- Pinned Memory Usage: ------------------------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max CurrentMem max ------------------------------------------------------------------------------------------------------------------------------------- VisMF::Write(FabArray) 80 80 1631 KiB 1728 KiB 1829 KiB 62 MiB 62 MiB 62 MiB 0 B The_Pinned_Arena::Initialize() 4 4 0 B 0 B 1 B 8192 KiB 8192 KiB 8192 KiB 0 B FillBoundary_nowait() 36400 36356 274 KiB 283 KiB 291 KiB 5917 KiB 5917 KiB 5918 KiB 13 KiB FillBoundary_finish() 4936 4892 47 KiB 47 KiB 48 KiB 51 KiB 52 KiB 52 KiB 13 KiB MultiFab::contains_nan() 52 48 256 B 256 B 258 B 768 B 768 B 768 B 256 B REMORA::estTimeStep() 88 88 1 B 1 B 1 B 512 B 512 B 512 B 0 B MultiFab::contains_inf() 40 40 0 B 0 B 0 B 256 B 256 B 256 B 0 B ------------------------------------------------------------------------------------------------------------------------------------- Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [3315 ... 3315] [The Arena] max space (MB) allocated spread across MPI: [1024 ... 1024] [The Arena] max space (MB) used spread across MPI: [891 ... 891] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [133 ... 133] [The Pinned Arena] max space (MB) used spread across MPI: [62 ... 62] AMReX (26.06-11-g9ed35ab0e584) finalized