Initializing AMReX (26.05-56-gb8117c772be8)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.05-56-gb8117c772be8) initialized REMORA git hash: v1.5-139-g612f474 AMReX git hash: 26.05-56-gb8117c772b Making level 0 from scratch GRIDS AT LEVEL 0 ARE (BoxArray maxbox(4) m_ref->m_hash_sig(0) ((0,0,0) (159,159,63) (0,0,0)) ((160,0,0) (319,159,63) (0,0,0)) ((0,160,0) (159,319,63) (0,0,0)) ((160,160,0) (319,319,63) (0,0,0)) ) Writing plotfile Seamount64-xy_plt00000 Coarse STEP 1 starts ... Coarse STEP 1 ends. TIME = 60 DT = 60 Coarse STEP 2 starts ... Coarse STEP 2 ends. TIME = 120 DT = 60 Coarse STEP 3 starts ... Coarse STEP 3 ends. TIME = 180 DT = 60 Coarse STEP 4 starts ... Coarse STEP 4 ends. TIME = 240 DT = 60 Coarse STEP 5 starts ... Coarse STEP 5 ends. TIME = 300 DT = 60 Coarse STEP 6 starts ... Coarse STEP 6 ends. TIME = 360 DT = 60 Coarse STEP 7 starts ... Coarse STEP 7 ends. TIME = 420 DT = 60 Coarse STEP 8 starts ... Coarse STEP 8 ends. TIME = 480 DT = 60 Coarse STEP 9 starts ... Coarse STEP 9 ends. TIME = 540 DT = 60 Coarse STEP 10 starts ... Coarse STEP 10 ends. TIME = 600 DT = 60 Writing plotfile Seamount64-xy_plt00010 TinyProfiler total time across processes [min...avg...max]: 7.543 ... 7.546 ... 7.552 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- FillBoundary_finish() 3935 1.484 1.608 1.745 23.10% FillBoundary_nowait() 3935 1.362 1.494 1.607 21.27% REMORA::advance2d() 590 1.438 1.451 1.462 19.35% REMORAPhysBCFunct::() 1896 0.1818 0.3033 0.415 5.50% REMORA::uv3dmix() 590 0.3256 0.331 0.3352 4.44% VisMF::Write(FabArray) 4 0.2936 0.3072 0.3212 4.25% REMORA::rhs_uv_2d() 580 0.2502 0.2712 0.2861 3.79% REMORA::vert_visc_3d() 50 0.2764 0.2795 0.282 3.73% REMORA::advance_3d() 10 0.1919 0.1973 0.2011 2.66% REMORA::prestep() 10 0.1545 0.1578 0.1612 2.13% REMORA::t3dmix2_s() 20 0.1504 0.153 0.1555 2.06% REMORA::setup_step() 10 0.1384 0.1445 0.1474 1.95% REMORA::rhs_t_3d() 30 0.1359 0.136 0.1361 1.80% REMORA::rhs_uv_3d() 10 0.1264 0.1281 0.1291 1.71% REMORA::prestep_t_advection() 30 0.1084 0.1176 0.1268 1.68% REMORA::estTimeStep() 11 0.03839 0.07319 0.1149 1.52% REMORA::InitData() 2 0.04943 0.07539 0.1022 1.35% FabArray::setVal() 171 0.08014 0.08157 0.08295 1.10% REMORA::Evolve() 1 0.03661 0.03982 0.04314 0.57% amrex::Copy() 52 0.03518 0.0366 0.03795 0.50% REMORA::stretch_transform() 11 0.0335 0.03365 0.03393 0.45% REMORA::update_massflux_3d() 20 0.02081 0.02267 0.0255 0.34% main() 1 0.0001348 0.006071 0.02387 0.32% REMORA::prsgrd() 10 0.01026 0.01511 0.01948 0.26% MultiFab::contains_nan() 10 0.01344 0.01399 0.01494 0.20% MultiFab::contains_inf() 10 0.01045 0.01045 0.01045 0.14% REMORA::advance_3d_ml() 10 0.004565 0.007368 0.009607 0.13% REMORA::set_analytic_vmix() 11 0.00853 0.008754 0.008954 0.12% WriteMultiLevelPlotfileWithBathymetry() 2 0.001019 0.002287 0.004706 0.06% REMORA::init_only() 1 0.00344 0.003817 0.004132 0.05% REMORA::FillPatch() 1866 0.003475 0.003629 0.00376 0.05% REMORA::set_zeta_to_Ztavg() 10 0.003612 0.003632 0.003651 0.05% FillPatchSingleLevel 1998 0.002057 0.002176 0.002248 0.03% FabArray::FillBoundary() 3313 0.00181 0.001916 0.002063 0.03% FabArray::EnforcePeriodicity 619 0.0003275 0.0003527 0.0003825 0.01% REMORA::Advance() 10 0.0002909 0.0003074 0.0003213 0.00% REMORA::FillPatchNoBC() 132 0.0001264 0.000133 0.0001431 0.00% MultiFab::Copy() 39 1.68e-05 1.776e-05 1.937e-05 0.00% Other 4794 0.02021 0.023 0.0268 0.35% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 7.543 7.546 7.552 100.00% REMORA::Evolve() 1 7.131 7.152 7.168 94.91% REMORA::Advance() 10 6.891 6.894 6.898 91.34% REMORA::advance2d() 590 4.734 4.734 4.736 62.71% FabArray::FillBoundary() 3313 2.618 2.685 2.771 36.69% REMORA::FillPatch() 1866 2.035 2.133 2.232 29.55% FillPatchSingleLevel 1998 1.456 1.612 1.786 23.65% FillBoundary_finish() 3935 1.484 1.608 1.745 23.10% FillBoundary_nowait() 3935 1.363 1.496 1.609 21.30% REMORA::setup_step() 10 1.142 1.149 1.156 15.31% REMORA::advance_3d_ml() 10 1.008 1.011 1.014 13.43% REMORA::advance_3d() 10 0.8798 0.8828 0.8845 11.71% FabArray::EnforcePeriodicity 619 0.3701 0.4204 0.4578 6.06% REMORAPhysBCFunct::() 1896 0.1949 0.3155 0.426 5.64% REMORA::InitData() 2 0.384 0.388 0.3909 5.18% REMORA::uv3dmix() 590 0.3256 0.331 0.3352 4.44% WriteMultiLevelPlotfileWithBathymetry() 2 0.2967 0.3113 0.326 4.32% VisMF::Write(FabArray) 4 0.2936 0.3072 0.3212 4.25% REMORA::prestep() 10 0.2942 0.2998 0.3056 4.05% REMORA::rhs_uv_2d() 580 0.2502 0.2712 0.2861 3.79% REMORA::vert_visc_3d() 50 0.2764 0.2795 0.282 3.73% REMORA::FillPatchNoBC() 132 0.1932 0.1989 0.2105 2.79% REMORA::t3dmix2_s() 20 0.1504 0.153 0.1555 2.06% REMORA::rhs_t_3d() 30 0.1369 0.1369 0.1369 1.81% REMORA::rhs_uv_3d() 10 0.1264 0.1281 0.1291 1.71% REMORA::prestep_t_advection() 30 0.1084 0.1176 0.1268 1.68% REMORA::estTimeStep() 11 0.03839 0.07319 0.1149 1.52% REMORA::stretch_transform() 11 0.09553 0.09677 0.09764 1.29% FabArray::setVal() 171 0.08014 0.08157 0.08295 1.10% REMORA::set_analytic_vmix() 11 0.07723 0.07723 0.07723 1.02% REMORA::init_only() 1 0.05904 0.05905 0.05905 0.78% amrex::Copy() 52 0.03518 0.0366 0.03795 0.50% MultiFab::Copy() 39 0.02803 0.02894 0.0297 0.39% REMORA::update_massflux_3d() 20 0.02081 0.02267 0.0255 0.34% REMORA::prsgrd() 10 0.01026 0.01511 0.01948 0.26% MultiFab::contains_nan() 10 0.01344 0.01399 0.01494 0.20% MultiFab::contains_inf() 10 0.01045 0.01045 0.01045 0.14% REMORA::set_zeta_to_Ztavg() 10 0.009079 0.009619 0.01016 0.13% Other 4794 0.05671 0.06245 0.06881 0.91% ------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::amr.checkpoint_files_output(nvals = 1) :: [0] Device Memory Usage: ---------------------------------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max CurrentMem max ---------------------------------------------------------------------------------------------------------------------------------------------- The_Arena::Initialize() 4 4 100 B 119 B 144 B 1024 MiB 1024 MiB 1024 MiB 0 B REMORA::InitData() 352 320 625 MiB 626 MiB 627 MiB 786 MiB 786 MiB 786 MiB 174 MiB REMORA::Evolve() 12 12 2874 KiB 3245 KiB 3547 KiB 141 MiB 141 MiB 141 MiB 0 B REMORA::advance_3d() 760 760 8627 KiB 8671 KiB 8711 KiB 107 MiB 107 MiB 107 MiB 0 B REMORA::prestep_t_advection() 120 120 2413 KiB 2511 KiB 2610 KiB 105 MiB 105 MiB 105 MiB 0 B REMORA::prestep() 160 160 2924 KiB 2982 KiB 3042 KiB 79 MiB 79 MiB 79 MiB 0 B REMORA::prsgrd() 40 40 1000 KiB 1072 KiB 1141 KiB 78 MiB 78 MiB 78 MiB 0 B REMORA::setup_step() 240 240 7703 KiB 7725 KiB 7734 KiB 67 MiB 67 MiB 67 MiB 0 B REMORA::rhs_t_3d() 480 480 989 KiB 1060 KiB 1167 KiB 65 MiB 65 MiB 65 MiB 0 B REMORA::t3dmix2_s() 160 160 762 KiB 781 KiB 806 KiB 62 MiB 62 MiB 62 MiB 0 B WriteMultiLevelPlotfileWithBathymetry() 8 8 2434 KiB 2545 KiB 2666 KiB 62 MiB 62 MiB 62 MiB 0 B REMORA::uv3dmix() 2360 2360 781 KiB 809 KiB 836 KiB 52 MiB 52 MiB 52 MiB 0 B REMORA::rhs_uv_3d() 160 160 767 KiB 790 KiB 807 KiB 50 MiB 50 MiB 50 MiB 0 B REMORA::estTimeStep() 132 132 188 KiB 360 KiB 565 KiB 37 MiB 37 MiB 37 MiB 0 B ResizeRandomSeed 4 4 9204 KiB 9205 KiB 9207 KiB 9216 KiB 9216 KiB 9216 KiB 0 B REMORA::advance2d() 9440 9440 1690 KiB 1691 KiB 1692 KiB 3222 KiB 3222 KiB 3222 KiB 0 B REMORA::rhs_uv_2d() 2320 2320 60 KiB 62 KiB 63 KiB 840 KiB 840 KiB 840 KiB 0 B REMORA::update_massflux_3d() 80 80 1597 B 1688 B 1827 B 420 KiB 420 KiB 420 KiB 0 B VisMF::Write(FabArray) 64 64 35 B 60 B 80 B 96 KiB 96 KiB 96 KiB 0 B FillBoundary_nowait() 4936 4892 47 KiB 47 KiB 48 KiB 51 KiB 52 KiB 52 KiB 13 KiB FillBoundary_finish() 4936 4892 47 KiB 47 KiB 48 KiB 51 KiB 52 KiB 52 KiB 13 KiB MultiFab::contains_nan() 52 48 261 B 263 B 265 B 6656 B 6656 B 6656 B 256 B MultiFab::contains_inf() 40 40 6 B 6 B 7 B 6144 B 6144 B 6144 B 0 B impose_cons_bcs() 88 88 0 B 0 B 0 B 256 B 256 B 256 B 0 B impose_xvel_bcs() 44 44 0 B 0 B 0 B 256 B 256 B 256 B 0 B impose_yvel_bcs() 44 44 0 B 0 B 0 B 256 B 256 B 256 B 0 B ---------------------------------------------------------------------------------------------------------------------------------------------- Async Memory Usage: ------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ------------------------------------------------------------------------------------------------------------- REMORA::prsgrd() 40 105 KiB 156 KiB 201 KiB 78 MiB 78 MiB 78 MiB REMORA::advance_3d() 600 2968 KiB 2994 KiB 3016 KiB 53 MiB 53 MiB 53 MiB REMORA::prestep_t_advection() 120 747 KiB 810 KiB 873 KiB 52 MiB 52 MiB 52 MiB REMORA::rhs_t_3d() 480 419 KiB 439 KiB 453 KiB 52 MiB 52 MiB 52 MiB REMORA::uv3dmix() 2360 138 KiB 165 KiB 180 KiB 52 MiB 52 MiB 52 MiB REMORA::rhs_uv_3d() 160 662 KiB 665 KiB 668 KiB 50 MiB 50 MiB 50 MiB REMORA::t3dmix2_s() 160 500 KiB 516 KiB 532 KiB 50 MiB 50 MiB 50 MiB REMORA::setup_step() 80 569 KiB 578 KiB 587 KiB 13 MiB 13 MiB 13 MiB REMORA::prestep() 40 200 KiB 216 KiB 231 KiB 13 MiB 13 MiB 13 MiB REMORA::rhs_uv_2d() 2320 26 KiB 29 KiB 30 KiB 840 KiB 840 KiB 840 KiB REMORA::advance2d() 2360 6041 B 6091 B 6161 B 215 KiB 215 KiB 215 KiB REMORA::update_massflux_3d() 80 573 B 623 B 702 B 210 KiB 210 KiB 210 KiB ------------------------------------------------------------------------------------------------------------- Managed Memory Usage: --------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------- The_Managed_Arena::Initialize() 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------- Pinned Memory Usage: ------------------------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max CurrentMem max ------------------------------------------------------------------------------------------------------------------------------------- VisMF::Write(FabArray) 80 80 1663 KiB 1723 KiB 1793 KiB 62 MiB 62 MiB 62 MiB 0 B The_Pinned_Arena::Initialize() 4 4 0 B 0 B 1 B 8192 KiB 8192 KiB 8192 KiB 0 B FillBoundary_nowait() 36400 36356 274 KiB 286 KiB 296 KiB 5917 KiB 5917 KiB 5918 KiB 13 KiB FillBoundary_finish() 4936 4892 47 KiB 47 KiB 48 KiB 51 KiB 52 KiB 52 KiB 13 KiB MultiFab::contains_nan() 52 48 253 B 254 B 256 B 768 B 768 B 768 B 256 B REMORA::estTimeStep() 88 88 1 B 1 B 3 B 512 B 512 B 512 B 0 B MultiFab::contains_inf() 40 40 0 B 0 B 0 B 256 B 256 B 256 B 0 B ------------------------------------------------------------------------------------------------------------------------------------- Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [3315 ... 3315] [The Arena] max space (MB) allocated spread across MPI: [1024 ... 1024] [The Arena] max space (MB) used spread across MPI: [891 ... 891] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [133 ... 133] [The Pinned Arena] max space (MB) used spread across MPI: [62 ... 62] AMReX (26.05-56-gb8117c772be8) finalized