Initializing AMReX (26.06-23-g6de3ba7ac2ba)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.06-23-g6de3ba7ac2ba) initialized REMORA git hash: v1.6-2-g4c07cf1 AMReX git hash: 26.06-23-g6de3ba7ac2 Making level 0 from scratch GRIDS AT LEVEL 0 ARE (BoxArray maxbox(6) m_ref->m_hash_sig(0) ((0,0,0) (16,23,12) (0,0,0)) ((0,24,0) (16,47,12) (0,0,0)) ((17,0,0) (32,23,12) (0,0,0)) ((17,24,0) (32,47,12) (0,0,0)) ((33,0,0) (48,23,12) (0,0,0)) ((33,24,0) (48,47,12) (0,0,0)) ) Writing plotfile Seamount-xy_plt00000 Coarse STEP 1 starts ... Coarse STEP 1 ends. TIME = 60 DT = 60 Coarse STEP 2 starts ... Coarse STEP 2 ends. TIME = 120 DT = 60 Coarse STEP 3 starts ... Coarse STEP 3 ends. TIME = 180 DT = 60 Coarse STEP 4 starts ... Coarse STEP 4 ends. TIME = 240 DT = 60 Coarse STEP 5 starts ... Coarse STEP 5 ends. TIME = 300 DT = 60 Coarse STEP 6 starts ... Coarse STEP 6 ends. TIME = 360 DT = 60 Coarse STEP 7 starts ... Coarse STEP 7 ends. TIME = 420 DT = 60 Coarse STEP 8 starts ... Coarse STEP 8 ends. TIME = 480 DT = 60 Coarse STEP 9 starts ... Coarse STEP 9 ends. TIME = 540 DT = 60 Coarse STEP 10 starts ... Coarse STEP 10 ends. TIME = 600 DT = 60 Writing plotfile Seamount-xy_plt00010 TinyProfiler total time across processes [min...avg...max]: 6.309 ... 6.311 ... 6.314 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- FillBoundary_finish() 3935 1.477 2.017 2.579 40.85% REMORA::advance2d() 590 1.329 1.596 1.88 29.77% FillBoundary_nowait() 3935 1.011 1.093 1.185 18.76% REMORA::rhs_uv_2d() 870 0.2235 0.3664 0.5091 8.06% REMORAPhysBCFunct::() 1896 0.2172 0.2904 0.3601 5.70% REMORA::uv3dmix() 885 0.2574 0.275 0.2943 4.66% REMORA::advance_3d() 10 0.0883 0.1051 0.1223 1.94% REMORA::vert_visc_3d() 75 0.06368 0.0784 0.09343 1.48% FabArray::setVal() 171 0.06216 0.07042 0.07856 1.24% REMORA::InitData() 2 0.04371 0.0594 0.07668 1.21% REMORA::rhs_t_3d() 45 0.04408 0.05109 0.05875 0.93% REMORA::setup_step() 10 0.0318 0.04097 0.05048 0.80% REMORA::rhs_uv_3d() 15 0.0273 0.03403 0.04076 0.65% REMORA::estTimeStep() 11 0.01921 0.02905 0.0387 0.61% amrex::Copy() 52 0.01535 0.02323 0.03043 0.48% REMORA::t3dmix2_s() 30 0.02051 0.02485 0.02967 0.47% REMORA::prestep() 10 0.01846 0.02323 0.02803 0.44% REMORA::stretch_transform() 11 0.01617 0.01995 0.02384 0.38% REMORA::Evolve() 1 0.008756 0.01323 0.01878 0.30% VisMF::Write(FabArray) 4 0.005608 0.01036 0.01518 0.24% impose_cons_bcs() 33 0.004256 0.009959 0.01492 0.24% REMORA::prestep_t_advection() 45 0.01003 0.01141 0.01266 0.20% REMORA::update_massflux_3d() 30 0.007553 0.0083 0.009142 0.14% MultiFab::contains_nan() 10 0.006597 0.007406 0.008575 0.14% impose_yvel_bcs() 16 0.001884 0.004521 0.007336 0.12% REMORA::FillPatch() 1866 0.003171 0.003844 0.004593 0.07% REMORA::set_analytic_vmix() 11 0.003809 0.003939 0.004017 0.06% REMORA::set_zeta_to_Ztavg() 10 0.003584 0.003607 0.003627 0.06% REMORA::advance_3d_ml() 10 0.001199 0.002361 0.003528 0.06% WriteMultiLevelPlotfileWithBathymetry() 2 0.0009632 0.001596 0.003074 0.05% REMORA::init_only() 1 0.00221 0.002304 0.002404 0.04% FillPatchSingleLevel 1998 0.001692 0.001798 0.001927 0.03% FabArray::FillBoundary() 3313 0.001627 0.001767 0.00186 0.03% REMORA::set_hmixcoef() 1 0.0001295 0.0002922 0.0004552 0.01% FabArray::EnforcePeriodicity 619 0.0002755 0.0002961 0.0003353 0.01% REMORA::Advance() 10 0.0002492 0.0002783 0.0003124 0.00% main() 1 0.000126 0.0001763 0.0002375 0.00% REMORA::FillPatchNoBC() 132 7.22e-05 7.525e-05 8.172e-05 0.00% MultiFab::Copy() 39 1.238e-05 1.343e-05 1.45e-05 0.00% REMORA::set_smflux() 10 9.275e-06 1.056e-05 1.23e-05 0.00% Other 5145 0.01959 0.02544 0.03065 0.49% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 6.309 6.311 6.314 100.00% REMORA::Evolve() 1 6.129 6.134 6.138 97.21% REMORA::Advance() 10 6.078 6.079 6.08 96.29% REMORA::advance2d() 590 5.211 5.214 5.215 82.60% FabArray::FillBoundary() 3313 2.162 2.735 3.323 52.62% FillBoundary_finish() 3935 1.477 2.017 2.579 40.85% REMORA::FillPatch() 1866 1.683 1.998 2.324 36.80% FillPatchSingleLevel 1998 1.071 1.434 1.803 28.55% FillBoundary_nowait() 3935 1.013 1.095 1.186 18.79% REMORA::rhs_uv_2d() 870 0.2235 0.3664 0.5091 8.06% REMORA::advance_3d_ml() 10 0.4727 0.4755 0.477 7.55% FabArray::EnforcePeriodicity 619 0.3296 0.3774 0.4216 6.68% REMORA::setup_step() 10 0.3887 0.3901 0.3919 6.21% REMORA::advance_3d() 10 0.3881 0.3893 0.3905 6.18% REMORAPhysBCFunct::() 1896 0.2271 0.3089 0.3871 6.13% REMORA::uv3dmix() 885 0.2574 0.275 0.2943 4.66% REMORA::InitData() 2 0.1719 0.1774 0.1817 2.88% REMORA::FillPatchNoBC() 132 0.09565 0.104 0.1133 1.79% REMORA::vert_visc_3d() 75 0.06368 0.0784 0.09343 1.48% FabArray::setVal() 171 0.06216 0.07042 0.07856 1.24% REMORA::init_only() 1 0.05262 0.06459 0.0766 1.21% REMORA::prestep() 10 0.04224 0.05193 0.06179 0.98% REMORA::rhs_t_3d() 45 0.0448 0.05203 0.05991 0.95% REMORA::stretch_transform() 11 0.05476 0.05477 0.05479 0.87% REMORA::set_analytic_vmix() 11 0.04672 0.04673 0.04673 0.74% REMORA::rhs_uv_3d() 15 0.0273 0.03403 0.04076 0.65% REMORA::estTimeStep() 11 0.01921 0.02905 0.0387 0.61% amrex::Copy() 52 0.01535 0.02323 0.03043 0.48% REMORA::t3dmix2_s() 30 0.02051 0.02485 0.02967 0.47% MultiFab::Copy() 39 0.01296 0.01935 0.02547 0.40% WriteMultiLevelPlotfileWithBathymetry() 2 0.008888 0.01258 0.0172 0.27% VisMF::Write(FabArray) 4 0.005608 0.01036 0.01518 0.24% impose_cons_bcs() 33 0.004256 0.009959 0.01492 0.24% REMORA::prestep_t_advection() 45 0.01003 0.01141 0.01266 0.20% REMORA::set_zeta_to_Ztavg() 10 0.00954 0.009808 0.01007 0.16% REMORA::set_hmixcoef() 1 0.009735 0.0099 0.01007 0.16% REMORA::update_massflux_3d() 30 0.007553 0.0083 0.009142 0.14% MultiFab::contains_nan() 10 0.006597 0.007406 0.008575 0.14% REMORA::set_smflux() 10 0.007069 0.007355 0.007709 0.12% impose_yvel_bcs() 16 0.001884 0.004521 0.007336 0.12% Other 5145 0.03915 0.04842 0.05658 0.90% ------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::amr.checkpoint_files_output(nvals = 1) :: [0] Device Memory Usage: ---------------------------------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max CurrentMem max ---------------------------------------------------------------------------------------------------------------------------------------------- The_Arena::Initialize() 4 4 124 B 134 B 157 B 1024 MiB 1024 MiB 1024 MiB 0 B ResizeRandomSeed 4 4 9200 KiB 9202 KiB 9204 KiB 9216 KiB 9216 KiB 9216 KiB 0 B REMORA::InitData() 514 466 3501 KiB 5332 KiB 7162 KiB 4173 KiB 6360 KiB 8547 KiB 2332 KiB REMORA::Evolve() 18 18 1431 B 2888 B 4388 B 520 KiB 794 KiB 1069 KiB 0 B REMORA::advance_3d() 1140 1140 17 KiB 25 KiB 33 KiB 501 KiB 764 KiB 1028 KiB 0 B REMORA::prestep() 240 240 1857 B 3306 B 4752 B 346 KiB 527 KiB 709 KiB 0 B REMORA::prestep_t_advection() 180 180 895 B 911 B 942 B 455 KiB 577 KiB 694 KiB 0 B REMORA::setup_step() 360 360 12 KiB 19 KiB 25 KiB 317 KiB 484 KiB 651 KiB 0 B REMORA::rhs_t_3d() 720 720 1393 B 2015 B 2631 B 285 KiB 405 KiB 524 KiB 0 B WriteMultiLevelPlotfileWithBathymetry() 12 12 183 B 585 B 1049 B 195 KiB 298 KiB 402 KiB 0 B REMORA::prsgrd() 60 60 494 B 651 B 803 B 341 KiB 349 KiB 358 KiB 0 B REMORA::estTimeStep() 154 154 667 B 739 B 794 B 165 KiB 227 KiB 289 KiB 0 B REMORA::uv3dmix() 3540 3540 1714 B 2621 B 3532 B 227 KiB 233 KiB 239 KiB 0 B REMORA::t3dmix2_s() 240 240 486 B 590 B 708 B 205 KiB 212 KiB 218 KiB 0 B REMORA::rhs_uv_3d() 240 240 687 B 880 B 1073 B 172 KiB 177 KiB 183 KiB 0 B REMORA::advance2d() 14160 14160 53 KiB 75 KiB 97 KiB 76 KiB 116 KiB 157 KiB 0 B FillBoundary_nowait() 7392 7327 46 KiB 69 KiB 94 KiB 50 KiB 75 KiB 102 KiB 24 KiB VisMF::Write(FabArray) 96 96 63 B 89 B 106 B 96 KiB 96 KiB 96 KiB 0 B FillBoundary_finish() 4935 4892 45 KiB 54 KiB 66 KiB 50 KiB 59 KiB 72 KiB 17 KiB REMORA::rhs_uv_2d() 3480 3480 1287 B 1752 B 2209 B 17 KiB 18 KiB 18 KiB 0 B FabArray::setVal() 234 220 0 B 8656 B 16 KiB 0 B 9472 B 18 KiB 1792 B REMORA::update_massflux_3d() 120 120 10 B 13 B 16 B 9216 B 11 KiB 13 KiB 0 B MultiFab::contains_nan() 48 46 243 B 245 B 247 B 6400 B 6528 B 6656 B 256 B MultiFab::contains_inf() 40 40 2 B 3 B 5 B 6144 B 6144 B 6144 B 0 B amrex::Copy() 70 68 0 B 362 B 724 B 0 B 768 B 1536 B 256 B impose_cons_bcs() 132 132 0 B 0 B 0 B 256 B 256 B 256 B 0 B impose_xvel_bcs() 66 66 0 B 0 B 0 B 256 B 256 B 256 B 0 B impose_yvel_bcs() 66 66 0 B 0 B 0 B 256 B 256 B 256 B 0 B ---------------------------------------------------------------------------------------------------------------------------------------------- Async Memory Usage: ------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ------------------------------------------------------------------------------------------------------------- REMORA::prsgrd() 60 209 B 248 B 283 B 341 KiB 349 KiB 358 KiB REMORA::advance_3d() 900 4857 B 5956 B 7012 B 245 KiB 251 KiB 257 KiB REMORA::prestep_t_advection() 180 364 B 411 B 470 B 227 KiB 233 KiB 238 KiB REMORA::rhs_t_3d() 720 1002 B 1188 B 1359 B 227 KiB 233 KiB 238 KiB REMORA::uv3dmix() 3540 836 B 942 B 1043 B 227 KiB 233 KiB 238 KiB REMORA::rhs_uv_3d() 240 582 B 742 B 903 B 172 KiB 177 KiB 182 KiB REMORA::t3dmix2_s() 240 321 B 393 B 475 B 164 KiB 169 KiB 174 KiB REMORA::setup_step() 120 645 B 818 B 993 B 65 KiB 67 KiB 68 KiB REMORA::prestep() 60 141 B 152 B 167 B 61 KiB 62 KiB 64 KiB REMORA::rhs_uv_2d() 3480 608 B 1012 B 1413 B 17 KiB 17 KiB 18 KiB REMORA::advance2d() 3540 168 B 202 B 236 B 5280 B 5400 B 5520 B REMORA::update_massflux_3d() 120 5 B 5 B 6 B 4480 B 4592 B 4704 B ------------------------------------------------------------------------------------------------------------- Managed Memory Usage: --------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------- The_Managed_Arena::Initialize() 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------- Pinned Memory Usage: ------------------------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max CurrentMem max ------------------------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 1 B 1 B 2 B 8192 KiB 8192 KiB 8192 KiB 0 B FillBoundary_nowait() 38856 38791 53 KiB 75 KiB 99 KiB 216 KiB 277 KiB 348 KiB 24 KiB VisMF::Write(FabArray) 120 120 17 B 34 B 48 B 195 KiB 201 KiB 207 KiB 0 B FillBoundary_finish() 4935 4892 45 KiB 54 KiB 66 KiB 50 KiB 59 KiB 72 KiB 17 KiB FabArray::setVal() 234 220 0 B 8656 B 16 KiB 0 B 9472 B 18 KiB 1792 B amrex::Copy() 70 68 0 B 362 B 724 B 0 B 768 B 1536 B 256 B MultiFab::contains_nan() 48 46 240 B 240 B 241 B 512 B 640 B 768 B 256 B REMORA::estTimeStep() 88 88 0 B 0 B 1 B 512 B 512 B 512 B 0 B MultiFab::contains_inf() 40 40 0 B 0 B 0 B 256 B 256 B 256 B 0 B ------------------------------------------------------------------------------------------------------------------------------------- Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [3315 ... 3315] [The Arena] max space (MB) allocated spread across MPI: [1024 ... 1024] [The Arena] max space (MB) used spread across MPI: [13 ... 18] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Pinned Arena] max space (MB) used spread across MPI: [0 ... 0] AMReX (26.06-23-g6de3ba7ac2ba) finalized