Initializing AMReX (26.06-13-g87eac2327e9f)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.06-13-g87eac2327e9f) initialized REMORA git hash: v1.5-149-g3b71c66 AMReX git hash: 26.06-13-g87eac2327e Making level 0 from scratch GRIDS AT LEVEL 0 ARE (BoxArray maxbox(4) m_ref->m_hash_sig(0) ((0,0,0) (159,159,63) (0,0,0)) ((160,0,0) (319,159,63) (0,0,0)) ((0,160,0) (159,319,63) (0,0,0)) ((160,160,0) (319,319,63) (0,0,0)) ) Writing plotfile Seamount64-xy_plt00000 Coarse STEP 1 starts ... Coarse STEP 1 ends. TIME = 60 DT = 60 Coarse STEP 2 starts ... Coarse STEP 2 ends. TIME = 120 DT = 60 Coarse STEP 3 starts ... Coarse STEP 3 ends. TIME = 180 DT = 60 Coarse STEP 4 starts ... Coarse STEP 4 ends. TIME = 240 DT = 60 Coarse STEP 5 starts ... Coarse STEP 5 ends. TIME = 300 DT = 60 Coarse STEP 6 starts ... Coarse STEP 6 ends. TIME = 360 DT = 60 Coarse STEP 7 starts ... Coarse STEP 7 ends. TIME = 420 DT = 60 Coarse STEP 8 starts ... Coarse STEP 8 ends. TIME = 480 DT = 60 Coarse STEP 9 starts ... Coarse STEP 9 ends. TIME = 540 DT = 60 Coarse STEP 10 starts ... Coarse STEP 10 ends. TIME = 600 DT = 60 Writing plotfile Seamount64-xy_plt00010 TinyProfiler total time across processes [min...avg...max]: 8.202 ... 8.221 ... 8.232 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- FillBoundary_finish() 3935 1.475 1.617 1.764 21.43% FillBoundary_nowait() 3935 1.426 1.495 1.639 19.90% REMORA::advance2d() 590 1.452 1.461 1.471 17.87% VisMF::Write(FabArray) 4 0.965 1.001 1.022 12.41% REMORAPhysBCFunct::() 1896 0.2069 0.3034 0.3656 4.44% REMORA::uv3dmix() 590 0.3298 0.3333 0.3374 4.10% REMORA::rhs_uv_2d() 580 0.2625 0.2729 0.2873 3.49% REMORA::vert_visc_3d() 50 0.269 0.2793 0.2832 3.44% REMORA::advance_3d() 10 0.1944 0.1989 0.2048 2.49% REMORA::prestep() 10 0.1547 0.158 0.1607 1.95% REMORA::t3dmix2_s() 20 0.1513 0.1541 0.1564 1.90% REMORA::setup_step() 10 0.1371 0.1445 0.1483 1.80% REMORA::rhs_t_3d() 30 0.1364 0.1364 0.1364 1.66% REMORA::rhs_uv_3d() 10 0.1273 0.1286 0.1297 1.58% REMORA::prestep_t_advection() 30 0.11 0.1176 0.1253 1.52% FabArray::setVal() 171 0.08056 0.08165 0.08361 1.02% REMORA::estTimeStep() 11 0.03925 0.05069 0.07105 0.86% REMORA::InitData() 2 0.04427 0.04956 0.05612 0.68% REMORA::Evolve() 1 0.03324 0.03946 0.0485 0.59% amrex::Copy() 52 0.03517 0.03636 0.03811 0.46% REMORA::stretch_transform() 11 0.03348 0.03375 0.03417 0.42% main() 1 0.0002082 0.007894 0.03078 0.37% REMORA::update_massflux_3d() 20 0.0194 0.02274 0.02798 0.34% REMORA::prsgrd() 10 0.01047 0.01512 0.02057 0.25% MultiFab::contains_nan() 10 0.01136 0.01317 0.01479 0.18% DistributionMapping::LeastUsedCPUs() 1 7.906e-06 0.005729 0.01466 0.18% REMORA::set_analytic_vmix() 11 0.008617 0.008778 0.009001 0.11% REMORA::advance_3d_ml() 10 0.005701 0.00711 0.008519 0.10% WriteMultiLevelPlotfileWithBathymetry() 2 0.001423 0.002335 0.004709 0.06% REMORA::init_only() 1 0.00328 0.003822 0.004471 0.05% REMORA::set_zeta_to_Ztavg() 10 0.003648 0.003658 0.003674 0.04% REMORA::FillPatch() 1866 0.003318 0.003387 0.003456 0.04% FillPatchSingleLevel 1998 0.001962 0.002055 0.002156 0.03% FabArray::FillBoundary() 3313 0.001719 0.001848 0.002049 0.02% FabArray::EnforcePeriodicity 619 0.0003234 0.0003435 0.0003667 0.00% REMORA::Advance() 10 0.0002625 0.0002789 0.0002899 0.00% REMORA::FillPatchNoBC() 132 0.0001128 0.0001198 0.0001245 0.00% MultiFab::Copy() 39 1.32e-05 1.494e-05 1.623e-05 0.00% AmrMesh::MakeDistributionMap() 1 1.283e-06 3.461e-06 5.299e-06 0.00% DistributionMapping::SFCProcessorMapDoIt() 1 1.936e-06 2.546e-06 3.748e-06 0.00% Other 4801 0.02785 0.0315 0.0364 0.44% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 8.202 8.221 8.232 100.00% REMORA::Evolve() 1 7.497 7.52 7.541 91.60% REMORA::Advance() 10 6.92 6.923 6.926 84.13% REMORA::advance2d() 590 4.757 4.758 4.759 57.81% FabArray::FillBoundary() 3313 2.617 2.694 2.779 33.75% REMORA::FillPatch() 1866 2.072 2.138 2.202 26.75% FillBoundary_finish() 3935 1.475 1.617 1.764 21.43% FillPatchSingleLevel 1998 1.506 1.616 1.746 21.21% FillBoundary_nowait() 3935 1.428 1.496 1.641 19.93% REMORA::setup_step() 10 1.149 1.151 1.154 14.02% WriteMultiLevelPlotfileWithBathymetry() 2 0.9688 1.005 1.025 12.45% VisMF::Write(FabArray) 4 0.965 1.001 1.022 12.41% REMORA::advance_3d_ml() 10 1.011 1.013 1.015 12.33% REMORA::advance_3d() 10 0.8829 0.8845 0.8855 10.76% REMORA::InitData() 2 0.6862 0.6928 0.701 8.51% FabArray::EnforcePeriodicity 619 0.402 0.4205 0.4499 5.46% REMORAPhysBCFunct::() 1896 0.2169 0.3154 0.3781 4.59% REMORA::uv3dmix() 590 0.3298 0.3333 0.3374 4.10% REMORA::prestep() 10 0.2954 0.3001 0.3043 3.70% REMORA::rhs_uv_2d() 580 0.2625 0.2729 0.2873 3.49% REMORA::vert_visc_3d() 50 0.269 0.2793 0.2832 3.44% REMORA::FillPatchNoBC() 132 0.195 0.1987 0.2022 2.46% REMORA::t3dmix2_s() 20 0.1513 0.1541 0.1564 1.90% REMORA::rhs_t_3d() 30 0.1372 0.1372 0.1372 1.67% REMORA::rhs_uv_3d() 10 0.1273 0.1286 0.1297 1.58% REMORA::prestep_t_advection() 30 0.11 0.1176 0.1253 1.52% REMORA::stretch_transform() 11 0.09595 0.09688 0.09772 1.19% FabArray::setVal() 171 0.08056 0.08165 0.08361 1.02% REMORA::set_analytic_vmix() 11 0.07671 0.07672 0.07672 0.93% REMORA::estTimeStep() 11 0.03925 0.05069 0.07105 0.86% REMORA::init_only() 1 0.05766 0.05786 0.05807 0.71% amrex::Copy() 52 0.03517 0.03636 0.03811 0.46% MultiFab::Copy() 39 0.02818 0.02888 0.02958 0.36% REMORA::update_massflux_3d() 20 0.0194 0.02274 0.02798 0.34% REMORA::prsgrd() 10 0.01047 0.01512 0.02057 0.25% MultiFab::contains_nan() 10 0.01136 0.01317 0.01479 0.18% AmrMesh::MakeDistributionMap() 1 1.399e-05 0.005735 0.01467 0.18% DistributionMapping::SFCProcessorMapDoIt() 1 1.052e-05 0.005732 0.01467 0.18% DistributionMapping::LeastUsedCPUs() 1 7.906e-06 0.005729 0.01466 0.18% REMORA::set_zeta_to_Ztavg() 10 0.009325 0.009773 0.01023 0.12% Other 4801 0.06336 0.06913 0.07614 0.92% ------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::amr.checkpoint_files_output(nvals = 1) :: [0] Device Memory Usage: ---------------------------------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max CurrentMem max ---------------------------------------------------------------------------------------------------------------------------------------------- The_Arena::Initialize() 4 4 75 B 117 B 182 B 1024 MiB 1024 MiB 1024 MiB 0 B REMORA::InitData() 352 320 630 MiB 631 MiB 632 MiB 786 MiB 786 MiB 786 MiB 174 MiB REMORA::Evolve() 12 12 8367 KiB 8759 KiB 9114 KiB 141 MiB 141 MiB 141 MiB 0 B REMORA::advance_3d() 760 760 7950 KiB 7978 KiB 8048 KiB 107 MiB 107 MiB 107 MiB 0 B REMORA::prestep_t_advection() 120 120 2230 KiB 2306 KiB 2379 KiB 105 MiB 105 MiB 105 MiB 0 B REMORA::prestep() 160 160 2695 KiB 2739 KiB 2780 KiB 79 MiB 79 MiB 79 MiB 0 B REMORA::prsgrd() 40 40 925 KiB 989 KiB 1067 KiB 78 MiB 78 MiB 78 MiB 0 B REMORA::setup_step() 240 240 7098 KiB 7112 KiB 7131 KiB 67 MiB 67 MiB 67 MiB 0 B REMORA::rhs_t_3d() 480 480 861 KiB 975 KiB 1155 KiB 65 MiB 65 MiB 65 MiB 0 B REMORA::t3dmix2_s() 160 160 705 KiB 722 KiB 732 KiB 62 MiB 62 MiB 62 MiB 0 B WriteMultiLevelPlotfileWithBathymetry() 8 8 7290 KiB 7559 KiB 7718 KiB 62 MiB 62 MiB 62 MiB 0 B REMORA::uv3dmix() 2360 2360 719 KiB 746 KiB 774 KiB 52 MiB 52 MiB 52 MiB 0 B REMORA::rhs_uv_3d() 160 160 707 KiB 728 KiB 744 KiB 50 MiB 50 MiB 50 MiB 0 B REMORA::estTimeStep() 132 132 177 KiB 228 KiB 320 KiB 37 MiB 37 MiB 37 MiB 0 B ResizeRandomSeed 4 4 9200 KiB 9202 KiB 9202 KiB 9216 KiB 9216 KiB 9216 KiB 0 B REMORA::advance2d() 9440 9440 1560 KiB 1560 KiB 1561 KiB 3222 KiB 3222 KiB 3222 KiB 0 B REMORA::rhs_uv_2d() 2320 2320 56 KiB 57 KiB 58 KiB 840 KiB 840 KiB 840 KiB 0 B REMORA::update_massflux_3d() 80 80 1408 B 1558 B 1794 B 420 KiB 420 KiB 420 KiB 0 B VisMF::Write(FabArray) 64 64 29 B 105 B 153 B 96 KiB 96 KiB 96 KiB 0 B FillBoundary_nowait() 4936 4892 46 KiB 47 KiB 47 KiB 51 KiB 52 KiB 52 KiB 13 KiB FillBoundary_finish() 4936 4892 46 KiB 47 KiB 47 KiB 51 KiB 52 KiB 52 KiB 13 KiB MultiFab::contains_nan() 52 48 281 B 281 B 282 B 6656 B 6656 B 6656 B 256 B MultiFab::contains_inf() 40 40 5 B 5 B 6 B 6144 B 6144 B 6144 B 0 B impose_cons_bcs() 88 88 0 B 0 B 0 B 256 B 256 B 256 B 0 B impose_xvel_bcs() 44 44 0 B 0 B 0 B 256 B 256 B 256 B 0 B impose_yvel_bcs() 44 44 0 B 0 B 0 B 256 B 256 B 256 B 0 B ---------------------------------------------------------------------------------------------------------------------------------------------- Async Memory Usage: ------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ------------------------------------------------------------------------------------------------------------- REMORA::prsgrd() 40 99 KiB 143 KiB 195 KiB 78 MiB 78 MiB 78 MiB REMORA::advance_3d() 600 2680 KiB 2749 KiB 2780 KiB 53 MiB 53 MiB 53 MiB REMORA::prestep_t_advection() 120 696 KiB 744 KiB 792 KiB 52 MiB 52 MiB 52 MiB REMORA::rhs_t_3d() 480 369 KiB 404 KiB 426 KiB 52 MiB 52 MiB 52 MiB REMORA::uv3dmix() 2360 128 KiB 152 KiB 170 KiB 52 MiB 52 MiB 52 MiB REMORA::rhs_uv_3d() 160 609 KiB 613 KiB 616 KiB 50 MiB 50 MiB 50 MiB REMORA::t3dmix2_s() 160 451 KiB 478 KiB 490 KiB 50 MiB 50 MiB 50 MiB REMORA::setup_step() 80 525 KiB 533 KiB 543 KiB 13 MiB 13 MiB 13 MiB REMORA::prestep() 40 186 KiB 198 KiB 210 KiB 13 MiB 13 MiB 13 MiB REMORA::rhs_uv_2d() 2320 25 KiB 26 KiB 28 KiB 840 KiB 840 KiB 840 KiB REMORA::advance2d() 2360 5589 B 5626 B 5662 B 215 KiB 215 KiB 215 KiB REMORA::update_massflux_3d() 80 490 B 574 B 707 B 210 KiB 210 KiB 210 KiB ------------------------------------------------------------------------------------------------------------- Managed Memory Usage: --------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------- The_Managed_Arena::Initialize() 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------- Pinned Memory Usage: ------------------------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max CurrentMem max ------------------------------------------------------------------------------------------------------------------------------------- VisMF::Write(FabArray) 80 80 1574 KiB 1669 KiB 1783 KiB 62 MiB 62 MiB 62 MiB 0 B The_Pinned_Arena::Initialize() 4 4 0 B 0 B 1 B 8192 KiB 8192 KiB 8192 KiB 0 B FillBoundary_nowait() 36400 36356 260 KiB 267 KiB 282 KiB 5917 KiB 5917 KiB 5918 KiB 13 KiB FillBoundary_finish() 4936 4892 46 KiB 47 KiB 47 KiB 51 KiB 52 KiB 52 KiB 13 KiB MultiFab::contains_nan() 52 48 273 B 274 B 275 B 768 B 768 B 768 B 256 B REMORA::estTimeStep() 88 88 0 B 0 B 1 B 512 B 512 B 512 B 0 B MultiFab::contains_inf() 40 40 0 B 0 B 0 B 256 B 256 B 256 B 0 B ------------------------------------------------------------------------------------------------------------------------------------- Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [3315 ... 4339] [The Arena] max space (MB) allocated spread across MPI: [1024 ... 1024] [The Arena] max space (MB) used spread across MPI: [891 ... 891] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [133 ... 133] [The Pinned Arena] max space (MB) used spread across MPI: [62 ... 62] AMReX (26.06-13-g87eac2327e9f) finalized