Initializing AMReX (26.06-11-g9ed35ab0e584)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.06-11-g9ed35ab0e584) initialized REMORA git hash: v1.5-149-g3b71c66 AMReX git hash: 26.06-11-g9ed35ab0e5 Making level 0 from scratch GRIDS AT LEVEL 0 ARE (BoxArray maxbox(4) m_ref->m_hash_sig(0) ((0,0,0) (159,159,63) (0,0,0)) ((160,0,0) (319,159,63) (0,0,0)) ((0,160,0) (159,319,63) (0,0,0)) ((160,160,0) (319,319,63) (0,0,0)) ) Writing plotfile Seamount64-xy_plt00000 Coarse STEP 1 starts ... Coarse STEP 1 ends. TIME = 60 DT = 60 Coarse STEP 2 starts ... Coarse STEP 2 ends. TIME = 120 DT = 60 Coarse STEP 3 starts ... Coarse STEP 3 ends. TIME = 180 DT = 60 Coarse STEP 4 starts ... Coarse STEP 4 ends. TIME = 240 DT = 60 Coarse STEP 5 starts ... Coarse STEP 5 ends. TIME = 300 DT = 60 Coarse STEP 6 starts ... Coarse STEP 6 ends. TIME = 360 DT = 60 Coarse STEP 7 starts ... Coarse STEP 7 ends. TIME = 420 DT = 60 Coarse STEP 8 starts ... Coarse STEP 8 ends. TIME = 480 DT = 60 Coarse STEP 9 starts ... Coarse STEP 9 ends. TIME = 540 DT = 60 Coarse STEP 10 starts ... Coarse STEP 10 ends. TIME = 600 DT = 60 Writing plotfile Seamount64-xy_plt00010 TinyProfiler total time across processes [min...avg...max]: 7.485 ... 7.491 ... 7.501 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- FillBoundary_finish() 3935 1.582 1.61 1.641 21.87% FillBoundary_nowait() 3935 1.421 1.49 1.523 20.31% REMORA::advance2d() 590 1.439 1.45 1.464 19.52% REMORAPhysBCFunct::() 1896 0.2127 0.3023 0.3809 5.08% REMORA::uv3dmix() 590 0.3249 0.3285 0.3337 4.45% VisMF::Write(FabArray) 4 0.2987 0.3109 0.3243 4.32% REMORA::rhs_uv_2d() 580 0.2616 0.2722 0.2863 3.82% REMORA::vert_visc_3d() 50 0.2766 0.2793 0.2811 3.75% REMORA::advance_3d() 10 0.1948 0.1975 0.2006 2.67% REMORA::prestep() 10 0.1528 0.1583 0.1629 2.17% REMORA::t3dmix2_s() 20 0.1532 0.1557 0.1581 2.11% REMORA::setup_step() 10 0.1466 0.1493 0.1541 2.05% REMORA::rhs_t_3d() 30 0.1362 0.1362 0.1362 1.82% REMORA::prestep_t_advection() 30 0.1054 0.1176 0.1328 1.77% REMORA::rhs_uv_3d() 10 0.1271 0.1284 0.13 1.73% FabArray::setVal() 171 0.08027 0.08131 0.08361 1.11% REMORA::estTimeStep() 11 0.04024 0.04674 0.06136 0.82% REMORA::InitData() 2 0.04398 0.04899 0.05594 0.75% REMORA::Evolve() 1 0.03187 0.03818 0.04584 0.61% amrex::Copy() 52 0.03552 0.03679 0.03773 0.50% REMORA::stretch_transform() 11 0.03302 0.03339 0.03374 0.45% REMORA::update_massflux_3d() 20 0.01978 0.02265 0.02453 0.33% main() 1 0.0001268 0.005146 0.02017 0.27% REMORA::prsgrd() 10 0.0102 0.01266 0.01735 0.23% MultiFab::contains_nan() 10 0.01176 0.01304 0.01389 0.19% MultiFab::contains_inf() 10 0.009671 0.009675 0.009678 0.13% REMORA::advance_3d_ml() 10 0.006338 0.007181 0.009138 0.12% REMORA::set_analytic_vmix() 11 0.008642 0.00875 0.008943 0.12% WriteMultiLevelPlotfileWithBathymetry() 2 0.00107 0.002133 0.004508 0.06% REMORA::init_only() 1 0.003189 0.003894 0.004498 0.06% REMORA::set_zeta_to_Ztavg() 10 0.003598 0.003633 0.003656 0.05% REMORA::FillPatch() 1866 0.00338 0.003427 0.003484 0.05% FillPatchSingleLevel 1998 0.001943 0.001993 0.002042 0.03% FabArray::FillBoundary() 3313 0.001738 0.001813 0.001929 0.03% FabArray::EnforcePeriodicity 619 0.0003056 0.0003286 0.0003446 0.00% REMORA::Advance() 10 0.0002681 0.0002793 0.0002907 0.00% REMORA::FillPatchNoBC() 132 0.0001161 0.0001247 0.0001333 0.00% MultiFab::Copy() 39 1.418e-05 1.514e-05 1.573e-05 0.00% Other 4794 0.01874 0.02237 0.0256 0.34% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 7.485 7.491 7.501 100.00% REMORA::Evolve() 1 7.135 7.153 7.162 95.47% REMORA::Advance() 10 6.891 6.894 6.901 91.99% REMORA::advance2d() 590 4.734 4.735 4.737 63.15% FabArray::FillBoundary() 3313 2.666 2.683 2.702 36.03% REMORA::FillPatch() 1866 2.109 2.132 2.158 28.77% FillPatchSingleLevel 1998 1.588 1.612 1.642 21.90% FillBoundary_finish() 3935 1.582 1.61 1.641 21.87% FillBoundary_nowait() 3935 1.423 1.492 1.525 20.33% REMORA::setup_step() 10 1.145 1.148 1.155 15.40% REMORA::advance_3d_ml() 10 1.009 1.01 1.012 13.49% REMORA::advance_3d() 10 0.88 0.8819 0.8833 11.78% FabArray::EnforcePeriodicity 619 0.3948 0.42 0.4518 6.02% REMORAPhysBCFunct::() 1896 0.2247 0.3145 0.3934 5.24% REMORA::InitData() 2 0.3279 0.3326 0.3396 4.53% REMORA::uv3dmix() 590 0.3249 0.3285 0.3337 4.45% WriteMultiLevelPlotfileWithBathymetry() 2 0.3023 0.3149 0.3283 4.38% VisMF::Write(FabArray) 4 0.2987 0.3109 0.3243 4.32% REMORA::prestep() 10 0.2929 0.3002 0.3097 4.13% REMORA::rhs_uv_2d() 580 0.2616 0.2722 0.2863 3.82% REMORA::vert_visc_3d() 50 0.2766 0.2793 0.2811 3.75% REMORA::FillPatchNoBC() 132 0.1929 0.1983 0.205 2.73% REMORA::t3dmix2_s() 20 0.1532 0.1557 0.1581 2.11% REMORA::rhs_t_3d() 30 0.1369 0.1369 0.1369 1.83% REMORA::prestep_t_advection() 30 0.1054 0.1176 0.1328 1.77% REMORA::rhs_uv_3d() 10 0.1271 0.1284 0.13 1.73% REMORA::stretch_transform() 11 0.09482 0.09646 0.09774 1.30% FabArray::setVal() 171 0.08027 0.08131 0.08361 1.11% REMORA::set_analytic_vmix() 11 0.07734 0.07734 0.07734 1.03% REMORA::estTimeStep() 11 0.04024 0.04674 0.06136 0.82% REMORA::init_only() 1 0.05901 0.05923 0.05944 0.79% amrex::Copy() 52 0.03552 0.03679 0.03773 0.50% MultiFab::Copy() 39 0.02863 0.029 0.02941 0.39% REMORA::update_massflux_3d() 20 0.01978 0.02265 0.02453 0.33% REMORA::prsgrd() 10 0.0102 0.01266 0.01735 0.23% MultiFab::contains_nan() 10 0.01176 0.01304 0.01389 0.19% REMORA::set_zeta_to_Ztavg() 10 0.008987 0.009704 0.0106 0.14% MultiFab::contains_inf() 10 0.009671 0.009675 0.009678 0.13% Other 4794 0.05518 0.06174 0.06826 0.91% ------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::amr.checkpoint_files_output(nvals = 1) :: [0] Device Memory Usage: ---------------------------------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max CurrentMem max ---------------------------------------------------------------------------------------------------------------------------------------------- The_Arena::Initialize() 4 4 84 B 102 B 146 B 1024 MiB 1024 MiB 1024 MiB 0 B REMORA::InitData() 352 320 625 MiB 626 MiB 626 MiB 786 MiB 786 MiB 786 MiB 174 MiB REMORA::Evolve() 12 12 2969 KiB 3310 KiB 3466 KiB 141 MiB 141 MiB 141 MiB 0 B REMORA::advance_3d() 760 760 8689 KiB 8740 KiB 8773 KiB 107 MiB 107 MiB 107 MiB 0 B REMORA::prestep_t_advection() 120 120 2404 KiB 2535 KiB 2702 KiB 105 MiB 105 MiB 105 MiB 0 B REMORA::prestep() 160 160 2934 KiB 3011 KiB 3110 KiB 79 MiB 79 MiB 79 MiB 0 B REMORA::prsgrd() 40 40 1061 KiB 1109 KiB 1154 KiB 78 MiB 78 MiB 78 MiB 0 B REMORA::setup_step() 240 240 7783 KiB 7791 KiB 7805 KiB 67 MiB 67 MiB 67 MiB 0 B REMORA::rhs_t_3d() 480 480 962 KiB 1070 KiB 1141 KiB 65 MiB 65 MiB 65 MiB 0 B REMORA::t3dmix2_s() 160 160 745 KiB 789 KiB 838 KiB 62 MiB 62 MiB 62 MiB 0 B WriteMultiLevelPlotfileWithBathymetry() 8 8 2494 KiB 2596 KiB 2711 KiB 62 MiB 62 MiB 62 MiB 0 B REMORA::uv3dmix() 2360 2360 811 KiB 828 KiB 846 KiB 52 MiB 52 MiB 52 MiB 0 B REMORA::rhs_uv_3d() 160 160 758 KiB 777 KiB 803 KiB 50 MiB 50 MiB 50 MiB 0 B REMORA::estTimeStep() 132 132 199 KiB 231 KiB 304 KiB 37 MiB 37 MiB 37 MiB 0 B ResizeRandomSeed 4 4 9205 KiB 9206 KiB 9206 KiB 9216 KiB 9216 KiB 9216 KiB 0 B REMORA::advance2d() 9440 9440 1705 KiB 1706 KiB 1707 KiB 3222 KiB 3222 KiB 3222 KiB 0 B REMORA::rhs_uv_2d() 2320 2320 61 KiB 62 KiB 64 KiB 840 KiB 840 KiB 840 KiB 0 B REMORA::update_massflux_3d() 80 80 1564 B 1705 B 1799 B 420 KiB 420 KiB 420 KiB 0 B VisMF::Write(FabArray) 64 64 57 B 105 B 161 B 96 KiB 96 KiB 96 KiB 0 B FillBoundary_nowait() 4936 4892 47 KiB 47 KiB 48 KiB 51 KiB 52 KiB 52 KiB 13 KiB FillBoundary_finish() 4936 4892 47 KiB 47 KiB 48 KiB 51 KiB 52 KiB 52 KiB 13 KiB MultiFab::contains_nan() 52 48 261 B 261 B 262 B 6656 B 6656 B 6656 B 256 B MultiFab::contains_inf() 40 40 6 B 6 B 7 B 6144 B 6144 B 6144 B 0 B impose_cons_bcs() 88 88 0 B 0 B 0 B 256 B 256 B 256 B 0 B impose_xvel_bcs() 44 44 0 B 0 B 0 B 256 B 256 B 256 B 0 B impose_yvel_bcs() 44 44 0 B 0 B 0 B 256 B 256 B 256 B 0 B ---------------------------------------------------------------------------------------------------------------------------------------------- Async Memory Usage: ------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ------------------------------------------------------------------------------------------------------------- REMORA::prsgrd() 40 106 KiB 131 KiB 180 KiB 78 MiB 78 MiB 78 MiB REMORA::advance_3d() 600 2994 KiB 3019 KiB 3031 KiB 53 MiB 53 MiB 53 MiB REMORA::prestep_t_advection() 120 732 KiB 817 KiB 923 KiB 52 MiB 52 MiB 52 MiB REMORA::rhs_t_3d() 480 428 KiB 442 KiB 463 KiB 52 MiB 52 MiB 52 MiB REMORA::uv3dmix() 2360 128 KiB 144 KiB 172 KiB 52 MiB 52 MiB 52 MiB REMORA::rhs_uv_3d() 160 666 KiB 672 KiB 678 KiB 50 MiB 50 MiB 50 MiB REMORA::t3dmix2_s() 160 505 KiB 516 KiB 542 KiB 50 MiB 50 MiB 50 MiB REMORA::setup_step() 80 572 KiB 578 KiB 587 KiB 13 MiB 13 MiB 13 MiB REMORA::prestep() 40 196 KiB 217 KiB 244 KiB 13 MiB 13 MiB 13 MiB REMORA::rhs_uv_2d() 2320 28 KiB 29 KiB 31 KiB 840 KiB 840 KiB 840 KiB REMORA::advance2d() 2360 6091 B 6134 B 6169 B 215 KiB 215 KiB 215 KiB REMORA::update_massflux_3d() 80 549 B 628 B 681 B 210 KiB 210 KiB 210 KiB ------------------------------------------------------------------------------------------------------------- Managed Memory Usage: --------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------- The_Managed_Arena::Initialize() 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------- Pinned Memory Usage: ------------------------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max CurrentMem max ------------------------------------------------------------------------------------------------------------------------------------- VisMF::Write(FabArray) 80 80 1674 KiB 1729 KiB 1782 KiB 62 MiB 62 MiB 62 MiB 0 B The_Pinned_Arena::Initialize() 4 4 0 B 0 B 1 B 8192 KiB 8192 KiB 8192 KiB 0 B FillBoundary_nowait() 36400 36356 276 KiB 287 KiB 293 KiB 5917 KiB 5917 KiB 5918 KiB 13 KiB FillBoundary_finish() 4936 4892 47 KiB 47 KiB 48 KiB 51 KiB 52 KiB 52 KiB 13 KiB MultiFab::contains_nan() 52 48 253 B 253 B 254 B 768 B 768 B 768 B 256 B REMORA::estTimeStep() 88 88 1 B 1 B 1 B 512 B 512 B 512 B 0 B MultiFab::contains_inf() 40 40 0 B 0 B 0 B 256 B 256 B 256 B 0 B ------------------------------------------------------------------------------------------------------------------------------------- Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [3315 ... 3315] [The Arena] max space (MB) allocated spread across MPI: [1024 ... 1024] [The Arena] max space (MB) used spread across MPI: [891 ... 891] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [133 ... 133] [The Pinned Arena] max space (MB) used spread across MPI: [62 ... 62] AMReX (26.06-11-g9ed35ab0e584) finalized