Initializing AMReX (26.09-175-gf2e8bf0a946a)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.09-175-gf2e8bf0a946a) initialized Successfully read inputs file ... REMORA git hash: v1.7-139-gf0e94a91 AMReX git hash: 26.09-175-gf2e8bf0a94 WARNING: subcycling refined levels in time is EXPERIMENTAL. Multi-level answers are not yet considered production quality; check them against a single-level run before relying on them. remora.do_substep = 0 selects the older lockstep driver instead. [REMORA] Active E-P source: bulk evap-rain diagnostic. Refinement ratio at level 0 set to be 3 3 1 Creating new distribution map on level: 0 Making level 0 from scratch GRIDS AT LEVEL 0 ARE (BoxArray maxbox(4) m_ref->m_hash_sig(0) ((0,0,0) (4,4,3) (0,0,0)) ((5,0,0) (9,4,3) (0,0,0)) ((0,5,0) (4,9,3) (0,0,0)) ((5,5,0) (9,9,3) (0,0,0)) ) Reading high resolution land-sea mask Loading high resolution land/sea mask from NetCDF file hires_grd.nc Done reading in high resolution land-sea mask Reading high resolution bathymetry and grid data Loading high resolution bathymetry from NetCDF file hires_grd.nc Loading high resolution grid variables from NetCDF file hires_grd.nc Done reading in high resolution bathymetry and grid data Reading high resolution initial data Loading initial solution data from NetCDF file hires_ini.nc Loading initial sea surface height from NetCDF file hires_ini.nc Done reading in high resolution initial data Land-sea masks are consistent across 1 level(s) Writing plotfile IdealMiniGrid_synth_hires_mask-restart_plt00000 TIME = 0 KIN. ENG. = 1336297.39277395 VOLUME = 9506731758.71372 MAX. VEL. = 0.0509512910885042 Using cfl = 0.8 advective limit at level 0: 9045.579144 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 Coarse STEP 1 starts ... Using cfl = 0.8 advective limit at level 0: 9045.579144 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 Timestep hierarchy (remora.do_substep = 1, remora.ndtfast = 10, nfast = 15) ================== Level Ref ratio Substeps Fast steps Slow dt Fast dt per lev-1 per lev-0 (s) (s) 0 --- 1 10 10.0000 1.0000 [Level 0 step 1] ADVANCE from time = 0 to 10 with dt = 10 [Level 0 step 1] Advanced 400 cells Coarse STEP 1 ends. TIME = 10 DT = 10 Timestep time = 0.401724628 seconds. TIME = 10 KIN. ENG. = 1340257.53414811 VOLUME = 9506691527.70458 MAX. VEL. = 0.0509910162936999 Coarse STEP 2 starts ... Using cfl = 0.8 advective limit at level 0: 8842.13496 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 [Level 0 step 2] ADVANCE from time = 10 to 20 with dt = 10 [Level 0 step 2] Advanced 400 cells Coarse STEP 2 ends. TIME = 20 DT = 10 Timestep time = 0.395140729 seconds. TIME = 20 KIN. ENG. = 1430419.42592461 VOLUME = 9506594049.5866 MAX. VEL. = 0.0513597871094688 Coarse STEP 3 starts ... Using cfl = 0.8 advective limit at level 0: 8422.957484 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 Land-sea masks are consistent across 1 level(s) [Level 0 step 3] ADVANCE from time = 20 to 30 with dt = 10 [Level 0 step 3] Advanced 400 cells Coarse STEP 3 ends. TIME = 30 DT = 10 Timestep time = 0.404331774 seconds. TIME = 30 KIN. ENG. = 1539924.65209297 VOLUME = 9506447867.8891 MAX. VEL. = 0.0529137178576718 Coarse STEP 4 starts ... Using cfl = 0.8 advective limit at level 0: 8043.587555 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 [Level 0 step 4] ADVANCE from time = 30 to 40 with dt = 10 [Level 0 step 4] Advanced 400 cells Coarse STEP 4 ends. TIME = 40 DT = 10 Timestep time = 0.395023921 seconds. TIME = 40 KIN. ENG. = 1690624.55477588 VOLUME = 9506287812.42694 MAX. VEL. = 0.0554574934182323 Coarse STEP 5 starts ... Using cfl = 0.8 advective limit at level 0: 7588.769378 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 Land-sea masks are consistent across 1 level(s) [Level 0 step 5] ADVANCE from time = 40 to 50 with dt = 10 [Level 0 step 5] Advanced 400 cells Coarse STEP 5 ends. TIME = 50 DT = 10 Timestep time = 0.404317015 seconds. TIME = 50 KIN. ENG. = 1952572.4723585 VOLUME = 9506152500.7227 MAX. VEL. = 0.0594326008044683 Coarse STEP 6 starts ... Using cfl = 0.8 advective limit at level 0: 6913.349021 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 [Level 0 step 6] ADVANCE from time = 50 to 60 with dt = 10 [Level 0 step 6] Advanced 400 cells Coarse STEP 6 ends. TIME = 60 DT = 10 Timestep time = 0.39512895 seconds. TIME = 60 KIN. ENG. = 2358212.53029683 VOLUME = 9506071414.43333 MAX. VEL. = 0.0639705423679357 Coarse STEP 7 starts ... Using cfl = 0.8 advective limit at level 0: 6351.68573 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 Land-sea masks are consistent across 1 level(s) [Level 0 step 7] ADVANCE from time = 60 to 70 with dt = 10 [Level 0 step 7] Advanced 400 cells Coarse STEP 7 ends. TIME = 70 DT = 10 Timestep time = 0.404267813 seconds. TIME = 70 KIN. ENG. = 2945495.68596525 VOLUME = 9506064763.19713 MAX. VEL. = 0.066587368139088 Coarse STEP 8 starts ... Using cfl = 0.8 advective limit at level 0: 6043.928651 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 [Level 0 step 8] ADVANCE from time = 70 to 80 with dt = 10 [Level 0 step 8] Advanced 400 cells Coarse STEP 8 ends. TIME = 80 DT = 10 Timestep time = 0.394908301 seconds. TIME = 80 KIN. ENG. = 3718858.10992821 VOLUME = 9506140445.08016 MAX. VEL. = 0.0708275704130908 Coarse STEP 9 starts ... Using cfl = 0.8 advective limit at level 0: 5896.404601 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 Land-sea masks are consistent across 1 level(s) [Level 0 step 9] ADVANCE from time = 80 to 90 with dt = 10 [Level 0 step 9] Advanced 400 cells Coarse STEP 9 ends. TIME = 90 DT = 10 Timestep time = 0.404086579 seconds. TIME = 90 KIN. ENG. = 4669567.01973744 VOLUME = 9506291435.6012 MAX. VEL. = 0.0756018223066024 Coarse STEP 10 starts ... Using cfl = 0.8 advective limit at level 0: 5548.209341 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 [Level 0 step 10] ADVANCE from time = 90 to 100 with dt = 10 [Level 0 step 10] Advanced 400 cells Coarse STEP 10 ends. TIME = 100 DT = 10 Timestep time = 0.394941794 seconds. TIME = 100 KIN. ENG. = 5845181.82263203 VOLUME = 9506497422.13087 MAX. VEL. = 0.0793376038354893 Writing plotfile IdealMiniGrid_synth_hires_mask-restart_plt00010 Total Time: 4.545374687 Unused ParmParse Variables: [TOP]::amr.check_int(nvals = 1) :: [5] [TOP]::amr.checkpoint_files_output(nvals = 1) :: [1] Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [3315 ... 6387] [The Arena] max space (MB) allocated spread across MPI: [1024 ... 1024] [The Arena] max space (MB) used spread across MPI: [9 ... 9] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Pinned Arena] max space (MB) used spread across MPI: [0 ... 0] AMReX (26.09-175-gf2e8bf0a946a) finalized Initializing AMReX (26.09-175-gf2e8bf0a946a)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.09-175-gf2e8bf0a946a) initialized Successfully read inputs file ... REMORA git hash: v1.7-139-gf0e94a91 AMReX git hash: 26.09-175-gf2e8bf0a94 WARNING: subcycling refined levels in time is EXPERIMENTAL. Multi-level answers are not yet considered production quality; check them against a single-level run before relying on them. remora.do_substep = 0 selects the older lockstep driver instead. [REMORA] Active E-P source: bulk evap-rain diagnostic. Refinement ratio at level 0 set to be 3 3 1 Creating new distribution map on level: 0 Making level 0 from scratch GRIDS AT LEVEL 0 ARE (BoxArray maxbox(4) m_ref->m_hash_sig(0) ((0,0,0) (4,4,3) (0,0,0)) ((5,0,0) (9,4,3) (0,0,0)) ((0,5,0) (4,9,3) (0,0,0)) ((5,5,0) (9,9,3) (0,0,0)) ) Reading high resolution land-sea mask Loading high resolution land/sea mask from NetCDF file hires_grd.nc Done reading in high resolution land-sea mask Reading high resolution bathymetry and grid data Loading high resolution bathymetry from NetCDF file hires_grd.nc Loading high resolution grid variables from NetCDF file hires_grd.nc Done reading in high resolution bathymetry and grid data Reading high resolution initial data Loading initial solution data from NetCDF file hires_ini.nc Loading initial sea surface height from NetCDF file hires_ini.nc Done reading in high resolution initial data Land-sea masks are consistent across 1 level(s) Writing plotfile IdealMiniGrid_synth_hires_mask-restart_plt00000 TIME = 0 KIN. ENG. = 1336297.39277395 VOLUME = 9506731758.71372 MAX. VEL. = 0.0509512910885042 Using cfl = 0.8 advective limit at level 0: 9045.579144 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 Coarse STEP 1 starts ... Using cfl = 0.8 advective limit at level 0: 9045.579144 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 Timestep hierarchy (remora.do_substep = 1, remora.ndtfast = 10, nfast = 15) ================== Level Ref ratio Substeps Fast steps Slow dt Fast dt per lev-1 per lev-0 (s) (s) 0 --- 1 10 10.0000 1.0000 [Level 0 step 1] ADVANCE from time = 0 to 10 with dt = 10 [Level 0 step 1] Advanced 400 cells Coarse STEP 1 ends. TIME = 10 DT = 10 Timestep time = 0.399157071 seconds. TIME = 10 KIN. ENG. = 1340257.53414811 VOLUME = 9506691527.70458 MAX. VEL. = 0.0509910162936999 Coarse STEP 2 starts ... Using cfl = 0.8 advective limit at level 0: 8842.13496 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 [Level 0 step 2] ADVANCE from time = 10 to 20 with dt = 10 [Level 0 step 2] Advanced 400 cells Coarse STEP 2 ends. TIME = 20 DT = 10 Timestep time = 0.395061243 seconds. TIME = 20 KIN. ENG. = 1430419.42592461 VOLUME = 9506594049.5866 MAX. VEL. = 0.0513597871094688 Coarse STEP 3 starts ... Using cfl = 0.8 advective limit at level 0: 8422.957484 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 Land-sea masks are consistent across 1 level(s) [Level 0 step 3] ADVANCE from time = 20 to 30 with dt = 10 [Level 0 step 3] Advanced 400 cells Coarse STEP 3 ends. TIME = 30 DT = 10 Timestep time = 0.404310726 seconds. TIME = 30 KIN. ENG. = 1539924.65209297 VOLUME = 9506447867.8891 MAX. VEL. = 0.0529137178576718 Coarse STEP 4 starts ... Using cfl = 0.8 advective limit at level 0: 8043.587555 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 [Level 0 step 4] ADVANCE from time = 30 to 40 with dt = 10 [Level 0 step 4] Advanced 400 cells Coarse STEP 4 ends. TIME = 40 DT = 10 Timestep time = 0.39485183 seconds. TIME = 40 KIN. ENG. = 1690624.55477588 VOLUME = 9506287812.42694 MAX. VEL. = 0.0554574934182323 Coarse STEP 5 starts ... Using cfl = 0.8 advective limit at level 0: 7588.769378 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 Land-sea masks are consistent across 1 level(s) [Level 0 step 5] ADVANCE from time = 40 to 50 with dt = 10 [Level 0 step 5] Advanced 400 cells Coarse STEP 5 ends. TIME = 50 DT = 10 Timestep time = 0.404221571 seconds. TIME = 50 KIN. ENG. = 1952572.4723585 VOLUME = 9506152500.7227 MAX. VEL. = 0.0594326008044683 Coarse STEP 6 starts ... Using cfl = 0.8 advective limit at level 0: 6913.349021 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 [Level 0 step 6] ADVANCE from time = 50 to 60 with dt = 10 [Level 0 step 6] Advanced 400 cells Coarse STEP 6 ends. TIME = 60 DT = 10 Timestep time = 0.39488711 seconds. TIME = 60 KIN. ENG. = 2358212.53029683 VOLUME = 9506071414.43333 MAX. VEL. = 0.0639705423679357 Coarse STEP 7 starts ... Using cfl = 0.8 advective limit at level 0: 6351.68573 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 Land-sea masks are consistent across 1 level(s) [Level 0 step 7] ADVANCE from time = 60 to 70 with dt = 10 [Level 0 step 7] Advanced 400 cells Coarse STEP 7 ends. TIME = 70 DT = 10 Timestep time = 0.404271544 seconds. TIME = 70 KIN. ENG. = 2945495.68596525 VOLUME = 9506064763.19713 MAX. VEL. = 0.066587368139088 Coarse STEP 8 starts ... Using cfl = 0.8 advective limit at level 0: 6043.928651 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 [Level 0 step 8] ADVANCE from time = 70 to 80 with dt = 10 [Level 0 step 8] Advanced 400 cells Coarse STEP 8 ends. TIME = 80 DT = 10 Timestep time = 0.394920782 seconds. TIME = 80 KIN. ENG. = 3718858.10992821 VOLUME = 9506140445.08016 MAX. VEL. = 0.0708275704130908 Coarse STEP 9 starts ... Using cfl = 0.8 advective limit at level 0: 5896.404601 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 Land-sea masks are consistent across 1 level(s) [Level 0 step 9] ADVANCE from time = 80 to 90 with dt = 10 [Level 0 step 9] Advanced 400 cells Coarse STEP 9 ends. TIME = 90 DT = 10 Timestep time = 0.404204934 seconds. TIME = 90 KIN. ENG. = 4669567.01973744 VOLUME = 9506291435.6012 MAX. VEL. = 0.0756018223066024 Coarse STEP 10 starts ... Using cfl = 0.8 advective limit at level 0: 5548.209341 gravity wave limit at level 0: 41.5920109 (ndtfast = 10) Slow dt at level 0 would be: 41.5920109 Fixed dt at level 0 is: 10 [Level 0 step 10] ADVANCE from time = 90 to 100 with dt = 10 [Level 0 step 10] Advanced 400 cells Coarse STEP 10 ends. TIME = 100 DT = 10 Timestep time = 0.395086811 seconds. TIME = 100 KIN. ENG. = 5845181.82263203 VOLUME = 9506497422.13087 MAX. VEL. = 0.0793376038354893 Writing plotfile IdealMiniGrid_synth_hires_mask-restart_plt00010 Total Time: 4.38392172 Unused ParmParse Variables: [TOP]::amr.checkpoint_files_output(nvals = 1) :: [0] [TOP]::amr.restart(nvals = 1) :: [IdealMiniGrid_synth_hires_mask-restart_chk00005] Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [3315 ... 4339] [The Arena] max space (MB) allocated spread across MPI: [1024 ... 1024] [The Arena] max space (MB) used spread across MPI: [9 ... 9] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Pinned Arena] max space (MB) used spread across MPI: [0 ... 0] AMReX (26.09-175-gf2e8bf0a946a) finalized