Initializing AMReX (26.09-185-g19d64c697f1a)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.09-185-g19d64c697f1a) initialized Successfully read inputs file ... Successfully read inputs file ... Starting to call amrex_probinit ... Successfully run amrex_probinit Now regridding at level lbase = 0 grid_places() time: 0.019026963 new finest: 1 STEP = 0 TIME = 0 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 INITIAL GRIDS Level 0 16 grids 4194304 cells 100 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 PLOTFILE: file = CNS-RT_plt00000 grid_places() time: 0.019698156 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 0 Total y-momentum is 0 Total z-momentum is 0 Total energy is 7176192.0007176176 Write plotfile time = 0.15347515 seconds [Level 0 step 1] ADVANCE at time 0 with dt = 0.0002870776155 [Level 0 step 1] Advanced 4194304 cells [Level 1 step 1] ADVANCE at time 0 with dt = 0.0001435388078 [Level 1 step 1] Advanced 4194304 cells [Level 1 step 2] ADVANCE at time 0.0001435388078 with dt = 0.0001435388078 [Level 1 step 2] Advanced 4194304 cells STEP = 1 TIME = 0.0002870776155 DT = 0.0002870776155 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 3.4600423210458683e-21 Total y-momentum is 2.0479171591388077e-21 Total z-momentum is 1.0569463681968465 Total energy is 7176192.0005832035 [STEP 1] Coarse TimeStep time: 0.689194447 [STEP 1] FAB kilobyte spread across MPI nodes: [536184 ... 544157] [Level 0 step 2] ADVANCE at time 0.0002870776155 with dt = 0.0002870672688 [Level 0 step 2] Advanced 4194304 cells [Level 1 step 3] ADVANCE at time 0.0002870776155 with dt = 0.0001435336344 [Level 1 step 3] Advanced 4194304 cells [Level 1 step 4] ADVANCE at time 0.0004306112499 with dt = 0.0001435336344 [Level 1 step 4] Advanced 4194304 cells STEP = 2 TIME = 0.0005741448843 DT = 0.0002870672688 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -2.0746744576564437e-20 Total y-momentum is 7.1848555606412404e-21 Total z-momentum is 2.3209418694909116 Total energy is 7176192.0001132824 [STEP 2] Coarse TimeStep time: 0.666669029 [STEP 2] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 2 TIME = 0.0005741448843 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 3] ADVANCE at time 0.0005741448843 with dt = 0.0002870608582 [Level 0 step 3] Advanced 4194304 cells [Level 1 step 5] ADVANCE at time 0.0005741448843 with dt = 0.0001435304291 [Level 1 step 5] Advanced 4194304 cells [Level 1 step 6] ADVANCE at time 0.0007176753134 with dt = 0.0001435304291 [Level 1 step 6] Advanced 4194304 cells STEP = 3 TIME = 0.0008612057425 DT = 0.0002870608582 grid_places() time: 0.015928904 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 7.5708260554279181e-20 Total y-momentum is 1.0853671400560117e-19 Total z-momentum is 3.7623957612122796 Total energy is 7176191.9992529377 [STEP 3] Coarse TimeStep time: 0.703326381 [STEP 3] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 4] ADVANCE at time 0.0008612057425 with dt = 0.00028705672 [Level 0 step 4] Advanced 4194304 cells [Level 1 step 7] ADVANCE at time 0.0008612057425 with dt = 0.00014352836 [Level 1 step 7] Advanced 4194304 cells [Level 1 step 8] ADVANCE at time 0.001004734103 with dt = 0.00014352836 [Level 1 step 8] Advanced 4194304 cells STEP = 4 TIME = 0.001148262463 DT = 0.00028705672 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 7.3619098490785573e-20 Total y-momentum is -3.3770302497149155e-19 Total z-momentum is 5.3553134404616571 Total energy is 7176191.9979552217 [STEP 4] Coarse TimeStep time: 0.664765862 [STEP 4] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 4 TIME = 0.001148262463 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 5] ADVANCE at time 0.001148262463 with dt = 0.0002870541933 [Level 0 step 5] Advanced 4194304 cells [Level 1 step 9] ADVANCE at time 0.001148262463 with dt = 0.0001435270967 [Level 1 step 9] Advanced 4194304 cells [Level 1 step 10] ADVANCE at time 0.001291789559 with dt = 0.0001435270967 [Level 1 step 10] Advanced 4194304 cells STEP = 5 TIME = 0.001435316656 DT = 0.0002870541933 grid_places() time: 0.01602515 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -3.2608222937573232e-19 Total y-momentum is -2.6424116081489047e-19 Total z-momentum is 7.0771445174316776 Total energy is 7176191.9961801395 [STEP 5] Coarse TimeStep time: 0.699901159 [STEP 5] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 6] ADVANCE at time 0.001435316656 with dt = 0.0002870527674 [Level 0 step 6] Advanced 4194304 cells [Level 1 step 11] ADVANCE at time 0.001435316656 with dt = 0.0001435263837 [Level 1 step 11] Advanced 4194304 cells [Level 1 step 12] ADVANCE at time 0.00157884304 with dt = 0.0001435263837 [Level 1 step 12] Advanced 4194304 cells STEP = 6 TIME = 0.001722369423 DT = 0.0002870527674 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 2.1894415462558567e-19 Total y-momentum is 1.0192467340569854e-18 Total z-momentum is 8.9084964335707468 Total energy is 7176191.9938936941 [STEP 6] Coarse TimeStep time: 0.662514668 [STEP 6] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 6 TIME = 0.001722369423 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 7] ADVANCE at time 0.001722369423 with dt = 0.0002870520599 [Level 0 step 7] Advanced 4194304 cells [Level 1 step 13] ADVANCE at time 0.001722369423 with dt = 0.00014352603 [Level 1 step 13] Advanced 4194304 cells [Level 1 step 14] ADVANCE at time 0.001865895453 with dt = 0.00014352603 [Level 1 step 14] Advanced 4194304 cells STEP = 7 TIME = 0.002009421483 DT = 0.0002870520599 grid_places() time: 0.01612765 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -7.6990262974807339e-19 Total y-momentum is -1.5253840639083534e-18 Total z-momentum is 10.832804311097531 Total energy is 7176191.9910670239 [STEP 7] Coarse TimeStep time: 0.698823481 [STEP 7] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 8] ADVANCE at time 0.002009421483 with dt = 0.0002870517947 [Level 0 step 8] Advanced 4194304 cells [Level 1 step 15] ADVANCE at time 0.002009421483 with dt = 0.0001435258973 [Level 1 step 15] Advanced 4194304 cells [Level 1 step 16] ADVANCE at time 0.002152947381 with dt = 0.0001435258973 [Level 1 step 16] Advanced 4194304 cells STEP = 8 TIME = 0.002296473278 DT = 0.0002870517947 [CNS] Total mass is 1302338.0233411889 Total x-momentum is -1.7306562615022928e-18 Total y-momentum is 2.646116891001415e-18 Total z-momentum is 12.835981478053229 Total energy is 7176191.9876756454 [STEP 8] Coarse TimeStep time: 0.66212134 [STEP 8] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 8 TIME = 0.002296473278 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 9] ADVANCE at time 0.002296473278 with dt = 0.00028705178 [Level 0 step 9] Advanced 4194304 cells [Level 1 step 17] ADVANCE at time 0.002296473278 with dt = 0.00014352589 [Level 1 step 17] Advanced 4194304 cells [Level 1 step 18] ADVANCE at time 0.002439999168 with dt = 0.00014352589 [Level 1 step 18] Advanced 4194304 cells STEP = 9 TIME = 0.002583525058 DT = 0.00028705178 grid_places() time: 0.016235801 new finest: 1 [CNS] Total mass is 1302338.0233411889 Total x-momentum is 5.6810854521708086e-19 Total y-momentum is 1.2936874203733604e-18 Total z-momentum is 14.906076194007598 Total energy is 7176191.9836987816 [STEP 9] Coarse TimeStep time: 0.697820163 [STEP 9] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 10] ADVANCE at time 0.002583525058 with dt = 0.0002870518889 [Level 0 step 10] Advanced 4194304 cells [Level 1 step 19] ADVANCE at time 0.002583525058 with dt = 0.0001435259444 [Level 1 step 19] Advanced 4194304 cells [Level 1 step 20] ADVANCE at time 0.002727051002 with dt = 0.0001435259444 [Level 1 step 20] Advanced 4194304 cells STEP = 10 TIME = 0.002870576947 DT = 0.0002870518889 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 1.376067658812568e-18 Total y-momentum is 8.7411424597322119e-19 Total z-momentum is 17.032951759614733 Total energy is 7176191.979118811 [STEP 10] Coarse TimeStep time: 0.660813075 [STEP 10] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 10 TIME = 0.002870576947 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 11] ADVANCE at time 0.002870576947 with dt = 0.0002870520422 [Level 0 step 11] Advanced 4194304 cells [Level 1 step 21] ADVANCE at time 0.002870576947 with dt = 0.0001435260211 [Level 1 step 21] Advanced 4194304 cells [Level 1 step 22] ADVANCE at time 0.003014102968 with dt = 0.0001435260211 [Level 1 step 22] Advanced 4194304 cells STEP = 11 TIME = 0.003157628989 DT = 0.0002870520422 grid_places() time: 0.0162526 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 8.8714661369221303e-19 Total y-momentum is 4.7158369086842284e-18 Total z-momentum is 19.208000131229269 Total energy is 7176191.9739207756 [STEP 11] Coarse TimeStep time: 0.697573271 [STEP 11] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 12] ADVANCE at time 0.003157628989 with dt = 0.0002870521946 [Level 0 step 12] Advanced 4194304 cells [Level 1 step 23] ADVANCE at time 0.003157628989 with dt = 0.0001435260973 [Level 1 step 23] Advanced 4194304 cells [Level 1 step 24] ADVANCE at time 0.003301155086 with dt = 0.0001435260973 [Level 1 step 24] Advanced 4194304 cells STEP = 12 TIME = 0.003444681184 DT = 0.0002870521946 [CNS] Total mass is 1302338.0233411889 Total x-momentum is 2.198810717002711e-18 Total y-momentum is 7.0752732394503774e-18 Total z-momentum is 21.423893568568744 Total energy is 7176191.9680919927 [STEP 12] Coarse TimeStep time: 0.660136939 [STEP 12] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 12 TIME = 0.003444681184 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 13] ADVANCE at time 0.003444681184 with dt = 0.0002870523239 [Level 0 step 13] Advanced 4194304 cells [Level 1 step 25] ADVANCE at time 0.003444681184 with dt = 0.0001435261619 [Level 1 step 25] Advanced 4194304 cells [Level 1 step 26] ADVANCE at time 0.003588207345 with dt = 0.0001435261619 [Level 1 step 26] Advanced 4194304 cells STEP = 13 TIME = 0.003731733507 DT = 0.0002870523239 grid_places() time: 0.016344961 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -4.0390711608099386e-18 Total y-momentum is -7.3165912148932354e-18 Total z-momentum is 23.674374717272059 Total energy is 7176191.9616217129 [STEP 13] Coarse TimeStep time: 0.701399657 [STEP 13] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 14] ADVANCE at time 0.003731733507 with dt = 0.000287052422 [Level 0 step 14] Advanced 4194304 cells [Level 1 step 27] ADVANCE at time 0.003731733507 with dt = 0.000143526211 [Level 1 step 27] Advanced 4194304 cells [Level 1 step 28] ADVANCE at time 0.003875259718 with dt = 0.000143526211 [Level 1 step 28] Advanced 4194304 cells STEP = 14 TIME = 0.004018785929 DT = 0.000287052422 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -1.4254982508844783e-18 Total y-momentum is 1.015700761020574e-18 Total z-momentum is 25.95408273901171 Total energy is 7176191.9545008428 [STEP 14] Coarse TimeStep time: 0.664170595 [STEP 14] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 14 TIME = 0.004018785929 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 15] ADVANCE at time 0.004018785929 with dt = 0.0002870524893 [Level 0 step 15] Advanced 4194304 cells [Level 1 step 29] ADVANCE at time 0.004018785929 with dt = 0.0001435262446 [Level 1 step 29] Advanced 4194304 cells [Level 1 step 30] ADVANCE at time 0.004162312174 with dt = 0.0001435262446 [Level 1 step 30] Advanced 4194304 cells STEP = 15 TIME = 0.004305838419 DT = 0.0002870524893 grid_places() time: 0.016386669 new finest: 1 [CNS] Total mass is 1302338.0233411889 Total x-momentum is -3.8096209963360823e-19 Total y-momentum is -8.7918912827820473e-19 Total z-momentum is 28.258411443578304 Total energy is 7176191.9467217075 [STEP 15] Coarse TimeStep time: 0.701020533 [STEP 15] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 16] ADVANCE at time 0.004305838419 with dt = 0.0002870525306 [Level 0 step 16] Advanced 4194304 cells [Level 1 step 31] ADVANCE at time 0.004305838419 with dt = 0.0001435262653 [Level 1 step 31] Advanced 4194304 cells [Level 1 step 32] ADVANCE at time 0.004449364684 with dt = 0.0001435262653 [Level 1 step 32] Advanced 4194304 cells STEP = 16 TIME = 0.004592890949 DT = 0.0002870525306 [CNS] Total mass is 1302338.0233411889 Total x-momentum is 5.4730419547846157e-18 Total y-momentum is 6.7296237746774245e-18 Total z-momentum is 30.583394611415127 Total energy is 7176191.9382778564 [STEP 16] Coarse TimeStep time: 0.663519132 [STEP 16] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 16 TIME = 0.004592890949 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 17] ADVANCE at time 0.004592890949 with dt = 0.000287052552 [Level 0 step 17] Advanced 4194304 cells [Level 1 step 33] ADVANCE at time 0.004592890949 with dt = 0.000143526276 [Level 1 step 33] Advanced 4194304 cells [Level 1 step 34] ADVANCE at time 0.004736417225 with dt = 0.000143526276 [Level 1 step 34] Advanced 4194304 cells STEP = 17 TIME = 0.004879943501 DT = 0.000287052552 grid_places() time: 0.016223483 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -9.2035317838578576e-18 Total y-momentum is -2.6665374729341175e-18 Total z-momentum is 32.925613582717489 Total energy is 7176191.9291638918 [STEP 17] Coarse TimeStep time: 0.699623174 [STEP 17] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 18] ADVANCE at time 0.004879943501 with dt = 0.0002870525597 [Level 0 step 18] Advanced 4194304 cells [Level 1 step 35] ADVANCE at time 0.004879943501 with dt = 0.0001435262798 [Level 1 step 35] Advanced 4194304 cells [Level 1 step 36] ADVANCE at time 0.005023469781 with dt = 0.0001435262798 [Level 1 step 36] Advanced 4194304 cells STEP = 18 TIME = 0.005166996061 DT = 0.0002870525597 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -1.3908635798758459e-17 Total y-momentum is -1.6212768957360782e-18 Total z-momentum is 35.282122513822877 Total energy is 7176191.9193753228 [STEP 18] Coarse TimeStep time: 0.662694526 [STEP 18] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 18 TIME = 0.005166996061 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 19] ADVANCE at time 0.005166996061 with dt = 0.0002870525588 [Level 0 step 19] Advanced 4194304 cells [Level 1 step 37] ADVANCE at time 0.005166996061 with dt = 0.0001435262794 [Level 1 step 37] Advanced 4194304 cells [Level 1 step 38] ADVANCE at time 0.00531052234 with dt = 0.0001435262794 [Level 1 step 38] Advanced 4194304 cells STEP = 19 TIME = 0.00545404862 DT = 0.0002870525588 grid_places() time: 0.016221292 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -3.043424857689809e-19 Total y-momentum is 1.1848493735178941e-19 Total z-momentum is 37.650387281643575 Total energy is 7176191.9089084426 [STEP 19] Coarse TimeStep time: 0.699016443 [STEP 19] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 20] ADVANCE at time 0.00545404862 with dt = 0.0002870525536 [Level 0 step 20] Advanced 4194304 cells [Level 1 step 39] ADVANCE at time 0.00545404862 with dt = 0.0001435262768 [Level 1 step 39] Advanced 4194304 cells [Level 1 step 40] ADVANCE at time 0.005597574896 with dt = 0.0001435262768 [Level 1 step 40] Advanced 4194304 cells STEP = 20 TIME = 0.005741101173 DT = 0.0002870525536 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 7.3173620118177381e-18 Total y-momentum is -7.3889842784956805e-18 Total z-momentum is 40.028234710671775 Total energy is 7176191.8977602143 [STEP 20] Coarse TimeStep time: 0.661901568 [STEP 20] FAB kilobyte spread across MPI nodes: [575320 ... 575320] PLOTFILE: file = CNS-RT_plt00020 Write plotfile time = 0.145603385 seconds Run Time total = 14.03015656 Run Time init = 0.270023547 Run Time advance = 13.61428737 TinyProfiler total time across processes [min...avg...max]: 14.02 ... 14.02 ... 14.03 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- CNS::compute_dSdt() 120 8.031 8.173 8.344 59.47% FabArray::ParallelCopy_finish() 572 0.5057 1.105 1.72 12.26% StateDataPhysBCFunct::() 220 0.001079 0.8556 1.716 12.23% FabArray::ParallelCopy_nowait() 572 0.4699 0.7044 0.9557 6.81% FluxRegister::Reflux() 120 0.6218 0.6525 0.683 4.87% RungeKutta2 60 0.3086 0.3434 0.3796 2.71% VisMF::Write(FabArray) 4 0.2742 0.2774 0.2823 2.01% FabArray::setVal() 282 0.2183 0.2515 0.2801 2.00% CNS::computeTemp() 144 0.1504 0.1882 0.2324 1.66% FillBoundary_finish() 80 0.1402 0.1772 0.2221 1.58% FillBoundary_nowait() 80 0.1574 0.1795 0.2109 1.50% CellConservativeLinear::interp() 120 0.03213 0.09031 0.1499 1.07% FillPatcher::fillCFB 80 0.09879 0.1205 0.138 0.98% FillPatchIterator::Initialize 60 0.1145 0.1175 0.1213 0.86% amrex::Copy() 88 0.08063 0.105 0.1207 0.86% FillPatchInterp(Fab) 80 0.01728 0.05549 0.1206 0.86% Amr::regrid() 10 0.1078 0.1083 0.109 0.78% amrex::average_down 21 0.07609 0.09404 0.1088 0.78% CNS::estTimeStep() 41 0.05825 0.07403 0.09352 0.67% FabArray::sum() 105 0.05591 0.07371 0.08993 0.64% TagBoxArray::collate() 11 0.001752 0.03748 0.07362 0.52% CNS::advance() 60 0.06563 0.06953 0.07349 0.52% FabArray::setDomainBndry() 60 0.0001879 0.02423 0.04979 0.35% post_timestep 60 0.02997 0.03395 0.03743 0.27% StateData::define() 12 0.02153 0.02335 0.02488 0.18% CNS::errorEst() 11 0.02073 0.02212 0.0234 0.17% Amr::bldFineLevels() 1 0.01258 0.01263 0.01269 0.09% AmrLevel::writePlotFile() 4 0.003838 0.006526 0.01033 0.07% TagBoxArray::mapPRD 11 0.003894 0.00658 0.009454 0.07% AmrMesh::MakeNewGrids() 11 0.006078 0.007666 0.009434 0.07% Amr::writePlotFile() 2 0.001345 0.003237 0.004266 0.03% Amr::coarseTimeStep() 20 0.0002799 0.001245 0.002403 0.02% Amr::timeStep() 60 0.000628 0.0008694 0.001403 0.01% AmrLevel::FillPatch() 60 0.000643 0.0007055 0.0008006 0.01% FabArray::ParallelCopy() 572 0.0005333 0.0005792 0.000609 0.00% FillPatchSingleLevel 140 0.0002351 0.0002728 0.0003423 0.00% AmrLevel::FillPatcherFill() 120 0.0002571 0.0002817 0.0002952 0.00% main() 1 0.0001878 0.0002332 0.0002672 0.00% FillPatcher::fill() 80 0.0001438 0.0001594 0.0001879 0.00% FabArray::FillBoundary() 80 0.0001295 0.0001435 0.0001642 0.00% postCoarseTimeStep() 20 3.515e-05 5.211e-05 7.484e-05 0.00% FillPatchIterator::FillFromLevel0() 51 6.115e-05 6.509e-05 6.953e-05 0.00% Amr::grid_places() 11 5.528e-05 5.805e-05 5.93e-05 0.00% AmrLevel::RK() 60 3.903e-05 4.432e-05 5.401e-05 0.00% CNS::avgDown() 21 2.993e-05 4.037e-05 5.06e-05 0.00% Amr::FinalizeInit() 1 5.721e-06 1.609e-05 3.606e-05 0.00% AmrLevel::AmrLevel(dm) 12 2.696e-05 2.958e-05 3.162e-05 0.00% amrex::average_down_w_geom 21 1.828e-05 2.07e-05 2.29e-05 0.00% CNS::writePlotFile() 4 6.277e-06 7.23e-06 8.978e-06 0.00% Amr::init() 1 1.1e-06 1.242e-06 1.557e-06 0.00% Amr::initialInit() 1 8.23e-07 9.74e-07 1.09e-06 0.00% Other 1197 0.01009 0.02331 0.04306 0.31% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 14.02 14.02 14.03 100.00% Amr::coarseTimeStep() 20 13.61 13.61 13.61 97.04% Amr::timeStep() 60 13.44 13.47 13.5 96.26% CNS::advance() 60 11.94 11.96 11.96 85.25% AmrLevel::RK() 60 11.88 11.89 11.89 84.77% RungeKutta2 60 11.88 11.89 11.89 84.77% CNS::compute_dSdt() 120 8.262 9.102 9.942 70.86% AmrLevel::FillPatcherFill() 120 1.43 2.275 3.107 22.15% FabArray::ParallelCopy() 572 1.031 1.812 2.592 18.48% FillPatchSingleLevel 140 0.6042 1.459 2.339 16.67% AmrLevel::FillPatch() 60 0.4832 1.337 2.208 15.73% FillPatchIterator::Initialize 60 0.3738 1.243 2.12 15.11% FillPatchIterator::FillFromLevel0() 51 0.2411 1.091 1.95 13.90% StateDataPhysBCFunct::() 220 0.001079 0.8601 1.725 12.30% FabArray::ParallelCopy_finish() 572 0.5057 1.105 1.72 12.26% post_timestep 60 1.174 1.21 1.245 8.87% FillPatcher::fill() 80 1.043 1.075 1.113 7.93% FluxRegister::Reflux() 120 0.9147 0.9473 0.98 6.98% FabArray::ParallelCopy_nowait() 572 0.4715 0.7064 0.9577 6.83% FillPatcher::fillCFB 80 0.6958 0.7173 0.7609 5.42% FabArray::FillBoundary() 80 0.3465 0.3569 0.3797 2.71% Amr::regrid() 10 0.3383 0.3438 0.3489 2.49% Amr::writePlotFile() 2 0.2988 0.2992 0.2998 2.14% CNS::writePlotFile() 4 0.2953 0.2959 0.2974 2.12% AmrLevel::writePlotFile() 4 0.2953 0.2959 0.2974 2.12% VisMF::Write(FabArray) 4 0.2742 0.2774 0.2823 2.01% FabArray::setVal() 282 0.2183 0.2515 0.2801 2.00% Amr::init() 1 0.2564 0.2602 0.2694 1.92% CNS::avgDown() 21 0.2019 0.225 0.244 1.74% CNS::computeTemp() 144 0.1504 0.1882 0.2324 1.66% FillBoundary_finish() 80 0.1402 0.1772 0.2221 1.58% amrex::average_down_w_geom 21 0.1965 0.2094 0.2208 1.57% amrex::average_down 21 0.1964 0.2094 0.2208 1.57% FillBoundary_nowait() 80 0.1575 0.1796 0.211 1.50% Amr::grid_places() 11 0.1837 0.1841 0.1845 1.31% AmrMesh::MakeNewGrids() 11 0.1836 0.184 0.1844 1.31% FillPatchInterp(Fab) 80 0.09838 0.1458 0.1717 1.22% CNS::errorEst() 11 0.06878 0.1123 0.1561 1.11% CellConservativeLinear::interp() 120 0.03213 0.09031 0.1499 1.07% amrex::Copy() 88 0.08063 0.105 0.1207 0.86% Amr::initialInit() 1 0.1023 0.1067 0.116 0.83% Amr::FinalizeInit() 1 0.09554 0.0959 0.0964 0.69% CNS::estTimeStep() 41 0.05825 0.07403 0.09352 0.67% FabArray::sum() 105 0.05591 0.07371 0.08993 0.64% postCoarseTimeStep() 20 0.05333 0.07042 0.08711 0.62% Amr::bldFineLevels() 1 0.07721 0.0776 0.07798 0.56% TagBoxArray::collate() 11 0.001752 0.03748 0.07362 0.52% FabArray::setDomainBndry() 60 0.0001879 0.02423 0.04979 0.35% TagBoxArray::mapPRD 11 0.01205 0.0191 0.02591 0.18% AmrLevel::AmrLevel(dm) 12 0.02156 0.02338 0.02491 0.18% StateData::define() 12 0.02153 0.02335 0.02488 0.18% Other 1197 0.04371 0.07594 0.1317 0.94% ------------------------------------------------------------------------------------------- Device Memory Usage: ---------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ---------------------------------------------------------------------------------------------------------------- CNS::compute_dSdt() 19200 127 MiB 140 MiB 154 MiB 261 MiB 261 MiB 261 MiB StateData::define() 192 133 MiB 133 MiB 133 MiB 201 MiB 201 MiB 201 MiB CNS::advance() 176 117 MiB 117 MiB 117 MiB 134 MiB 134 MiB 134 MiB FluxRegister::Reflux() 1920 3543 KiB 3716 KiB 3912 KiB 56 MiB 67 MiB 71 MiB RungeKutta2 1244 46 MiB 46 MiB 46 MiB 70 MiB 70 MiB 70 MiB FillPatchIterator::Initialize 960 1897 KiB 5866 KiB 9910 KiB 67 MiB 67 MiB 67 MiB AmrLevel::writePlotFile() 64 1183 KiB 1184 KiB 1187 KiB 56 MiB 56 MiB 56 MiB FillPatcher::fillCFB 760 4275 KiB 6443 KiB 8628 KiB 7998 KiB 11 MiB 15 MiB ResizeRandomSeed 4 9209 KiB 9210 KiB 9211 KiB 9216 KiB 9216 KiB 9216 KiB CNS::errorEst() 176 39 KiB 66 KiB 94 KiB 8985 KiB 8985 KiB 8985 KiB post_timestep 320 574 KiB 581 KiB 589 KiB 8192 KiB 8192 KiB 8192 KiB amrex::average_down 428 81 KiB 90 KiB 97 KiB 7169 KiB 7169 KiB 7169 KiB CellConservativeLinear::interp() 960 11 KiB 19 KiB 33 KiB 1355 KiB 2064 KiB 2773 KiB Amr::regrid() 960 1328 KiB 1328 KiB 1328 KiB 2688 KiB 2688 KiB 2688 KiB StateDataPhysBCFunct::() 816 0 B 48 KiB 96 KiB 0 B 736 KiB 1473 KiB Amr::bldFineLevels() 96 4410 B 4420 B 4451 B 1344 KiB 1344 KiB 1344 KiB AmrMesh::MakeNewGrids() 352 13 KiB 13 KiB 13 KiB 1025 KiB 1025 KiB 1025 KiB FabArray::ParallelCopy_nowait() 2202 16 KiB 55 KiB 95 KiB 26 KiB 99 KiB 173 KiB VisMF::Write(FabArray) 448 140 B 162 B 190 B 96 KiB 96 KiB 96 KiB CNS::estTimeStep() 172 1143 B 1149 B 1162 B 49 KiB 49 KiB 49 KiB FabArray::sum() 420 193 B 254 B 310 B 48 KiB 48 KiB 48 KiB FabArray::ParallelCopy_finish() 1568 5160 B 6477 B 8299 B 9216 B 13 KiB 17 KiB FillBoundary_nowait() 160 7567 B 7573 B 7580 B 10 KiB 10 KiB 10 KiB FillBoundary_finish() 80 3777 B 3778 B 3781 B 5120 B 5120 B 5120 B FabArray::setVal() 1144 2091 B 2119 B 2149 B 3072 B 3072 B 3072 B amrex::Copy() 348 1956 B 1958 B 1960 B 3072 B 3072 B 3072 B TagBoxArray::mapPRD 352 1 B 2 B 3 B 2048 B 2048 B 2048 B TagBoxArray::collate() 88 0 B 0 B 0 B 256 B 896 B 1536 B OwnerMask() 176 0 B 0 B 0 B 1024 B 1024 B 1024 B StateData::FillBoundary(geom) 1224 0 B 10 B 22 B 0 B 256 B 512 B Amr::InitAmr() 8 505 B 505 B 505 B 512 B 512 B 512 B ---------------------------------------------------------------------------------------------------------------- Async Memory Usage: ------------------------------------------------------------------------------------------------------ Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ------------------------------------------------------------------------------------------------------ CNS::compute_dSdt() 3840 4713 KiB 8464 KiB 11 MiB 65 MiB 65 MiB 65 MiB FluxRegister::Reflux() 1920 2595 KiB 3134 KiB 3499 KiB 56 MiB 56 MiB 56 MiB RungeKutta2 960 46 MiB 46 MiB 46 MiB 56 MiB 56 MiB 56 MiB post_timestep 320 547 KiB 566 KiB 584 KiB 8192 KiB 8192 KiB 8192 KiB amrex::average_down 336 80 KiB 89 KiB 97 KiB 7168 KiB 7168 KiB 7168 KiB ------------------------------------------------------------------------------------------------------ Managed Memory Usage: --------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------- The_Managed_Arena::Initialize() 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------- Pinned Memory Usage: ---------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ---------------------------------------------------------------------------------------------------------------- FabArray::ParallelCopy_nowait() 5496 292 KiB 507 KiB 708 KiB 7189 KiB 11 MiB 14 MiB VisMF::Write(FabArray) 512 214 KiB 226 KiB 245 KiB 14 MiB 14 MiB 14 MiB The_Pinned_Arena::Initialize() 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB FillBoundary_nowait() 800 189 KiB 195 KiB 207 KiB 7514 KiB 7514 KiB 7514 KiB FabArray::ParallelCopy_finish() 1568 5160 B 6477 B 8299 B 9216 B 13 KiB 17 KiB FillBoundary_finish() 80 3777 B 3778 B 3781 B 5120 B 5120 B 5120 B CNS::compute_dSdt() 1920 127 B 180 B 234 B 3072 B 3072 B 3072 B amrex::Copy() 348 1956 B 1958 B 1960 B 3072 B 3072 B 3072 B FabArray::setVal() 1144 2091 B 2119 B 2149 B 3072 B 3072 B 3072 B TagBoxArray::collate() 154 0 B 2 B 4 B 512 B 1344 B 1792 B amrex::average_down 92 562 B 563 B 564 B 1536 B 1536 B 1536 B RungeKutta2 284 1076 B 1076 B 1077 B 1536 B 1536 B 1536 B CNS::estTimeStep() 172 1010 B 1010 B 1010 B 1280 B 1280 B 1280 B FillPatcher::fillCFB 280 393 B 466 B 539 B 768 B 896 B 1024 B StateData::FillBoundary(geom) 1224 0 B 10 B 22 B 0 B 256 B 512 B CellConservativeLinear::interp() 480 1 B 1 B 2 B 256 B 384 B 512 B FabArray::sum() 420 1 B 1 B 1 B 256 B 256 B 256 B ---------------------------------------------------------------------------------------------------------------- Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [4227 ... 4227] [The Arena] max space (MB) allocated spread across MPI: [741 ... 766] [The Arena] max space (MB) used spread across MPI: [603 ... 611] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [32 ... 36] [The Pinned Arena] max space (MB) used spread across MPI: [14 ... 14] AMReX (26.09-185-g19d64c697f1a) finalized