Initializing AMReX (26.09-120-g66028f892b7d)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.09-120-g66028f892b7d) initialized Successfully read inputs file ... Successfully read inputs file ... Starting to call amrex_probinit ... Successfully run amrex_probinit Now regridding at level lbase = 0 grid_places() time: 0.020125782 new finest: 1 STEP = 0 TIME = 0 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 INITIAL GRIDS Level 0 16 grids 4194304 cells 100 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 PLOTFILE: file = CNS-RT_plt00000 grid_places() time: 0.019808027 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 0 Total y-momentum is 0 Total z-momentum is 0 Total energy is 7176192.0007176176 Write plotfile time = 0.15263465 seconds [Level 0 step 1] ADVANCE at time 0 with dt = 0.0002870776155 [Level 0 step 1] Advanced 4194304 cells [Level 1 step 1] ADVANCE at time 0 with dt = 0.0001435388078 [Level 1 step 1] Advanced 4194304 cells [Level 1 step 2] ADVANCE at time 0.0001435388078 with dt = 0.0001435388078 [Level 1 step 2] Advanced 4194304 cells STEP = 1 TIME = 0.0002870776155 DT = 0.0002870776155 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 3.4600423210458683e-21 Total y-momentum is 2.0479171591388077e-21 Total z-momentum is 1.0569463681968465 Total energy is 7176192.0005832035 [STEP 1] Coarse TimeStep time: 0.69126374 [STEP 1] FAB kilobyte spread across MPI nodes: [536184 ... 544157] [Level 0 step 2] ADVANCE at time 0.0002870776155 with dt = 0.0002870672688 [Level 0 step 2] Advanced 4194304 cells [Level 1 step 3] ADVANCE at time 0.0002870776155 with dt = 0.0001435336344 [Level 1 step 3] Advanced 4194304 cells [Level 1 step 4] ADVANCE at time 0.0004306112499 with dt = 0.0001435336344 [Level 1 step 4] Advanced 4194304 cells STEP = 2 TIME = 0.0005741448843 DT = 0.0002870672688 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -2.0746744576564437e-20 Total y-momentum is 7.1848555606412404e-21 Total z-momentum is 2.3209418694909116 Total energy is 7176192.0001132824 [STEP 2] Coarse TimeStep time: 0.671183663 [STEP 2] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 2 TIME = 0.0005741448843 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 3] ADVANCE at time 0.0005741448843 with dt = 0.0002870608582 [Level 0 step 3] Advanced 4194304 cells [Level 1 step 5] ADVANCE at time 0.0005741448843 with dt = 0.0001435304291 [Level 1 step 5] Advanced 4194304 cells [Level 1 step 6] ADVANCE at time 0.0007176753134 with dt = 0.0001435304291 [Level 1 step 6] Advanced 4194304 cells STEP = 3 TIME = 0.0008612057425 DT = 0.0002870608582 grid_places() time: 0.016045817 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 7.5708260554279181e-20 Total y-momentum is 1.0853671400560117e-19 Total z-momentum is 3.7623957612122796 Total energy is 7176191.9992529377 [STEP 3] Coarse TimeStep time: 0.707558052 [STEP 3] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 4] ADVANCE at time 0.0008612057425 with dt = 0.00028705672 [Level 0 step 4] Advanced 4194304 cells [Level 1 step 7] ADVANCE at time 0.0008612057425 with dt = 0.00014352836 [Level 1 step 7] Advanced 4194304 cells [Level 1 step 8] ADVANCE at time 0.001004734103 with dt = 0.00014352836 [Level 1 step 8] Advanced 4194304 cells STEP = 4 TIME = 0.001148262463 DT = 0.00028705672 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 7.3619098490785573e-20 Total y-momentum is -3.3770302497149155e-19 Total z-momentum is 5.3553134404616571 Total energy is 7176191.9979552217 [STEP 4] Coarse TimeStep time: 0.669464794 [STEP 4] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 4 TIME = 0.001148262463 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 5] ADVANCE at time 0.001148262463 with dt = 0.0002870541933 [Level 0 step 5] Advanced 4194304 cells [Level 1 step 9] ADVANCE at time 0.001148262463 with dt = 0.0001435270967 [Level 1 step 9] Advanced 4194304 cells [Level 1 step 10] ADVANCE at time 0.001291789559 with dt = 0.0001435270967 [Level 1 step 10] Advanced 4194304 cells STEP = 5 TIME = 0.001435316656 DT = 0.0002870541933 grid_places() time: 0.01615537 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -3.2608222937573232e-19 Total y-momentum is -2.6424116081489047e-19 Total z-momentum is 7.0771445174316776 Total energy is 7176191.9961801395 [STEP 5] Coarse TimeStep time: 0.705394736 [STEP 5] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 6] ADVANCE at time 0.001435316656 with dt = 0.0002870527674 [Level 0 step 6] Advanced 4194304 cells [Level 1 step 11] ADVANCE at time 0.001435316656 with dt = 0.0001435263837 [Level 1 step 11] Advanced 4194304 cells [Level 1 step 12] ADVANCE at time 0.00157884304 with dt = 0.0001435263837 [Level 1 step 12] Advanced 4194304 cells STEP = 6 TIME = 0.001722369423 DT = 0.0002870527674 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 2.1894415462558567e-19 Total y-momentum is 1.0192467340569854e-18 Total z-momentum is 8.9084964335707468 Total energy is 7176191.9938936941 [STEP 6] Coarse TimeStep time: 0.667642757 [STEP 6] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 6 TIME = 0.001722369423 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 7] ADVANCE at time 0.001722369423 with dt = 0.0002870520599 [Level 0 step 7] Advanced 4194304 cells [Level 1 step 13] ADVANCE at time 0.001722369423 with dt = 0.00014352603 [Level 1 step 13] Advanced 4194304 cells [Level 1 step 14] ADVANCE at time 0.001865895453 with dt = 0.00014352603 [Level 1 step 14] Advanced 4194304 cells STEP = 7 TIME = 0.002009421483 DT = 0.0002870520599 grid_places() time: 0.01615887 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -7.6990262974807339e-19 Total y-momentum is -1.5253840639083534e-18 Total z-momentum is 10.832804311097531 Total energy is 7176191.9910670239 [STEP 7] Coarse TimeStep time: 0.703390092 [STEP 7] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 8] ADVANCE at time 0.002009421483 with dt = 0.0002870517947 [Level 0 step 8] Advanced 4194304 cells [Level 1 step 15] ADVANCE at time 0.002009421483 with dt = 0.0001435258973 [Level 1 step 15] Advanced 4194304 cells [Level 1 step 16] ADVANCE at time 0.002152947381 with dt = 0.0001435258973 [Level 1 step 16] Advanced 4194304 cells STEP = 8 TIME = 0.002296473278 DT = 0.0002870517947 [CNS] Total mass is 1302338.0233411889 Total x-momentum is -1.7306562615022928e-18 Total y-momentum is 2.646116891001415e-18 Total z-momentum is 12.835981478053229 Total energy is 7176191.9876756454 [STEP 8] Coarse TimeStep time: 0.665773598 [STEP 8] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 8 TIME = 0.002296473278 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 9] ADVANCE at time 0.002296473278 with dt = 0.00028705178 [Level 0 step 9] Advanced 4194304 cells [Level 1 step 17] ADVANCE at time 0.002296473278 with dt = 0.00014352589 [Level 1 step 17] Advanced 4194304 cells [Level 1 step 18] ADVANCE at time 0.002439999168 with dt = 0.00014352589 [Level 1 step 18] Advanced 4194304 cells STEP = 9 TIME = 0.002583525058 DT = 0.00028705178 grid_places() time: 0.016308198 new finest: 1 [CNS] Total mass is 1302338.0233411889 Total x-momentum is 5.6810854521708086e-19 Total y-momentum is 1.2936874203733604e-18 Total z-momentum is 14.906076194007598 Total energy is 7176191.9836987816 [STEP 9] Coarse TimeStep time: 0.70256012 [STEP 9] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 10] ADVANCE at time 0.002583525058 with dt = 0.0002870518889 [Level 0 step 10] Advanced 4194304 cells [Level 1 step 19] ADVANCE at time 0.002583525058 with dt = 0.0001435259444 [Level 1 step 19] Advanced 4194304 cells [Level 1 step 20] ADVANCE at time 0.002727051002 with dt = 0.0001435259444 [Level 1 step 20] Advanced 4194304 cells STEP = 10 TIME = 0.002870576947 DT = 0.0002870518889 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 1.376067658812568e-18 Total y-momentum is 8.7411424597322119e-19 Total z-momentum is 17.032951759614733 Total energy is 7176191.979118811 [STEP 10] Coarse TimeStep time: 0.665145351 [STEP 10] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 10 TIME = 0.002870576947 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 11] ADVANCE at time 0.002870576947 with dt = 0.0002870520422 [Level 0 step 11] Advanced 4194304 cells [Level 1 step 21] ADVANCE at time 0.002870576947 with dt = 0.0001435260211 [Level 1 step 21] Advanced 4194304 cells [Level 1 step 22] ADVANCE at time 0.003014102968 with dt = 0.0001435260211 [Level 1 step 22] Advanced 4194304 cells STEP = 11 TIME = 0.003157628989 DT = 0.0002870520422 grid_places() time: 0.01630383 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 8.8714661369221303e-19 Total y-momentum is 4.7158369086842284e-18 Total z-momentum is 19.208000131229269 Total energy is 7176191.9739207756 [STEP 11] Coarse TimeStep time: 0.703249302 [STEP 11] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 12] ADVANCE at time 0.003157628989 with dt = 0.0002870521946 [Level 0 step 12] Advanced 4194304 cells [Level 1 step 23] ADVANCE at time 0.003157628989 with dt = 0.0001435260973 [Level 1 step 23] Advanced 4194304 cells [Level 1 step 24] ADVANCE at time 0.003301155086 with dt = 0.0001435260973 [Level 1 step 24] Advanced 4194304 cells STEP = 12 TIME = 0.003444681184 DT = 0.0002870521946 [CNS] Total mass is 1302338.0233411889 Total x-momentum is 2.198810717002711e-18 Total y-momentum is 7.0752732394503774e-18 Total z-momentum is 21.423893568568744 Total energy is 7176191.9680919927 [STEP 12] Coarse TimeStep time: 0.664148973 [STEP 12] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 12 TIME = 0.003444681184 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 13] ADVANCE at time 0.003444681184 with dt = 0.0002870523239 [Level 0 step 13] Advanced 4194304 cells [Level 1 step 25] ADVANCE at time 0.003444681184 with dt = 0.0001435261619 [Level 1 step 25] Advanced 4194304 cells [Level 1 step 26] ADVANCE at time 0.003588207345 with dt = 0.0001435261619 [Level 1 step 26] Advanced 4194304 cells STEP = 13 TIME = 0.003731733507 DT = 0.0002870523239 grid_places() time: 0.016250971 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -4.0390711608099386e-18 Total y-momentum is -7.3165912148932354e-18 Total z-momentum is 23.674374717272059 Total energy is 7176191.9616217129 [STEP 13] Coarse TimeStep time: 0.702345516 [STEP 13] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 14] ADVANCE at time 0.003731733507 with dt = 0.000287052422 [Level 0 step 14] Advanced 4194304 cells [Level 1 step 27] ADVANCE at time 0.003731733507 with dt = 0.000143526211 [Level 1 step 27] Advanced 4194304 cells [Level 1 step 28] ADVANCE at time 0.003875259718 with dt = 0.000143526211 [Level 1 step 28] Advanced 4194304 cells STEP = 14 TIME = 0.004018785929 DT = 0.000287052422 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -1.4254982508844783e-18 Total y-momentum is 1.015700761020574e-18 Total z-momentum is 25.95408273901171 Total energy is 7176191.9545008428 [STEP 14] Coarse TimeStep time: 0.66832456 [STEP 14] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 14 TIME = 0.004018785929 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 15] ADVANCE at time 0.004018785929 with dt = 0.0002870524893 [Level 0 step 15] Advanced 4194304 cells [Level 1 step 29] ADVANCE at time 0.004018785929 with dt = 0.0001435262446 [Level 1 step 29] Advanced 4194304 cells [Level 1 step 30] ADVANCE at time 0.004162312174 with dt = 0.0001435262446 [Level 1 step 30] Advanced 4194304 cells STEP = 15 TIME = 0.004305838419 DT = 0.0002870524893 grid_places() time: 0.016091275 new finest: 1 [CNS] Total mass is 1302338.0233411889 Total x-momentum is -3.8096209963360823e-19 Total y-momentum is -8.7918912827820473e-19 Total z-momentum is 28.258411443578304 Total energy is 7176191.9467217075 [STEP 15] Coarse TimeStep time: 0.704615421 [STEP 15] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 16] ADVANCE at time 0.004305838419 with dt = 0.0002870525306 [Level 0 step 16] Advanced 4194304 cells [Level 1 step 31] ADVANCE at time 0.004305838419 with dt = 0.0001435262653 [Level 1 step 31] Advanced 4194304 cells [Level 1 step 32] ADVANCE at time 0.004449364684 with dt = 0.0001435262653 [Level 1 step 32] Advanced 4194304 cells STEP = 16 TIME = 0.004592890949 DT = 0.0002870525306 [CNS] Total mass is 1302338.0233411889 Total x-momentum is 5.4730419547846157e-18 Total y-momentum is 6.7296237746774245e-18 Total z-momentum is 30.583394611415127 Total energy is 7176191.9382778564 [STEP 16] Coarse TimeStep time: 0.667601553 [STEP 16] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 16 TIME = 0.004592890949 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 17] ADVANCE at time 0.004592890949 with dt = 0.000287052552 [Level 0 step 17] Advanced 4194304 cells [Level 1 step 33] ADVANCE at time 0.004592890949 with dt = 0.000143526276 [Level 1 step 33] Advanced 4194304 cells [Level 1 step 34] ADVANCE at time 0.004736417225 with dt = 0.000143526276 [Level 1 step 34] Advanced 4194304 cells STEP = 17 TIME = 0.004879943501 DT = 0.000287052552 grid_places() time: 0.016414114 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -9.2035317838578576e-18 Total y-momentum is -2.6665374729341175e-18 Total z-momentum is 32.925613582717489 Total energy is 7176191.9291638918 [STEP 17] Coarse TimeStep time: 0.70462014 [STEP 17] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 18] ADVANCE at time 0.004879943501 with dt = 0.0002870525597 [Level 0 step 18] Advanced 4194304 cells [Level 1 step 35] ADVANCE at time 0.004879943501 with dt = 0.0001435262798 [Level 1 step 35] Advanced 4194304 cells [Level 1 step 36] ADVANCE at time 0.005023469781 with dt = 0.0001435262798 [Level 1 step 36] Advanced 4194304 cells STEP = 18 TIME = 0.005166996061 DT = 0.0002870525597 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -1.3908635798758459e-17 Total y-momentum is -1.6212768957360782e-18 Total z-momentum is 35.282122513822877 Total energy is 7176191.9193753228 [STEP 18] Coarse TimeStep time: 0.666932561 [STEP 18] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 18 TIME = 0.005166996061 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 19] ADVANCE at time 0.005166996061 with dt = 0.0002870525588 [Level 0 step 19] Advanced 4194304 cells [Level 1 step 37] ADVANCE at time 0.005166996061 with dt = 0.0001435262794 [Level 1 step 37] Advanced 4194304 cells [Level 1 step 38] ADVANCE at time 0.00531052234 with dt = 0.0001435262794 [Level 1 step 38] Advanced 4194304 cells STEP = 19 TIME = 0.00545404862 DT = 0.0002870525588 grid_places() time: 0.016174616 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -3.043424857689809e-19 Total y-momentum is 1.1848493735178941e-19 Total z-momentum is 37.650387281643575 Total energy is 7176191.9089084426 [STEP 19] Coarse TimeStep time: 0.703326524 [STEP 19] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 20] ADVANCE at time 0.00545404862 with dt = 0.0002870525536 [Level 0 step 20] Advanced 4194304 cells [Level 1 step 39] ADVANCE at time 0.00545404862 with dt = 0.0001435262768 [Level 1 step 39] Advanced 4194304 cells [Level 1 step 40] ADVANCE at time 0.005597574896 with dt = 0.0001435262768 [Level 1 step 40] Advanced 4194304 cells STEP = 20 TIME = 0.005741101173 DT = 0.0002870525536 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 7.3173620118177381e-18 Total y-momentum is -7.3889842784956805e-18 Total z-momentum is 40.028234710671775 Total energy is 7176191.8977602143 [STEP 20] Coarse TimeStep time: 0.665824923 [STEP 20] FAB kilobyte spread across MPI nodes: [575320 ... 575320] PLOTFILE: file = CNS-RT_plt00020 Write plotfile time = 0.134824134 seconds Run Time total = 14.10051076 Run Time init = 0.267589099 Run Time advance = 13.69786339 TinyProfiler total time across processes [min...avg...max]: 14.09 ... 14.1 ... 14.1 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- CNS::compute_dSdt() 120 8.11 8.276 8.48 60.14% StateDataPhysBCFunct::() 220 0.001075 0.862 1.729 12.26% FabArray::ParallelCopy_finish() 572 0.5316 1.077 1.63 11.56% FabArray::ParallelCopy_nowait() 572 0.4751 0.7017 0.9684 6.87% FluxRegister::Reflux() 120 0.596 0.6531 0.6869 4.87% RungeKutta2 60 0.3071 0.3427 0.3615 2.56% FabArray::setVal() 282 0.2219 0.2519 0.2787 1.98% VisMF::Write(FabArray) 4 0.2658 0.2681 0.2695 1.91% FillBoundary_finish() 80 0.1194 0.2088 0.2631 1.87% CNS::computeTemp() 144 0.1441 0.1758 0.204 1.45% CellConservativeLinear::interp() 120 0.03544 0.09559 0.171 1.21% FillBoundary_nowait() 80 0.1194 0.1475 0.1694 1.20% CNS::estTimeStep() 41 0.04888 0.08677 0.1581 1.12% FillPatcher::fillCFB 80 0.09631 0.1283 0.1543 1.09% FillPatchIterator::Initialize 60 0.1175 0.1246 0.1342 0.95% FillPatchInterp(Fab) 80 0.01765 0.05611 0.1226 0.87% amrex::Copy() 88 0.08891 0.1045 0.1186 0.84% Amr::regrid() 10 0.1086 0.1091 0.1096 0.78% amrex::average_down 21 0.07796 0.09691 0.1095 0.78% CNS::advance() 60 0.07 0.07197 0.07395 0.52% TagBoxArray::collate() 11 0.002026 0.03761 0.07391 0.52% FabArray::sum() 105 0.00917 0.04617 0.065 0.46% FabArray::setDomainBndry() 60 0.0001942 0.02432 0.05222 0.37% post_timestep 60 0.03028 0.03706 0.04846 0.34% CNS::errorEst() 11 0.02096 0.02216 0.02326 0.16% Amr::bldFineLevels() 1 0.0126 0.01266 0.01271 0.09% AmrLevel::writePlotFile() 4 0.003903 0.005606 0.01025 0.07% AmrMesh::MakeNewGrids() 11 0.005887 0.007938 0.01009 0.07% Amr::coarseTimeStep() 20 0.0002747 0.002993 0.01001 0.07% TagBoxArray::mapPRD 11 0.004029 0.006552 0.009157 0.06% Amr::writePlotFile() 2 0.002092 0.00363 0.004334 0.03% Amr::timeStep() 60 0.0006274 0.0008604 0.001359 0.01% AmrLevel::FillPatch() 60 0.0006469 0.0006978 0.00076 0.01% FabArray::ParallelCopy() 572 0.0005061 0.0005464 0.000603 0.00% FillPatchSingleLevel 140 0.0002353 0.0002676 0.0003144 0.00% AmrLevel::FillPatcherFill() 120 0.0002645 0.000287 0.0003126 0.00% main() 1 0.0002052 0.0002437 0.0002656 0.00% FillPatcher::fill() 80 0.0001329 0.0001522 0.0001793 0.00% FabArray::FillBoundary() 80 0.0001247 0.0001387 0.0001556 0.00% postCoarseTimeStep() 20 3.602e-05 4.663e-05 7.251e-05 0.00% FillPatchIterator::FillFromLevel0() 51 6.634e-05 6.772e-05 6.992e-05 0.00% AmrLevel::RK() 60 4.513e-05 5.053e-05 6.183e-05 0.00% CNS::avgDown() 21 3.584e-05 4.381e-05 5.614e-05 0.00% Amr::grid_places() 11 4.972e-05 5.357e-05 5.6e-05 0.00% Amr::FinalizeInit() 1 4.437e-06 1.89e-05 4.302e-05 0.00% amrex::average_down_w_geom 21 1.704e-05 1.813e-05 2.081e-05 0.00% CNS::writePlotFile() 4 5.889e-06 6.467e-06 6.839e-06 0.00% Amr::init() 1 1.305e-06 1.411e-06 1.54e-06 0.00% Amr::initialInit() 1 9.26e-07 1.005e-06 1.051e-06 0.00% Other 1221 0.03346 0.04681 0.06172 0.44% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 14.09 14.1 14.1 100.00% Amr::coarseTimeStep() 20 13.7 13.7 13.7 97.14% Amr::timeStep() 60 13.53 13.57 13.6 96.48% CNS::advance() 60 12.03 12.04 12.05 85.46% AmrLevel::RK() 60 11.95 11.97 11.98 84.95% RungeKutta2 60 11.95 11.97 11.98 84.95% CNS::compute_dSdt() 120 8.343 9.165 10.01 71.02% AmrLevel::FillPatcherFill() 120 1.455 2.311 3.126 22.17% FabArray::ParallelCopy() 572 1.009 1.781 2.533 17.97% FillPatchSingleLevel 140 0.5634 1.464 2.351 16.67% AmrLevel::FillPatch() 60 0.4708 1.351 2.214 15.70% FillPatchIterator::Initialize 60 0.3606 1.256 2.129 15.10% FillPatchIterator::FillFromLevel0() 51 0.2325 1.097 1.952 13.84% StateDataPhysBCFunct::() 220 0.001075 0.8674 1.74 12.34% FabArray::ParallelCopy_finish() 572 0.5316 1.077 1.63 11.56% post_timestep 60 1.176 1.218 1.243 8.82% FillPatcher::fill() 80 1.079 1.097 1.127 7.99% FluxRegister::Reflux() 120 0.9203 0.9536 0.9863 6.99% FabArray::ParallelCopy_nowait() 572 0.4771 0.7037 0.9705 6.88% FillPatcher::fillCFB 80 0.6859 0.74 0.8367 5.93% FabArray::FillBoundary() 80 0.2891 0.3565 0.399 2.83% Amr::regrid() 10 0.3419 0.3451 0.35 2.48% Amr::writePlotFile() 2 0.2868 0.2873 0.2878 2.04% CNS::writePlotFile() 4 0.2825 0.2836 0.2854 2.02% AmrLevel::writePlotFile() 4 0.2825 0.2836 0.2853 2.02% FabArray::setVal() 282 0.2219 0.2519 0.2787 1.98% VisMF::Write(FabArray) 4 0.2658 0.2681 0.2695 1.91% FillBoundary_finish() 80 0.1194 0.2088 0.2631 1.87% Amr::init() 1 0.2579 0.2593 0.2614 1.85% CNS::avgDown() 21 0.2168 0.2251 0.2363 1.68% amrex::average_down_w_geom 21 0.2029 0.2081 0.214 1.52% amrex::average_down 21 0.2029 0.2081 0.214 1.52% CNS::computeTemp() 144 0.1441 0.1758 0.204 1.45% FillPatchInterp(Fab) 80 0.1012 0.1517 0.1886 1.34% Amr::grid_places() 11 0.1851 0.1854 0.1858 1.32% AmrMesh::MakeNewGrids() 11 0.185 0.1854 0.1858 1.32% CellConservativeLinear::interp() 120 0.03544 0.09559 0.171 1.21% FillBoundary_nowait() 80 0.1195 0.1476 0.1695 1.20% CNS::estTimeStep() 41 0.04888 0.08677 0.1581 1.12% CNS::errorEst() 11 0.06907 0.113 0.1563 1.11% amrex::Copy() 88 0.08891 0.1045 0.1186 0.84% Amr::initialInit() 1 0.1052 0.1069 0.1094 0.78% Amr::FinalizeInit() 1 0.09756 0.09828 0.09902 0.70% Amr::bldFineLevels() 1 0.07852 0.07891 0.07928 0.56% TagBoxArray::collate() 11 0.002026 0.03761 0.07391 0.52% FabArray::sum() 105 0.00917 0.04617 0.065 0.46% postCoarseTimeStep() 20 0.004471 0.04266 0.06036 0.43% FabArray::setDomainBndry() 60 0.0001942 0.02432 0.05222 0.37% TagBoxArray::mapPRD 11 0.01282 0.01924 0.0257 0.18% Other 1221 0.09517 0.115 0.139 0.99% ------------------------------------------------------------------------------------------- Device Memory Usage: ---------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ---------------------------------------------------------------------------------------------------------------- CNS::compute_dSdt() 19200 128 MiB 141 MiB 155 MiB 261 MiB 261 MiB 261 MiB StateData::define() 192 133 MiB 133 MiB 133 MiB 201 MiB 201 MiB 201 MiB CNS::advance() 176 117 MiB 117 MiB 117 MiB 134 MiB 134 MiB 134 MiB FluxRegister::Reflux() 1920 3515 KiB 3710 KiB 3899 KiB 56 MiB 67 MiB 71 MiB RungeKutta2 1244 46 MiB 46 MiB 46 MiB 70 MiB 70 MiB 70 MiB FillPatchIterator::Initialize 960 1841 KiB 5893 KiB 9884 KiB 67 MiB 67 MiB 67 MiB AmrLevel::writePlotFile() 64 1123 KiB 1129 KiB 1134 KiB 56 MiB 56 MiB 56 MiB FillPatcher::fillCFB 760 4277 KiB 6453 KiB 8614 KiB 7998 KiB 11 MiB 15 MiB ResizeRandomSeed 4 9210 KiB 9210 KiB 9210 KiB 9216 KiB 9216 KiB 9216 KiB CNS::errorEst() 176 39 KiB 67 KiB 94 KiB 8985 KiB 8985 KiB 8985 KiB post_timestep 320 575 KiB 580 KiB 590 KiB 8192 KiB 8192 KiB 8192 KiB amrex::average_down 428 80 KiB 89 KiB 102 KiB 7169 KiB 7169 KiB 7169 KiB CellConservativeLinear::interp() 960 11 KiB 19 KiB 33 KiB 1355 KiB 2064 KiB 2773 KiB Amr::regrid() 960 1328 KiB 1328 KiB 1329 KiB 2688 KiB 2688 KiB 2688 KiB StateDataPhysBCFunct::() 816 0 B 48 KiB 96 KiB 0 B 736 KiB 1473 KiB Amr::bldFineLevels() 96 4403 B 4413 B 4445 B 1344 KiB 1344 KiB 1344 KiB AmrMesh::MakeNewGrids() 352 13 KiB 13 KiB 13 KiB 1025 KiB 1025 KiB 1025 KiB FabArray::ParallelCopy_nowait() 2202 16 KiB 55 KiB 95 KiB 26 KiB 99 KiB 173 KiB VisMF::Write(FabArray) 448 45 B 57 B 74 B 96 KiB 96 KiB 96 KiB CNS::estTimeStep() 172 1120 B 1155 B 1211 B 49 KiB 49 KiB 49 KiB FabArray::sum() 420 31 B 158 B 223 B 48 KiB 48 KiB 48 KiB FabArray::ParallelCopy_finish() 1568 5172 B 6475 B 8300 B 9216 B 13 KiB 17 KiB FillBoundary_nowait() 160 7563 B 7571 B 7577 B 10 KiB 10 KiB 10 KiB FillBoundary_finish() 80 3776 B 3776 B 3778 B 5120 B 5120 B 5120 B FabArray::setVal() 1144 2092 B 2120 B 2149 B 3072 B 3072 B 3072 B amrex::Copy() 348 1956 B 1958 B 1960 B 3072 B 3072 B 3072 B TagBoxArray::mapPRD 352 1 B 2 B 3 B 2048 B 2048 B 2048 B TagBoxArray::collate() 88 0 B 0 B 0 B 256 B 896 B 1536 B OwnerMask() 176 0 B 0 B 0 B 1024 B 1024 B 1024 B StateData::FillBoundary(geom) 1224 0 B 11 B 22 B 0 B 256 B 512 B Amr::InitAmr() 8 505 B 505 B 505 B 512 B 512 B 512 B ---------------------------------------------------------------------------------------------------------------- Async Memory Usage: ------------------------------------------------------------------------------------------------------ Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ------------------------------------------------------------------------------------------------------ CNS::compute_dSdt() 3840 4782 KiB 8483 KiB 11 MiB 65 MiB 65 MiB 65 MiB FluxRegister::Reflux() 1920 2602 KiB 3138 KiB 3499 KiB 56 MiB 56 MiB 56 MiB RungeKutta2 960 46 MiB 46 MiB 46 MiB 56 MiB 56 MiB 56 MiB post_timestep 320 548 KiB 567 KiB 586 KiB 8192 KiB 8192 KiB 8192 KiB amrex::average_down 336 80 KiB 89 KiB 101 KiB 7168 KiB 7168 KiB 7168 KiB ------------------------------------------------------------------------------------------------------ Managed Memory Usage: --------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------- The_Managed_Arena::Initialize() 4 0 B 0 B 1 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------- Pinned Memory Usage: ---------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ---------------------------------------------------------------------------------------------------------------- FabArray::ParallelCopy_nowait() 5496 275 KiB 506 KiB 691 KiB 7189 KiB 11 MiB 14 MiB VisMF::Write(FabArray) 512 206 KiB 219 KiB 230 KiB 14 MiB 14 MiB 14 MiB The_Pinned_Arena::Initialize() 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB FillBoundary_nowait() 800 157 KiB 193 KiB 216 KiB 7514 KiB 7514 KiB 7514 KiB FabArray::ParallelCopy_finish() 1568 5172 B 6475 B 8300 B 9216 B 13 KiB 17 KiB FillBoundary_finish() 80 3776 B 3776 B 3778 B 5120 B 5120 B 5120 B CNS::compute_dSdt() 1920 129 B 181 B 235 B 3072 B 3072 B 3072 B amrex::Copy() 348 1956 B 1958 B 1960 B 3072 B 3072 B 3072 B FabArray::setVal() 1144 2092 B 2120 B 2149 B 3072 B 3072 B 3072 B TagBoxArray::collate() 154 0 B 2 B 4 B 512 B 1344 B 1792 B amrex::average_down 92 562 B 563 B 564 B 1536 B 1536 B 1536 B RungeKutta2 284 1076 B 1077 B 1078 B 1536 B 1536 B 1536 B CNS::estTimeStep() 172 1011 B 1011 B 1011 B 1280 B 1280 B 1280 B FillPatcher::fillCFB 280 396 B 467 B 540 B 768 B 896 B 1024 B StateData::FillBoundary(geom) 1224 0 B 11 B 22 B 0 B 256 B 512 B CellConservativeLinear::interp() 480 1 B 1 B 2 B 256 B 384 B 512 B FabArray::sum() 420 0 B 0 B 1 B 256 B 256 B 256 B ---------------------------------------------------------------------------------------------------------------- Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [4227 ... 4227] [The Arena] max space (MB) allocated spread across MPI: [741 ... 766] [The Arena] max space (MB) used spread across MPI: [603 ... 611] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [32 ... 36] [The Pinned Arena] max space (MB) used spread across MPI: [14 ... 14] AMReX (26.09-120-g66028f892b7d) finalized