Initializing AMReX (26.09-122-ga846c6a9c672)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.09-122-ga846c6a9c672) initialized Successfully read inputs file ... Successfully read inputs file ... Starting to call amrex_probinit ... Successfully run amrex_probinit Now regridding at level lbase = 0 grid_places() time: 0.019454885 new finest: 1 STEP = 0 TIME = 0 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 INITIAL GRIDS Level 0 16 grids 4194304 cells 100 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 PLOTFILE: file = CNS-RT_plt00000 grid_places() time: 0.019898377 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 0 Total y-momentum is 0 Total z-momentum is 0 Total energy is 7176192.0007176176 Write plotfile time = 0.173903712 seconds [Level 0 step 1] ADVANCE at time 0 with dt = 0.0002870776155 [Level 0 step 1] Advanced 4194304 cells [Level 1 step 1] ADVANCE at time 0 with dt = 0.0001435388078 [Level 1 step 1] Advanced 4194304 cells [Level 1 step 2] ADVANCE at time 0.0001435388078 with dt = 0.0001435388078 [Level 1 step 2] Advanced 4194304 cells STEP = 1 TIME = 0.0002870776155 DT = 0.0002870776155 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 3.4600423210458683e-21 Total y-momentum is 2.0479171591388077e-21 Total z-momentum is 1.0569463681968465 Total energy is 7176192.0005832035 [STEP 1] Coarse TimeStep time: 0.70170983 [STEP 1] FAB kilobyte spread across MPI nodes: [536184 ... 544157] [Level 0 step 2] ADVANCE at time 0.0002870776155 with dt = 0.0002870672688 [Level 0 step 2] Advanced 4194304 cells [Level 1 step 3] ADVANCE at time 0.0002870776155 with dt = 0.0001435336344 [Level 1 step 3] Advanced 4194304 cells [Level 1 step 4] ADVANCE at time 0.0004306112499 with dt = 0.0001435336344 [Level 1 step 4] Advanced 4194304 cells STEP = 2 TIME = 0.0005741448843 DT = 0.0002870672688 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -2.0746744576564437e-20 Total y-momentum is 7.1848555606412404e-21 Total z-momentum is 2.3209418694909116 Total energy is 7176192.0001132824 [STEP 2] Coarse TimeStep time: 0.670981188 [STEP 2] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 2 TIME = 0.0005741448843 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 3] ADVANCE at time 0.0005741448843 with dt = 0.0002870608582 [Level 0 step 3] Advanced 4194304 cells [Level 1 step 5] ADVANCE at time 0.0005741448843 with dt = 0.0001435304291 [Level 1 step 5] Advanced 4194304 cells [Level 1 step 6] ADVANCE at time 0.0007176753134 with dt = 0.0001435304291 [Level 1 step 6] Advanced 4194304 cells STEP = 3 TIME = 0.0008612057425 DT = 0.0002870608582 grid_places() time: 0.016255578 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 7.5708260554279181e-20 Total y-momentum is 1.0853671400560117e-19 Total z-momentum is 3.7623957612122796 Total energy is 7176191.9992529377 [STEP 3] Coarse TimeStep time: 0.707465399 [STEP 3] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 4] ADVANCE at time 0.0008612057425 with dt = 0.00028705672 [Level 0 step 4] Advanced 4194304 cells [Level 1 step 7] ADVANCE at time 0.0008612057425 with dt = 0.00014352836 [Level 1 step 7] Advanced 4194304 cells [Level 1 step 8] ADVANCE at time 0.001004734103 with dt = 0.00014352836 [Level 1 step 8] Advanced 4194304 cells STEP = 4 TIME = 0.001148262463 DT = 0.00028705672 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 7.3619098490785573e-20 Total y-momentum is -3.3770302497149155e-19 Total z-momentum is 5.3553134404616571 Total energy is 7176191.9979552217 [STEP 4] Coarse TimeStep time: 0.669218849 [STEP 4] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 4 TIME = 0.001148262463 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 5] ADVANCE at time 0.001148262463 with dt = 0.0002870541933 [Level 0 step 5] Advanced 4194304 cells [Level 1 step 9] ADVANCE at time 0.001148262463 with dt = 0.0001435270967 [Level 1 step 9] Advanced 4194304 cells [Level 1 step 10] ADVANCE at time 0.001291789559 with dt = 0.0001435270967 [Level 1 step 10] Advanced 4194304 cells STEP = 5 TIME = 0.001435316656 DT = 0.0002870541933 grid_places() time: 0.016249294 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -3.2608222937573232e-19 Total y-momentum is -2.6424116081489047e-19 Total z-momentum is 7.0771445174316776 Total energy is 7176191.9961801395 [STEP 5] Coarse TimeStep time: 0.705052939 [STEP 5] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 6] ADVANCE at time 0.001435316656 with dt = 0.0002870527674 [Level 0 step 6] Advanced 4194304 cells [Level 1 step 11] ADVANCE at time 0.001435316656 with dt = 0.0001435263837 [Level 1 step 11] Advanced 4194304 cells [Level 1 step 12] ADVANCE at time 0.00157884304 with dt = 0.0001435263837 [Level 1 step 12] Advanced 4194304 cells STEP = 6 TIME = 0.001722369423 DT = 0.0002870527674 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 2.1894415462558567e-19 Total y-momentum is 1.0192467340569854e-18 Total z-momentum is 8.9084964335707468 Total energy is 7176191.9938936941 [STEP 6] Coarse TimeStep time: 0.668095416 [STEP 6] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 6 TIME = 0.001722369423 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 7] ADVANCE at time 0.001722369423 with dt = 0.0002870520599 [Level 0 step 7] Advanced 4194304 cells [Level 1 step 13] ADVANCE at time 0.001722369423 with dt = 0.00014352603 [Level 1 step 13] Advanced 4194304 cells [Level 1 step 14] ADVANCE at time 0.001865895453 with dt = 0.00014352603 [Level 1 step 14] Advanced 4194304 cells STEP = 7 TIME = 0.002009421483 DT = 0.0002870520599 grid_places() time: 0.01623769 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -7.6990262974807339e-19 Total y-momentum is -1.5253840639083534e-18 Total z-momentum is 10.832804311097531 Total energy is 7176191.9910670239 [STEP 7] Coarse TimeStep time: 0.703730683 [STEP 7] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 8] ADVANCE at time 0.002009421483 with dt = 0.0002870517947 [Level 0 step 8] Advanced 4194304 cells [Level 1 step 15] ADVANCE at time 0.002009421483 with dt = 0.0001435258973 [Level 1 step 15] Advanced 4194304 cells [Level 1 step 16] ADVANCE at time 0.002152947381 with dt = 0.0001435258973 [Level 1 step 16] Advanced 4194304 cells STEP = 8 TIME = 0.002296473278 DT = 0.0002870517947 [CNS] Total mass is 1302338.0233411889 Total x-momentum is -1.7306562615022928e-18 Total y-momentum is 2.646116891001415e-18 Total z-momentum is 12.835981478053229 Total energy is 7176191.9876756454 [STEP 8] Coarse TimeStep time: 0.665930541 [STEP 8] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 8 TIME = 0.002296473278 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 9] ADVANCE at time 0.002296473278 with dt = 0.00028705178 [Level 0 step 9] Advanced 4194304 cells [Level 1 step 17] ADVANCE at time 0.002296473278 with dt = 0.00014352589 [Level 1 step 17] Advanced 4194304 cells [Level 1 step 18] ADVANCE at time 0.002439999168 with dt = 0.00014352589 [Level 1 step 18] Advanced 4194304 cells STEP = 9 TIME = 0.002583525058 DT = 0.00028705178 grid_places() time: 0.016381736 new finest: 1 [CNS] Total mass is 1302338.0233411889 Total x-momentum is 5.6810854521708086e-19 Total y-momentum is 1.2936874203733604e-18 Total z-momentum is 14.906076194007598 Total energy is 7176191.9836987816 [STEP 9] Coarse TimeStep time: 0.702578739 [STEP 9] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 10] ADVANCE at time 0.002583525058 with dt = 0.0002870518889 [Level 0 step 10] Advanced 4194304 cells [Level 1 step 19] ADVANCE at time 0.002583525058 with dt = 0.0001435259444 [Level 1 step 19] Advanced 4194304 cells [Level 1 step 20] ADVANCE at time 0.002727051002 with dt = 0.0001435259444 [Level 1 step 20] Advanced 4194304 cells STEP = 10 TIME = 0.002870576947 DT = 0.0002870518889 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 1.376067658812568e-18 Total y-momentum is 8.7411424597322119e-19 Total z-momentum is 17.032951759614733 Total energy is 7176191.979118811 [STEP 10] Coarse TimeStep time: 0.665014964 [STEP 10] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 10 TIME = 0.002870576947 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 11] ADVANCE at time 0.002870576947 with dt = 0.0002870520422 [Level 0 step 11] Advanced 4194304 cells [Level 1 step 21] ADVANCE at time 0.002870576947 with dt = 0.0001435260211 [Level 1 step 21] Advanced 4194304 cells [Level 1 step 22] ADVANCE at time 0.003014102968 with dt = 0.0001435260211 [Level 1 step 22] Advanced 4194304 cells STEP = 11 TIME = 0.003157628989 DT = 0.0002870520422 grid_places() time: 0.016232652 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 8.8714661369221303e-19 Total y-momentum is 4.7158369086842284e-18 Total z-momentum is 19.208000131229269 Total energy is 7176191.9739207756 [STEP 11] Coarse TimeStep time: 0.702112456 [STEP 11] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 12] ADVANCE at time 0.003157628989 with dt = 0.0002870521946 [Level 0 step 12] Advanced 4194304 cells [Level 1 step 23] ADVANCE at time 0.003157628989 with dt = 0.0001435260973 [Level 1 step 23] Advanced 4194304 cells [Level 1 step 24] ADVANCE at time 0.003301155086 with dt = 0.0001435260973 [Level 1 step 24] Advanced 4194304 cells STEP = 12 TIME = 0.003444681184 DT = 0.0002870521946 [CNS] Total mass is 1302338.0233411889 Total x-momentum is 2.198810717002711e-18 Total y-momentum is 7.0752732394503774e-18 Total z-momentum is 21.423893568568744 Total energy is 7176191.9680919927 [STEP 12] Coarse TimeStep time: 0.663885098 [STEP 12] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 12 TIME = 0.003444681184 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 13] ADVANCE at time 0.003444681184 with dt = 0.0002870523239 [Level 0 step 13] Advanced 4194304 cells [Level 1 step 25] ADVANCE at time 0.003444681184 with dt = 0.0001435261619 [Level 1 step 25] Advanced 4194304 cells [Level 1 step 26] ADVANCE at time 0.003588207345 with dt = 0.0001435261619 [Level 1 step 26] Advanced 4194304 cells STEP = 13 TIME = 0.003731733507 DT = 0.0002870523239 grid_places() time: 0.016212322 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -4.0390711608099386e-18 Total y-momentum is -7.3165912148932354e-18 Total z-momentum is 23.674374717272059 Total energy is 7176191.9616217129 [STEP 13] Coarse TimeStep time: 0.700697262 [STEP 13] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 14] ADVANCE at time 0.003731733507 with dt = 0.000287052422 [Level 0 step 14] Advanced 4194304 cells [Level 1 step 27] ADVANCE at time 0.003731733507 with dt = 0.000143526211 [Level 1 step 27] Advanced 4194304 cells [Level 1 step 28] ADVANCE at time 0.003875259718 with dt = 0.000143526211 [Level 1 step 28] Advanced 4194304 cells STEP = 14 TIME = 0.004018785929 DT = 0.000287052422 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -1.4254982508844783e-18 Total y-momentum is 1.015700761020574e-18 Total z-momentum is 25.95408273901171 Total energy is 7176191.9545008428 [STEP 14] Coarse TimeStep time: 0.664348853 [STEP 14] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 14 TIME = 0.004018785929 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 15] ADVANCE at time 0.004018785929 with dt = 0.0002870524893 [Level 0 step 15] Advanced 4194304 cells [Level 1 step 29] ADVANCE at time 0.004018785929 with dt = 0.0001435262446 [Level 1 step 29] Advanced 4194304 cells [Level 1 step 30] ADVANCE at time 0.004162312174 with dt = 0.0001435262446 [Level 1 step 30] Advanced 4194304 cells STEP = 15 TIME = 0.004305838419 DT = 0.0002870524893 grid_places() time: 0.01639044 new finest: 1 [CNS] Total mass is 1302338.0233411889 Total x-momentum is -3.8096209963360823e-19 Total y-momentum is -8.7918912827820473e-19 Total z-momentum is 28.258411443578304 Total energy is 7176191.9467217075 [STEP 15] Coarse TimeStep time: 0.700480771 [STEP 15] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 16] ADVANCE at time 0.004305838419 with dt = 0.0002870525306 [Level 0 step 16] Advanced 4194304 cells [Level 1 step 31] ADVANCE at time 0.004305838419 with dt = 0.0001435262653 [Level 1 step 31] Advanced 4194304 cells [Level 1 step 32] ADVANCE at time 0.004449364684 with dt = 0.0001435262653 [Level 1 step 32] Advanced 4194304 cells STEP = 16 TIME = 0.004592890949 DT = 0.0002870525306 [CNS] Total mass is 1302338.0233411889 Total x-momentum is 5.4730419547846157e-18 Total y-momentum is 6.7296237746774245e-18 Total z-momentum is 30.583394611415127 Total energy is 7176191.9382778564 [STEP 16] Coarse TimeStep time: 0.662267096 [STEP 16] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 16 TIME = 0.004592890949 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 17] ADVANCE at time 0.004592890949 with dt = 0.000287052552 [Level 0 step 17] Advanced 4194304 cells [Level 1 step 33] ADVANCE at time 0.004592890949 with dt = 0.000143526276 [Level 1 step 33] Advanced 4194304 cells [Level 1 step 34] ADVANCE at time 0.004736417225 with dt = 0.000143526276 [Level 1 step 34] Advanced 4194304 cells STEP = 17 TIME = 0.004879943501 DT = 0.000287052552 grid_places() time: 0.016395623 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -9.2035317838578576e-18 Total y-momentum is -2.6665374729341175e-18 Total z-momentum is 32.925613582717489 Total energy is 7176191.9291638918 [STEP 17] Coarse TimeStep time: 0.699612888 [STEP 17] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 18] ADVANCE at time 0.004879943501 with dt = 0.0002870525597 [Level 0 step 18] Advanced 4194304 cells [Level 1 step 35] ADVANCE at time 0.004879943501 with dt = 0.0001435262798 [Level 1 step 35] Advanced 4194304 cells [Level 1 step 36] ADVANCE at time 0.005023469781 with dt = 0.0001435262798 [Level 1 step 36] Advanced 4194304 cells STEP = 18 TIME = 0.005166996061 DT = 0.0002870525597 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -1.3908635798758459e-17 Total y-momentum is -1.6212768957360782e-18 Total z-momentum is 35.282122513822877 Total energy is 7176191.9193753228 [STEP 18] Coarse TimeStep time: 0.662134817 [STEP 18] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 18 TIME = 0.005166996061 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 19] ADVANCE at time 0.005166996061 with dt = 0.0002870525588 [Level 0 step 19] Advanced 4194304 cells [Level 1 step 37] ADVANCE at time 0.005166996061 with dt = 0.0001435262794 [Level 1 step 37] Advanced 4194304 cells [Level 1 step 38] ADVANCE at time 0.00531052234 with dt = 0.0001435262794 [Level 1 step 38] Advanced 4194304 cells STEP = 19 TIME = 0.00545404862 DT = 0.0002870525588 grid_places() time: 0.016397837 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -3.043424857689809e-19 Total y-momentum is 1.1848493735178941e-19 Total z-momentum is 37.650387281643575 Total energy is 7176191.9089084426 [STEP 19] Coarse TimeStep time: 0.698822523 [STEP 19] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 20] ADVANCE at time 0.00545404862 with dt = 0.0002870525536 [Level 0 step 20] Advanced 4194304 cells [Level 1 step 39] ADVANCE at time 0.00545404862 with dt = 0.0001435262768 [Level 1 step 39] Advanced 4194304 cells [Level 1 step 40] ADVANCE at time 0.005597574896 with dt = 0.0001435262768 [Level 1 step 40] Advanced 4194304 cells STEP = 20 TIME = 0.005741101173 DT = 0.0002870525536 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 7.3173620118177381e-18 Total y-momentum is -7.3889842784956805e-18 Total z-momentum is 40.028234710671775 Total energy is 7176191.8977602143 [STEP 20] Coarse TimeStep time: 0.662150263 [STEP 20] FAB kilobyte spread across MPI nodes: [575320 ... 575320] PLOTFILE: file = CNS-RT_plt00020 Write plotfile time = 0.146217712 seconds Run Time total = 14.11956828 Run Time init = 0.298360294 Run Time advance = 13.67479739 TinyProfiler total time across processes [min...avg...max]: 14.1 ... 14.11 ... 14.12 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- CNS::compute_dSdt() 120 8.085 8.269 8.49 60.13% StateDataPhysBCFunct::() 220 0.001068 0.8581 1.726 12.22% FabArray::ParallelCopy_finish() 572 0.5729 1.048 1.566 11.09% FabArray::ParallelCopy_nowait() 572 0.4823 0.7135 0.9844 6.97% FluxRegister::Reflux() 120 0.642 0.6574 0.6853 4.85% RungeKutta2 60 0.2955 0.3449 0.374 2.65% VisMF::Write(FabArray) 4 0.299 0.2995 0.3007 2.13% FabArray::setVal() 282 0.2223 0.2522 0.2807 1.99% CNS::computeTemp() 144 0.163 0.1867 0.2354 1.67% FillBoundary_nowait() 80 0.1518 0.18 0.2168 1.54% FillBoundary_finish() 80 0.1338 0.1779 0.2018 1.43% CellConservativeLinear::interp() 120 0.03308 0.09126 0.1617 1.15% FillPatcher::fillCFB 80 0.09427 0.1217 0.1435 1.02% FillPatchInterp(Fab) 80 0.01706 0.05596 0.1248 0.88% FillPatchIterator::Initialize 60 0.114 0.1182 0.1229 0.87% amrex::Copy() 88 0.09089 0.1048 0.118 0.84% amrex::average_down 21 0.07647 0.09421 0.1093 0.77% Amr::regrid() 10 0.1082 0.1088 0.1092 0.77% CNS::estTimeStep() 41 0.05764 0.07425 0.09065 0.64% FabArray::sum() 105 0.05406 0.07262 0.08821 0.62% TagBoxArray::collate() 11 0.001417 0.03784 0.0735 0.52% CNS::advance() 60 0.0673 0.06974 0.07123 0.50% FabArray::setDomainBndry() 60 0.0002033 0.02381 0.04953 0.35% post_timestep 60 0.03202 0.03411 0.03641 0.26% StateData::define() 12 0.02192 0.02348 0.02556 0.18% CNS::errorEst() 11 0.02069 0.02226 0.02346 0.17% Amr::bldFineLevels() 1 0.01259 0.01265 0.0127 0.09% TagBoxArray::mapPRD 11 0.003505 0.006758 0.009834 0.07% AmrMesh::MakeNewGrids() 11 0.006313 0.007749 0.009462 0.07% AmrLevel::writePlotFile() 4 0.004745 0.006746 0.009273 0.07% Amr::writePlotFile() 2 0.002188 0.003057 0.003499 0.02% Amr::coarseTimeStep() 20 0.00028 0.001369 0.002417 0.02% Amr::timeStep() 60 0.0006119 0.0008669 0.001381 0.01% AmrLevel::FillPatch() 60 0.0006706 0.0007134 0.0007747 0.01% FabArray::ParallelCopy() 572 0.000531 0.0005704 0.0006109 0.00% main() 1 0.0002357 0.0003274 0.0004035 0.00% FillPatchSingleLevel 140 0.0002296 0.0002682 0.0003292 0.00% AmrLevel::FillPatcherFill() 120 0.0002617 0.0002852 0.0003069 0.00% FillPatcher::fill() 80 0.0001445 0.0001593 0.0001998 0.00% FabArray::FillBoundary() 80 0.000131 0.0001405 0.0001527 0.00% FillPatchIterator::FillFromLevel0() 51 5.955e-05 6.577e-05 6.942e-05 0.00% postCoarseTimeStep() 20 3.91e-05 5.116e-05 6.917e-05 0.00% Amr::FinalizeInit() 1 5.815e-06 2.285e-05 6.755e-05 0.00% Amr::grid_places() 11 5.541e-05 5.955e-05 6.138e-05 0.00% CNS::avgDown() 21 3.194e-05 4.195e-05 5.252e-05 0.00% AmrLevel::RK() 60 4.717e-05 4.914e-05 5.163e-05 0.00% AmrLevel::AmrLevel(dm) 12 2.922e-05 3.079e-05 3.267e-05 0.00% amrex::average_down_w_geom 21 1.745e-05 1.966e-05 2.265e-05 0.00% CNS::writePlotFile() 4 7.133e-06 7.5e-06 7.779e-06 0.00% Amr::init() 1 1.374e-06 1.504e-06 1.557e-06 0.00% Amr::initialInit() 1 9.24e-07 1.162e-06 1.441e-06 0.00% Other 1197 0.01104 0.02972 0.04717 0.33% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 14.1 14.11 14.12 100.00% Amr::coarseTimeStep() 20 13.67 13.67 13.67 96.85% Amr::timeStep() 60 13.5 13.53 13.57 96.11% CNS::advance() 60 12 12.01 12.02 85.11% AmrLevel::RK() 60 11.93 11.94 11.95 84.61% RungeKutta2 60 11.93 11.94 11.95 84.61% CNS::compute_dSdt() 120 8.315 9.149 10 70.82% AmrLevel::FillPatcherFill() 120 1.439 2.282 3.109 22.02% FabArray::ParallelCopy() 572 1.058 1.764 2.467 17.47% FillPatchSingleLevel 140 0.5972 1.463 2.329 16.50% AmrLevel::FillPatch() 60 0.4771 1.342 2.21 15.65% FillPatchIterator::Initialize 60 0.369 1.246 2.125 15.05% FillPatchIterator::FillFromLevel0() 51 0.235 1.094 1.951 13.82% StateDataPhysBCFunct::() 220 0.001068 0.8628 1.735 12.29% FabArray::ParallelCopy_finish() 572 0.5729 1.048 1.566 11.09% post_timestep 60 1.179 1.215 1.249 8.85% FillPatcher::fill() 80 1.049 1.078 1.11 7.86% FabArray::ParallelCopy_nowait() 572 0.4843 0.7156 0.9867 6.99% FluxRegister::Reflux() 120 0.9173 0.952 0.9859 6.98% FillPatcher::fillCFB 80 0.6941 0.7189 0.7585 5.37% FabArray::FillBoundary() 80 0.3508 0.3582 0.3697 2.62% Amr::regrid() 10 0.3416 0.3459 0.3509 2.49% Amr::writePlotFile() 2 0.3193 0.3197 0.3203 2.27% CNS::writePlotFile() 4 0.3161 0.3166 0.3179 2.25% AmrLevel::writePlotFile() 4 0.3161 0.3166 0.3179 2.25% VisMF::Write(FabArray) 4 0.299 0.2995 0.3007 2.13% Amr::init() 1 0.2786 0.2879 0.295 2.09% FabArray::setVal() 282 0.2223 0.2522 0.2807 1.99% CNS::avgDown() 21 0.2083 0.2269 0.244 1.73% CNS::computeTemp() 144 0.163 0.1867 0.2354 1.67% amrex::average_down_w_geom 21 0.2009 0.2116 0.2199 1.56% amrex::average_down 21 0.2009 0.2116 0.2198 1.56% FillBoundary_nowait() 80 0.1519 0.1801 0.2169 1.54% FillBoundary_finish() 80 0.1338 0.1779 0.2018 1.43% Amr::grid_places() 11 0.1853 0.1857 0.1861 1.32% AmrMesh::MakeNewGrids() 11 0.1853 0.1856 0.186 1.32% FillPatchInterp(Fab) 80 0.09714 0.1472 0.1788 1.27% CellConservativeLinear::interp() 120 0.03308 0.09126 0.1617 1.15% CNS::errorEst() 11 0.06914 0.1132 0.1584 1.12% Amr::initialInit() 1 0.1056 0.1144 0.1211 0.86% amrex::Copy() 88 0.09089 0.1048 0.118 0.84% Amr::FinalizeInit() 1 0.09784 0.09803 0.09829 0.70% CNS::estTimeStep() 41 0.05764 0.07425 0.09065 0.64% FabArray::sum() 105 0.05406 0.07262 0.08821 0.62% postCoarseTimeStep() 20 0.05015 0.06908 0.08538 0.60% Amr::bldFineLevels() 1 0.078 0.0784 0.07877 0.56% TagBoxArray::collate() 11 0.001417 0.03784 0.0735 0.52% FabArray::setDomainBndry() 60 0.0002033 0.02381 0.04953 0.35% TagBoxArray::mapPRD 11 0.0115 0.01926 0.02637 0.19% AmrLevel::AmrLevel(dm) 12 0.02195 0.02351 0.0256 0.18% StateData::define() 12 0.02192 0.02348 0.02556 0.18% Other 1197 0.04956 0.09731 0.1352 0.96% ------------------------------------------------------------------------------------------- Device Memory Usage: ---------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ---------------------------------------------------------------------------------------------------------------- CNS::compute_dSdt() 19200 127 MiB 140 MiB 154 MiB 261 MiB 261 MiB 261 MiB StateData::define() 192 133 MiB 133 MiB 133 MiB 201 MiB 201 MiB 201 MiB CNS::advance() 176 116 MiB 116 MiB 116 MiB 134 MiB 134 MiB 134 MiB FluxRegister::Reflux() 1920 3527 KiB 3723 KiB 3916 KiB 56 MiB 67 MiB 71 MiB RungeKutta2 1244 46 MiB 46 MiB 46 MiB 70 MiB 70 MiB 70 MiB FillPatchIterator::Initialize 960 1855 KiB 5849 KiB 9858 KiB 67 MiB 67 MiB 67 MiB AmrLevel::writePlotFile() 64 1257 KiB 1259 KiB 1262 KiB 56 MiB 56 MiB 56 MiB FillPatcher::fillCFB 760 4268 KiB 6431 KiB 8610 KiB 7998 KiB 11 MiB 15 MiB ResizeRandomSeed 4 9211 KiB 9211 KiB 9211 KiB 9216 KiB 9216 KiB 9216 KiB CNS::errorEst() 176 39 KiB 67 KiB 95 KiB 8985 KiB 8985 KiB 8985 KiB post_timestep 320 572 KiB 581 KiB 589 KiB 8192 KiB 8192 KiB 8192 KiB amrex::average_down 428 82 KiB 90 KiB 101 KiB 7169 KiB 7169 KiB 7169 KiB CellConservativeLinear::interp() 960 10 KiB 19 KiB 33 KiB 1355 KiB 2064 KiB 2773 KiB Amr::regrid() 960 1327 KiB 1328 KiB 1328 KiB 2688 KiB 2688 KiB 2688 KiB StateDataPhysBCFunct::() 816 0 B 47 KiB 96 KiB 0 B 736 KiB 1473 KiB Amr::bldFineLevels() 96 4421 B 4421 B 4422 B 1344 KiB 1344 KiB 1344 KiB AmrMesh::MakeNewGrids() 352 13 KiB 13 KiB 13 KiB 1025 KiB 1025 KiB 1025 KiB FabArray::ParallelCopy_nowait() 2202 16 KiB 55 KiB 95 KiB 26 KiB 99 KiB 173 KiB VisMF::Write(FabArray) 448 22 B 80 B 116 B 96 KiB 96 KiB 96 KiB CNS::estTimeStep() 172 1137 B 1147 B 1166 B 49 KiB 49 KiB 49 KiB FabArray::sum() 420 185 B 248 B 302 B 48 KiB 48 KiB 48 KiB FabArray::ParallelCopy_finish() 1568 5149 B 6459 B 8274 B 9216 B 13 KiB 17 KiB FillBoundary_nowait() 160 7551 B 7557 B 7566 B 10 KiB 10 KiB 10 KiB FillBoundary_finish() 80 3770 B 3770 B 3773 B 5120 B 5120 B 5120 B FabArray::setVal() 1144 2088 B 2116 B 2146 B 3072 B 3072 B 3072 B amrex::Copy() 348 1954 B 1956 B 1958 B 3072 B 3072 B 3072 B TagBoxArray::mapPRD 352 1 B 2 B 3 B 2048 B 2048 B 2048 B TagBoxArray::collate() 88 0 B 0 B 0 B 256 B 896 B 1536 B OwnerMask() 176 0 B 0 B 0 B 1024 B 1024 B 1024 B StateData::FillBoundary(geom) 1224 0 B 11 B 22 B 0 B 256 B 512 B Amr::InitAmr() 8 505 B 505 B 505 B 512 B 512 B 512 B ---------------------------------------------------------------------------------------------------------------- Async Memory Usage: ------------------------------------------------------------------------------------------------------ Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ------------------------------------------------------------------------------------------------------ CNS::compute_dSdt() 3840 4721 KiB 8420 KiB 11 MiB 65 MiB 65 MiB 65 MiB FluxRegister::Reflux() 1920 2614 KiB 3129 KiB 3506 KiB 56 MiB 56 MiB 56 MiB RungeKutta2 960 46 MiB 46 MiB 46 MiB 56 MiB 56 MiB 56 MiB post_timestep 320 545 KiB 565 KiB 585 KiB 8192 KiB 8192 KiB 8192 KiB amrex::average_down 336 82 KiB 90 KiB 100 KiB 7168 KiB 7168 KiB 7168 KiB ------------------------------------------------------------------------------------------------------ Managed Memory Usage: --------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------- The_Managed_Arena::Initialize() 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------- Pinned Memory Usage: ---------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ---------------------------------------------------------------------------------------------------------------- FabArray::ParallelCopy_nowait() 5496 294 KiB 503 KiB 698 KiB 7189 KiB 11 MiB 14 MiB VisMF::Write(FabArray) 512 232 KiB 245 KiB 258 KiB 14 MiB 14 MiB 14 MiB The_Pinned_Arena::Initialize() 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB FillBoundary_nowait() 800 190 KiB 194 KiB 201 KiB 7514 KiB 7514 KiB 7514 KiB FabArray::ParallelCopy_finish() 1568 5149 B 6459 B 8274 B 9216 B 13 KiB 17 KiB FillBoundary_finish() 80 3770 B 3770 B 3773 B 5120 B 5120 B 5120 B CNS::compute_dSdt() 1920 127 B 179 B 231 B 3072 B 3072 B 3072 B amrex::Copy() 348 1954 B 1956 B 1958 B 3072 B 3072 B 3072 B FabArray::setVal() 1144 2088 B 2116 B 2146 B 3072 B 3072 B 3072 B TagBoxArray::collate() 154 0 B 2 B 4 B 512 B 1344 B 1792 B amrex::average_down 92 563 B 563 B 565 B 1536 B 1536 B 1536 B RungeKutta2 284 1074 B 1074 B 1075 B 1536 B 1536 B 1536 B CNS::estTimeStep() 172 1010 B 1010 B 1010 B 1280 B 1280 B 1280 B FillPatcher::fillCFB 280 393 B 465 B 539 B 768 B 896 B 1024 B StateData::FillBoundary(geom) 1224 0 B 11 B 23 B 0 B 256 B 512 B CellConservativeLinear::interp() 480 1 B 1 B 2 B 256 B 384 B 512 B FabArray::sum() 420 0 B 0 B 1 B 256 B 256 B 256 B ---------------------------------------------------------------------------------------------------------------- Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [4227 ... 4227] [The Arena] max space (MB) allocated spread across MPI: [741 ... 766] [The Arena] max space (MB) used spread across MPI: [603 ... 611] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [32 ... 36] [The Pinned Arena] max space (MB) used spread across MPI: [14 ... 14] AMReX (26.09-122-ga846c6a9c672) finalized