Initializing AMReX (26.09-163-gfca855037a21)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.09-163-gfca855037a21) initialized Successfully read inputs file ... Successfully read inputs file ... Starting to call amrex_probinit ... Successfully run amrex_probinit Now regridding at level lbase = 0 grid_places() time: 0.019138295 new finest: 1 STEP = 0 TIME = 0 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 INITIAL GRIDS Level 0 16 grids 4194304 cells 100 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 PLOTFILE: file = CNS-RT_plt00000 grid_places() time: 0.019927106 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 0 Total y-momentum is 0 Total z-momentum is 0 Total energy is 7176192.0007176176 Write plotfile time = 0.156207558 seconds [Level 0 step 1] ADVANCE at time 0 with dt = 0.0002870776155 [Level 0 step 1] Advanced 4194304 cells [Level 1 step 1] ADVANCE at time 0 with dt = 0.0001435388078 [Level 1 step 1] Advanced 4194304 cells [Level 1 step 2] ADVANCE at time 0.0001435388078 with dt = 0.0001435388078 [Level 1 step 2] Advanced 4194304 cells STEP = 1 TIME = 0.0002870776155 DT = 0.0002870776155 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 3.4600423210458683e-21 Total y-momentum is 2.0479171591388077e-21 Total z-momentum is 1.0569463681968465 Total energy is 7176192.0005832035 [STEP 1] Coarse TimeStep time: 0.694033634 [STEP 1] FAB kilobyte spread across MPI nodes: [536184 ... 544157] [Level 0 step 2] ADVANCE at time 0.0002870776155 with dt = 0.0002870672688 [Level 0 step 2] Advanced 4194304 cells [Level 1 step 3] ADVANCE at time 0.0002870776155 with dt = 0.0001435336344 [Level 1 step 3] Advanced 4194304 cells [Level 1 step 4] ADVANCE at time 0.0004306112499 with dt = 0.0001435336344 [Level 1 step 4] Advanced 4194304 cells STEP = 2 TIME = 0.0005741448843 DT = 0.0002870672688 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -2.0746744576564437e-20 Total y-momentum is 7.1848555606412404e-21 Total z-momentum is 2.3209418694909116 Total energy is 7176192.0001132824 [STEP 2] Coarse TimeStep time: 0.671338998 [STEP 2] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 2 TIME = 0.0005741448843 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 3] ADVANCE at time 0.0005741448843 with dt = 0.0002870608582 [Level 0 step 3] Advanced 4194304 cells [Level 1 step 5] ADVANCE at time 0.0005741448843 with dt = 0.0001435304291 [Level 1 step 5] Advanced 4194304 cells [Level 1 step 6] ADVANCE at time 0.0007176753134 with dt = 0.0001435304291 [Level 1 step 6] Advanced 4194304 cells STEP = 3 TIME = 0.0008612057425 DT = 0.0002870608582 grid_places() time: 0.01624515 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 7.5708260554279181e-20 Total y-momentum is 1.0853671400560117e-19 Total z-momentum is 3.7623957612122796 Total energy is 7176191.9992529377 [STEP 3] Coarse TimeStep time: 0.707785612 [STEP 3] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 4] ADVANCE at time 0.0008612057425 with dt = 0.00028705672 [Level 0 step 4] Advanced 4194304 cells [Level 1 step 7] ADVANCE at time 0.0008612057425 with dt = 0.00014352836 [Level 1 step 7] Advanced 4194304 cells [Level 1 step 8] ADVANCE at time 0.001004734103 with dt = 0.00014352836 [Level 1 step 8] Advanced 4194304 cells STEP = 4 TIME = 0.001148262463 DT = 0.00028705672 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 7.3619098490785573e-20 Total y-momentum is -3.3770302497149155e-19 Total z-momentum is 5.3553134404616571 Total energy is 7176191.9979552217 [STEP 4] Coarse TimeStep time: 0.664559647 [STEP 4] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 4 TIME = 0.001148262463 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 5] ADVANCE at time 0.001148262463 with dt = 0.0002870541933 [Level 0 step 5] Advanced 4194304 cells [Level 1 step 9] ADVANCE at time 0.001148262463 with dt = 0.0001435270967 [Level 1 step 9] Advanced 4194304 cells [Level 1 step 10] ADVANCE at time 0.001291789559 with dt = 0.0001435270967 [Level 1 step 10] Advanced 4194304 cells STEP = 5 TIME = 0.001435316656 DT = 0.0002870541933 grid_places() time: 0.016082228 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -3.2608222937573232e-19 Total y-momentum is -2.6424116081489047e-19 Total z-momentum is 7.0771445174316776 Total energy is 7176191.9961801395 [STEP 5] Coarse TimeStep time: 0.699725773 [STEP 5] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 6] ADVANCE at time 0.001435316656 with dt = 0.0002870527674 [Level 0 step 6] Advanced 4194304 cells [Level 1 step 11] ADVANCE at time 0.001435316656 with dt = 0.0001435263837 [Level 1 step 11] Advanced 4194304 cells [Level 1 step 12] ADVANCE at time 0.00157884304 with dt = 0.0001435263837 [Level 1 step 12] Advanced 4194304 cells STEP = 6 TIME = 0.001722369423 DT = 0.0002870527674 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 2.1894415462558567e-19 Total y-momentum is 1.0192467340569854e-18 Total z-momentum is 8.9084964335707468 Total energy is 7176191.9938936941 [STEP 6] Coarse TimeStep time: 0.662512445 [STEP 6] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 6 TIME = 0.001722369423 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 7] ADVANCE at time 0.001722369423 with dt = 0.0002870520599 [Level 0 step 7] Advanced 4194304 cells [Level 1 step 13] ADVANCE at time 0.001722369423 with dt = 0.00014352603 [Level 1 step 13] Advanced 4194304 cells [Level 1 step 14] ADVANCE at time 0.001865895453 with dt = 0.00014352603 [Level 1 step 14] Advanced 4194304 cells STEP = 7 TIME = 0.002009421483 DT = 0.0002870520599 grid_places() time: 0.015941503 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -7.6990262974807339e-19 Total y-momentum is -1.5253840639083534e-18 Total z-momentum is 10.832804311097531 Total energy is 7176191.9910670239 [STEP 7] Coarse TimeStep time: 0.698861105 [STEP 7] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 8] ADVANCE at time 0.002009421483 with dt = 0.0002870517947 [Level 0 step 8] Advanced 4194304 cells [Level 1 step 15] ADVANCE at time 0.002009421483 with dt = 0.0001435258973 [Level 1 step 15] Advanced 4194304 cells [Level 1 step 16] ADVANCE at time 0.002152947381 with dt = 0.0001435258973 [Level 1 step 16] Advanced 4194304 cells STEP = 8 TIME = 0.002296473278 DT = 0.0002870517947 [CNS] Total mass is 1302338.0233411889 Total x-momentum is -1.7306562615022928e-18 Total y-momentum is 2.646116891001415e-18 Total z-momentum is 12.835981478053229 Total energy is 7176191.9876756454 [STEP 8] Coarse TimeStep time: 0.661842485 [STEP 8] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 8 TIME = 0.002296473278 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 9] ADVANCE at time 0.002296473278 with dt = 0.00028705178 [Level 0 step 9] Advanced 4194304 cells [Level 1 step 17] ADVANCE at time 0.002296473278 with dt = 0.00014352589 [Level 1 step 17] Advanced 4194304 cells [Level 1 step 18] ADVANCE at time 0.002439999168 with dt = 0.00014352589 [Level 1 step 18] Advanced 4194304 cells STEP = 9 TIME = 0.002583525058 DT = 0.00028705178 grid_places() time: 0.016079974 new finest: 1 [CNS] Total mass is 1302338.0233411889 Total x-momentum is 5.6810854521708086e-19 Total y-momentum is 1.2936874203733604e-18 Total z-momentum is 14.906076194007598 Total energy is 7176191.9836987816 [STEP 9] Coarse TimeStep time: 0.697764046 [STEP 9] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 10] ADVANCE at time 0.002583525058 with dt = 0.0002870518889 [Level 0 step 10] Advanced 4194304 cells [Level 1 step 19] ADVANCE at time 0.002583525058 with dt = 0.0001435259444 [Level 1 step 19] Advanced 4194304 cells [Level 1 step 20] ADVANCE at time 0.002727051002 with dt = 0.0001435259444 [Level 1 step 20] Advanced 4194304 cells STEP = 10 TIME = 0.002870576947 DT = 0.0002870518889 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 1.376067658812568e-18 Total y-momentum is 8.7411424597322119e-19 Total z-momentum is 17.032951759614733 Total energy is 7176191.979118811 [STEP 10] Coarse TimeStep time: 0.660209921 [STEP 10] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 10 TIME = 0.002870576947 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 11] ADVANCE at time 0.002870576947 with dt = 0.0002870520422 [Level 0 step 11] Advanced 4194304 cells [Level 1 step 21] ADVANCE at time 0.002870576947 with dt = 0.0001435260211 [Level 1 step 21] Advanced 4194304 cells [Level 1 step 22] ADVANCE at time 0.003014102968 with dt = 0.0001435260211 [Level 1 step 22] Advanced 4194304 cells STEP = 11 TIME = 0.003157628989 DT = 0.0002870520422 grid_places() time: 0.01593406 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 8.8714661369221303e-19 Total y-momentum is 4.7158369086842284e-18 Total z-momentum is 19.208000131229269 Total energy is 7176191.9739207756 [STEP 11] Coarse TimeStep time: 0.698669062 [STEP 11] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 12] ADVANCE at time 0.003157628989 with dt = 0.0002870521946 [Level 0 step 12] Advanced 4194304 cells [Level 1 step 23] ADVANCE at time 0.003157628989 with dt = 0.0001435260973 [Level 1 step 23] Advanced 4194304 cells [Level 1 step 24] ADVANCE at time 0.003301155086 with dt = 0.0001435260973 [Level 1 step 24] Advanced 4194304 cells STEP = 12 TIME = 0.003444681184 DT = 0.0002870521946 [CNS] Total mass is 1302338.0233411889 Total x-momentum is 2.198810717002711e-18 Total y-momentum is 7.0752732394503774e-18 Total z-momentum is 21.423893568568744 Total energy is 7176191.9680919927 [STEP 12] Coarse TimeStep time: 0.659476402 [STEP 12] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 12 TIME = 0.003444681184 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 13] ADVANCE at time 0.003444681184 with dt = 0.0002870523239 [Level 0 step 13] Advanced 4194304 cells [Level 1 step 25] ADVANCE at time 0.003444681184 with dt = 0.0001435261619 [Level 1 step 25] Advanced 4194304 cells [Level 1 step 26] ADVANCE at time 0.003588207345 with dt = 0.0001435261619 [Level 1 step 26] Advanced 4194304 cells STEP = 13 TIME = 0.003731733507 DT = 0.0002870523239 grid_places() time: 0.016059355 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -4.0390711608099386e-18 Total y-momentum is -7.3165912148932354e-18 Total z-momentum is 23.674374717272059 Total energy is 7176191.9616217129 [STEP 13] Coarse TimeStep time: 0.695825876 [STEP 13] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 14] ADVANCE at time 0.003731733507 with dt = 0.000287052422 [Level 0 step 14] Advanced 4194304 cells [Level 1 step 27] ADVANCE at time 0.003731733507 with dt = 0.000143526211 [Level 1 step 27] Advanced 4194304 cells [Level 1 step 28] ADVANCE at time 0.003875259718 with dt = 0.000143526211 [Level 1 step 28] Advanced 4194304 cells STEP = 14 TIME = 0.004018785929 DT = 0.000287052422 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -1.4254982508844783e-18 Total y-momentum is 1.015700761020574e-18 Total z-momentum is 25.95408273901171 Total energy is 7176191.9545008428 [STEP 14] Coarse TimeStep time: 0.658913561 [STEP 14] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 14 TIME = 0.004018785929 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 15] ADVANCE at time 0.004018785929 with dt = 0.0002870524893 [Level 0 step 15] Advanced 4194304 cells [Level 1 step 29] ADVANCE at time 0.004018785929 with dt = 0.0001435262446 [Level 1 step 29] Advanced 4194304 cells [Level 1 step 30] ADVANCE at time 0.004162312174 with dt = 0.0001435262446 [Level 1 step 30] Advanced 4194304 cells STEP = 15 TIME = 0.004305838419 DT = 0.0002870524893 grid_places() time: 0.016091729 new finest: 1 [CNS] Total mass is 1302338.0233411889 Total x-momentum is -3.8096209963360823e-19 Total y-momentum is -8.7918912827820473e-19 Total z-momentum is 28.258411443578304 Total energy is 7176191.9467217075 [STEP 15] Coarse TimeStep time: 0.695546567 [STEP 15] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 16] ADVANCE at time 0.004305838419 with dt = 0.0002870525306 [Level 0 step 16] Advanced 4194304 cells [Level 1 step 31] ADVANCE at time 0.004305838419 with dt = 0.0001435262653 [Level 1 step 31] Advanced 4194304 cells [Level 1 step 32] ADVANCE at time 0.004449364684 with dt = 0.0001435262653 [Level 1 step 32] Advanced 4194304 cells STEP = 16 TIME = 0.004592890949 DT = 0.0002870525306 [CNS] Total mass is 1302338.0233411889 Total x-momentum is 5.4730419547846157e-18 Total y-momentum is 6.7296237746774245e-18 Total z-momentum is 30.583394611415127 Total energy is 7176191.9382778564 [STEP 16] Coarse TimeStep time: 0.658853994 [STEP 16] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 16 TIME = 0.004592890949 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 17] ADVANCE at time 0.004592890949 with dt = 0.000287052552 [Level 0 step 17] Advanced 4194304 cells [Level 1 step 33] ADVANCE at time 0.004592890949 with dt = 0.000143526276 [Level 1 step 33] Advanced 4194304 cells [Level 1 step 34] ADVANCE at time 0.004736417225 with dt = 0.000143526276 [Level 1 step 34] Advanced 4194304 cells STEP = 17 TIME = 0.004879943501 DT = 0.000287052552 grid_places() time: 0.016154933 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -9.2035317838578576e-18 Total y-momentum is -2.6665374729341175e-18 Total z-momentum is 32.925613582717489 Total energy is 7176191.9291638918 [STEP 17] Coarse TimeStep time: 0.694454581 [STEP 17] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 18] ADVANCE at time 0.004879943501 with dt = 0.0002870525597 [Level 0 step 18] Advanced 4194304 cells [Level 1 step 35] ADVANCE at time 0.004879943501 with dt = 0.0001435262798 [Level 1 step 35] Advanced 4194304 cells [Level 1 step 36] ADVANCE at time 0.005023469781 with dt = 0.0001435262798 [Level 1 step 36] Advanced 4194304 cells STEP = 18 TIME = 0.005166996061 DT = 0.0002870525597 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -1.3908635798758459e-17 Total y-momentum is -1.6212768957360782e-18 Total z-momentum is 35.282122513822877 Total energy is 7176191.9193753228 [STEP 18] Coarse TimeStep time: 0.658572969 [STEP 18] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 18 TIME = 0.005166996061 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 19] ADVANCE at time 0.005166996061 with dt = 0.0002870525588 [Level 0 step 19] Advanced 4194304 cells [Level 1 step 37] ADVANCE at time 0.005166996061 with dt = 0.0001435262794 [Level 1 step 37] Advanced 4194304 cells [Level 1 step 38] ADVANCE at time 0.00531052234 with dt = 0.0001435262794 [Level 1 step 38] Advanced 4194304 cells STEP = 19 TIME = 0.00545404862 DT = 0.0002870525588 grid_places() time: 0.015910408 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -3.043424857689809e-19 Total y-momentum is 1.1848493735178941e-19 Total z-momentum is 37.650387281643575 Total energy is 7176191.9089084426 [STEP 19] Coarse TimeStep time: 0.693602394 [STEP 19] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 20] ADVANCE at time 0.00545404862 with dt = 0.0002870525536 [Level 0 step 20] Advanced 4194304 cells [Level 1 step 39] ADVANCE at time 0.00545404862 with dt = 0.0001435262768 [Level 1 step 39] Advanced 4194304 cells [Level 1 step 40] ADVANCE at time 0.005597574896 with dt = 0.0001435262768 [Level 1 step 40] Advanced 4194304 cells STEP = 20 TIME = 0.005741101173 DT = 0.0002870525536 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 7.3173620118177381e-18 Total y-momentum is -7.3889842784956805e-18 Total z-momentum is 40.028234710671775 Total energy is 7176191.8977602143 [STEP 20] Coarse TimeStep time: 0.65659543 [STEP 20] FAB kilobyte spread across MPI nodes: [575320 ... 575320] PLOTFILE: file = CNS-RT_plt00020 Write plotfile time = 0.143041442 seconds Run Time total = 13.9907318 Run Time init = 0.269681888 Run Time advance = 13.57778016 TinyProfiler total time across processes [min...avg...max]: 13.98 ... 13.99 ... 13.99 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- CNS::compute_dSdt() 120 8.002 8.127 8.242 58.91% FabArray::ParallelCopy_finish() 572 0.5226 1.138 1.736 12.41% StateDataPhysBCFunct::() 220 0.001211 0.8537 1.711 12.23% FabArray::ParallelCopy_nowait() 572 0.4842 0.6897 0.892 6.38% FluxRegister::Reflux() 120 0.6185 0.6558 0.673 4.81% RungeKutta2 60 0.3161 0.3411 0.3872 2.77% FabArray::setVal() 282 0.2259 0.2525 0.2817 2.01% VisMF::Write(FabArray) 4 0.2731 0.2768 0.2789 1.99% FillBoundary_finish() 80 0.1132 0.189 0.2217 1.58% CNS::computeTemp() 144 0.1439 0.1823 0.2184 1.56% FillBoundary_nowait() 80 0.1454 0.1692 0.1938 1.39% CellConservativeLinear::interp() 120 0.03208 0.09421 0.1655 1.18% FillPatcher::fillCFB 80 0.107 0.1241 0.1456 1.04% FillPatchIterator::Initialize 60 0.1151 0.1213 0.1301 0.93% amrex::Copy() 88 0.08644 0.102 0.1139 0.81% Amr::regrid() 10 0.1075 0.1078 0.1082 0.77% amrex::average_down 21 0.07674 0.08901 0.1068 0.76% FillPatchInterp(Fab) 80 0.01695 0.05063 0.09691 0.69% CNS::estTimeStep() 41 0.05753 0.07468 0.09226 0.66% FabArray::sum() 105 0.05527 0.06879 0.08578 0.61% CNS::advance() 60 0.06811 0.0702 0.07437 0.53% TagBoxArray::collate() 11 0.001988 0.03706 0.0719 0.51% FabArray::setDomainBndry() 60 0.0002087 0.02406 0.04825 0.34% post_timestep 60 0.03174 0.03467 0.03749 0.27% CNS::errorEst() 11 0.02096 0.02209 0.02309 0.17% Amr::bldFineLevels() 1 0.0126 0.01266 0.01272 0.09% Amr::coarseTimeStep() 20 0.0002783 0.003078 0.009769 0.07% AmrMesh::MakeNewGrids() 11 0.005968 0.007626 0.009222 0.07% TagBoxArray::mapPRD 11 0.004149 0.00662 0.008878 0.06% AmrLevel::writePlotFile() 4 0.005143 0.006275 0.007849 0.06% Amr::writePlotFile() 2 0.002166 0.002911 0.003315 0.02% Amr::timeStep() 60 0.0006247 0.0008788 0.00142 0.01% AmrLevel::FillPatch() 60 0.0006651 0.0007154 0.0007818 0.01% FabArray::ParallelCopy() 572 0.000542 0.0005911 0.0006383 0.00% FillPatchSingleLevel 140 0.0002618 0.0002833 0.0003285 0.00% AmrLevel::FillPatcherFill() 120 0.0002609 0.0002889 0.000315 0.00% main() 1 0.0001876 0.0002214 0.0002523 0.00% FillPatcher::fill() 80 0.0001448 0.0001644 0.0002018 0.00% FabArray::FillBoundary() 80 0.0001326 0.0001536 0.0001746 0.00% FillPatchIterator::FillFromLevel0() 51 6.773e-05 7.006e-05 7.237e-05 0.00% postCoarseTimeStep() 20 3.926e-05 4.925e-05 6.248e-05 0.00% Amr::grid_places() 11 5.627e-05 5.813e-05 5.98e-05 0.00% Amr::FinalizeInit() 1 4.646e-06 1.669e-05 5.172e-05 0.00% CNS::avgDown() 21 3.641e-05 4.244e-05 5.011e-05 0.00% AmrLevel::RK() 60 3.995e-05 4.475e-05 4.844e-05 0.00% AmrLevel::AmrLevel(dm) 12 2.697e-05 2.814e-05 2.977e-05 0.00% amrex::average_down_w_geom 21 1.837e-05 2.01e-05 2.108e-05 0.00% CNS::writePlotFile() 4 6.888e-06 7.335e-06 7.989e-06 0.00% Amr::init() 1 1.271e-06 1.451e-06 1.717e-06 0.00% Amr::initialInit() 1 8.84e-07 1.015e-06 1.244e-06 0.00% Other 1209 0.03377 0.04694 0.06044 0.43% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 13.98 13.99 13.99 100.00% Amr::coarseTimeStep() 20 13.58 13.58 13.58 97.05% Amr::timeStep() 60 13.4 13.44 13.47 96.29% CNS::advance() 60 11.92 11.93 11.93 85.29% AmrLevel::RK() 60 11.85 11.85 11.86 84.80% RungeKutta2 60 11.85 11.85 11.86 84.80% CNS::compute_dSdt() 120 8.233 9.071 9.909 70.82% AmrLevel::FillPatcherFill() 120 1.442 2.282 3.113 22.25% FabArray::ParallelCopy() 572 1.04 1.83 2.631 18.80% FillPatchSingleLevel 140 0.5774 1.459 2.336 16.70% AmrLevel::FillPatch() 60 0.468 1.335 2.194 15.68% FillPatchIterator::Initialize 60 0.3647 1.245 2.11 15.08% FillPatchIterator::FillFromLevel0() 51 0.2358 1.09 1.935 13.83% FabArray::ParallelCopy_finish() 572 0.5226 1.138 1.736 12.41% StateDataPhysBCFunct::() 220 0.001211 0.8583 1.72 12.29% post_timestep 60 1.171 1.209 1.242 8.88% FillPatcher::fill() 80 1.067 1.083 1.1 7.86% FluxRegister::Reflux() 120 0.914 0.9453 0.9788 7.00% FabArray::ParallelCopy_nowait() 572 0.4859 0.6918 0.8941 6.39% FillPatcher::fillCFB 80 0.6993 0.7234 0.7914 5.66% FabArray::FillBoundary() 80 0.3073 0.3585 0.3954 2.83% Amr::regrid() 10 0.3398 0.3422 0.3455 2.47% Amr::writePlotFile() 2 0.2985 0.2989 0.2994 2.14% CNS::writePlotFile() 4 0.2955 0.2959 0.2971 2.12% AmrLevel::writePlotFile() 4 0.2955 0.2959 0.2971 2.12% FabArray::setVal() 282 0.2259 0.2525 0.2817 2.01% VisMF::Write(FabArray) 4 0.2731 0.2768 0.2789 1.99% Amr::init() 1 0.2592 0.2631 0.2657 1.90% CNS::avgDown() 21 0.2028 0.2252 0.2464 1.76% FillBoundary_finish() 80 0.1132 0.189 0.2217 1.58% amrex::average_down_w_geom 21 0.196 0.2102 0.2192 1.57% amrex::average_down 21 0.196 0.2102 0.2191 1.57% CNS::computeTemp() 144 0.1439 0.1823 0.2184 1.56% FillBoundary_nowait() 80 0.1456 0.1693 0.1939 1.39% Amr::grid_places() 11 0.1827 0.1832 0.1836 1.31% AmrMesh::MakeNewGrids() 11 0.1827 0.1831 0.1835 1.31% FillPatchInterp(Fab) 80 0.1033 0.1448 0.183 1.31% CellConservativeLinear::interp() 120 0.03208 0.09421 0.1655 1.18% CNS::errorEst() 11 0.06911 0.1117 0.1544 1.10% amrex::Copy() 88 0.08644 0.102 0.1139 0.81% Amr::initialInit() 1 0.1034 0.1074 0.1102 0.79% Amr::FinalizeInit() 1 0.09592 0.09631 0.09705 0.69% CNS::estTimeStep() 41 0.05753 0.07468 0.09226 0.66% FabArray::sum() 105 0.05527 0.06879 0.08578 0.61% postCoarseTimeStep() 20 0.05138 0.06509 0.08081 0.58% Amr::bldFineLevels() 1 0.07758 0.07794 0.07831 0.56% TagBoxArray::collate() 11 0.001988 0.03706 0.0719 0.51% FabArray::setDomainBndry() 60 0.0002087 0.02406 0.04825 0.34% TagBoxArray::mapPRD 11 0.01249 0.01915 0.02543 0.18% AmrLevel::AmrLevel(dm) 12 0.02256 0.02317 0.02398 0.17% Other 1209 0.07358 0.1004 0.1222 0.87% ------------------------------------------------------------------------------------------- Device Memory Usage: ---------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ---------------------------------------------------------------------------------------------------------------- CNS::compute_dSdt() 19200 127 MiB 140 MiB 154 MiB 261 MiB 261 MiB 261 MiB StateData::define() 192 133 MiB 133 MiB 133 MiB 201 MiB 201 MiB 201 MiB CNS::advance() 176 117 MiB 117 MiB 117 MiB 134 MiB 134 MiB 134 MiB FluxRegister::Reflux() 1920 3560 KiB 3739 KiB 3896 KiB 56 MiB 67 MiB 71 MiB RungeKutta2 1244 46 MiB 46 MiB 46 MiB 70 MiB 70 MiB 70 MiB FillPatchIterator::Initialize 960 1851 KiB 5873 KiB 9879 KiB 67 MiB 67 MiB 67 MiB AmrLevel::writePlotFile() 64 1186 KiB 1188 KiB 1189 KiB 56 MiB 56 MiB 56 MiB FillPatcher::fillCFB 760 4278 KiB 6449 KiB 8637 KiB 7998 KiB 11 MiB 15 MiB ResizeRandomSeed 4 9210 KiB 9210 KiB 9210 KiB 9216 KiB 9216 KiB 9216 KiB CNS::errorEst() 176 39 KiB 66 KiB 93 KiB 8985 KiB 8985 KiB 8985 KiB post_timestep 320 574 KiB 584 KiB 590 KiB 8192 KiB 8192 KiB 8192 KiB amrex::average_down 428 79 KiB 90 KiB 99 KiB 7169 KiB 7169 KiB 7169 KiB CellConservativeLinear::interp() 960 11 KiB 18 KiB 29 KiB 1355 KiB 2064 KiB 2773 KiB Amr::regrid() 960 1328 KiB 1328 KiB 1329 KiB 2688 KiB 2688 KiB 2688 KiB StateDataPhysBCFunct::() 816 0 B 48 KiB 97 KiB 0 B 736 KiB 1473 KiB Amr::bldFineLevels() 96 4420 B 4451 B 4462 B 1344 KiB 1344 KiB 1344 KiB AmrMesh::MakeNewGrids() 352 13 KiB 13 KiB 13 KiB 1025 KiB 1025 KiB 1025 KiB FabArray::ParallelCopy_nowait() 2202 16 KiB 55 KiB 95 KiB 26 KiB 99 KiB 173 KiB VisMF::Write(FabArray) 448 173 B 204 B 236 B 96 KiB 96 KiB 96 KiB CNS::estTimeStep() 172 1135 B 1149 B 1163 B 49 KiB 49 KiB 49 KiB FabArray::sum() 420 191 B 237 B 297 B 48 KiB 48 KiB 48 KiB FabArray::ParallelCopy_finish() 1568 5163 B 6478 B 8301 B 9216 B 13 KiB 17 KiB FillBoundary_nowait() 160 7565 B 7573 B 7579 B 10 KiB 10 KiB 10 KiB FillBoundary_finish() 80 3778 B 3778 B 3780 B 5120 B 5120 B 5120 B FabArray::setVal() 1144 2092 B 2120 B 2150 B 3072 B 3072 B 3072 B amrex::Copy() 348 1956 B 1957 B 1959 B 3072 B 3072 B 3072 B TagBoxArray::mapPRD 352 1 B 2 B 3 B 2048 B 2048 B 2048 B TagBoxArray::collate() 88 0 B 0 B 0 B 256 B 896 B 1536 B OwnerMask() 176 0 B 0 B 0 B 1024 B 1024 B 1024 B StateData::FillBoundary(geom) 1224 0 B 10 B 21 B 0 B 256 B 512 B Amr::InitAmr() 8 505 B 505 B 505 B 512 B 512 B 512 B ---------------------------------------------------------------------------------------------------------------- Async Memory Usage: ------------------------------------------------------------------------------------------------------ Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ------------------------------------------------------------------------------------------------------ CNS::compute_dSdt() 3840 4730 KiB 8476 KiB 11 MiB 65 MiB 65 MiB 65 MiB FluxRegister::Reflux() 1920 2788 KiB 3138 KiB 3502 KiB 56 MiB 56 MiB 56 MiB RungeKutta2 960 46 MiB 46 MiB 46 MiB 56 MiB 56 MiB 56 MiB post_timestep 320 548 KiB 566 KiB 586 KiB 8192 KiB 8192 KiB 8192 KiB amrex::average_down 336 78 KiB 90 KiB 99 KiB 7168 KiB 7168 KiB 7168 KiB ------------------------------------------------------------------------------------------------------ Managed Memory Usage: --------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------- The_Managed_Arena::Initialize() 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------- Pinned Memory Usage: ---------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ---------------------------------------------------------------------------------------------------------------- FabArray::ParallelCopy_nowait() 5496 292 KiB 514 KiB 707 KiB 7189 KiB 11 MiB 14 MiB VisMF::Write(FabArray) 512 217 KiB 224 KiB 238 KiB 14 MiB 14 MiB 14 MiB The_Pinned_Arena::Initialize() 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB FillBoundary_nowait() 800 169 KiB 196 KiB 216 KiB 7514 KiB 7514 KiB 7514 KiB FabArray::ParallelCopy_finish() 1568 5163 B 6478 B 8301 B 9216 B 13 KiB 17 KiB FillBoundary_finish() 80 3778 B 3778 B 3780 B 5120 B 5120 B 5120 B CNS::compute_dSdt() 1920 127 B 181 B 235 B 3072 B 3072 B 3072 B amrex::Copy() 348 1956 B 1957 B 1959 B 3072 B 3072 B 3072 B FabArray::setVal() 1144 2092 B 2120 B 2150 B 3072 B 3072 B 3072 B TagBoxArray::collate() 154 0 B 2 B 4 B 512 B 1344 B 1792 B amrex::average_down 92 563 B 563 B 565 B 1536 B 1536 B 1536 B RungeKutta2 284 1076 B 1076 B 1077 B 1536 B 1536 B 1536 B CNS::estTimeStep() 172 1011 B 1011 B 1011 B 1280 B 1280 B 1280 B FillPatcher::fillCFB 280 394 B 467 B 540 B 768 B 896 B 1024 B StateData::FillBoundary(geom) 1224 0 B 10 B 21 B 0 B 256 B 512 B CellConservativeLinear::interp() 480 1 B 1 B 2 B 256 B 384 B 512 B FabArray::sum() 420 0 B 0 B 1 B 256 B 256 B 256 B ---------------------------------------------------------------------------------------------------------------- Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [4227 ... 4227] [The Arena] max space (MB) allocated spread across MPI: [741 ... 766] [The Arena] max space (MB) used spread across MPI: [603 ... 611] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [32 ... 36] [The Pinned Arena] max space (MB) used spread across MPI: [14 ... 14] AMReX (26.09-163-gfca855037a21) finalized