Initializing AMReX (26.04-109-g60df5ae33414)... MPI initialized with 4 MPI processes MPI initialized with thread support level 0 Initializing CUDA... CUDA initialized with 1 device. AMReX (26.04-109-g60df5ae33414) initialized Successfully read inputs file ... Successfully read inputs file ... Starting to call amrex_probinit ... Successfully run amrex_probinit Now regridding at level lbase = 0 grid_places() time: 0.019094825 new finest: 1 STEP = 0 TIME = 0 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 INITIAL GRIDS Level 0 16 grids 4194304 cells 100 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 PLOTFILE: file = CNS-RT_plt00000 grid_places() time: 0.018665812 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 0 Total y-momentum is 0 Total z-momentum is 0 Total energy is 7176192.0007176176 Write plotfile time = 0.148642095 seconds [Level 0 step 1] ADVANCE at time 0 with dt = 0.0002870776155 [Level 0 step 1] Advanced 4194304 cells [Level 1 step 1] ADVANCE at time 0 with dt = 0.0001435388078 [Level 1 step 1] Advanced 4194304 cells [Level 1 step 2] ADVANCE at time 0.0001435388078 with dt = 0.0001435388078 [Level 1 step 2] Advanced 4194304 cells STEP = 1 TIME = 0.0002870776155 DT = 0.0002870776155 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 3.4600423210458683e-21 Total y-momentum is 2.0479171591388077e-21 Total z-momentum is 1.0569463681968465 Total energy is 7176192.0005832035 [STEP 1] Coarse TimeStep time: 0.688987136 [STEP 1] FAB kilobyte spread across MPI nodes: [536184 ... 544157] [Level 0 step 2] ADVANCE at time 0.0002870776155 with dt = 0.0002870783094 [Level 0 step 2] Advanced 4194304 cells [Level 1 step 3] ADVANCE at time 0.0002870776155 with dt = 0.0001435391547 [Level 1 step 3] Advanced 4194304 cells [Level 1 step 4] ADVANCE at time 0.0004306167702 with dt = 0.0001435391547 [Level 1 step 4] Advanced 4194304 cells STEP = 2 TIME = 0.0005741559249 DT = 0.0002870783094 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 3.9958311725556041e-20 Total y-momentum is 1.3460343874667005e-20 Total z-momentum is 2.3209944805054441 Total energy is 7176192.0001132581 [STEP 2] Coarse TimeStep time: 0.667865622 [STEP 2] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 2 TIME = 0.0005741559249 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 3] ADVANCE at time 0.0005741559249 with dt = 0.0002870780246 [Level 0 step 3] Advanced 4194304 cells [Level 1 step 5] ADVANCE at time 0.0005741559249 with dt = 0.0001435390123 [Level 1 step 5] Advanced 4194304 cells [Level 1 step 6] ADVANCE at time 0.0007176949372 with dt = 0.0001435390123 [Level 1 step 6] Advanced 4194304 cells STEP = 3 TIME = 0.0008612339496 DT = 0.0002870780246 grid_places() time: 0.016023914 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -1.5772450013167652e-20 Total y-momentum is -3.3612896367759814e-20 Total z-momentum is 3.7625461629590609 Total energy is 7176191.9992528344 [STEP 3] Coarse TimeStep time: 0.704441114 [STEP 3] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 4] ADVANCE at time 0.0008612339496 with dt = 0.0002870774356 [Level 0 step 4] Advanced 4194304 cells [Level 1 step 7] ADVANCE at time 0.0008612339496 with dt = 0.0001435387178 [Level 1 step 7] Advanced 4194304 cells [Level 1 step 8] ADVANCE at time 0.001004772667 with dt = 0.0001435387178 [Level 1 step 8] Advanced 4194304 cells STEP = 4 TIME = 0.001148311385 DT = 0.0002870774356 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -1.6642168363118369e-19 Total y-momentum is -5.342067168146162e-19 Total z-momentum is 5.355597903262626 Total energy is 7176191.9979549646 [STEP 4] Coarse TimeStep time: 0.670057999 [STEP 4] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 4 TIME = 0.001148311385 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 5] ADVANCE at time 0.001148311385 with dt = 0.0002870767217 [Level 0 step 5] Advanced 4194304 cells [Level 1 step 9] ADVANCE at time 0.001148311385 with dt = 0.0001435383608 [Level 1 step 9] Advanced 4194304 cells [Level 1 step 10] ADVANCE at time 0.001291849746 with dt = 0.0001435383608 [Level 1 step 10] Advanced 4194304 cells STEP = 5 TIME = 0.001435388107 DT = 0.0002870767217 grid_places() time: 0.016285372 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -1.358812720720659e-19 Total y-momentum is -2.0160459479448667e-21 Total z-momentum is 7.0775892529197408 Total energy is 7176191.9961796384 [STEP 5] Coarse TimeStep time: 0.706635908 [STEP 5] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 6] ADVANCE at time 0.001435388107 with dt = 0.0002870760002 [Level 0 step 6] Advanced 4194304 cells [Level 1 step 11] ADVANCE at time 0.001435388107 with dt = 0.0001435380001 [Level 1 step 11] Advanced 4194304 cells [Level 1 step 12] ADVANCE at time 0.001578926107 with dt = 0.0001435380001 [Level 1 step 12] Advanced 4194304 cells STEP = 6 TIME = 0.001722464107 DT = 0.0002870760002 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 7.2906006961982036e-19 Total y-momentum is -1.2884571541290133e-18 Total z-momentum is 8.9091186975067487 Total energy is 7176191.9938928559 [STEP 6] Coarse TimeStep time: 0.6686652 [STEP 6] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 6 TIME = 0.001722464107 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 7] ADVANCE at time 0.001722464107 with dt = 0.0002870753412 [Level 0 step 7] Advanced 4194304 cells [Level 1 step 13] ADVANCE at time 0.001722464107 with dt = 0.0001435376706 [Level 1 step 13] Advanced 4194304 cells [Level 1 step 14] ADVANCE at time 0.001866001778 with dt = 0.0001435376706 [Level 1 step 14] Advanced 4194304 cells STEP = 7 TIME = 0.002009539448 DT = 0.0002870753412 grid_places() time: 0.016091801 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -8.5144132686239444e-19 Total y-momentum is -1.3986777590613538e-20 Total z-momentum is 10.83361436279802 Total energy is 7176191.9910657536 [STEP 7] Coarse TimeStep time: 0.704014797 [STEP 7] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 8] ADVANCE at time 0.002009539448 with dt = 0.00028707478 [Level 0 step 8] Advanced 4194304 cells [Level 1 step 15] ADVANCE at time 0.002009539448 with dt = 0.00014353739 [Level 1 step 15] Advanced 4194304 cells [Level 1 step 16] ADVANCE at time 0.002153076838 with dt = 0.00014353739 [Level 1 step 16] Advanced 4194304 cells STEP = 8 TIME = 0.002296614228 DT = 0.00028707478 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -7.2204597721556348e-19 Total y-momentum is -2.4427946815143561e-18 Total z-momentum is 12.836984615501322 Total energy is 7176191.9876738414 [STEP 8] Coarse TimeStep time: 0.666357648 [STEP 8] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 8 TIME = 0.002296614228 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 9] ADVANCE at time 0.002296614228 with dt = 0.0002870743281 [Level 0 step 9] Advanced 4194304 cells [Level 1 step 17] ADVANCE at time 0.002296614228 with dt = 0.000143537164 [Level 1 step 17] Advanced 4194304 cells [Level 1 step 18] ADVANCE at time 0.002440151392 with dt = 0.000143537164 [Level 1 step 18] Advanced 4194304 cells STEP = 9 TIME = 0.002583688556 DT = 0.0002870743281 grid_places() time: 0.01625512 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 1.2783712819635498e-18 Total y-momentum is 1.0977113243582025e-19 Total z-momentum is 14.907274492170664 Total energy is 7176191.9836963499 [STEP 9] Coarse TimeStep time: 0.703219225 [STEP 9] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 10] ADVANCE at time 0.002583688556 with dt = 0.0002870739818 [Level 0 step 10] Advanced 4194304 cells [Level 1 step 19] ADVANCE at time 0.002583688556 with dt = 0.0001435369909 [Level 1 step 19] Advanced 4194304 cells [Level 1 step 20] ADVANCE at time 0.002727225547 with dt = 0.0001435369909 [Level 1 step 20] Advanced 4194304 cells STEP = 10 TIME = 0.002870762538 DT = 0.0002870739818 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -2.0429037700670925e-18 Total y-momentum is 6.4881480403821162e-19 Total z-momentum is 17.034345387467006 Total energy is 7176191.9791156547 [STEP 10] Coarse TimeStep time: 0.665259128 [STEP 10] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 10 TIME = 0.002870762538 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 11] ADVANCE at time 0.002870762538 with dt = 0.0002870737289 [Level 0 step 11] Advanced 4194304 cells [Level 1 step 21] ADVANCE at time 0.002870762538 with dt = 0.0001435368644 [Level 1 step 21] Advanced 4194304 cells [Level 1 step 22] ADVANCE at time 0.003014299402 with dt = 0.0001435368644 [Level 1 step 22] Advanced 4194304 cells STEP = 11 TIME = 0.003157836267 DT = 0.0002870737289 grid_places() time: 0.016309108 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -2.3110753741976848e-18 Total y-momentum is 8.5020291015675668e-19 Total z-momentum is 19.209588257631538 Total energy is 7176191.9739167988 [STEP 11] Coarse TimeStep time: 0.703120596 [STEP 11] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 12] ADVANCE at time 0.003157836267 with dt = 0.0002870735531 [Level 0 step 12] Advanced 4194304 cells [Level 1 step 23] ADVANCE at time 0.003157836267 with dt = 0.0001435367766 [Level 1 step 23] Advanced 4194304 cells [Level 1 step 24] ADVANCE at time 0.003301373043 with dt = 0.0001435367766 [Level 1 step 24] Advanced 4194304 cells STEP = 12 TIME = 0.00344490982 DT = 0.0002870735531 [CNS] Total mass is 1302338.0233411889 Total x-momentum is 1.1878774904533169e-18 Total y-momentum is 6.1642905454492962e-18 Total z-momentum is 21.425674928290547 Total energy is 7176191.9680870995 [STEP 12] Coarse TimeStep time: 0.66527702 [STEP 12] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 12 TIME = 0.00344490982 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 13] ADVANCE at time 0.00344490982 with dt = 0.0002870734377 [Level 0 step 13] Advanced 4194304 cells [Level 1 step 25] ADVANCE at time 0.00344490982 with dt = 0.0001435367188 [Level 1 step 25] Advanced 4194304 cells [Level 1 step 26] ADVANCE at time 0.003588446539 with dt = 0.0001435367188 [Level 1 step 26] Advanced 4194304 cells STEP = 13 TIME = 0.003731983258 DT = 0.0002870734377 grid_places() time: 0.016247055 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 1.9370850150886611e-18 Total y-momentum is -3.7199697674035681e-18 Total z-momentum is 23.676347925915085 Total energy is 7176191.9616158046 [STEP 13] Coarse TimeStep time: 0.701102628 [STEP 13] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 14] ADVANCE at time 0.003731983258 with dt = 0.000287073367 [Level 0 step 14] Advanced 4194304 cells [Level 1 step 27] ADVANCE at time 0.003731983258 with dt = 0.0001435366835 [Level 1 step 27] Advanced 4194304 cells [Level 1 step 28] ADVANCE at time 0.003875519941 with dt = 0.0001435366835 [Level 1 step 28] Advanced 4194304 cells STEP = 14 TIME = 0.004019056625 DT = 0.000287073367 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 1.9618247652419002e-18 Total y-momentum is 2.1637474215199119e-18 Total z-momentum is 25.956246439946725 Total energy is 7176191.9544938197 [STEP 14] Coarse TimeStep time: 0.664222215 [STEP 14] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 14 TIME = 0.004019056625 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 15] ADVANCE at time 0.004019056625 with dt = 0.0002870733276 [Level 0 step 15] Advanced 4194304 cells [Level 1 step 29] ADVANCE at time 0.004019056625 with dt = 0.0001435366638 [Level 1 step 29] Advanced 4194304 cells [Level 1 step 30] ADVANCE at time 0.004162593288 with dt = 0.0001435366638 [Level 1 step 30] Advanced 4194304 cells STEP = 15 TIME = 0.004306129952 DT = 0.0002870733276 grid_places() time: 0.016303784 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is 2.5164719770389941e-18 Total y-momentum is -1.9526228567543407e-18 Total z-momentum is 28.260764354252117 Total energy is 7176191.9467134727 [STEP 15] Coarse TimeStep time: 0.701130533 [STEP 15] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 16] ADVANCE at time 0.004306129952 with dt = 0.000287073309 [Level 0 step 16] Advanced 4194304 cells [Level 1 step 31] ADVANCE at time 0.004306129952 with dt = 0.0001435366545 [Level 1 step 31] Advanced 4194304 cells [Level 1 step 32] ADVANCE at time 0.004449666607 with dt = 0.0001435366545 [Level 1 step 32] Advanced 4194304 cells STEP = 16 TIME = 0.004593203261 DT = 0.000287073309 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -7.0597052721974554e-18 Total y-momentum is 1.599185848405719e-18 Total z-momentum is 30.58593551816789 Total energy is 7176191.9382683076 [STEP 16] Coarse TimeStep time: 0.663649397 [STEP 16] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 16 TIME = 0.004593203261 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 17] ADVANCE at time 0.004593203261 with dt = 0.0002870733031 [Level 0 step 17] Advanced 4194304 cells [Level 1 step 33] ADVANCE at time 0.004593203261 with dt = 0.0001435366516 [Level 1 step 33] Advanced 4194304 cells [Level 1 step 34] ADVANCE at time 0.004736739913 with dt = 0.0001435366516 [Level 1 step 34] Advanced 4194304 cells STEP = 17 TIME = 0.004880276564 DT = 0.0002870733031 grid_places() time: 0.016315088 new finest: 1 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -1.9438899178277888e-18 Total y-momentum is -6.0749455267165249e-18 Total z-momentum is 32.928341316525405 Total energy is 7176191.9291529283 [STEP 17] Coarse TimeStep time: 0.699733449 [STEP 17] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 18] ADVANCE at time 0.004880276564 with dt = 0.0002870733041 [Level 0 step 18] Advanced 4194304 cells [Level 1 step 35] ADVANCE at time 0.004880276564 with dt = 0.0001435366521 [Level 1 step 35] Advanced 4194304 cells [Level 1 step 36] ADVANCE at time 0.005023813216 with dt = 0.0001435366521 [Level 1 step 36] Advanced 4194304 cells STEP = 18 TIME = 0.005167349868 DT = 0.0002870733041 [CNS] Total mass is 1302338.0233411889 Total x-momentum is 5.6810548860320156e-18 Total y-momentum is -1.5707062577299444e-17 Total z-momentum is 35.285035925910549 Total energy is 7176191.9193628421 [STEP 18] Coarse TimeStep time: 0.662666311 [STEP 18] FAB kilobyte spread across MPI nodes: [575320 ... 575320] Now regridding at level lbase = 0 STEP = 18 TIME = 0.005167349868 : REGRID with lbase = 0 Level 1 16 grids 4194304 cells 12.5 % of domain smallest grid: 64 x 64 x 64 biggest grid: 64 x 64 x 64 [Level 0 step 19] ADVANCE at time 0.005167349868 with dt = 0.0002870733082 [Level 0 step 19] Advanced 4194304 cells [Level 1 step 37] ADVANCE at time 0.005167349868 with dt = 0.0001435366541 [Level 1 step 37] Advanced 4194304 cells [Level 1 step 38] ADVANCE at time 0.005310886523 with dt = 0.0001435366541 [Level 1 step 38] Advanced 4194304 cells STEP = 19 TIME = 0.005454423177 DT = 0.0002870733082 grid_places() time: 0.016314842 new finest: 1 [CNS] Total mass is 1302338.0233411889 Total x-momentum is 8.3932117135403259e-18 Total y-momentum is -2.0684121055476387e-17 Total z-momentum is 37.653485228064916 Total energy is 7176191.9088943424 [STEP 19] Coarse TimeStep time: 0.699456048 [STEP 19] FAB kilobyte spread across MPI nodes: [575320 ... 575320] [Level 0 step 20] ADVANCE at time 0.005454423177 with dt = 0.0002870733131 [Level 0 step 20] Advanced 4194304 cells [Level 1 step 39] ADVANCE at time 0.005454423177 with dt = 0.0001435366565 [Level 1 step 39] Advanced 4194304 cells [Level 1 step 40] ADVANCE at time 0.005597959833 with dt = 0.0001435366565 [Level 1 step 40] Advanced 4194304 cells STEP = 20 TIME = 0.00574149649 DT = 0.0002870733131 [CNS] Total mass is 1302338.0233411891 Total x-momentum is -6.3501140143874955e-18 Total y-momentum is -1.7382606036736604e-18 Total z-momentum is 40.031516048437133 Total energy is 7176191.8977443911 [STEP 20] Coarse TimeStep time: 0.66245087 [STEP 20] FAB kilobyte spread across MPI nodes: [575320 ... 575320] PLOTFILE: file = CNS-RT_plt00020 Write plotfile time = 0.139106919 seconds Run Time total = 14.06564928 Run Time init = 0.260623483 Run Time advance = 13.66566884 TinyProfiler total time across processes [min...avg...max]: 14.05 ... 14.06 ... 14.07 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- CNS::compute_dSdt() 120 8.082 8.232 8.394 59.68% StateDataPhysBCFunct::() 220 0.001163 0.8596 1.725 12.26% FabArray::ParallelCopy_finish() 572 0.5014 1.072 1.63 11.59% FabArray::ParallelCopy_nowait() 572 0.4813 0.7102 0.9652 6.86% FluxRegister::Reflux() 120 0.6354 0.6647 0.6823 4.85% RungeKutta2 60 0.3179 0.3434 0.3659 2.60% FabArray::setVal() 282 0.226 0.2527 0.2789 1.98% VisMF::Write(FabArray) 4 0.2647 0.2678 0.2714 1.93% FillBoundary_finish() 80 0.1284 0.1891 0.2113 1.50% CNS::computeTemp() 144 0.1522 0.1753 0.2062 1.47% FillBoundary_nowait() 80 0.1547 0.1708 0.1938 1.38% CellConservativeLinear::interp() 120 0.03234 0.09371 0.1705 1.21% FillPatcher::fillCFB 80 0.1081 0.1281 0.1573 1.12% FillPatchIterator::Initialize 60 0.1153 0.1217 0.1283 0.91% amrex::Copy() 88 0.0925 0.1023 0.1145 0.81% amrex::average_down 21 0.07609 0.08911 0.1089 0.77% Amr::regrid() 10 0.1075 0.1079 0.1086 0.77% FillPatchInterp(Fab) 80 0.01741 0.05206 0.0997 0.71% FabArray::sum() 105 0.05652 0.07549 0.09206 0.65% CNS::estTimeStep() 41 0.05619 0.07126 0.08699 0.62% CNS::advance() 60 0.06389 0.07144 0.07779 0.55% TagBoxArray::collate() 11 0.001352 0.03691 0.07174 0.51% FabArray::setDomainBndry() 60 0.0002144 0.02558 0.05257 0.37% post_timestep 60 0.03334 0.03661 0.04578 0.33% CNS::errorEst() 11 0.02043 0.02211 0.02328 0.17% Amr::bldFineLevels() 1 0.01258 0.01263 0.01268 0.09% AmrLevel::writePlotFile() 4 0.004052 0.006128 0.01103 0.08% AmrMesh::MakeNewGrids() 11 0.005788 0.007506 0.009231 0.07% TagBoxArray::mapPRD 11 0.003359 0.006469 0.009098 0.06% Amr::writePlotFile() 2 0.001404 0.002864 0.003818 0.03% Amr::coarseTimeStep() 20 0.0002887 0.001184 0.002161 0.02% Amr::timeStep() 60 0.0006471 0.0008698 0.001381 0.01% AmrLevel::FillPatch() 60 0.0006244 0.000674 0.000712 0.01% FabArray::ParallelCopy() 572 0.0005437 0.0005914 0.0006314 0.00% FillPatchSingleLevel 140 0.00024 0.0002681 0.0003188 0.00% AmrLevel::FillPatcherFill() 120 0.0002553 0.0002639 0.0002739 0.00% main() 1 0.0001793 0.0002168 0.0002387 0.00% FillPatcher::fill() 80 0.0001387 0.000155 0.0001815 0.00% FabArray::FillBoundary() 80 0.0001178 0.0001394 0.0001575 0.00% FillPatchIterator::FillFromLevel0() 51 6.23e-05 6.984e-05 7.861e-05 0.00% postCoarseTimeStep() 20 3.634e-05 4.977e-05 7.12e-05 0.00% AmrLevel::RK() 60 4.24e-05 4.786e-05 5.825e-05 0.00% Amr::grid_places() 11 5.06e-05 5.32e-05 5.504e-05 0.00% CNS::avgDown() 21 3.057e-05 4.113e-05 5.237e-05 0.00% Amr::FinalizeInit() 1 6.558e-06 1.791e-05 4.242e-05 0.00% AmrLevel::AmrLevel(dm) 12 2.819e-05 3.001e-05 3.175e-05 0.00% amrex::average_down_w_geom 21 1.589e-05 1.841e-05 2.004e-05 0.00% CNS::writePlotFile() 4 6.621e-06 7.771e-06 8.289e-06 0.00% Amr::init() 1 1.266e-06 1.324e-06 1.39e-06 0.00% Amr::initialInit() 1 9.43e-07 1.061e-06 1.249e-06 0.00% Other 1209 0.03155 0.04582 0.0642 0.46% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 14.05 14.06 14.07 100.00% Amr::coarseTimeStep() 20 13.67 13.67 13.67 97.16% Amr::timeStep() 60 13.49 13.52 13.56 96.39% CNS::advance() 60 11.98 12 12.01 85.38% AmrLevel::RK() 60 11.92 11.93 11.94 84.86% RungeKutta2 60 11.92 11.93 11.94 84.86% CNS::compute_dSdt() 120 8.314 9.135 9.966 70.86% AmrLevel::FillPatcherFill() 120 1.466 2.297 3.112 22.13% FabArray::ParallelCopy() 572 1.033 1.785 2.547 18.11% FillPatchSingleLevel 140 0.5926 1.467 2.335 16.60% AmrLevel::FillPatch() 60 0.4796 1.345 2.208 15.70% FillPatchIterator::Initialize 60 0.3747 1.253 2.12 15.08% FillPatchIterator::FillFromLevel0() 51 0.2428 1.096 1.948 13.85% StateDataPhysBCFunct::() 220 0.001163 0.8642 1.734 12.33% FabArray::ParallelCopy_finish() 572 0.5014 1.072 1.63 11.59% post_timestep 60 1.18 1.216 1.256 8.93% FillPatcher::fill() 80 1.081 1.089 1.109 7.88% FluxRegister::Reflux() 120 0.9164 0.9518 0.9836 6.99% FabArray::ParallelCopy_nowait() 572 0.4831 0.7123 0.9674 6.88% FillPatcher::fillCFB 80 0.7013 0.7283 0.7858 5.59% FabArray::FillBoundary() 80 0.3224 0.3601 0.3787 2.69% Amr::regrid() 10 0.3385 0.3426 0.3463 2.46% Amr::writePlotFile() 2 0.2874 0.2877 0.2883 2.05% CNS::writePlotFile() 4 0.2844 0.2848 0.286 2.03% AmrLevel::writePlotFile() 4 0.2844 0.2848 0.286 2.03% FabArray::setVal() 282 0.226 0.2527 0.2789 1.98% VisMF::Write(FabArray) 4 0.2647 0.2678 0.2714 1.93% Amr::init() 1 0.2488 0.2501 0.2536 1.80% CNS::avgDown() 21 0.1986 0.2232 0.2402 1.71% amrex::average_down_w_geom 21 0.1932 0.2083 0.2209 1.57% amrex::average_down 21 0.1931 0.2083 0.2208 1.57% FillBoundary_finish() 80 0.1284 0.1891 0.2113 1.50% CNS::computeTemp() 144 0.1522 0.1753 0.2062 1.47% FillBoundary_nowait() 80 0.1548 0.1709 0.1939 1.38% FillPatchInterp(Fab) 80 0.1054 0.1458 0.1886 1.34% Amr::grid_places() 11 0.183 0.1835 0.1839 1.31% AmrMesh::MakeNewGrids() 11 0.183 0.1834 0.1838 1.31% CellConservativeLinear::interp() 120 0.03234 0.09371 0.1705 1.21% CNS::errorEst() 11 0.06957 0.1124 0.1567 1.11% amrex::Copy() 88 0.0925 0.1023 0.1145 0.81% Amr::initialInit() 1 0.09997 0.1016 0.1054 0.75% Amr::FinalizeInit() 1 0.09199 0.09291 0.09331 0.66% FabArray::sum() 105 0.05652 0.07549 0.09206 0.65% postCoarseTimeStep() 20 0.05446 0.07195 0.08761 0.62% CNS::estTimeStep() 41 0.05619 0.07126 0.08699 0.62% Amr::bldFineLevels() 1 0.07506 0.07544 0.07581 0.54% TagBoxArray::collate() 11 0.001352 0.03691 0.07174 0.51% FabArray::setDomainBndry() 60 0.0002144 0.02558 0.05257 0.37% TagBoxArray::mapPRD 11 0.0117 0.01907 0.02568 0.18% AmrLevel::AmrLevel(dm) 12 0.02151 0.02331 0.02455 0.17% Other 1209 0.069 0.09062 0.1182 0.84% ------------------------------------------------------------------------------------------- Device Memory Usage: ---------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ---------------------------------------------------------------------------------------------------------------- CNS::compute_dSdt() 19200 127 MiB 140 MiB 154 MiB 261 MiB 265 MiB 280 MiB StateData::define() 192 133 MiB 133 MiB 133 MiB 201 MiB 201 MiB 201 MiB CNS::advance() 176 117 MiB 117 MiB 117 MiB 134 MiB 134 MiB 134 MiB FluxRegister::Reflux() 1920 3578 KiB 3756 KiB 3922 KiB 56 MiB 67 MiB 71 MiB RungeKutta2 1244 46 MiB 46 MiB 46 MiB 70 MiB 70 MiB 70 MiB FillPatchIterator::Initialize 960 1889 KiB 5888 KiB 9891 KiB 67 MiB 67 MiB 67 MiB AmrLevel::writePlotFile() 64 1130 KiB 1137 KiB 1140 KiB 56 MiB 56 MiB 56 MiB FillPatcher::fillCFB 760 4279 KiB 6450 KiB 8638 KiB 7998 KiB 11 MiB 15 MiB ResizeRandomSeed 4 9209 KiB 9210 KiB 9211 KiB 9216 KiB 9216 KiB 9216 KiB CNS::errorEst() 176 39 KiB 66 KiB 94 KiB 8985 KiB 8985 KiB 8985 KiB post_timestep 320 574 KiB 586 KiB 592 KiB 8192 KiB 8192 KiB 8192 KiB amrex::average_down 428 77 KiB 89 KiB 99 KiB 7169 KiB 7169 KiB 7169 KiB CellConservativeLinear::interp() 960 10 KiB 18 KiB 29 KiB 1355 KiB 2064 KiB 2773 KiB Amr::regrid() 960 1328 KiB 1329 KiB 1329 KiB 2688 KiB 2688 KiB 2688 KiB StateDataPhysBCFunct::() 816 0 B 48 KiB 97 KiB 0 B 736 KiB 1473 KiB Amr::bldFineLevels() 96 4190 B 4219 B 4230 B 1344 KiB 1344 KiB 1344 KiB AmrMesh::MakeNewGrids() 352 13 KiB 13 KiB 13 KiB 1025 KiB 1025 KiB 1025 KiB FabArray::ParallelCopy_nowait() 2202 16 KiB 55 KiB 95 KiB 26 KiB 99 KiB 173 KiB VisMF::Write(FabArray) 448 90 B 113 B 135 B 96 KiB 96 KiB 96 KiB CNS::estTimeStep() 172 1119 B 1145 B 1177 B 49 KiB 49 KiB 49 KiB FabArray::sum() 420 194 B 259 B 317 B 48 KiB 48 KiB 48 KiB FabArray::ParallelCopy_finish() 1568 5170 B 6478 B 8302 B 9216 B 13 KiB 17 KiB FillBoundary_nowait() 160 7569 B 7579 B 7586 B 10 KiB 10 KiB 10 KiB FillBoundary_finish() 80 3780 B 3781 B 3783 B 5120 B 5120 B 5120 B FabArray::setVal() 1144 2079 B 2115 B 2151 B 3072 B 3072 B 3072 B amrex::Copy() 348 1957 B 1958 B 1961 B 3072 B 3072 B 3072 B TagBoxArray::mapPRD 352 1 B 2 B 3 B 2048 B 2048 B 2048 B TagBoxArray::collate() 88 0 B 0 B 0 B 256 B 896 B 1536 B OwnerMask() 176 0 B 0 B 0 B 1024 B 1024 B 1024 B StateData::FillBoundary(geom) 1224 0 B 10 B 21 B 0 B 256 B 512 B Amr::InitAmr() 8 505 B 505 B 505 B 512 B 512 B 512 B ---------------------------------------------------------------------------------------------------------------- Async Memory Usage: ------------------------------------------------------------------------------------------------------ Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ------------------------------------------------------------------------------------------------------ CNS::compute_dSdt() 3840 4755 KiB 8483 KiB 11 MiB 65 MiB 65 MiB 65 MiB FluxRegister::Reflux() 1920 2760 KiB 3139 KiB 3515 KiB 56 MiB 56 MiB 56 MiB RungeKutta2 960 46 MiB 46 MiB 46 MiB 56 MiB 56 MiB 56 MiB post_timestep 320 547 KiB 568 KiB 586 KiB 8192 KiB 8192 KiB 8192 KiB ------------------------------------------------------------------------------------------------------ Managed Memory Usage: --------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------- The_Managed_Arena::Initialize() 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------- Pinned Memory Usage: ---------------------------------------------------------------------------------------------------------------- Name Nalloc AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max ---------------------------------------------------------------------------------------------------------------- FabArray::ParallelCopy_nowait() 5496 288 KiB 510 KiB 704 KiB 7189 KiB 11 MiB 14 MiB VisMF::Write(FabArray) 512 203 KiB 215 KiB 227 KiB 14 MiB 14 MiB 14 MiB The_Pinned_Arena::Initialize() 4 0 B 0 B 1 B 8192 KiB 8192 KiB 8192 KiB FillBoundary_nowait() 800 176 KiB 196 KiB 206 KiB 7514 KiB 7514 KiB 7514 KiB FabArray::ParallelCopy_finish() 1568 5170 B 6478 B 8302 B 9216 B 13 KiB 17 KiB FillBoundary_finish() 80 3780 B 3781 B 3783 B 5120 B 5120 B 5120 B CNS::compute_dSdt() 1920 119 B 174 B 230 B 3072 B 3072 B 3072 B amrex::Copy() 348 1957 B 1958 B 1961 B 3072 B 3072 B 3072 B FabArray::setVal() 1144 2079 B 2115 B 2151 B 3072 B 3072 B 3072 B TagBoxArray::collate() 154 0 B 2 B 4 B 512 B 1344 B 1792 B amrex::average_down 92 562 B 563 B 564 B 1536 B 1536 B 1536 B RungeKutta2 284 1075 B 1075 B 1076 B 1536 B 1536 B 1536 B CNS::estTimeStep() 172 1011 B 1011 B 1011 B 1280 B 1280 B 1280 B FillPatcher::fillCFB 280 394 B 466 B 540 B 768 B 896 B 1024 B StateData::FillBoundary(geom) 1224 0 B 10 B 21 B 0 B 256 B 512 B CellConservativeLinear::interp() 480 1 B 1 B 2 B 256 B 384 B 512 B FabArray::sum() 420 1 B 1 B 1 B 256 B 256 B 256 B ---------------------------------------------------------------------------------------------------------------- Total GPU global memory (MB) spread across MPI: [7965 ... 7965] Free GPU global memory (MB) spread across MPI: [4207 ... 4207] [The Arena] max space (MB) allocated spread across MPI: [741 ... 766] [The Arena] max space (MB) used spread across MPI: [603 ... 630] [The Managed Arena] max space (MB) allocated spread across MPI: [8 ... 8] [The Managed Arena] max space (MB) used spread across MPI: [0 ... 0] [The Pinned Arena] max space (MB) allocated spread across MPI: [32 ... 36] [The Pinned Arena] max space (MB) used spread across MPI: [14 ... 14] AMReX (26.04-109-g60df5ae33414) finalized