m_memory_estimate Module

Static (no-run) GPU memory footprint formulas for the CUDA backend.

Byte-exact sizes for the field, halo, and cuFFT/cuFFTMp spectral-array terms, mirroring the allocate statements in src/allocator.f90, src/backend/cuda/backend.f90, and src/backend/cuda/poisson_fft.f90. These formulas were originally written and tested inside tests/ as part of a runtime memory probe, since merged into src/memcheck.f90's --build Tier 3 path; they live here so both that real build measurement and the static, no build Tiers 1-2 estimate can share them instead of duplicating them.


Uses

  • module~~m_memory_estimate~~UsesGraph module~m_memory_estimate m_memory_estimate module~m_allocator m_allocator module~m_memory_estimate->module~m_allocator module~m_common m_common module~m_memory_estimate->module~m_common module~m_config m_config module~m_memory_estimate->module~m_config module~m_time_integrator m_time_integrator module~m_memory_estimate->module~m_time_integrator module~m_allocator->module~m_common module~m_field m_field module~m_allocator->module~m_field m_mpi m_mpi module~m_common->m_mpi module~m_logger m_logger module~m_common->module~m_logger module~m_config->module~m_common iso_fortran_env iso_fortran_env module~m_config->iso_fortran_env module~m_config->module~m_logger module~m_time_integrator->module~m_allocator module~m_time_integrator->module~m_common module~m_base_backend m_base_backend module~m_time_integrator->module~m_base_backend module~m_time_integrator->module~m_field module~m_time_integrator->module~m_logger module~m_base_backend->module~m_allocator module~m_base_backend->module~m_common module~m_base_backend->m_mpi module~m_base_backend->module~m_field module~m_base_backend->module~m_logger iso_c_binding iso_c_binding module~m_base_backend->iso_c_binding module~m_mesh m_mesh module~m_base_backend->module~m_mesh module~m_poisson_fft m_poisson_fft module~m_base_backend->module~m_poisson_fft module~m_tdsops m_tdsops module~m_base_backend->module~m_tdsops module~m_field->module~m_common module~m_field->iso_c_binding module~m_logger->iso_fortran_env module~m_logger->m_mpi module~m_mesh->module~m_common module~m_mesh->m_mpi module~m_mesh->module~m_field module~m_mesh->module~m_logger module~m_mesh_content m_mesh_content module~m_mesh->module~m_mesh_content module~m_poisson_fft->module~m_common module~m_poisson_fft->module~m_field module~m_poisson_fft->module~m_logger module~m_poisson_fft->module~m_mesh module~m_poisson_fft->module~m_tdsops module~m_tdsops->module~m_common module~m_tdsops->module~m_logger module~m_mesh_content->module~m_common module~m_mesh_content->module~m_logger

Used by

  • module~~m_memory_estimate~~UsedByGraph module~m_memory_estimate m_memory_estimate module~m_cuda_memcheck_device m_cuda_memcheck_device module~m_cuda_memcheck_device->module~m_memory_estimate module~m_memcheck_build m_memcheck_build module~m_memcheck_build->module~m_memory_estimate module~m_memcheck_estimate m_memcheck_estimate module~m_memcheck_build->module~m_memcheck_estimate module~m_memcheck_report m_memcheck_report module~m_memcheck_build->module~m_memcheck_report module~m_memcheck_context m_memcheck_context module~m_memcheck_build->module~m_memcheck_context module~m_memcheck_estimate->module~m_memory_estimate module~m_memcheck_estimate->module~m_memcheck_context module~m_memcheck_report->module~m_memory_estimate module~m_memcheck_report->module~m_cuda_memcheck_device module~m_memcheck_report->module~m_memcheck_estimate module~m_memcheck_report->module~m_memcheck_context program~x3d2_memcheck x3d2_memcheck program~x3d2_memcheck->module~m_memory_estimate program~x3d2_memcheck->module~m_memcheck_build program~x3d2_memcheck->module~m_memcheck_estimate program~x3d2_memcheck->module~m_memcheck_report program~x3d2_memcheck->module~m_memcheck_context module~m_memcheck_context->module~m_cuda_memcheck_device

Variables

Type Visibility Attributes Name Initial
integer(kind=i8), public, parameter :: fft_xtdesc_step_bytes = 512_i8*1024_i8**2

The xtdesc buffer grows the heap in steps of about 0.5 GiB.
integer(kind=i8), public, parameter :: fft_probe_margin_bytes = 1024_i8**3

The free memory the full-grid FFT probe keeps beyond the predicted heap. The probe context and the libraries that stay after cufftDestroy use it.

Functions

public pure function padded_cells(dims, sz) result(n)

Cell count of one padded field at dims (x, y padded to sz, z untouched). i8 so callers can multiply by field count/nbytes without overflow, as the allocator's own int32 ngrid does for very large grids.

Arguments

Type IntentOptional Attributes Name
integer, intent(in) :: dims(3)
integer, intent(in) :: sz

Return Value integer(kind=i8)

public pure function spectral_slab_bytes(n_spec) result(nbytes8)

Bytes of ONE complex spectral array (waves_dev, and c_dev on the plain-cuFFT fallback path) of the shape n_spec. n_spec comes from cuda_fft_spectral_dims in src/backend/cuda/poisson_fft.f90, the routine the solver init calls, so the shape per BC class and per number of ranks is the solver's. Whether a decomposition is valid is checked by cuda_fft_decomp_reason, not here.

Arguments

Type IntentOptional Attributes Name
integer, intent(in) :: n_spec(3)

Return Value integer(kind=i8)

public pure function round_up_bytes(n_bytes, step_bytes) result(nbytes8)

The smallest multiple of step_bytes that is at least n_bytes.

Arguments

Type IntentOptional Attributes Name
integer(kind=i8), intent(in) :: n_bytes
integer(kind=i8), intent(in) :: step_bytes

Return Value integer(kind=i8)

public pure function predicted_fft_heap_bytes(plan_bytes, worksize_bytes) result(nbytes8)

The NVSHMEM heap of the forward and backward cuFFTMp plans with the workspace allocated and without the xtdesc buffer: the plan memory plus the workspace rounded up to 1 GiB. This is the definition of the measured heap of the normal FFT workspace query.

Arguments

Type IntentOptional Attributes Name
integer(kind=i8), intent(in) :: plan_bytes
integer(kind=i8), intent(in) :: worksize_bytes

Return Value integer(kind=i8)

public pure function mirror_buffer_bytes_100(n_spec, num_gpus) result(nbytes8)

100-case multi rank only mirror/exchange buffers, zero at num_gpus<=1 - matches the nproc>1 guard at src/backend/cuda/poisson_fft.f90:332-349: c_mirror_dev + c_slab_send_dev: (nx_spec, ny_spec, nz_spec) x2 c_plane_send_dev + c_plane_recv_dev: (nx_spec, nz_spec) x2 all complex(dp). n_spec is the spectral shape of the 100 case.

Arguments

Type IntentOptional Attributes Name
integer, intent(in) :: n_spec(3)
integer, intent(in) :: num_gpus

Return Value integer(kind=i8)

public pure function spectral_extra_bytes_110(n_spec, cdims) result(nbytes8)

110-case bytes on top of the waves_dev slab (never runs at num_gpus>1 in this solver): a SECOND complex array of the same spectral shape n_spec, c_dev (src/backend/cuda/poisson_fft.f90:416-418), plus a real, globally-shaped transposed workspace r_dev_110 (:414), which has the same total element count as cdims, just permuted.

Arguments

Type IntentOptional Attributes Name
integer, intent(in) :: n_spec(3)
integer, intent(in) :: cdims(3)

Return Value integer(kind=i8)

public pure function stretched_y_matrix_bytes(bc_is_010, stretching_y, lowmem_fft, n_spec) result(nbytes8)

010-case, non-uniform y-stretching Poisson coefficient matrices, src/poisson_fft.f90:176-186 (only 010 sets stretched_y, and only at num_gpus=1 since 010 error-stops at nproc>1) and :320-323,424-428 (allocation shapes), copied to the device at src/backend/cuda/poisson_fft.f90:390-411: 'bottom': a_re, a_im, shape (nx_spec, ny_spec, nz_spec, 5), real(dp) otherwise ('centred', 'top-bottom'): a_odd_re/im, a_even_re/im, shape (nx_spec, ny_spec/2, nz_spec, 5), real(dp) doubled by the store_a_*_dev copies when lowmem_fft is .false. (the default).

Arguments

Type IntentOptional Attributes Name
logical, intent(in) :: bc_is_010
character(len=*), intent(in) :: stretching_y
logical, intent(in) :: lowmem_fft
integer, intent(in) :: n_spec(3)

Return Value integer(kind=i8)

public pure function halo_bytes(padded_dims, sz, n_halo) result(nbytes8)

CUDA-backend halo-exchange buffer bytes. Mirrors the 24 allocate statements in src/backend/cuda/backend.f90:128-152: 12 arrays shaped (sz, n_halo, n_groups) for u/v/w send/recv start/end, plus 12 more shaped (sz, 1, n_groups) for du/dud/d2u send/recv start/end - all real(dp). n_groups is the max over X/Y/Z of ny_pnz_p/sz, nx_pnz_p/ sz, nx_pny_p/sz (src/allocator.f90:81-83, allocator_t% n_groups_dir/get_n_groups), where _p are the sz-padded dims (x/y padded via padded_dim, z untouched, same rule padded_cells uses).

Arguments

Type IntentOptional Attributes Name
integer, intent(in) :: padded_dims(3)

[nx_padded, ny_padded, nz_padded]
integer, intent(in) :: sz
integer, intent(in) :: n_halo

Return Value integer(kind=i8)

public pure function padded_halo_bytes(local_dims, sz, n_halo) result(nbytes8)

halo_bytes for a local block of local_dims cells: pads x and y to sz (z untouched) the way the allocator does, then sizes the halo-exchange buffers of those padded dims.

Arguments

Type IntentOptional Attributes Name
integer, intent(in) :: local_dims(3)
integer, intent(in) :: sz
integer, intent(in) :: n_halo

Return Value integer(kind=i8)

public pure function gpu_io_staging_bytes(local_dims, checkpoint_cfg, device_write) result(bytes)

Device memory the GPU-aware ADIOS2 write path (src/io/adios2/io.f90, write_device_field_adios2) holds on top of the solver's own peak: a single reused staging buffer (file_handle%device_staging), sized to one unpadded LOCAL field in whichever precision that write uses. The buffer is grown, never shrunk (see write_device_field_adios2), so its steady-state size is the LARGEST single write it is asked to hold: the build precision whenever a checkpoint is enabled (always native, and always at least as large as a single-precision snapshot), otherwise whatever precision the snapshot uses.

Read more…

Arguments

Type IntentOptional Attributes Name
integer, intent(in) :: local_dims(3)
type(checkpoint_config_t), intent(in) :: checkpoint_cfg
logical, intent(in) :: device_write

Return Value integer(kind=i8)

public function peak_fields_lookup(flow_case_name, time_intg, n_species, les_on, ibm_on, output_vorticity, output_qcriterion, lowmem_transeq) result(n)

Static estimate of the allocator's work-field high-water mark (allocator_t%next_id after one substep - see src/allocator.f90), evaluated purely from config flags, no build/run required.

Read more…

Arguments

Type IntentOptional Attributes Name
character(len=*), intent(in) :: flow_case_name
character(len=3), intent(in) :: time_intg
integer, intent(in) :: n_species
logical, intent(in) :: les_on
logical, intent(in) :: ibm_on
logical, intent(in) :: output_vorticity
logical, intent(in) :: output_qcriterion
logical, intent(in) :: lowmem_transeq

Return Value integer


Subroutines

public pure subroutine gpu_io_staging_parts(checkpoint_cfg, snapshot_dev, checkpoint_dev)

Which writes can take the device path: a snapshot only at unit stride (striding falls back to the host path in snapshot_manager.f90), a checkpoint never strides. Shared by gpu_io_staging_bytes and the x3d2-memcheck report.

Arguments

Type IntentOptional Attributes Name
type(checkpoint_config_t), intent(in) :: checkpoint_cfg
logical, intent(out) :: snapshot_dev
logical, intent(out) :: checkpoint_dev