m_cuda_spectral Module

Forward paired split of one dim2 mode against its mirror. The rule is symmetric, so the partner entry is pair_fw(partner, own, i2_rev).

Arguments

Return Value complex(kind=dp)

Backward counterpart of pair_fw, symmetric in the same way.

Arguments

Return Value complex(kind=dp)

Forward postprocess of one spectral entry in dim3 then dim1.

Arguments

Backward counterpart of postprocess_dim13_fw.

Arguments

Divide one entry by its spectral equivalence constant, with the eps_wave guard that keeps roundoff noise out of the pressure field.

Arguments

Copy data between x3d2 padded arrays and cuFFTMp descriptors

Arguments

Copy with transpose: src(nx, ny, nz) -> dst(ny, nx, nz) Used for 100 case forward FFT

Arguments

Copy with transpose back: src(ny, nx, nz) -> dst(nx, ny, nz) Used for 100 case backward FFT

Arguments

Used before R2C FFT for 110 case to put Z (periodic) in fast dim.

Arguments

Transpose back: src(nz, nx, ny) -> dst(nx, ny, nz) Used after C2R FFT for 110 case.

Arguments

Post-processes the divergence of velocity in spectral space, including scaling w.r.t. grid size.

Arguments

Post-processes the divergence of velocity in spectral space, including scaling w.r.t. grid size.

Arguments

Post-processes the divergence of velocity in spectral space, including scaling w.r.t. grid size.

Arguments

Solve the Poisson equation at cell centres with non-periodic BC along y

Arguments

Post-processes the divergence of velocity in spectral space, including scaling w.r.t. grid size.

Arguments

Stage 1 (forward): normalisation followed by the dim3 and dim1 postprocess. Every mode is handled on its own here.

Arguments

Stage 2 (forward): the dim2 paired split, then the Poisson solve.

Arguments

Stage 3 (backward): the dim2 paired recombination, then the dim3 and dim1 postprocess.

Arguments

Copies the first dim2 plane of the local spectral slab into a contiguous buffer so it can be handed to MPI.

Arguments

Step 1 (forward): normalise + Z periodic post-process Z is dim1 (serial j loop), periodic R2C — no sign flip needed since j only goes to nz/2+1. nx, ny are the local loop bounds (equal to nx_glob, ny_glob on a single rank); the normalisation divisor always needs the globals.

Arguments

Step 2 (forward): X paired even/odd split X is dim2 (thread i). Only i in [2, nx/2+1] executes. Writes to i and nx-i+2. Race-free (pair doesn't enter).

Arguments

Step 3 (forward): Y paired even/odd split Y is dim3 (blockIdx%y = k). Only k in [2, ny/2+1] executes. Writes to k and ny-k+2. Race-free (pair block doesn't enter).

Arguments

Step 4: Poisson solve — divide by waves nx is the local loop bound; the Nyquist test always needs nx_glob.

Arguments

Step 5 (backward): Y paired even/odd recombine

Arguments

Step 6 (backward): X paired even/odd recombine

Arguments

Multi-rank counterpart of process_spectral_110_x_pair_fw: the X paired split, but the partner mode lives on another rank, reached through the dim2 mirror exchange (see exchange_dim2_mirror). Unlike the single-rank kernel this only updates this rank's own modes; the partner rank updates its own copy the same way. At the global Nyquist mode (even grid size) the mirror partner is this rank's own pre-update value, so the formula below self-pairs it exactly as the single-rank kernel does.

Arguments

Multi-rank counterpart of process_spectral_110_x_pair_bw, see process_spectral_110_x_pair_fw_mirror.

Arguments

Step 7 (backward): Z periodic undo

Arguments

Packs the physical Z-slab field block (nx, ny, nz_l) into the MPI_Alltoall send buffer, selecting for each destination rank s the Y range [sny_l+1, (s+1)ny_l] it owns in the Y-slab layout.

Arguments

Unpacks the MPI_Alltoall receive buffer into the padded real view (2(nz/2+1), nx, ny_l) of the cuFFTMp Y-slab descriptor: data received from source rank s reconstructs the global Z range [snz_l+1, (s+1)*nz_l].

Arguments

Inverse of unpack_110_yslab: packs the Y-slab real view into the MPI_Alltoall send buffer, selecting for each destination rank s the Z range it originally owned in the physical Z-slab layout.

Arguments

Inverse of pack_110_xyz_to_yslab: unpacks into the physical Z-slab field block (nx, ny, nz_l). dst must be zeroed by the caller first, as fft_backward_110_cuda already does for the single rank path.

Arguments

Arguments

Arguments

Arguments

Arguments

Combined X and Y periodicity enforcement (interleave shuffle).

Arguments

Combined X and Y periodicity undo (reverse interleave shuffle).

Arguments