Copy data between x3d2 padded arrays and cuFFTMp descriptors
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=dp), | intent(inout), | device, dimension(:, :, :) | :: | dst | ||
| real(kind=dp), | intent(in), | device, dimension(:, :, :) | :: | src | ||
| integer, | intent(in), | value | :: | nx | ||
| integer, | intent(in), | value | :: | ny | ||
| integer, | intent(in), | value | :: | nz |
Copy with transpose: src(nx, ny, nz) -> dst(ny, nx, nz) Used for 100 case forward FFT
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=dp), | intent(out), | device, dimension(:, :, :) | :: | dst | ||
| real(kind=dp), | intent(in), | device, dimension(:, :, :) | :: | src | ||
| integer, | intent(in), | value | :: | nx | ||
| integer, | intent(in), | value | :: | ny | ||
| integer, | intent(in), | value | :: | nz |
Copy with transpose back: src(ny, nx, nz) -> dst(nx, ny, nz) Used for 100 case backward FFT
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=dp), | intent(out), | device, dimension(:, :, :) | :: | dst | ||
| real(kind=dp), | intent(in), | device, dimension(:, :, :) | :: | src | ||
| integer, | intent(in), | value | :: | nx | ||
| integer, | intent(in), | value | :: | ny | ||
| integer, | intent(in), | value | :: | nz |
Used before R2C FFT for 110 case to put Z (periodic) in fast dim.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=dp), | intent(out), | device, dimension(:, :, :) | :: | dst | ||
| real(kind=dp), | intent(in), | device, dimension(:, :, :) | :: | src | ||
| integer, | intent(in), | value | :: | nx | ||
| integer, | intent(in), | value | :: | ny | ||
| integer, | intent(in), | value | :: | nz |
Transpose back: src(nz, nx, ny) -> dst(nx, ny, nz) Used after C2R FFT for 110 case.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=dp), | intent(out), | device, dimension(:, :, :) | :: | dst | ||
| real(kind=dp), | intent(in), | device, dimension(:, :, :) | :: | src | ||
| integer, | intent(in), | value | :: | nx | ||
| integer, | intent(in), | value | :: | ny | ||
| integer, | intent(in), | value | :: | nz |
Post-processes the divergence of velocity in spectral space, including scaling w.r.t. grid size.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| complex(kind=dp), | intent(inout), | device, dimension(:, :, :) | :: | div_u |
Divergence of velocity in spectral space |
|
| complex(kind=dp), | intent(in), | device, dimension(:, :, :) | :: | waves |
Spectral equivalence constants |
|
| integer, | intent(in), | value | :: | nx_spec |
Grid size in spectral space |
|
| integer, | intent(in), | value | :: | ny_spec |
Grid size in spectral space |
|
| integer, | intent(in), | value | :: | y_sp_st |
Offset in y direction in the permuted slabs in spectral space |
|
| integer, | intent(in), | value | :: | nx |
Grid size |
|
| integer, | intent(in), | value | :: | ny |
Grid size |
|
| integer, | intent(in), | value | :: | nz |
Grid size |
|
| real(kind=dp), | intent(in), | device, dimension(:) | :: | ax | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | bx | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | ay | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | by | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | az | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | bz |
Post-processes the divergence of velocity in spectral space, including scaling w.r.t. grid size.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| complex(kind=dp), | intent(inout), | device, dimension(:, :, :) | :: | div_u |
Divergence of velocity in spectral space |
|
| complex(kind=dp), | intent(in), | device, dimension(:, :, :) | :: | waves |
Spectral equivalence constants |
|
| integer, | intent(in), | value | :: | nx_spec |
Grid size in spectral space |
|
| integer, | intent(in), | value | :: | ny_spec |
Grid size in spectral space |
|
| integer, | intent(in), | value | :: | y_sp_st |
Offset in y direction in the permuted slabs in spectral space |
|
| integer, | intent(in), | value | :: | nx |
Grid size |
|
| integer, | intent(in), | value | :: | ny |
Grid size |
|
| integer, | intent(in), | value | :: | nz |
Grid size |
|
| real(kind=dp), | intent(in), | device, dimension(:) | :: | ax | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | bx | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | ay | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | by | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | az | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | bz |
Post-processes the divergence of velocity in spectral space, including scaling w.r.t. grid size.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| complex(kind=dp), | intent(inout), | device, dimension(:, :, :) | :: | div_u |
Divergence of velocity in spectral space |
|
| integer, | intent(in), | value | :: | nx_spec |
Grid size in spectral space |
|
| integer, | intent(in), | value | :: | ny_spec |
Grid size in spectral space |
|
| integer, | intent(in), | value | :: | y_sp_st |
Offset in y direction in the permuted slabs in spectral space |
|
| integer, | intent(in), | value | :: | nx |
Grid size |
|
| integer, | intent(in), | value | :: | ny |
Grid size |
|
| integer, | intent(in), | value | :: | nz |
Grid size |
|
| real(kind=dp), | intent(in), | device, dimension(:) | :: | ax |
Spectral equivalence constants |
|
| real(kind=dp), | intent(in), | device, dimension(:) | :: | bx |
Spectral equivalence constants |
|
| real(kind=dp), | intent(in), | device, dimension(:) | :: | ay |
Spectral equivalence constants |
|
| real(kind=dp), | intent(in), | device, dimension(:) | :: | by |
Spectral equivalence constants |
|
| real(kind=dp), | intent(in), | device, dimension(:) | :: | az |
Spectral equivalence constants |
|
| real(kind=dp), | intent(in), | device, dimension(:) | :: | bz |
Spectral equivalence constants |
Solve the Poisson equation at cell centres with non-perioic BC along y
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| complex(kind=dp), | intent(inout), | device, dimension(:, :, :) | :: | div_u |
Divergence of velocity in spectral space |
|
| real(kind=dp), | intent(inout), | device, dimension(:, :, :, :) | :: | a_re |
Spectral equivalence constants |
|
| real(kind=dp), | intent(inout), | device, dimension(:, :, :, :) | :: | a_im |
Spectral equivalence constants |
|
| integer, | intent(in), | value | :: | off |
offset and increment. increment is 2 when considering only odd or even |
|
| integer, | intent(in), | value | :: | inc |
offset and increment. increment is 2 when considering only odd or even |
|
| integer, | intent(in), | value | :: | nx_spec |
Grid size in spectral space |
|
| integer, | intent(in), | value | :: | n |
Grid size in spectral space |
|
| integer, | intent(in), | value | :: | nx |
Grid size in spectral space |
|
| integer, | intent(in), | value | :: | ny |
Grid size in spectral space |
|
| integer, | intent(in), | value | :: | nz |
Grid size in spectral space |
Post-processes the divergence of velocity in spectral space, including scaling w.r.t. grid size.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| complex(kind=dp), | intent(inout), | device, dimension(:, :, :) | :: | div_u |
Divergence of velocity in spectral space |
|
| integer, | intent(in), | value | :: | nx_spec |
Grid size in spectral space |
|
| integer, | intent(in), | value | :: | ny_spec |
Grid size in spectral space |
|
| integer, | intent(in), | value | :: | y_sp_st |
Offset in y direction in the permuted slabs in spectral space |
|
| integer, | intent(in), | value | :: | nx |
Grid size |
|
| integer, | intent(in), | value | :: | ny |
Grid size |
|
| integer, | intent(in), | value | :: | nz |
Grid size |
|
| real(kind=dp), | intent(in), | device, dimension(:) | :: | ax |
Spectral equivalence constants |
|
| real(kind=dp), | intent(in), | device, dimension(:) | :: | bx |
Spectral equivalence constants |
|
| real(kind=dp), | intent(in), | device, dimension(:) | :: | ay |
Spectral equivalence constants |
|
| real(kind=dp), | intent(in), | device, dimension(:) | :: | by |
Spectral equivalence constants |
|
| real(kind=dp), | intent(in), | device, dimension(:) | :: | az |
Spectral equivalence constants |
|
| real(kind=dp), | intent(in), | device, dimension(:) | :: | bz |
Spectral equivalence constants |
Step 1 (forward): normalise + Z periodic post-process Z is dim1 (serial j loop), periodic R2C — no sign flip needed since j only goes to nz/2+1.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| complex(kind=dp), | intent(inout), | device, dimension(:, :, :) | :: | div_u | ||
| integer, | intent(in), | value | :: | nz_h | ||
| integer, | intent(in), | value | :: | nx | ||
| integer, | intent(in), | value | :: | ny | ||
| integer, | intent(in), | value | :: | nz | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | az | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | bz |
Step 2 (forward): X paired even/odd split X is dim2 (thread i). Only i in [2, nx/2+1] executes. Writes to i and nx-i+2. Race-free (pair doesn't enter).
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| complex(kind=dp), | intent(inout), | device, dimension(:, :, :) | :: | div_u | ||
| integer, | intent(in), | value | :: | nz_h | ||
| integer, | intent(in), | value | :: | nx | ||
| integer, | intent(in), | value | :: | ny | ||
| integer, | intent(in), | value | :: | x_sp_st | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | ax | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | bx |
Step 3 (forward): Y paired even/odd split Y is dim3 (blockIdx%y = k). Only k in [2, ny/2+1] executes. Writes to k and ny-k+2. Race-free (pair block doesn't enter).
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| complex(kind=dp), | intent(inout), | device, dimension(:, :, :) | :: | div_u | ||
| integer, | intent(in), | value | :: | nz_h | ||
| integer, | intent(in), | value | :: | nx | ||
| integer, | intent(in), | value | :: | ny | ||
| integer, | intent(in), | value | :: | y_sp_st | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | ay | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | by |
Step 4: Poisson solve — divide by waves
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| complex(kind=dp), | intent(inout), | device, dimension(:, :, :) | :: | div_u | ||
| complex(kind=dp), | intent(in), | device, dimension(:, :, :) | :: | waves | ||
| integer, | intent(in), | value | :: | nz_h | ||
| integer, | intent(in), | value | :: | nx | ||
| integer, | intent(in), | value | :: | ny | ||
| integer, | intent(in), | value | :: | nz | ||
| integer, | intent(in), | value | :: | x_sp_st |
Step 5 (backward): Y paired even/odd recombine
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| complex(kind=dp), | intent(inout), | device, dimension(:, :, :) | :: | div_u | ||
| integer, | intent(in), | value | :: | nz_h | ||
| integer, | intent(in), | value | :: | nx | ||
| integer, | intent(in), | value | :: | ny | ||
| integer, | intent(in), | value | :: | y_sp_st | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | ay | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | by |
Step 6 (backward): X paired even/odd recombine
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| complex(kind=dp), | intent(inout), | device, dimension(:, :, :) | :: | div_u | ||
| integer, | intent(in), | value | :: | nz_h | ||
| integer, | intent(in), | value | :: | nx | ||
| integer, | intent(in), | value | :: | ny | ||
| integer, | intent(in), | value | :: | x_sp_st | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | ax | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | bx |
Step 7 (backward): Z periodic undo
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| complex(kind=dp), | intent(inout), | device, dimension(:, :, :) | :: | div_u | ||
| integer, | intent(in), | value | :: | nz_h | ||
| integer, | intent(in), | value | :: | nx | ||
| integer, | intent(in), | value | :: | ny | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | az | ||
| real(kind=dp), | intent(in), | device, dimension(:) | :: | bz |
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=dp), | intent(out), | device, dimension(:, :, :) | :: | f_out | ||
| real(kind=dp), | intent(in), | device, dimension(:, :, :) | :: | f_in | ||
| integer, | intent(in), | value | :: | nx |
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=dp), | intent(out), | device, dimension(:, :, :) | :: | f_out | ||
| real(kind=dp), | intent(in), | device, dimension(:, :, :) | :: | f_in | ||
| integer, | intent(in), | value | :: | nx |
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=dp), | intent(out), | device, dimension(:, :, :) | :: | f_out | ||
| real(kind=dp), | intent(in), | device, dimension(:, :, :) | :: | f_in | ||
| integer, | intent(in), | value | :: | ny |
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=dp), | intent(out), | device, dimension(:, :, :) | :: | f_out | ||
| real(kind=dp), | intent(in), | device, dimension(:, :, :) | :: | f_in | ||
| integer, | intent(in), | value | :: | ny |
Combined X and Y periodicity enforcement (interleave shuffle).
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=dp), | intent(out), | device, dimension(:, :, :) | :: | f_out | ||
| real(kind=dp), | intent(in), | device, dimension(:, :, :) | :: | f_in | ||
| integer, | intent(in), | value | :: | nx | ||
| integer, | intent(in), | value | :: | ny |
Combined X and Y periodicity undo (reverse interleave shuffle).
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=dp), | intent(out), | device, dimension(:, :, :) | :: | f_out | ||
| real(kind=dp), | intent(in), | device, dimension(:, :, :) | :: | f_in | ||
| integer, | intent(in), | value | :: | nx | ||
| integer, | intent(in), | value | :: | ny |