Exact solve of the reduced system of the distributed tridiagonal solver.
The first phase of the distributed solve (der_univ_dist) reduces each
rank's block of a line so that its interior depends only on the block's
own first and last unknowns, x_s and x_e. What couples the ranks is then
a reduced system in the x_s and x_e of every rank along the line, in the
order x_s(0), x_e(0), x_s(1), x_e(1), ...:
where sa_1, sc_1, sa_n and sc_n are rank p's dist_sa(1), dist_sc(1),
dist_sa(n) and dist_sc(n), and d_s and d_e its first and last values
after the first phase. The system is cyclic: its corners couple the last
rank to the first. On a non-periodic line they vanish, because the
boundary rows have no outside neighbour.
The default second phase (der_univ_subs) solves this system only
approximately: it drops sc_1 and sa_n and pairs each block boundary with
its neighbour alone. The dropped entries decay geometrically with the
points per rank, at a rate set by how diagonally dominant the system is.
That is invisible for the derivative operators, but not for the low-pass
filter near its sharp limit filter_alpha -> 0.5, whose gain it then gets
wrong by about sa_n even at zero wavenumber, every time it is applied.
Here every rank gathers the reduced rows of its lines and solves the
2*nproc system exactly, keeping its own x_s and x_e. The system is small,
so solving it on every rank costs less than more rounds of messages.
Nodes of different colours represent the following:
Solid arrows point from a submodule to the (sub)module which it is
descended from. Dashed arrows point from a module or program unit to
modules which it uses.
Where possible, edges connecting nodes are
given different colours to make them easier to distinguish in
large graphs.
Nodes of different colours represent the following:
Solid arrows point from a submodule to the (sub)module which it is
descended from. Dashed arrows point from a module or program unit to
modules which it uses.
Where possible, edges connecting nodes are
given different colours to make them easier to distinguish in
large graphs.
Solve the reduced system of every line exactly. Line l of d_s and d_e
must be the same physical line on every rank along dir, as it is for
the neighbour exchange of the default second phase.
Arguments
Type
Intent
Optional
Attributes
Name
real(kind=dp),
intent(out)
::
x_s(:)
real(kind=dp),
intent(out)
::
x_e(:)
real(kind=dp),
intent(in)
::
d_s(:)
real(kind=dp),
intent(in)
::
d_e(:)
real(kind=dp),
intent(in)
::
sa_1
This rank's dist_sa(1), dist_sc(1), dist_sa(n) and dist_sc(n)
real(kind=dp),
intent(in)
::
sc_1
This rank's dist_sa(1), dist_sc(1), dist_sa(n) and dist_sc(n)
real(kind=dp),
intent(in)
::
sa_n
This rank's dist_sa(1), dist_sc(1), dist_sa(n) and dist_sc(n)
real(kind=dp),
intent(in)
::
sc_n
This rank's dist_sa(1), dist_sc(1), dist_sa(n) and dist_sc(n)