Skip to content

Block-triangular preconditioners for Maxwell solves - #13

Draft
Alexey-Voronin wants to merge 9 commits into
sandialabs:mainfrom
Alexey-Voronin:maxwell_solvers
Draft

Block-triangular preconditioners for Maxwell solves#13
Alexey-Voronin wants to merge 9 commits into
sandialabs:mainfrom
Alexey-Voronin:maxwell_solvers

Conversation

@Alexey-Voronin

Copy link
Copy Markdown
Contributor

Summary

This PR adds a block-triangular preconditioner for $2 \times 2$ mixed Maxwell systems in linear_algebra, where the pivot block, Schur approximation, and per-block solvers are all individually configurable.

The code is split into a self-contained block_prec/ module (types, validation, operators, assembly, Schur construction) and integration code inside LinearAlgebraInterface.
Currently supported solvers:

  • Pivot block solvers: AMG, RefMaxwell, Direct, Diagonal;
  • Schur block solvers: AMG, RefMaxwell, Direct.

Mathematical Form

For

$$\begin{bmatrix} J_{00} & J_{01} \\ J_{10} & J_{11} \end{bmatrix} \begin{bmatrix} x_0 \\ x_1 \end{bmatrix} = \begin{bmatrix} b_0 \\ b_1 \end{bmatrix}$$

the preconditioner factors the system into a pivot solve and a Schur-complement solve.

Currently, two Schur approximations are supported:

  • base: $S = J_{11}$
  • diag: $S = J_{11} - \gamma J_{10},\mathrm{diag}(J_{00})^{-1}J_{01}$, with optional lumped diagonal.

To make input decks easier to compose, the parameter Schur pivot block = 0 or 1 is used. After extraction, blocks are relabeled by role (pivot/target), not by original index: J00 = pivot diagonal, J11 = target diagonal, J10 = target-from-pivot, J01 = pivot-from-target.

Structure

The block_prec/ files are ordered by dependency, each one only pulls from what's above it. I used Teko's BlockedTpetraOperator to handle block extraction, MueLu to build AMG and auxiliary-space preconditioners for each block, and Tpetra/Xpetra/Ifpack2 to supply the needed maps, matrices, and smoother wrappers.

File Owns
BlockTypes.hpp Tpetra aliases, SchurVariant/BlockPrecType enums
ParamUtils.hpp Param. key validation, allowed-key sets, and related warnings
BlockOperators.hpp DiagonalInverseOperator, DirectSolveOperator, BlockDiagonalOperator, BlockTriangularOperator, import/export workspace helpers
BlockAssembly.hpp Teko block extraction, buildInverseDiagonal, buildLumpedM0inv, buildBlockSystemForSet, pivot/Schur block preconditioner dispatch
SchurApproximation.hpp schurBase, assembleSchurCoreFromInputs (union-sparsity two-pass), buildSchurApproximation entry point

Integration layer:

File Role
linearSolverContext.hpp Grouped SchurConfig / RefMaxwellData structs; phased parse with strict section validation
linearAlgebraInterface_blockprec.hpp buildBlockMaps, block-diagonal build, block-triangular setup (extract → Schur → pivot/Schur precs → assemble operator), validateRefMaxwellBlockInputs, reuse policy
linearAlgebraInterface_solvers.hpp buildOrUpdatePreconditioner dispatch, buildRefMaxwellPreconditioner (Xpetra wrapping, hierarchy build/reuse), Belos solver creation and status handling

Some other supporting changes had to be made outside of linear_algebra:

File(s) Change (simplified)
managers/assembly/assemblyManager*.hpp Added an optional flag to AssemblyManager::getWeightedMass to build the mass matrix with all weights = 1. RefMaxwell/ADS uses this in the block-triangular path to assemble the H(curl) M1 matrix with unit weights.
managers/solver/solverManager*.hpp SolverManager::setupBlockTriangularAuxiliary now assembles the RefMaxwell auxiliary operators/data (D0/M1/M2, nodal coords, and D1 for ADS), calls getWeightedMass with the unit-mass option when needed, and reuses that assembled context across the related linear solver contexts.
interfaces/user/userInterface.hpp Solver config loading now supports base + overrides: load the external “Solver input file” first, then re-apply the in-main “Solver” block to override settings. This allows sharing a common solver file without copying everything. See yaml config pattern below
Solver input file: common_solver.yaml
Solver:
  Max iterations: 50 # overrides whatever value may have been pre-set in common_solver.yaml

Design Decisions

The diag correction $J_{10},\mathrm{diag}(J_{00})^{-1}J_{01}$ can create fill beyond $J_{11}$'s sparsity pattern.
The problem is that inserting into a fillComplete'd copy of $J_{11}$ via sumIntoGlobalValues silently drops these entries. Instead, assembleSchurCoreFromInputs runs two passes: one to compute the union column set per row, one to accumulate and insert before fillComplete.

Block-triangular has its own none/update/full reuse inside setupBlockTriangularPreconditioner, independently of the outer reuse_preconditioner flag used by the monolithic AMG path. update skips rebuild when the Jacobian values haven't changed; full freezes the operator entirely. RefMaxwell hierarchies are cached per-role (refmaxwell_prec for pivot, schur_refmaxwell_prec for Schur) so reuse on one block doesn't contaminate the other.

Tests

Related regression tests have been added to regression/maxwell/linear_solvers. These a grouped into 3 categories

  • Positive:

    • blocktri_pivot0_direct_direct, blocktri_pivot1_direct_direct, blocktri_pivot1_refmaxwell_gs, blocktri_pivot1_refmaxwell_cheb
    • checks that the converge is within expected number of iterations (max linear iters: #) +2 to account for potential variability across machines. The accuracy is verified through output integrated quantities. Printing residual to .gold files is not a architecture/compiler robust regression test.
  • Negative (expected-failure):

    • blocktri_invalid_unknown_key, blocktri_invalid_schur_type, blocktri_invalid_refmaxwell_key
    • Makes sure setup fails when incorrect/incompatible parameters are supplied for block triangular solves.
  • Baseline: monolith_blockdiag_jacobi (monolithic vs block-diagonal consistency)

    • These two should be identical in terms of iteration to convergence.
  • Sandbox: sandbox/testCases/maxwell_solvers/summary/README.md

    • Parallel scalability and h-refinement scalability tests.
    • A few helpful scripts to help prototype solvers and measure their performance for different time step sizes and mesh sizes. See attached scaling results from a 12-MPI ranks runs on attaway -

iteration_count_summary.pdf
solve_time_summary.pdf

Future steps

Currently the block-triangular apply is implemented directly at the Tpetra level in BlockTriangularOperator. An obious next step would be to express the same 2x2 block operator using Teko block preconditioner factories (e.g. SIMPLE, etc..) with MueLu/RefMaxwell plugged in through Thyra adaptors.

@Alexey-Voronin
Alexey-Voronin marked this pull request as draft March 25, 2026 16:40
Alexey-Voronin and others added 4 commits March 31, 2026 12:58
…check inputs manually in mrhyde, use xml decks for block-solvers that get passed to MueLU skipping MrHyDE pre-processing step.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant