Document the NGWF optimisers - #31
Conversation
There was no page describing the outer NGWF loop at all: ngwf_cg_type was listed in the keyword reference and nowhere else, so a user had no way to find out what the conjugate-gradient variants do, let alone how to choose between them. This adds one page covering all three options at the same depth, placed after Crystal symmetry in the ground-state section. The shared material comes first, since it applies whichever optimiser is selected: the covariant gradient and how it is raised, the k-point-weighted contraction that forms every slope and coefficient, and the RMS gradient the convergence threshold is tested against. Each optimiser then gets its own section. For conjugate gradients that means both coefficients side by side and, more usefully, the line search itself, which is what a user actually sees in the output but which has never been written down: the trial step, the parabola through it, the fallback to a cubic, and how the trial length carries over between iterations. For L-BFGS it means the two-loop recursion, the deferred verdict, the pair admission and retirement rules, and the conditions under which the run declares a numerical floor rather than stopping on an error. Two things get their own sections because they cut across the optimisers. The pairing is not positive definite, since the kinetic-energy preconditioner is one-sided, and each optimiser copes with that differently; and fast density interacts with the deferred verdict in a way that changes a default. Every formula was checked against the source rather than reproduced from the literature, and the output and diagnostic blocks are real, not invented.
nickhine
left a comment
nickhine
left a comment
There was a problem hiding this comment.
Hi Brad,
Thanks for this - a really nice addition to the documentation. Definitely happy to merge once you have had a look at a couple of minor concerns. Firstly, I worry a bit that it is written in maybe too mathematical a style for some of our likely readers? It is very useful to have it all properly written out in full, but maybe this page could be divided up a bit more clearly between consequences/decisions for users and important background for developers, with the former clearly highlighted and put up front?
Also, the important link to kinetic energy preconditioning is left mostly implicit, with only a couple of incidental mentions. Should we have a link to https://docs.onetep.org/recip_precond.html here, with a more explicit statement that :math:g_{\alpha} gets preconditioned.
I have left a few other comments as well.
Nick
| .. code:: | ||
|
|
||
| ngwf_cg_type : NGWF_FLETCHER # conjugate gradients (default) | ||
| ngwf_cg_type : NGWF_POLAK # conjugate gradients, Hestenes-Stiefel |
There was a problem hiding this comment.
Why the name Hestenes-Steifel rather than Polak-Ribiere?
There was a problem hiding this comment.
Because internal_polak_cg_coeff divides by <d_prev, y> rather than by <g_prev, g_prev>, and the former is the Hestenes-Stiefel denominator. The two are the same when the line search finds the exact minimum along the direction, so it only matters because it confused me when I read the code haha -- i also fixed some issues with Polak that will be included in my NGWF-LBFGS commit later today that includes some dramatic speed ups, stability and works with and without all 'fast' features, including your ZnMnO system.
| :math:`\mathbf{k}`. Every such contraction pairs one covariant object with one | ||
| contravariant one, so both channels are carried through the optimisation. The | ||
| sum itself is symmetric, so the order in which the two are written below | ||
| carries no meaning; which channel each is drawn from does. |
There was a problem hiding this comment.
You use this terminology "channel" a lot here. Is it standard? I don't remember using it or seeing it in electronic structure literature. We talk about spin channels sometimes but not usually contravariant or covariant "channels".
There was a problem hiding this comment.
I think this may have been a Brad special term ahha, I couldn't think of the correct one so used this, it is removed.
| Convergence | ||
| ----------- | ||
|
|
||
| All three optimisers are tested against the same quantity, the root-mean-square |
There was a problem hiding this comment.
Maybe:
The progress of the optimisation is monitored by the same quantity in all three cases, the root-mean-square NGWF gradient
There was a problem hiding this comment.
i agree and i adopted it
| .. math:: | ||
| \mathrm{RMS} = \sqrt{\frac{\bigl| \langle g, g \rangle \bigr|}{N}} , | ||
|
|
||
| where :math:`N` is the number of psinc coefficients. Because the ruler is |
There was a problem hiding this comment.
"ruler" is odd terminology here too.
| ========================== | ||
|
|
||
| FFTW's ``MEASURE`` planner selects a transform plan by timing candidates at | ||
| runtime, so two runs of the same input on the same machine may take different |
There was a problem hiding this comment.
The transform plan is selected by timing candidates at runtime when calling fftw_plan_many_dft, not when calling fourier_apply_box. I don't think use of FFTW_MEASURE is the reason for any non-deterministic behaviour of results.
| ``ngwf_threshold_orig : 5E-7`` it costs a lithium cluster four extra | ||
| iterations and leaves a platinum cluster unconverged. | ||
|
|
||
| Deterministic FFT planning |
There was a problem hiding this comment.
Not a fan of this bit - I think it confuses things and may not be correct.
|
I agree it might be a bit too technical, it is a stripped version of something I wrote for the development section of my thesis; I will give it a once over to perhaps truncate bits and clarify things either here in comments or there. |
Nick's review raised six points. This addresses all of them and trims the page down considerably in the process. The page led with the mathematics, which is the wrong order for most readers. There is now a short section on choosing an optimiser before any of it, and the old summary that said the same thing at the bottom is gone. Kinetic energy preconditioning was mentioned only in passing, which understated it badly. The gradient section now says that the covariant gradient is preconditioned before either form is used, links to the preconditioning page, and notes that it is applied to the covariant gradient and not to its raised partner. Dropped "channel" and "ruler", neither of which is standard usage. The two gradients are named directly instead. Took Nick's wording for the sentence introducing the convergence measure. Cut the discussion of Hestenes-Stiefel against Polak-Ribiere entirely, along with the sections on the indefinite metric and the interaction with fast density. The naming point belongs in a code comment rather than in user documentation, and the other two were background nobody needs to run a calculation. The one consequence a user does need, that selecting L-BFGS with fast density pins the trimming threshold, survives as a line in the summary. Removed the section on FFT planning. Whether or not the underlying point stands, it is an implementation detail and it does not belong on a page about choosing an optimiser. Also fixed while checking every claim against the source: the steepest-descent section still described the trial energy as gating that step, which stopped being true when the accept test moved to the deferred verdict, and the list of conditions that certify a numerical floor was missing one of the four.
|
okay a re-structured and cleaned up with lots of the thesis remnant technicalities stripped! |
All three fast paths work with NGWF_LBFGS and on most systems make no difference to the iteration count. The fast NGWF gradient is the exception: it evaluates the gradient from trimmed NGWFs, and where that gradient and the energy used by the line search disagree, the optimiser spends iterations reconciling the two rather than lowering the energy. Worth knowing when L-BFGS takes longer than expected or stops short of the threshold, so the page says to try turning it off. Kept to what a user can act on; the mechanism belongs in the code, and the earlier review was right that this page is not the place for it.
|
small caveat added for LBFGS from user experience |
|
I am happy for this to be merged. In your hands now, @nickhine. |
|
I am merging this now, as @nickhine is seemingly away for the time being, and there's pressure from Biovia to complete the tarball + |
3 participants
There was no page describing the outer NGWF loop at all: ngwf_cg_type was listed in the keyword reference and nowhere else, so a user had no way to find out what the conjugate-gradient variants do, let alone how to choose between them.
This adds one page covering all three options at the same depth, placed after Crystal symmetry in the ground-state section.
The shared material comes first, since it applies whichever optimiser is selected: the covariant gradient and how it is raised, the k-point-weighted contraction that forms every slope and coefficient, and the RMS gradient the convergence threshold is tested against. Each optimiser then gets its own section. For conjugate gradients that means both coefficients side by side and, more usefully, the line search itself, which is what a user actually sees in the output but which has never been written down: the trial step, the parabola through it, the fallback to a cubic, and how the trial length carries over between iterations. For L-BFGS it means the two-loop recursion, the deferred verdict, the pair admission and retirement rules, and the conditions under which the run declares a numerical floor rather than stopping on an error.
Two things get their own sections because they cut across the optimisers. The pairing is not positive definite, since the kinetic-energy preconditioner is one-sided, and each optimiser copes with that differently; and fast density interacts with the deferred verdict in a way that changes a default.
Every formula was checked against the source as well as reproduced from the literature, and the output and diagnostic blocks are real, not invented.
this is obviously contingent on you accepting the NGWF-LBFGS commit
@JacekDziedzic