Return the support limit from ssd_q*() at the extreme quantiles - #196
Merged
Merged
Conversation
Release ssdtools 2.7.0, removing the ggtext dependency
Sync main with the released ssdtools 2.7.0 source
`.qd()` short-circuited `p = 0` and `p = 1` to `0` and `Inf` before dispatching to the distribution's own quantile function, which assumes every distribution has support unbounded above and approaching zero below. `invpareto` is bounded above at `scale`, so `ssd_qinvpareto(1, shape = 1, scale = 5)` returned `Inf` where `qinvpareto_ssd()` already returned the correct `5`, and the quantile function did not round trip through `ssd_pinvpareto()` at the upper support limit. The shortcut was really a workaround for the three quantile functions that solve numerically. `root()` uses `uniroot(lower = 0, upper = 1, extendInt = "upX")`, which extends the bracket rather than diverging, so `qlogis_logis_ssd()` returned -1073.742 and 41.95, `qlnorm_lnorm_ssd()` returned 10486.75 at `p = 1`, and `qmulti_list()` returned 5243.9. The other eight dispatch targets already return their own correct endpoints. Add an `endpoints()` helper that root finds the interior probabilities and substitutes the exact quantiles at `p = 0` and `p = 1`, use it in those three functions, and drop the blanket shortcut from `.qd()` so `qdist()` always defers to the distribution. `.lgt` is no longer read in `.qd()` or `.qdist()`, only in `qdist()` where it back transforms. `test_dist()` gains `lower` and `upper` arguments, since it asserted the same unbounded endpoints for every distribution. Its `ssd_qmulti()` assertions now supply a weight, which the shortcut had allowed them to omit by returning before the "at least one distribution must have a positive weight" check. Closes #195. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
joethorley
marked this pull request as ready for review
September 2, 2026 16:16
`pinvpareto_ssd()` returned `(q / scale)^shape` unclamped, so the CDF exceeded 1 above the upper support limit and was negative for negative `q` with odd `shape`: `ssd_pinvpareto(10, shape = 3, scale = 5)` was 8, and `ssd_hp()` on the `ccme_boron` inverse Pareto fit reported 117.5% of species affected at a concentration of 100. Return exactly 0 below and 1 above the support, the p-side counterpart of the q-side contract that `ssd_q*()` returns the support limit at p = 0 and p = 1. `qmulti_list()` hard coded the mixture support as (0, Inf) with a comment asserting that every distribution available for model averaging is unbounded. Derive the limits instead as the minimum and maximum of each positive-weight component's own quantile function at p = 0 and 1, which this branch makes exact, so a mixture weighted on a bounded component returns its finite limit and a future bounded distribution inherits correct model-averaged endpoints without touching this code. `ssd_hc()` accepts `proportion` of 0 and 1, so this is a user-facing path. `test_dist()` probed the CDF for monotonicity at q = 1, which for a distribution bounded above at 1 is the support limit where the clamped CDF is flat; probe strictly inside the support instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Restore `expect_identical()` for the endpoint assertions in `test_dist()`, which had been relaxed to `expect_equal()` when the `lower`/`upper` arguments were added; every distribution returns its limits exactly, and the tolerance would have accepted a saturated root of 1e-9 at p = 0, the bug class being fixed. Interpolate the limits with 17 significant digits so a value such as `exp(-3)` round trips, and start the p/q round trip at `lower` rather than 0. Drop the two test blocks that restated `test_dist()` assertions on the exported mixture functions, refer to the fork issue as `#195` since bare `#195` resolves to the upstream tracker, explain `root()`'s bracket extension once in the `endpoints()` header rather than at each call site, and propagate NaN through `endpoints()` instead of collapsing it to NA. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tiles' into joethorley/issue195-bounded-quantiles
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #195
Summary
.qd()short-circuitedp = 0andp = 1to0andInfbefore dispatching to the distribution's own quantile function. That assumes every distribution has support unbounded above and approaching zero below, which is true for most of the set but not forinvpareto, bounded above atscale:So the exported function discarded the correct answer its own backend had already computed, and did not round trip through
ssd_pinvpareto()at the upper support limit.Investigating why the shortcut was there turned up the real cause.
root()inR/internal.Rusesuniroot(lower = 0, upper = 1, extendInt = "upX"), which extends the bracket rather than diverging, so the three quantile functions that solve numerically saturate at the endpoints instead of returning infinity. The shortcut masked that, at the cost of overriding the distributions with genuinely finite support.Measured on the live
.qd()dispatch targets before this change:p = 0p = 1burrIII3,gamma,gompertz,lnorm,weibullgumbel,logis(both.lgt)invparetoscalelogis_logis(.lgt)lnorm_lnormmultiEight of eleven targets already returned their own correct endpoints unaided.
What changed
endpoints()inR/internal.Rroot finds the interior probabilities and substitutes the exact quantiles atp = 0andp = 1. It is used byqlogis_logis_ssd()(-Inf,Infon the log scale),qlnorm_lnorm_ssd()(0,Inf) andqmulti_list().qmulti_list()derives its limits from the components rather than hard coding(0, Inf): the mixture's support is the union of the positive-weight components' supports, and each component'sq*_ssd()now reports its own limits exactly, solowerandupperare the min and max of those atp = 0andp = 1. A mixture weighted entirely on a bounded component returns its finite limit, and any future bounded distribution inherits correct model-averaged endpoints without touching this code.The
p == 0/p == 1block is then removed from.qd(), soqdist()always defers to the distribution..lgtis no longer read in.qd()or.qdist()and was dropped from both;qdist()still uses it to back transform.pinvpareto_ssd()is clamped to[0, 1], the p-side counterpart of the same bounded-support contract. It returned(q / scale)^shapeunclamped, so the CDF exceeded 1 abovescaleand was negative for negativeqwith oddshape:ssd_pinvpareto(10, shape = 3, scale = 5)was 8, andssd_hp()on theccme_boroninverse Pareto fit (fittedscale75.3) reported 117.5% of species affected at a concentration of 100.test_dist()gainslowerandupperarguments, because it assertedq(0) == 0andq(1) == Inffor every distribution, which is the same assumption in test form. Its CDF monotonicity probe now sits strictly inside the support rather than atq = 1, which forinvparetowith defaultscale = 1is the flat region above the limit. Itsssd_qmulti()endpoint assertions now supplylnorm.weight = 1; they had relied on the shortcut returning beforenormalize_weights()could raise "at least one distribution must have a positive weight". A newtest-multi.Rblock checks that a single-component mixture reproduces that component's endpoints for every distribution inssd_dists_all(), and that a mixture on a bounded component returns its finite limit.This supersedes the direction suggested in #195, which proposed a support flag in
dist_data. That would change a user-visible dataset and would leave the three numerical quantile functions still wrong when called directly, so it treats the symptom rather than the cause.Behaviour changes
Beyond the fixes above, three exported behaviours change at the endpoints, all arguably corrections:
ssd_hc()acceptsproportionof 0 and 1 (chk_range()is inclusive), so for a model average these now flow throughqmulti_list()'s derived limits. For the current distribution set they are still0andInf.p = 0or1returnNaNrather than0/Inf, consistent with every otherp(e.g.ssd_qlnorm(0, sdlog = -1)).ssd_qmulti(0)andssd_qmulti(1)with no positive weight now error like every otherp, instead of returning0/Infbefore validation.Verification
R CMD checkreports 0 errors, 0 warnings and 0 notes. The full test suite is 1370 passing with 0 failures and 0 warnings.air format --check .is clean.Tests were written first and watched fail: 6 failures on the pre-fix code for the endpoint change (reporting
Infwherescalewas expected and the saturated-1073.7,42.0and10486.8where infinite endpoints were expected), and 4 for the clamp and derived-support change (8,1.2 2.0,-0.008from the unclamped CDF,Inffrom the hard-coded mixture limit).Endpoints across every exported quantile function after the change:
And the round trip that was broken:
Notes
Branched from
mainand targeted atdevper the fork convention.ltriangle(#169) has bounded support on both sides and inherits this fix when that branch merges; itstest_dist("ltriangle")call will then needlower = exp(-3), upper = exp(3), so the merge order is this PR todev,devintoadd-triangle-distribution, then that one-line change there.The pre-existing cost of rebuilding the mixture skeleton on every scalar
.qd()call, which the review of this PR measured, is filed separately as #197.