Skip to content

Fix float32 MLE optimization with autograd gradients#66

Open
AparajeetS wants to merge 1 commit into
EFS-OpenSource:mainfrom
AparajeetS:codex/fix-mle-float32-gradient
Open

Fix float32 MLE optimization with autograd gradients#66
AparajeetS wants to merge 1 commit into
EFS-OpenSource:mainfrom
AparajeetS:codex/fix-mle-float32-gradient

Conversation

@AparajeetS

@AparajeetS AparajeetS commented Jul 16, 2026

Copy link
Copy Markdown

Fixes #61.

What changed

  • compute the convex MLE objective gradient with PyTorch autograd
  • pass the exact Jacobian to scipy.optimize.minimize
  • use the same Jacobian path for masked re-optimization
  • add a deterministic regression test for TemperatureScaling and
    LogisticCalibration with both float32 and float64 inputs
  • verify that the repaired float32 loss agrees with the float64 control

Root cause

The MLE callback converted SciPy's float64 parameter vector to the prepared input
tensor's dtype. With float32 inputs, SciPy's default finite-difference perturbation near
one was rounded away by that conversion. The loss at w=1 and w=1+1e-8 became
identical, so SciPy reported a zero gradient after two evaluations and returned the
identity initialization with a successful status.

On a deterministic five-class overconfidence problem, the old float32 behavior kept the
weight at 1.0 and NLL at 1.8382. The autograd path fits a weight of 0.3286 and
reduces NLL to 1.3154, matching the float64 control (0.3285, 1.3154).

Validation

  • python -m unittest tests.test_mle_precision -v
  • direct patched-library reproduction for float32 and float64
  • python -m compileall -q netcal tests
  • git diff --check

The full executable diagnosis and recorded results are available in
Phason Evidence Audit PEA-005.

@AparajeetS
AparajeetS force-pushed the codex/fix-mle-float32-gradient branch from 9f670f7 to b6c42fb Compare July 18, 2026 09:54
@AparajeetS
AparajeetS marked this pull request as ready for review July 18, 2026 09:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TemperatureScaling and LogisticCalibration do not work correctly

1 participant