Undamped `Spline` silently return wrong predictions with scikit-learn >= 1.9

[nicolas-graves]

Description of the problem:

Context: During the bump of scikit-learn in Guix, some Verde tests started failing:

• test_chain.py::test_chain_double,
• test_spline.py::test_spline,
• all three test_spline.py::test_spline_cv[...]

Since scikit-learn 1.9, fitting verde.Spline (or verde.SplineCV whenever cross-validation selects damping=None, which is the common/default outcome for noise-free-ish data) with the default, undamped configuration produces predictions that no longer match the training data. No exception or

This also affects anything that chains an undamped Spline, e.g. Chain([..., ("spline", Spline())]).

Investigation (disclaimer : partly assisted by generative AI)
verde.base.least_squares.least_squares() fits the undamped case via:

regr = LinearRegression(fit_intercept=False)
regr.fit(jacobian, np.ravel(data), sample_weight=weights)

Internally, LinearRegression.fit()'s dense-data path does:

self.coef_, _, self.rank_, self.singular_ = linalg.lstsq(X, y, cond=self.tol)

self.tol defaults to 1e-6. Per scikit-learn's own docstring for
LinearRegression:

tol : float, default=1e-6
    The precision of the solution (`coef_`) is determined by `tol` which
    specifies the convergence criterion of the underlying solver. ...
    `tol` is set as `cond` of :func:`scipy.linalg.lstsq` when fitting on
    dense training data.

    .. versionadded:: 1.7
    .. versionchanged:: 1.9
       Now supported on dense data, interpreted as the `cond` parameter.

System information

• verde 1.9.0
• scikit-learn 1.9.0
• python 3.12

[leouieda]

Thank you for reporting! Yet another reason why we should move away from relying on the scikit-learn estimators and implement our own linear algebra. They change things a bit too fast for us.