Align gifteval protocol

[cyleroux]

This PR aligns the GiftEval dataset and the forecasting metrics with the GIFT-Eval leaderboard protocol, so that benchmark scores are directly comparable to the published leaderboard numbers.

Dataset (datasets/gifteval.py, benchmark_utils/forecasting_constants.py):

• rolling test windows follow the leaderboard: the default n_windows="gifteval" resolves to min(20, ceil(0.1 * min_series_length / pred_len)), pinned to 1 for the m4_* datasets
• seasonality uses the gluonts table the leaderboard uses (gift_eval_seasonality: D=1, W=1, sub-hourly divided by the multiplier, instead of the generic Monash-style table)
• NaN targets (9 datasets, e.g. car_parts, kdd_cup_2018) are imputed like the official GIFT-Eval notebooks (forward-fill, flat-fill, 0.0 for all-NaN channels); the original locations are kept in nan_mask so the metrics can exclude the imputed points

Metrics (benchmark_utils/metrics.py, objective.py):

• MASE: each window is scaled by the seasonal-naive MAE of the context available at its cutoff, and the score averages the per-point scaled errors, like gluonts.evaluate_forecasts(axis=None)
• sMAPE: gluonts formula 2|y - yhat| / (|y| + |yhat|), with 0/0 points dropped
• every forecasting metric accepts valid_mask to score the non-imputed points only; the objective builds it from the dataset nan_mask

Baseline (solvers/seasonal_naive.py):

• season_length="auto" resolves to the dataset seasonality, like the GIFT-Eval seasonal-naive baseline; this replaces the [1, 7, 12, 24] sweep
• 9-level quantile fan around the point forecast like the official baseline; no new dependency

Validation: MASE and WQL now match the official Seasonal_Naive results on all 97 combos (WQL median relative diff 2e-8); the remaining ~1e-3 residuals on bitbrains are the same context-imputation effect as MAE/sMAPE.

Tests: unit tests cover the NaN imputation, the seasonality table, the per-window scales, the masked metrics and the window-count resolution (pytest tests/).

[felixdivo]

I didn't read all of the code, but it looks really cool!

[tomMoral]

Is it still used somewhere?

[cyleroux]

Yes, it's still used:

• datasets/fev.py imports and calls it to resolve (canonical_freq, seasonality, default_h) from the pandas frequency; default_h is what FEV uses as the default prediction_length when none is set (pre-existing, untouched by this PR);
• from_aeon() in the same module delegates to it, and from_aeon is used by datasets/monash.py (with the same default_h fallback);
• it's covered by the existing parametrized tests in tests/benchmark_utils/test_forecasting_constants.py.

This PR only replaced its use in datasets/gifteval.py with gift_eval_seasonality, which is why the import disappears from that file.

[Jad-yehya]

I think we don't use the mask here ? We should either call the mae function directly or _masked_mean with the mask.

Either way we need to pass the mask to this function.

[cyleroux]

The mask is applied right after the division:

if valid_mask is not None:
    per_point = per_point[valid_mask]

and the objective passes valid_mask to every forecasting metric (built from the dataset nan_mask in _eval_forecasting). It doesn't go through _masked_mean because that helper uses np.mean, which would propagate the NaN of the 0/0 points; gluonts drops those points from the mean instead. I can extend _masked_mean to skip NaNs if you prefer the shared helper, but it's shared by mae/mse/mcis, so I kept the handling local to smape in this PR.

[Jad-yehya]

Not a problem with the way we do gifteval evaluation, but this seasonal_naive only does point forecast and won't do intervals, I believe the official implementation uses statsforecast.

[cyleroux]

Right, the SeasonalNaive solver here only outputs the median (quantile_levels=(0.5,)) while the official baseline wraps statsforecast.SeasonalNaive and produces real quantiles. This PR is scoped to the protocol alignment, so I validated the point metrics against the leaderboard; with a single quantile, the distributional metrics (CRPS/WQL/MCIS) are not meaningful for SeasonalNaive yet. Should I add proper quantiles in a follow-up PR (wrapping statsforecast, like the official baseline), or in this one?

[tomMoral]

Let's do in this one I think

[cyleroux]

Done: the solver now outputs a 9-level quantile fan like the official baseline, so CRPS/WQL/MCIS are no longer degenerate. Replicates the native statsforecast intervals, no new dependency needed. WQL matches the leaderboard Seasonal_Naive CSV on all 97 combos (same context-imputation residual as MAE/sMAPE on bitbrains).