You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
PiWindDtype is listed in ORDERED_MODELS in tests/test_model_runs.py, meant to run only after every other model in the same pytest session. Running PiWindDtype/test_1 in isolation (-k PiWindDtype) fails --check-results against expected_results/output, even on main with no code change involved.
Confirmed not a code/output issue: dispatched with -k PiWindDtype on both main and an unrelated PR branch — both failed identically, and a diagnostic sha256 hash of every generated output/* file matched exactly between the two. So the divergence is against expected_results itself, triggered by out-of-order/isolated execution, likely due to some state (JIT cache, clearcache side effect, shared session state) that expected_results implicitly depends on from earlier models running first.
Makes -k-filtered runs of PiWindDtype/test_1 unreliable for CI debugging/bisection.
Steps to Reproduce (Bugs only)
pytest tests/test_model_runs.py -k PiWindDtype --check-results on a clean main checkout.
Observe FAILED ...PiWindDtype/test_1 - Failed: --check-results: output differs from .../expected_results/output.
Compare to a full-suite run (no -k filter, same commit) — passes.
Issue Description
PiWindDtypeis listed inORDERED_MODELSintests/test_model_runs.py, meant to run only after every other model in the same pytest session. RunningPiWindDtype/test_1in isolation (-k PiWindDtype) fails--check-resultsagainstexpected_results/output, even onmainwith no code change involved.Confirmed not a code/output issue: dispatched with
-k PiWindDtypeon bothmainand an unrelated PR branch — both failed identically, and a diagnostic sha256 hash of every generatedoutput/*file matched exactly between the two. So the divergence is againstexpected_resultsitself, triggered by out-of-order/isolated execution, likely due to some state (JIT cache,clearcacheside effect, shared session state) thatexpected_resultsimplicitly depends on from earlier models running first.Makes
-k-filtered runs ofPiWindDtype/test_1unreliable for CI debugging/bisection.Steps to Reproduce (Bugs only)
pytest tests/test_model_runs.py -k PiWindDtype --check-resultson a cleanmaincheckout.FAILED ...PiWindDtype/test_1 - Failed: --check-results: output differs from .../expected_results/output.-kfilter, same commit) — passes.Version / Environment information
tests/test_model_runs.py(added in enhance/dtype env #46)OasisModels CLIworkflow (OasisLMF/.github/workflows/piwind-mdk.yml),workflow_dispatchwithpytest_params: -k PiWindDtypeExample data / logs
output/gul_S1_melt.csvsha256 identical on both branches tested (88755fec...0888c17), confirming outputs match each other but notexpected_results.