fix: correct the GES/LGES/TGES baselines - #39
Merged
Conversation
The GES, LGES and TGES baselines were substantially understated. - `ges_discovery` read causal-learn's adjacency matrix with the endpoints transposed, reversing directed contemporaneous edges and dropping directed lagged ones. - The vendored GES search behind LGES/TGES had defects in its CPDAG construction and its Insert, Delete and Turn operators, so the forward phase stopped short of the optimum and LGES never converged. - All three now apply temporal background knowledge (a variable can only cause another at an equal or later time step), which the lag-embedded search previously ignored. `ges_discovery` gains an `engine=` argument selecting the vendored search (default) or causal-learn. F1 on the baseline_comparison datasets moves to 0.636/0.917/0.949/0.435 for all three. Every correction was verified against upstream ges 1.1.1 -- no defect originated with the upstream authors -- and the corrected search returns CPDAGs identical to upstream's on random DAGs. Notebooks re-executed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The GES, LGES and TGES baselines were substantially understated. Two independent problems:
ges_discoveryread causal-learn's adjacency matrix with the endpoints transposed, so every directed contemporaneous edge came back reversed and every directed lagged edge was silently dropped.lges_discovery/tges_discoveryhad defects in its CPDAG construction and its Insert, Delete and Turn operators. The forward phase stopped short of the optimum, so LGES never converged — F1 sat at 0.35–0.52 onex2no matter how much data it was given.All three now also apply temporal background knowledge (a variable can only cause another at an equal or later time step), which the lag-embedded search previously ignored.
ges_discoverygains anengine=argument selecting the vendored search (default) or causal-learn.Results. On the
baseline_comparisondatasets, F1 moves from 0.125/0.154/0.522/0.333 (GES) and 0.400/0.417/0.526/0.333 (LGES/TGES) to 0.636/0.917/0.949/0.435 for all three. Any prior comparison against these baselines understates them.Verification. Every correction was checked against upstream
ges1.1.1 — no defect originated with the upstream authors. The corrected search returns CPDAGs identical to upstream's on random DAGs, and is faster than it. New tests cover the CPDAG pipeline exhaustively (every DAG on 4 and 5 nodes) plus convergence and differential agreement with causal-learn.Notebooks re-executed, with a new cell showing where GES, LGES and TGES actually diverge (they coincide on all four bundled datasets, but not on larger/noisier data).