Skip to content

Commit 5542b0e

Browse files
fix(learning): suppress dayfirst-coercion warning in column diff
pandas <2's Series.eq() between an object (string) column and a datetime64 column implicitly attempts to parse the string side as dates for comparison, which can emit a dayfirst-ambiguity UserWarning; pandas >=2 dropped that implicit coercion. Since freshdata's pytest config promotes freshdata-sourced UserWarnings to errors, this broke the Python 3.9 (pandas<2) CI matrix job across most of tests/learning/. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent ddd33da commit 5542b0e

1 file changed

Lines changed: 9 additions & 3 deletions

File tree

‎src/freshdata/learning/diff.py‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@
77

88
from __future__ import annotations
99

10+
import warnings
11+
1012
import pandas as pd
1113

1214
from .types import AlignedPair, DiffSummary, RowDiffSummary, SchemaDiffSummary, ValueDiff
@@ -26,9 +28,13 @@ def _column_diffs(messy: pd.Series, clean: pd.Series, column: str) -> list[Value
2628
raw_na = messy.isna()
2729
clean_na = clean.isna()
2830
# A cell differs when values are unequal, except both-missing which is equal.
29-
unequal = ~(messy.eq(clean) | (raw_na & clean_na))
30-
# eq() treats NaN != NaN, and mixed dtypes can raise inside eq; the mask
31-
# above already covers the common path, but guard object-dtype surprises.
31+
# eq() between an object (string) column and a datetime64 column makes
32+
# pandas <2 attempt an implicit to_datetime coercion on the string side,
33+
# which can emit a dayfirst-ambiguity UserWarning; pandas >=2 dropped that
34+
# coercion. Suppress here rather than requiring every caller to know why.
35+
with warnings.catch_warnings():
36+
warnings.simplefilter("ignore", UserWarning)
37+
unequal = ~(messy.eq(clean) | (raw_na & clean_na))
3238
if not unequal.any():
3339
return []
3440

0 commit comments

Comments
 (0)