From 0d34e378a44caaeec37f5e1df0baee1bcd269e2f Mon Sep 17 00:00:00 2001
From: Kevin Costner <120246174+kevincostner17@users.noreply.github.com>
Date: Tue, 15 Sep 2026 22:20:41 +0530
Subject: [PATCH 1/3] docs: regenerate HTML examples for current default output
The committed samples predated the report-only duplicate default, the
outlier flag default and the not-assessed quality-debt dimensions (#420):
rows now read 205 -> 205, revenue shows outliers -> flag (7 flagged), and
the quality_debt total is 0.34 with schema_drift/category_churn unknown.
Regenerated with scripts/generate_html_examples.py. The action timeline's
duration and the quality-debt run_at timestamp are run-dependent and no
renderer option omits them, so those lines change on every regeneration.
---
docs/examples/action_timeline.html | 8 ++++----
docs/examples/baseline_drift.html | 2 +-
docs/examples/compare_plans_grid.html | 2 +-
docs/examples/explain_diff.html | 10 +++++-----
docs/examples/quality_debt.html | 8 ++++----
docs/examples/suggest_plan_cards.html | 4 ++--
6 files changed, 17 insertions(+), 17 deletions(-)
diff --git a/docs/examples/action_timeline.html b/docs/examples/action_timeline.html
index fb787a7d..3f4c8f6f 100644
--- a/docs/examples/action_timeline.html
+++ b/docs/examples/action_timeline.html
@@ -29,7 +29,7 @@
.fd-del-pos{color:#1a7f37}.fd-del-neg{color:#cf222e}
.fd-mono{font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:.8rem}
-
freshdata clean report
backend=pandas · 0.015s
205 → 200
rows
5 → 5
columns
76 → 23
missing cells
5
duplicates removed
0
outliers handled
5
actions
-
Action timeline
fix_dtypessignup_date — converted to datetime64[ns] (205 affected)lowautomatic
filled 52 missing value(s) with sentinel "Missing" ('Missing')
revenue
outliers
low
70%
7
flagged 7 outlier(s), 3.4% of values (method=iqr, factor=1.5) in new column 'revenue_outlier'
+
Needs review
id column 'email' has 24 missing value(s) (11.7%); check whether those rows are joinable at all
\ No newline at end of file
diff --git a/docs/examples/baseline_drift.html b/docs/examples/baseline_drift.html
index 312a3f7c..ca575e34 100644
--- a/docs/examples/baseline_drift.html
+++ b/docs/examples/baseline_drift.html
@@ -32,4 +32,4 @@
\ No newline at end of file
+⬇ JSON
\ No newline at end of file
diff --git a/docs/examples/compare_plans_grid.html b/docs/examples/compare_plans_grid.html
index c41e10b2..b871166a 100644
--- a/docs/examples/compare_plans_grid.html
+++ b/docs/examples/compare_plans_grid.html
@@ -29,4 +29,4 @@
.fd-del-pos{color:#1a7f37}.fd-del-neg{color:#cf222e}
.fd-mono{font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:.8rem}
-
freshdata strategy comparison
strategy diff grid
column
strategy
missing_model
outlier_action
n_outliers
email
balanced
preserve
0
revenue
balanced
5
tier
balanced
sentinel
0
email
aggressive
preserve
0
revenue
aggressive
5
tier
aggressive
sentinel
0
\ No newline at end of file
+
freshdata strategy comparison
strategy diff grid
column
strategy
missing_model
outlier_action
n_outliers
email
balanced
preserve
0
revenue
balanced
flag
7
tier
balanced
sentinel
0
email
aggressive
preserve
0
revenue
aggressive
flag
7
tier
aggressive
sentinel
0
\ No newline at end of file
diff --git a/docs/examples/explain_diff.html b/docs/examples/explain_diff.html
index ed05fd9c..e5788493 100644
--- a/docs/examples/explain_diff.html
+++ b/docs/examples/explain_diff.html
@@ -29,8 +29,8 @@
.fd-del-pos{color:#1a7f37}.fd-del-neg{color:#cf222e}
.fd-mono{font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:.8rem}
-
\ No newline at end of file
diff --git a/docs/examples/quality_debt.html b/docs/examples/quality_debt.html
index 400abde9..491cefa2 100644
--- a/docs/examples/quality_debt.html
+++ b/docs/examples/quality_debt.html
@@ -29,7 +29,7 @@
.fd-del-pos{color:#1a7f37}.fd-del-neg{color:#cf222e}
.fd-mono{font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:.8rem}
-
\ No newline at end of file
diff --git a/docs/examples/suggest_plan_cards.html b/docs/examples/suggest_plan_cards.html
index c9117b20..d2b198a2 100644
--- a/docs/examples/suggest_plan_cards.html
+++ b/docs/examples/suggest_plan_cards.html
@@ -29,5 +29,5 @@
.fd-del-pos{color:#1a7f37}.fd-del-neg{color:#cf222e}
.fd-mono{font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:.8rem}
-
\ No newline at end of file
From f8f64147deca7831a8e60082ab44a57af35edc2d Mon Sep 17 00:00:00 2001
From: Kevin Costner <120246174+kevincostner17@users.noreply.github.com>
Date: Tue, 15 Sep 2026 22:20:59 +0530
Subject: [PATCH 2/3] test: skip benchmark tests when benchmarks/ is absent
The sdist ships tests/ but not benchmarks/. conftest.py only filters
module-level imports, so tests that read benchmarks/ inside the test body
failed or errored from an unpacked sdist. The bench_streaming fixture now
uses pytest.importorskip (as the cleanbench suites do), and the entity
resolution smoke test and benchmark-stream CLI test skip when the
benchmark file is missing. The CLI test also runs from the repo root, since
the command resolves benchmarks/ against the working directory.
---
tests/test_enterprise_entity_resolution_dedupe.py | 7 ++++---
tests/test_streaming_cli.py | 12 ++++++++----
2 files changed, 12 insertions(+), 7 deletions(-)
diff --git a/tests/test_enterprise_entity_resolution_dedupe.py b/tests/test_enterprise_entity_resolution_dedupe.py
index f36af65f..bd7601ca 100644
--- a/tests/test_enterprise_entity_resolution_dedupe.py
+++ b/tests/test_enterprise_entity_resolution_dedupe.py
@@ -494,9 +494,10 @@ def test_retail_template_and_overrides():
def test_benchmark_smoke():
root = pathlib.Path(__file__).resolve().parents[1]
- spec = importlib.util.spec_from_file_location(
- "bench_er", root / "benchmarks" / "bench_entity_resolution.py"
- )
+ bench_path = root / "benchmarks" / "bench_entity_resolution.py"
+ if not bench_path.is_file():
+ pytest.skip("benchmarks/ directory not present (e.g. running from the sdist)")
+ spec = importlib.util.spec_from_file_location("bench_er", bench_path)
assert spec and spec.loader
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
diff --git a/tests/test_streaming_cli.py b/tests/test_streaming_cli.py
index 008d832a..b9826c88 100644
--- a/tests/test_streaming_cli.py
+++ b/tests/test_streaming_cli.py
@@ -120,9 +120,8 @@ def test_cli_stream_parquet_roundtrip(tmp_path):
def bench_module():
sys.path.insert(0, str(BENCH_DIR))
try:
- import bench_streaming # type: ignore[import-not-found]
-
- yield bench_streaming
+ # benchmarks/ is repo-only (not in the sdist): skip rather than error there.
+ yield pytest.importorskip("bench_streaming")
finally:
sys.path.remove(str(BENCH_DIR))
sys.modules.pop("bench_streaming", None)
@@ -149,7 +148,12 @@ def test_benchmark_generator_emits_exact_row_count(bench_module):
# -- benchmark-stream CLI command ---------------------------------------------------
-def test_cli_benchmark_stream_writes_json(tmp_path):
+def test_cli_benchmark_stream_writes_json(tmp_path, monkeypatch):
+ # The command imports bench_streaming from ./benchmarks, which only a git
+ # checkout has; the sdist ships tests/ without it.
+ if not (BENCH_DIR / "bench_streaming.py").is_file():
+ pytest.skip("benchmarks/ directory not present (e.g. running from the sdist)")
+ monkeypatch.chdir(BENCH_DIR.parent)
report = tmp_path / "bench.json"
rc = main(["benchmark-stream", "--rows", "2000", "--batch-size", "1000",
"--cols", "6", "--report", str(report)])
From f149d84371012db53959219647b2f156ef30ff19 Mon Sep 17 00:00:00 2001
From: Kevin Costner <120246174+kevincostner17@users.noreply.github.com>
Date: Tue, 15 Sep 2026 22:21:13 +0530
Subject: [PATCH 3/3] docs: note Python 3.9 on Linux aarch64 builds the privacy
stack from source
thinc 8.3.4 and blis 1.2.0 publish no cp39 Linux aarch64 wheel, and spacy
3.8.7 (the py3.9 cap) requires thinc>=8.3.4,<8.4, so no pin avoids the
source build. Document it in the install guide and next to the py3.9 caps
in pyproject.toml. Dependency pins are unchanged.
---
docs/installation.md | 8 ++++++++
pyproject.toml | 3 +++
2 files changed, 11 insertions(+)
diff --git a/docs/installation.md b/docs/installation.md
index 0d0494a7..07b827ee 100644
--- a/docs/installation.md
+++ b/docs/installation.md
@@ -59,6 +59,14 @@ Install only what you need:
Pass a Polars DataFrame to `fd.clean` and get a Polars DataFrame back.
+!!! note "Python 3.9 on Linux aarch64: the `privacy` extra builds from source"
+ `freshdata-cleaner[privacy]` (and `[all]`) pulls in spaCy through Presidio.
+ On Python 3.9 spaCy is capped at 3.8.7, which requires `thinc>=8.3.4,<8.4`,
+ and neither thinc 8.3.4 nor blis 1.2.0 publishes a cp39 Linux aarch64 wheel.
+ pip therefore compiles thinc and blis from source there, which needs a C/C++
+ toolchain and takes several minutes. Python 3.10+ or x86-64 installs use
+ prebuilt wheels.
+
## Verify the installation
```bash
diff --git a/pyproject.toml b/pyproject.toml
index 72e52101..bebadcd0 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -168,6 +168,9 @@ excel = [
# Newer ones still declare Requires-Python >=3.9 but publish only an sdist for
# 3.9, and spacy >=3.8.13 cannot even build there (it needs thinc>=8.3.12, which
# is Python >=3.10). Keep these caps in sync with the `all` extra.
+# Even at these caps, thinc 8.3.4 and blis 1.2.0 have no cp39 Linux aarch64
+# wheel, so that platform compiles them from source (see docs/installation.md).
+# No pin avoids it: spacy 3.8.7 requires thinc>=8.3.4,<8.4.
privacy = [
"presidio-analyzer>=2.2",
"presidio-anonymizer>=2.2",