Skip to content

feat: add fd.clean_excel() companion to fd.clean_csv() - #198

Merged
kevincostner17 merged 2 commits into
mainfrom
feat/clean-excel
Sep 14, 2026
Merged

kevincostner17 merged 2 commits into
mainfrom
feat/clean-excel

Conversation

@kevincostner17

Copy link
Copy Markdown
Contributor

Closes #166

Summary

Adds fd.clean_excel(path, ...), the Excel companion to fd.clean_csv (#19) that #166 asks for.

  • Same API as clean_csv. The parameters match one-to-one, except read_csv_kwargs/to_csv_kwargs become read_excel_kwargs/to_excel_kwargs. A test checks that the two signatures stay in sync.
  • One sheet at a time. The first sheet is read by default, and read_excel_kwargs={"sheet_name": ...} picks another. A sheet_name that selects several sheets (None or a list) raises TypeError instead of passing a dict to fd.clean.
  • Formulas are sanitized by default. I checked this before writing the code: on py3.9/pandas 1.5 and py3.12/pandas 2.3 with openpyxl 3.1.5, a plain DataFrame.to_excel stores "=1+1" cells and "=..." headers as live formula cells (data_type == "f"). The written workbook therefore goes through the same sanitize_csv_formulas guard as clean_csv. sanitize_formulas=False opts out, and the returned DataFrame is never altered.
  • New excel extra. It adds openpyxl>=3.0.7, the minimum pandas 1.5 accepts. It's also in all, and in dev so CI runs the tests. freshdata never imports openpyxl itself; if it's missing, pandas raises its own ImportError.
  • Lockfile. uv.lock and constraints/ci.txt were refreshed with uv 0.12.10. The only additions are openpyxl and et-xmlfile.
  • Docs. docs/api-reference.md gets the new function, docs/threat-model.md gets a row in the sanitization table, and CHANGELOG.md has an entry.

Tests

New tests/test_clean_excel.py, 7 tests. The fixtures are built with openpyxl so = strings stay text on input.

  • The result matches fd.clean(pd.read_excel(src)), and the written workbook reads back the same.
  • return_report=True and the report=True alias both return a CleanReport.
  • Sheet selection works, and a multi-sheet sheet_name raises TypeError.
  • With sanitization on, the written cells are text with a ' prefix and there are no formula cells; the returned frame is unchanged. With sanitize_formulas=False, a formula cell is written.
  • The signature matches clean_csv, and clean_excel is exported.

Verification

  • pytest tests/test_clean_excel.py tests/test_csv_formula_sanitize.py tests/test_api.py tests/test_phase5_backcompat.py: all pass on Python 3.9 / pandas 1.5.3 and on Python 3.12 / pandas 2.3.3.
  • ruff check: clean. mypy src/freshdata: no issues in 201 files.
  • uv lock --check: passes. mkdocs build --strict: passes.

Read one sheet with pandas.read_excel, run fd.clean, and optionally
write the cleaned workbook. The signature mirrors clean_csv, with
read_excel_kwargs/to_excel_kwargs in place of the csv kwargs.

Formula sanitization is on by default for the written workbook: a raw
to_excel stores "=1+1" cells and "=..." headers as live formula cells.
A sheet_name that selects several sheets raises TypeError.

Adds an `excel` extra (openpyxl), also pulled into `all` and `dev`, and
refreshes uv.lock and constraints/ci.txt.

Closes #166
@coderabbitai

coderabbitai Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 815c18d4-ffb7-43b8-9b3c-4ffe91d67015


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

FreshData benchmark report — performance

  • freshdata: ?
  • python: ?
  • platform: ?
fixture n_rows n_cols p50 s p95 s peak MB repair % false-repair % preserve % trust monotonic export %

Authored-code reduction (Metric 6)

tests/truthbench/test_inventory.py requires every public name to be
classified, so collection failed with 'unclassified FreshData public
surface: clean_excel'. Register it as a decision surface on the
cleaning adapter, with a clean_excel operation mirroring clean_csv.
@kevincostner17
kevincostner17 merged commit 7af3251 into main Sep 14, 2026
21 checks passed
@kevincostner17
kevincostner17 deleted the feat/clean-excel branch September 14, 2026 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feature: add fd.clean_excel() companion to fd.clean_csv() (#19)

1 participant