Skip to content

[Enhancement - Junior Level] Recognize PostgreSQL \\N as a missing-value sentinel #20

Description

@JohnnyWilson16

The Problem: SQL and warehouse exports often use \\N to represent NULL. FreshData already handles many missing markers, but this common export format should be cleaned automatically.

The Solution: Add "\\N" support to the default sentinel registry so values like "\\N" and " \\N " are treated as missing only when the entire cell matches.

Acceptance Criteria:

  • Update src/freshdata/_sentinels.py with the new sentinel.
  • Add tests in tests/test_strings.py.
  • Confirm "\\N" and " \\N " become missing.
  • Confirm partial values like "A\\N42" are preserved.
  • python -m pytest tests/test_strings.py passes.

Skills Required: Python, PyTest, Pandas, string normalization

How to Get Started:

  1. Open src/freshdata/_sentinels.py and read the existing sentinel list.
  2. Open src/freshdata/steps/strings.py to see how matching works.
  3. Add the smallest possible tests in tests/test_strings.py before changing behavior.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions