Optimize array decoding for raw-string adapters

[oschwald]

Summary

• specialize raw-string adapter decoding so arrays expose exact Serde allocation hints after one up-front bounds check
• keep raw adapter values and array access separate from normal Serde hot types, avoiding mode branches or layout changes for typed decoding
• reject impossible raw array counts before the visitor can allocate
• add an Unreleased changelog entry

This benefits any raw-string format adapter that uses SeqAccess::size_hint() to preallocate a foreign-runtime array, not only the Python binding.

Performance

Persistent workers were pinned to one CPU and alternated baseline/candidate order over 21 matched rounds using installed GeoIP databases.

• Python random IPv4 full City lookup: 423,390 to 434,753 lookups/s, +2.80% paired median
• native LookupResult::decode::geoip2::City(): +0.08% paired median, effectively neutral

Validation

• default tests: 90 passed; 22 doc tests passed
• mmap + simdutf8: 91 passed; 23 doc tests passed
• mmap + unsafe-str-decode: 91 passed; 23 doc tests passed
• rustfmt, clippy with warnings denied, and rustdoc with warnings denied pass

Summary by CodeRabbit

• Bug Fixes

  • Improved array decoding performance for raw-string formats.
  • Added validation to reject impossible array sizes before processing begins, providing clearer errors and preventing invalid data from reaching application logic.

• Documentation

  • Added an Unreleased changelog entry describing the decoding performance improvement.

[qodo-code-review]

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 1f38a5d8-b6cc-4a7f-a847-226be956a5b3

📥 Commits

Reviewing files that changed from the base of the PR and between fceaa66 and fc3a2c3.

📒 Files selected for processing (2)

• CHANGELOG.md
• src/decoder.rs

---

Walkthrough

The raw-string decoding path now uses dedicated decoded values and entry points, validates declared array sizes before visitor dispatch, adds raw array sequence access, updates newtype routing, and includes regression coverage and a changelog entry.

Changes

Raw-string decoding

Layer / File(s)	Summary
Dedicated raw decoder and array validation src/decoder.rs	Raw-string values use a dedicated representation and decoding path, with validation rejecting array counts larger than the remaining data.
Raw array access and regression coverage src/decoder.rs, CHANGELOG.md	Raw newtype decoding uses the new decoder and RawArrayAccess; a regression test verifies impossible arrays fail before visitor invocation, with the change documented in the changelog.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• oschwald/maxminddb-rust#118: Introduced the raw-string mode and newtype marker used by this decoder refactor.

Sequence Diagram(s)

sequenceDiagram
  participant Deserializer
  participant decode_any_raw
  participant RawArrayAccess
  participant SerdeVisitor
  Deserializer->>decode_any_raw: decode raw-string newtype
  decode_any_raw->>RawArrayAccess: expose validated array elements
  RawArrayAccess->>SerdeVisitor: visit sequence elements

Loading

Poem

A rabbit hops through bytes so bright,
Raw strings race with less delay tonight.
Arrays count before they leap,
Bad sizes never wake the heap.
“Thump!” says the visitor, safe and sound.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: optimizing array decoding for raw-string adapters.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch greg/optimize-container-size-hints

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}