Skip to content

Use a linear-time substring search for in on str and bytes - #258

Open
Dilshad-Tanvi wants to merge 1 commit into
meta:mainfrom
Dilshad-Tanvi:linear-substring-in
Open

Dilshad-Tanvi wants to merge 1 commit into
meta:mainfrom
Dilshad-Tanvi:linear-substring-in

Conversation

@Dilshad-Tanvi

Copy link
Copy Markdown
Contributor

x in s for a str goes through fast_string::contains, which for a needle longer than one byte checks the needle against every candidate position, and x in b for bytes does the same with windows(). The worst case is O(len(haystack) * len(needle)): a needle that matches the haystack except for its last byte makes every comparison run the whole needle. Starlark has no unbounded loops or recursion, so a short script like this is enough to tie up an evaluator:

h = "a" * 640000
print(("a" * 320000 + "b") in h)

Release build, haystack 2n and needle n + 1:

n str before str after bytes before bytes after
160000 0.72s 0.03s 2.22s 0.01s
320000 4.38s 0.09s 2.15s 0.06s

Both now use memchr::memmem::find, which is guaranteed linear in haystack plus needle. memchr is already a workspace dependency used by fast_string. The str path had avoided str::contains because it is slow on short strings, and memmem::find takes a setup-free Rabin-Karp path for haystacks under 64 bytes, so the short case stays cheap. All other str searches (find, rfind, index, count, split, replace) already use std's linear searcher, so these two loops were the only quadratic ones. cargo test -p starlark and -p starlark_syntax pass, and the existing in tests for str and bytes cover the semantics.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 26, 2026
@meta-codesync

meta-codesync Bot commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

@jtbraun has imported this pull request. If you are a Meta employee, you can view this in D127576443. (Because this pull request was imported automatically, there will not be any future comments.)

@8Keep 8Keep left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants