Repository navigation
Look up str.strip chars in a set instead of rescanning them - #259
Closed
Dilshad-Tanvi wants to merge 1 commit into
Closed
Dilshad-Tanvi wants to merge 1 commit into
Dilshad-Tanvi wants to merge 1 commit into
Conversation
strip, lstrip and rstrip tested each character with `chars.contains(c)`, which scans the whole `chars` argument every time, so stripping cost O(len(S) * len(chars)). Build the set once per call instead: a bitmap for ASCII and a sorted Vec for the rest. Results are unchanged.
meta-cla
Bot
added
the
CLA Signed
Contributor
|
This pull request has been imported. If you are a Meta employee, you can view this in D128326955. (Because this pull request was imported automatically, there will not be any future comments.) |
Contributor
|
Super cool! I'll get this merged. Very nice work. |
meta-codesync Bot
pushed a commit
to meta/buck2
that referenced
this pull request
Oct 30, 2026
Summary:
`str.strip`, `lstrip` and `rstrip` with a `chars` argument test each character with `chars.contains(c)`, which scans all of `chars` every time, so stripping is O(len(S) * len(chars)). A string made of characters that only appear at the end of a long `chars` makes every test scan the whole argument. Starlark has no unbounded loops or recursion, so builtins are expected to cost about linear in their input, but this script ties up an evaluator for seconds:
```python
n = 320000
("é" * n).strip("ĩ" * n + "é")
```
Release `starlark` binary:
| n | before | after |
|---|---|---|
| 80000 | 0.34s | <0.01s |
| 160000 | 1.36s | <0.01s |
| 320000 | 5.42s | 0.01s |
| 5000000 | ~22 min (extrapolated) | 0.04s |
The ASCII form `("a" * n).strip("b" * n + "a")` takes 2.80s at n=320000 before and under 0.01s after.
The set is now built once per call: a 128-bit mask for ASCII and a sorted, deduplicated `Vec<char>` searched with `binary_search` for the rest, so the cost is O(len(S) log len(chars) + len(chars)) and an all-ASCII `chars` allocates nothing. starlark-go goes through Go's `strings.Trim`, which is linear for an ASCII cutset; CPython is quadratic here too, so this is faster than Python rather than matching it. These three methods are the only per-character membership scans over a user string in the workspace. The new test covers ASCII at both ends of the range, non-ASCII, duplicates and empty arguments, and passes both before and after the change since results are unchanged. `cargo test` and `cargo clippy` pass.
X-link: meta/starlark-rust#259
Reviewed By: ndmitchell
Differential Revision: D128326955
fbshipit-source-id: c57b1322064bcad9ad32a7ecd568ae433549043f
Contributor
|
This pull request has been merged in 6f222f5. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
str.strip,lstripandrstripwith acharsargument test each character withchars.contains(c), which scans all ofcharsevery time, so stripping is O(len(S) * len(chars)). A string made of characters that only appear at the end of a longcharsmakes every test scan the whole argument. Starlark has no unbounded loops or recursion, so builtins are expected to cost about linear in their input, but this script ties up an evaluator for seconds:Release
starlarkbinary:The ASCII form
("a" * n).strip("b" * n + "a")takes 2.80s at n=320000 before and under 0.01s after.The set is now built once per call: a 128-bit mask for ASCII and a sorted, deduplicated
Vec<char>searched withbinary_searchfor the rest, so the cost is O(len(S) log len(chars) + len(chars)) and an all-ASCIIcharsallocates nothing. starlark-go goes through Go'sstrings.Trim, which is linear for an ASCII cutset; CPython is quadratic here too, so this is faster than Python rather than matching it. These three methods are the only per-character membership scans over a user string in the workspace. The new test covers ASCII at both ends of the range, non-ASCII, duplicates and empty arguments, and passes both before and after the change since results are unchanged.cargo testandcargo clippypass.