Apply @rpkgoptskill to farm some better implementations

[MichaelChirico]

cc @dshkol. Gemini output:

---

Performance Optimization Report for {potools}

Following the principles and workflow in r-pkg-optimizer, we profiled {potools} workflows, diagnosed the dominant bottlenecks across R and C source extraction and serialization, implemented targeted optimizations preserving all idioms and public API contracts, and verified semantic equivalence and test suite integrity.

---

1. Workload & Bottleneck Diagnostics

Line and stage profiling (Rprof, summaryRprof, bench::mark) on local package extraction (get_message_data(".")) and large multi-file packages (tools, utils, grid) revealed several hot spots:

1. Interval Overlap Overhead in C Parser (R/get_src_messages.R):
   • is_outside_char_array() and match_parens() constructed single-point data.table(start = char_pos, end = char_pos) instances and ran foverlaps() against character array boundaries. Because quote arrays are sorted and non-overlapping, binary search via findInterval() performs the exact same point-in-interval check without building trees or allocating tables.
2. Reverse Joins & Table Copying in R AST Parser (R/get_r_messages.R):
   • get_named_args() ran named_args[expr_data, on = c('file', id = 'parent'), arg_value := i.text], forcing data.table to iterate through all rows in expr_data (often 10,000–500,000+ rows) to match a handful of named arguments.
   • get_call_args() used grouped .SD filtering (if (.N == 1L || 'NS_GET' %chin% token) .SD[...]) over every function call AST node.
   • get_strings_from_expr() iteratively grew strings with rbind(..., fill = TRUE) in a while loop.
   • get_dots_strings() recomputed exclude_parents and repeatedly subsetted expr_data[!exclude_parents] even when empty.
3. Repeated Comment Extraction in Call Builder (R/get_r_messages.R):
   • msg[, by = c('file', 'line1', 'col1', 'line2', 'col2'), call := build_call(...)] grouped over every extracted message and ran comments[.(.BY$file, .BY$line1:.BY$line2), .SD, nomatch=NULL]. For single-line calls (which represent >90% of messages), the grammar guarantees no comments can appear within the expression columns (col1:col2).
4. Redundant Regex Scanning (R/get_r_messages.R):
   • clean_text() unconditionally executed 5 consecutive Perl regex lookarounds across every string constant in the package, even though strings containing \ are rare.
5. String Formatting in PO Writer (R/write_po_file.R):
   • make_src_location() used glue("{files}:{lines}") inside a loop; replaced with vectorized paste0(files, ":", lines).

---

2. Implemented Optimizations

All changes were made in-place across 3 files:

• R/get_src_messages.R:
  • Replaced foverlaps() in is_outside_char_array() and match_parens() with findInterval() over sorted non-overlapping array boundaries (144x faster in isolation, 0 allocations).
• R/get_r_messages.R:
  • Removed unused 5-column secondary index setindexv(expr_data, c("file", "line1", "col1", "line2", "col2")).
  • Streamlined get_call_args() to bypass .SD grouping and return early on zero matches.
  • Replaced full expr_data scanning in get_named_args() with direct subset query: expr_data[named_args, on = c('file', parent = 'id'), mult = 'last', x.text].
  • Replaced loop-growth rbind() in get_strings_from_expr() with a list collector and single rbindlist(str_list, fill = TRUE).
  • Vectorized single-line call substring extraction by file in get_r_messages(), evaluating comment ranges only for multi-line expressions.
  • Guarded regex unescaping in clean_text() with grepl('\\', x, fixed = TRUE).
• R/write_po_file.R:
  • Replaced glue("{files}:{lines}") in make_src_location() with paste0(files, ":", lines).

---

3. Verification & Correctness

1. Semantic Equivalence:
   Verified 100% identical outputs (all.equal() == TRUE) across potools and diverse packages:
   • potools (86 messages)
   • tests/testthat/test_packages/r_cat_msg (16 messages)
   • tests/testthat/test_packages/r_msg (6 messages)
   • tests/testthat/test_packages/r_non_template (10 messages)
   • tests/testthat/test_packages/r_fuzzy (4 messages)
   • tests/testthat/test_packages/custom_translation (5 messages)
   • tools (1,242 R messages, 146 src messages)
   • utils (1,007 R messages, 157 src messages)
   • grid (294 R messages, 60 src messages)
   • parallel (55 R messages)
2. Full Test Suite:
   Executed pkgload::load_all('.'); testthat::test_dir('tests/testthat'):
   • [ FAIL 0 | WARN 0 | SKIP 2 | PASS 146 ] (all 146 tests passing, duration reduced from 13.4s to 12.1s).

---

4. Reproducible Before/After Benchmarks

Environment: Linux (x86_64), R Under development (unstable 2026-08-15 r90413), data.table 1.18.4, potools 0.2.4.

get_r_messages() Latency

Workload / Package	Baseline (Mean)	Optimized (Mean)	Speedup
potools (24 R files, 86 msgs)	0.2935 s	0.2300 s	1.28x
tools (48 R files, 1,242 msgs)	1.7033 s	1.2509 s	1.36x
utils (38 R files, 1,007 msgs)	1.0686 s	0.7227 s	1.48x
grid (22 R files, 294 msgs)	0.3916 s	0.3032 s	1.29x

Isolated Helper Benchmarks

Operation	Baseline (1000x)	Optimized (1000x)	Speedup
is_outside_char_array()	2.753 s	0.019 s	144x
clean_text() (1,149 strings)	0.124 s	0.068 s	1.82x
build_calls (100x on potools)	3.622 s	2.919 s	1.24x

[MichaelChirico]

@dshkol most of the intermediate commits are stylistic (and so would be captured by having better R style skills on my end), but some have meaningful potential improvements to the SKILL, please have CC take a look and distill.

[MichaelChirico]

From the benchmark result comparing the initial implementation to now:

---

Across packages, the additional refinements (vectorized foverlaps(which = TRUE) for multi-line calls, is_direct coordinate fast-path & NS_GET anti-joins, grepl tab subsetting, and vectorized call extraction) delivered further latency improvements and consistent 7% to 15% memory allocation reductions:

Package	luvykuuw Median	Current Median	Speedup	luvykuuw Alloc	Current Alloc	Memory Reduction
base	850 ms	764 ms	1.11x	217.5 MB	187.5 MB	-13.8%
utils	1,373 ms	1,276 ms	1.08x	253.2 MB	216.7 MB	-14.4%
potools	244 ms	239 ms	1.02x	41.2 MB	35.1 MB	-14.8%
tools	2,109 ms	2,089 ms	1.01x	500.1 MB	453.5 MB	-9.3%
stats	3,504 ms	3,491 ms	1.00x	508.5 MB	471.3 MB	-7.3%
graphics	991 ms	987 ms	1.00x	117.9 MB	113.3 MB	-3.9%
data.table	2,772 ms	3,274 ms	0.85x*	453.6 MB	447.4 MB	-1.4%

*Note: data.table timing variability was within normal GC noise on C-side parsing; overall memory allocation decreased across all packages.

---

3. Summary of Refinements in Current Tree

1. get_call_args() Direct Fast-Path & Anti-Join:
   • Un-prefixed direct calls are detected via coordinate matching i.line1 == x.line1 & i.col1 == x.col1, allowing >95% of calls to skip querying child tokens.
   • For prefixed calls, replaced grouped if (.N == 1L || 'NS_GET' %chin% token) inside j with a vectorized NS_GET anti-join.
2. build_call Extraction & foverlaps:
   • comments is pre-keyed on c("file", "line1", "line2").
   • Single-line calls are extracted vectorized by file, restricting adjust_tabs to only lines with \t.
   • Multi-line calls use foverlaps(multi, comments, which = TRUE) with split(ov$yid, ov$xid) to map comment rows at C speed.
3. clean_text() Early Return:
   • Immediately returns early if no backslashes are present (!any(has_backslash)).