Skip to content

Reduce chain generation and loading time - #625

Merged
qdeslandes merged 6 commits into
meta:mainfrom
qdeslandes:generation-optimizations
Oct 29, 2026
Merged

qdeslandes merged 6 commits into
meta:mainfrom
qdeslandes:generation-optimizations

Conversation

@qdeslandes

Copy link
Copy Markdown
Contributor

Summary

Reduce the work done to generate and load a chain, and add a benchmark to measure it.

  • Add a benchmark that measures bf_chain_set() on a chain of 1000 rules with counters. 4 rules match ip4.saddr against sets of 8, 65,536, or 1,048,576 elements. The other 996 rules each match one random ip4.saddr, and one of them logs. The benchmarks now set the log level to warning, so bpfilter's info logs don't appear in the benchmark output.
  • bf_cgen_load() calls bf_cgen_dump() only when debug logging is enabled. Before, every load called it, whatever the log level.
  • Request verifier logs only when a log buffer is provided. Before, every program load set log_level = 1, even without a buffer. A load that fails without logs is retried with a log buffer, so the error still includes the verifier output.
  • Copy chains and sets directly instead of serializing and deserializing them. The chain API copies the caller's chain before generating its program. New bf_rule_new_from_copy() and bf_matcher_new_from_copy() copy rules and matchers.
  • Cache successful BTF ID lookups by name until bf_btf_teardown(). The code generator looks up a BTF ID for every kfunc call it emits.

Related issue

None. The new benchmark measures how long bf_chain_set() takes to generate and load a large chain, and the other commits remove work from that path.

Testing

Full test suite passed.

Benchmark results before and after the changes:

Benchmark Before After Δ Δ%
chain_set__ip4_saddr__x_elem_set/8 10.32 ms 2.44 ms -7.88 ms -76.4%
chain_set__ip4_saddr__x_elem_set/65536 68.59 ms 55.01 ms -13.58 ms -19.8%
chain_set__ip4_saddr__x_elem_set/1048576 1551.22 ms 1445.98 ms -105.24 ms -6.8%
chain_set__1000_ip4_saddr__x_elem_sets_c_l/8 974.49 ms 832.14 ms -142.35 ms -14.6%
chain_set__1000_ip4_saddr__x_elem_sets_c_l/65536 1120.33 ms 965.03 ms -155.30 ms -13.9%
chain_set__1000_ip4_saddr__x_elem_sets_c_l/1048576 5030.59 ms 4520.33 ms -510.27 ms -10.1%

Notes for the reviewer

  • bf_btf_setup() now fails with -EBUSY when BTF is already loaded. Before, a second call replaced the loaded BTF without freeing it.
  • Without debug verbosity, a failed program load runs the verifier twice: first without logs, then with a log buffer to report the verifier output.

AI disclosure

Peer programming with Claude.

Checklist

  • I understand every line in this PR and can explain why it is correct
  • I built the project and ran the tests covering this change
  • make -C $BUILD test_bin test passes and the code follows the style guide
  • Commits are formatted as component: subcomponent: short description
  • I have disclosed any AI usage above

Prevent bpfilter from polluting the benchmark output by increase the
default log level during benchmarking to warning.
The userspace benchmarks currently focus on sets loading time (single
rule, large set). This change introduces a new benchmark to evaluate the
full generation and loading duration for a larger chain.

The benchmarked chain has 1000 rules, 4 sets of various size, uses
logging and counters. This will be used as a baseline for reducing the
duration of the program generation.
Replace the serialization round trip with direct copies of chains and
sets to avoid temporary pack buffers and serialization overhead.

Add copy constructors for rules and matchers to preserve configuration
and independently owned payloads. Reserve space for sets and their
elements before copying them.

Preserve the existing reset of runtime counters. Use the chain API to
add copied rules, validate set references, and recalculate derived
state. Leave the destination unchanged if copying fails.
@meta-cla meta-cla Bot added the cla signed label Oct 29, 2026
@qdeslandes
qdeslandes merged commit 8003e13 into meta:main Oct 29, 2026
38 checks passed
@qdeslandes
qdeslandes deleted the generation-optimizations branch October 29, 2026 13:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant