perf(ioc): avoid allocation-heavy match deduplication - #590
Merged
Merged
Conversation
Replace the formatted-key HashSet dedupe with a direct comparison against the matches already collected, checked before any string of the new match is allocated. Hash observables are collected into a three-slot SmallVec. Extend benches/ioc_observables.rs with hash feeds, match_hashes hit and miss, one hit case per event-matched kind, and events checked by a hash-only feed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #526 and related to #436.
Replace formatted-string HashSet deduplication with direct comparison against the small list of IOC matches already collected. Duplicate matches are rejected before allocating their owned strings. Hash observables now use an inline three-entry SmallVec, one slot per supported algorithm.
The IOC benchmark now covers event misses, hits for every event-matched indicator kind, computed hash hits and misses, and events evaluated by a hash-only feed.
Type of change
feat/enhancement- new featureperformance- performance improvementbug- bug fixrefactor- refactoring, no behaviour changedocumentation- docs onlyci- CI and release changesdependencies- dependency updatechore- other maintenancebreaking-change- breaking changeskip-changelog- release preparation or changes with no release-note valueTest plan
cargo test ioc --all-featurescargo clippy --all-targets --all-features -- -D warningsChecklist