Skip to content

Benchmarks and optimisations - #45

Merged
PyXiion merged 9 commits into
mainfrom
opt/mod-discovery
Jul 29, 2026
Merged

Benchmarks and optimisations#45
PyXiion merged 9 commits into
mainfrom
opt/mod-discovery

Conversation

@PyXiion

@PyXiion PyXiion commented Jul 29, 2026

Copy link
Copy Markdown
Owner

Optimised mod discovery

  1. find_about_xml — fast-path check About/About.xml with Path.is_file() before falling back to full directory iteration (covers the vast majority of real mods)
  2. scan_mod_directory — replace Path.iterdir() with os.scandir(), return dict[Path, Path] instead of list[Path] to cache each mod's About.xml path during the scan
  3. create_listed_mod_from_path — accept about_xml_path parameter to skip redundant find_about_xml call; callers (all three providers) pass the pre-found path from the scan
Before After Δ
1000 mods total 265.5ms 203.5ms -23%
2000 mods total 496.7ms 407.9ms -18%
scan avg 46.2 µs/mod 26.7 µs/mod -42%

Optimised topological sort

Replaced per-iteration list.sort() + list.pop(0) in Kahn's algorithm with heapq push/pop (O(n² log n) → O(n log n)), and flattened _build_deps_and_rev_deps from three edges_of_type() calls per node to a single outgoing() pass with inline dispatch.

Phase Before After Δ
build_deps 2.5ms
kahn_tier_2 54.1ms 2.1ms -96%
sort total (750 mods) 68.5ms 10.7ms -84%

Optimised XML parsing

Replaced the generic etree_to_dict recursive dict converter with a single-pass Element walker (_create_about_mod_from_element) that builds AboutXmlMod directly from lxml children, eliminating the intermediate nested dict and all value_extractor calls.

Pipeline change:
Before: ET.parseetree_to_dict → dict → value_extractorcreate_about_modAboutXmlMod
After: ET.parse_create_about_mod_from_elementAboutXmlMod

Before After Δ
Model construction (1000 mods) 41.4ms 25.4ms -39%
Full discovery parse (1000 mods) ~177ms 119ms -33%
parse avg ~177 µs/mod 119 µs/mod -33%

Benchmarks

  • tests/benchmarks/bench_sort.py — synthetic sort pipeline benchmark (200–2000 mods, community rules)
  • tests/benchmarks/bench_discovery.py — mod discovery scan + parse benchmark
  • Both run via just bench (all) or just bench <name> (filtered)

@PyXiion PyXiion added this to the 0.7.0 milestone Jul 29, 2026
@PyXiion PyXiion added enhancement New feature or request dependencies Pull requests that update a dependency file labels Jul 29, 2026
@PyXiion PyXiion self-assigned this Jul 29, 2026
@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 0 complexity · 0 duplication

Metric Results
Complexity 0
Duplication 0

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

PyXiion added 4 commits July 29, 2026 04:58
Replace the generic etree_to_dict recursive dict converter with a
single-pass Element walker that builds AboutXmlMod directly from
lxml children, eliminating the intermediate nested dict and
value_extractor calls.

Pipeline change:
  Before: ET.parse -> etree_to_dict -> dict -> value_extractor ->
          create_about_mod -> AboutXmlMod
  After:  ET.parse -> _create_about_mod_from_element -> AboutXmlMod

Microbenchmark (model construction only, 1000 mods):
  41.4 us/mod -> 25.4 us/mod  (1.6x faster)

Full discovery benchmark (1000 mods, incl. file I/O):
  157-197ms  -> 119ms  parse_xml
@PyXiion
PyXiion merged commit 4054e7f into main Jul 29, 2026
4 of 5 checks passed
@PyXiion
PyXiion deleted the opt/mod-discovery branch July 29, 2026 03:40
PyXiion added a commit that referenced this pull request Jul 29, 2026
Merge main's benchmarks & optimisations (PR #45) into feat/plugins, resolving
conflicts in main_window.py and justfile:

- main_window: keep HEAD's simple _auto_sort wrapper around
  CoreContext.auto_sort() over incoming UI-level duplication
- justfile: keep HEAD's build-js recipe + variadic build args, add
  bench recipe from main
- context: log sort time in ms instead of seconds
- packaging/build: add --nofollow-import-to=pytest,pygments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Startup and discovering 1341 mods takes 0.8s. Too slow Performance benchmark suite

1 participant