[FEATURE] GPU/Hardware Acceleration

[cmfc31]

Description

Hi Boris! I love your program but I think it's missing a critical feature.

1. GPU/Hardware Acceleration

What problem does this solve?

1. Currently users depend 100% on CPU to generate screenshots and video preview. Users with GPU can't take advantage of their hardware to accelerate these processes. Working to support something like this will make the Hub generation way faster to users with capable hardware.

Proposed Solution

1. For GPU acceleration, not sure if will be possible to implement due to the electron framework.

[whyboris]

Thank you @cmfc31 for the suggestion. The screenshot extraction depends entirely on FFmpeg

I don't know enough, but it seems like, from my quick research, "to extract a screenshot from a video using GPU acceleration with FFmpeg, you need a version of FFmpeg compiled with the appropriate hardware acceleration support" 🤔

I could play around to see if there's a way to get an Nvidia FFmpeg and if it works faster on my machine than the CPU version. If so, perhaps I could do two different PC releases 🤔

[fireworksordie]

Apologies for just emailing you moments ago about this same enhancement before even considering there would be a github!

VideoDuplicateFinder has hardware acceleration (both for discrete and iGPU) implemented - maybe worth peeking at how they've done it?

[whyboris]

Thank you @fireworksordie - I'll glance at the repository you shared to see if I can improve VHA 👍

[whyboris]

Looking forward to trying it soon 😊 with #805 🚀

[cmfc31]

Looking forward to trying it soon 😊 with #805 🚀

Hi @whyboris! I implemented this change on my fork, but after testing GPU-accelerated FFmpeg (hardware decode + hardware encode) on Windows/AMD, total import time barely improved. The main reason is how filmstrips are generated.

How filmstrips work today

For each video, the app builds one FFmpeg command with N separate inputs of the same file (-ss + -i repeated N times), then runs a filter_complex that scales/pads each frame and **hstack**s them into a single JPEG.

Why the GPU doesn’t help much here

• GPU only accelerates decode. Scale, pad, color conversion, hstack, and JPEG output all still run on the CPU.
• Frames are copied back from GPU to CPU before filtering, so decode savings are partially lost.
• Cost scales with frame count, more screenshots means more seeks, more decodes, and more filter work on the same file.
• Filmstrips don’t re-encode video, there’s no heavy encode step for the GPU to offload (unlike preview clips, where hardware H.264 encoding helps more).

Other limits on overall speed

• Videos are processed one at a time (thumbQueue concurrency = 1).
• Each video runs 4 sequential FFmpeg jobs: thumbnail → filmstrip → preview clip → clip poster.
• Disk I/O (especially on large/4K files or slow drives) adds overhead on top of the repeated seeks per filmstrip.

Bottom line

Filmstrip generation is CPU- and I/O-bound by design, not decode-bound. GPU acceleration helps individual decode/encode steps, but the filmstrip pipeline’s architecture, repeated input seeks, CPU-only filters, and serial processing, is why overall hub generation didn’t feel much faster.

---

Extra Features (User-Facing)

Here's a list of extra experimental features and I added to my fork. I wanted higher resolution and better quality thumbnails/filmstrips.

1. Higher-Quality FFmpeg Output

• JPEG thumbnails/filmstrips use quality level 1 (best).
• Preview clips use H.264 with CRF 20, slow preset, and 160k AAC.
• Optional AMD hardware encoding (h264_amf) when available.
• README section on how to regenerate thumbnails, filmstrips, and clips after upgrading.

2. Binary vs Decimal File Size Toggle

• New settings button: Windows-style binary sizes (1024-based, matches Explorer) vs decimal (1000-based).
• Shared utility: interfaces/file-size.util.ts.
• Updates all displayed file sizes when toggled; i18n strings in all locale files.

3. GPU Hardware Acceleration (Windows)

• Auto-detects d3d11va or dxva2 hardware decode.
• Uses GPU decode first, with automatic CPU fallback on failure.
• Prefers AMD H.264 encoder when available.

4. HDR / Color Correction Detection

• Detects HDR sources via ffprobe (smpte2084, arib-std-b67).
• Tone-maps HDR → SDR before generating JPEGs/previews.
• Correct color range/matrix handling for SDR vs HDR in scale/pad filters.

5. Missing Assets Regeneration + Analytics (Statistics Panel)

• New Missing assets section in Statistics with counts for:
  • Videos checked / videos with missing assets
  • Missing thumbnails, filmstrips, clips, clip thumbnails
  • Total missing assets
• Generate missing assets, re-extracts only what's missing (not a full re-import).
• Refresh stats button with loading state.
• IPC handlers: request-missing-assets-stats, missing-assets-stats.

6. Orphaned Asset Cleanup

• When generating missing assets, also removes thumbnails/assets for videos no longer in the hub.
• Refactored hash collection into getLiveVideoHashes() (excludes deleted items and folders).

7. Custom Title Bar Branding

• Window title changed from Video Hub App 3 to VHub 3 :: Neko's mod.

Build & Packaging Improvements

The building process was a failing on Windows, so I applied some changes to fix this.

8. Windows Build Fixes

• Updated electron-builder.json, package.json, and dependencies for reliable Windows builds.

9. FFmpeg/FFprobe Bundling for Packaged Apps

• New node/ffmpeg-paths.ts resolves executables correctly in dev and packaged builds.
• Handles ASAR unpacked paths and materializes binaries to a temp dir when needed.
• Cross-platform bundling in electron-builder.json.

10. Custom Windows Build Script (build.ps1)

• Admin-elevated PowerShell script.
• Clears winCodeSign cache, removes old release/ output, runs npm run electron.
• Fixed app/installer icons (src/favicon.ico).

11. Build Size Optimization

• Trimmed electron-builder.json config and refined build.ps1 to reduce installer size.

12. App Icon Handling

• Window icon path logic in main.ts (dev vs production).
• NSIS installer icons configured.

Commit Timeline

Date	Commit	Focus
May 26	02f516c	High-quality FFmpeg params
May 26	9f3b67a	Binary file size toggle
May 27	3f1e491	GPU acceleration + HDR detection
May 27	f7be25a	Missing assets UI + analytics
May 29	2b6bb11	Windows build fixes
May 29	903a13b	Custom dev logo (later removed)
May 31	67c4375	FFmpeg bundling for packaged app
May 31	a193404	Title bar rename
May 31	542ba77	Custom build script + icon fix
May 31	e200123	Build size optimization
May 31	1147afe	Delete orphaned assets on missing-assets check

[whyboris]

Hey @cmfc31 - amazing work 🤩

Thank you for diagnosing screenshot generation and why hardware acceleration might not speed things up ❤️

I'll look into your code some more and see what parts I might use for myself 🙇‍♂️

I spent a few days cleaning up my codebase - updating dependencies, etc. I have a PR to allow for nvidia acceleration: #805 that I just polished up a few minutes ago.

I would love for FFmpeg to generate a filmstrip without having to re-open the file many times, but I'm unsure if there's a way to get the filmstrip letterboxed to the VHA display aspect ratio. I'd love feedback if you happen to know 🤝 the current extraction command is really messy 😓

[whyboris]

I ran some performance tests but as far as I understood the outcome, including -hwaccel cuda in the FFmpeg command ends up increasing the time it takes to extract screenshots! 😅 -- perhaps because the screenshots get bussed from video card to CPU (as you said) whatever speedup occurs during decoding is wiped out by the other process.

I even tried an additional -hwaccel_output_format cuda but that resulted in screenshots not being generated at all 😝