feat(qairt): enable HTP multicore for QAIRT bundles - #1313
Open
MAN$I VERMA (mansiverma897993) wants to merge 2 commits into
Open
feat(qairt): enable HTP multicore for QAIRT bundles#1313MAN$I VERMA (mansiverma897993) wants to merge 2 commits into
MAN$I VERMA (mansiverma897993) wants to merge 2 commits into
Conversation
MAN$I VERMA (mansiverma897993)
force-pushed
the
feat/qairt-htp-multicore
branch
from
August 9, 2026 08:27
5a73be2 to
e69ce6d
Compare
Bumps the geniex-qairt submodule to the commit that plumbs a core-count knob (ModelConfig::num_cores, derived from the bundle htp_backend_ext_config.json devices[].cores list) through to QNN_HTP_GRAPH_CONFIG_OPTION_NUM_CORES on every loaded graph, logs device-reported and effective core counts at init, and warns on clamped or rejected requests instead of staying silent. Points at the upstream merge commit of qualcomm/geniex-qairt-plugin#31 (7aedc3e) on geniex-qairt-plugin main. Fixes qualcomm#1312 Signed-off-by: mansiverma897993 <vmansi756@gmail.com>
Covers the htp_backend_ext_config.json cores knob, what is settable at load time versus fixed at context-binary generation time, the init-time core-count logging, and the multicore-unavailable driver diagnostic. Signed-off-by: mansiverma897993 <vmansi756@gmail.com>
MAN$I VERMA (mansiverma897993)
force-pushed
the
feat/qairt-htp-multicore
branch
from
August 12, 2026 00:16
e69ce6d to
fb70b71
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
third-party/geniex-qairtto pick up HTP multicore support (feat(core): request HTP multicore execution via NUM_CORES graph config geniex-qairt-plugin#31):ModelConfig::num_coresauto-derives from the bundle'shtp_backend_ext_config.jsondevices[].coreslist and is applied to every loaded graph viaQNN_HTP_GRAPH_CONFIG_OPTION_NUM_CORES. All load paths honor the bundle knob — GenieX CLI/SDK (QairtLlm::createviamodelConfigFromDirectory), plugin example executables, and VLM vision encoders — with no FFI changes.Multicore support is unavailable).notes/run.mdand the plugin'sdocs/README.md.Depends on qualcomm/geniex-qairt-plugin#31 — the submodule pointer currently references that PR's head and should be retargeted to the upstream merge commit once it lands. A typed
geniex_LlmConfigfield + CLI flag can follow as a separate FFI-touching PR (per the sliding-window precedent in a10cdc7) if maintainers want a programmatic override on top of the bundle knob.Test plan
/std:c++20) against the vendored QNN headersllm_spec_loader_testunit suite (10 cases) passes locally — two runs, including after the final revisionModel::applyHtpNumCores(4 cases: JSON fallback, explicit override, no-clamp, default) passes against the link-time QnnApi stubTest report (local verification + on-device instructions for hardware testers)
Test report — HTP multicore support (#1312)
What changed
ModelConfig::num_cores(new): HTP cores to request per graph.0(default) = auto-derive from the bundle'shtp_backend_ext_config.jsondevices[].coreslist (parseHtpCoreCount) — viamodelConfigFromDirectoryor, for hand-built configs (example executables, embedders), at init fromhtp_config_path.1= force single core. Any explicit value overrides the JSON. The GenieX SDK path (QairtLlm::create) picks it up with no SDK/FFI changes, and every load path (CLI, SDK, plugin examples, VLM vision encoders) honors the same bundle knob.Model::initialize→applyHtpNumCores: logs the device-reported NSP core count, clamps requests above it with a warning, appliesQNN_HTP_GRAPH_CONFIG_OPTION_NUM_CORESto every loaded graph viaQnnGraph_setConfig(between context creation and first execute — the only valid window for graphs retrieved from prebuilt context binaries), and warns without failing init when the backend rejects the config.QnnApi::getHtpDeviceNumCores/QnnApi::setHtpNumCores(new QNN wrapper methods).docs/README.md§ "HTP Backend Config and Multicore Execution" (JSON schema, load-time vs generation-time split, theError code 1000 … key = 304diagnostic).Verified locally (no NPU required)
Host: Windows 11 x64, MSVC 19.29 (VS2019 BuildTools),
/std:c++20, vendored QNN headers.core/src/llm/llm_spec_loader.cppcore/src/model_init.cppqnn-api/src/QnnApi.cppllm_spec_loader_test(10 cases)Model::applyHtpNumCoresagainst the link-time QnnApi stubcore/+tests/filesUnit cases: missing file / malformed JSON / missing
devices/ non-arraydevices/ device withoutcores→ 0; single-core → 1; dual-core → 2; max across multiple devices → 3;modelConfigFromDirectorywiresnum_coresfrom the bundle JSON; defaults to 0 without the JSON.Behavioral probe (drives the exact init path hardware will hit, with the stub reporting
device cores = unknownand rejecting the graph config):num_cores=0+ 2-core JSON → init-time fallback parses the JSON, attempts multicore, logs the reject warning, init continues — ✅num_cores=1explicit + 2-core JSON → forces single core, never touches the backend — ✅num_cores=5explicit, device count unknown → no clamp, reject warning names 5 — ✅num_cores=0→ single-core default log only — ✅Design note: the knob is deliberately bundle-JSON + typed
ModelConfigonly — no env var. GenieX previously shippedGENIEX_SLIDING_WINDOWas an env var and maintainers replaced it with typed config (a10cdc70 fix(sdk)!: move sliding-window opt-in from env var to typed config); a typedgeniex_LlmConfigfield + CLI flag can follow as a separate FFI-touching PR if desired.Needs on-device validation (Snapdragon / QDC)
Not verifiable on x64 hosts — please run on real HTP hardware (e.g. via QDC):
GENIEX_LOG=info(CLI:--log info). Expect:htp_backend_ext_config.jsonto list a second entry indevices[0].cores({"core_id": 1, "perf_profile": "burst", "rpc_control_latency": 100}). Expect a clamp warning (Requested 2 HTP cores but the device exposes 1; clamping to 1) and a normal single-core run — no crash, no init failure. This works identically through the GenieX CLI (geniex infer), the SDK, and the plugin's example executables — all load paths honor the bundle JSON.numCores > 1(certain automotive/IoT parts). ExpectHTP multicore enabled: graphs will execute on 2 cores; verify output correctness and compare prefill/decode tok/s vs single core.Error code 1000 ... key = 304/Multicore support is unavailable, init must still succeed with the new warning (backend rejected QNN_HTP_GRAPH_CONFIG_OPTION_NUM_CORES; continuing on a single core).Note: context binaries compiled without a multicore graph config may not speed up even when the config is accepted — see the new docs section for the generation-time vs load-time split.
Closes #1312