Make Android scheduling load-aware

[ncylich]

Summary

Make Android CPU scheduling load-aware for production inference on top of current v2.

Key changes:

• Pin Android threadpool workers one-per-selected performance core instead of giving every worker the same broad performance-core mask.
• Split blocking Android parallel_for work into smaller fixed internal chunks so faster cores can pull additional work.
• Prepare production inference entrypoints by selecting a load-aware performance core for the caller thread.
• Initialize the worker pool before narrowing caller affinity so worker threads do not inherit a single-core caller mask.
• Preserve each caller thread's original allowed affinity and restore it before each load-aware selection, so repeated calls can move away from a newly busy core.
• Keep the policy internal and deterministic: no public API changes, no required environment variables, and no debug logging.

Base

This branch is based directly on current v2 after the orthogonal CQ4 LM-head layout PR merged. The PR diff against v2 is limited to:

• cactus-engine/src/complete.cpp
• cactus-kernels/src/threading.h

Rationale

The LM-head kernel optimization recovered the kernel-level cost, but Android end-to-end performance was still sensitive to where the OS placed the caller thread and how evenly worker work was distributed across asymmetric cores.

The previous default behavior could leave important work on a slow or contended core. A naive main-thread pin helped in benchmarks, but it was not robust under contention. This PR makes the production path use the same general idea safely: workers stay distributed across selected performance cores, blocking work is chunked so faster cores naturally take more work, and the caller thread is placed using recent per-core load rather than a fixed core choice.

Implementation Details

Worker scheduling:

• Detect Android core capacities from sysfs using the existing topology detection path.
• Limit the worker pool to selected performance cores.
• Pin each worker to an individual selected performance core.
• Use the existing threadpool for blocking Android parallel_for calls and divide each requested worker assignment into a fixed number of smaller tasks.

Caller scheduling:

• Keep each caller thread's original allowed affinity in a thread-local state object.
• Initialize the worker pool before caller pinning to avoid worker threads inheriting the caller's narrowed affinity mask.
• Restore the caller thread's original allowed affinity before each selection so prior single-core pinning does not restrict future choices.
• Take two short /proc/stat samples around a 20 ms interval to estimate recent per-core busy fraction.
• Score candidate performance cores by capacity adjusted for recent load, then pin the caller to the best allowed core.

Validation

Build and native tests after the cleanup:

• cactus build
• cactus build --android
• ./cactus-graph/build/cactus-kernels/test_matmul
• ./cactus-graph/build/test_ops
• git diff --check

Pixel sanity check after the cleanup:

• Generation on explain shrodingers cat produced coherent output.
• Gemma 512 benchmark: 10.73 tok/s decode, 69.06 tok/s prefill.
• Verified no leftover Pixel cactus_chat, cactus_llm_bench, or cpu_contention processes after the run.

Earlier contention testing also verified the main bug: pinning the caller before worker-pool initialization could cause all workers to inherit a one-core affinity mask. Initializing the pool first fixes that collapse while preserving load-aware caller placement.