Make normal single-order signing ~30× faster (1.95 ms → 63.9 µs)

[josusanmartin]

Result: ~30× faster normal single-order signing

Normal order latency: 1.95–2.02 ms → 63.9 µs

Sequential signing throughput: roughly 500 → 15,700 orders/sec

That is approximately 96.7% less latency without changing how callers sign an order.

This is the existing one-order-at-a-time SignCreateOrder / Python sign_create_order path. It requires no batching and no prepared nonces. The headline improvement is the normal path; batching and nonce preparation are optional secondary optimizations.

The 1.95–2.02 ms baseline is the original warm end-to-end caller measurement on this host. The 63.9 µs result is the public Python signing call over 7 × 1,000 sequential orders on the same host. Against the bundled signer's directly measured 1,644 µs public Python path, the new path is still 25.7× faster.

Important

The main C-to-Go callback speedup depends on Go's private _cgo_getstackbound hook via //go:linkname. This is unsupported runtime internals, has no compatibility guarantee, and must be explicitly accepted and revalidated on every Go toolchain upgrade. The implementation is pinned to the repository's Go 1.23 toolchain and disabled by a go1.26 guard until that version is audited.

Summary

Speed up the normal, one-order signing path exposed by the shared library. This is the primary change; callers do not need to batch or precompute anything to benefit.

• use the optimized fixed-generator Schnorr implementation from the linked Poseidon PR
• bypass an unused hash.Hash allocation in the built-in signer
• cache the public key and warm the fixed-generator table during client setup
• cache exact foreign-thread stack bounds around C-to-Go callbacks on supported libc implementations
• retain the public single-order SignCreateOrder API

On an Intel Xeon E-2286G (Go 1.23, CPython 3.12, glibc, main thread), the complete public Python sign_create_order call is 63.9 µs/order (median, 7 × 1,000 orders), about 15.7k orders/sec. The previous bundled signer measured 816.5 µs/order even after separately applying direct POSIX response cleanup, and 1,644 µs/order with its normal exported-Go cleanup.

The private-hook contribution is platform and thread dependent. In a no-op C-to-Go callback microbenchmark on this host, main-thread callback cost changed from about 279.5 µs to 1.54 µs; a worker-thread run changed from about 3.18 µs to 1.52 µs. macOS already has an O(1) runtime lookup, while Windows and libc fallbacks without exact pthread bounds intentionally do not install this cache.

The cache stores exact bounds per native thread and rechecks that the current stack pointer remains inside them on every callback. A coroutine or alternate stack falls through to the original runtime lookup. Setup uses atomic acquire/release publication, is idempotent, and has a version-gated no-op fallback.

Secondary improvements

Prepared nonces

An opt-in process-global pool can move Schnorr commitment generation outside a signing burst. Normal signing does not use it unless the pool is explicitly filled. Prepared single-order signing measured 14.8 µs/order on the same host. Nonces are single-use, process-bound, memory-only, and must not be serialized or shared across a fork.

Explicit batch API

SignCreateOrdersBatch signs independent orders with consecutive explicit nonces using up to GOMAXPROCS workers and returns one packed allocation. It is only used when the caller invokes the new batch API; a normal single SignCreateOrder call is never routed through batching.

For batches of 1,000 orders, the public Python path measured:

Mode	Median	Throughput
normal batch	10.5 µs/order	94.9k/sec
prepared batch	3.24 µs/order	308k/sec

Batch input is capped at 10,000, requires a non-negative explicit first nonce, rejects nonce overflow, uses the client's resolved account/API-key indices, and records per-order errors without corrupting adjacent results.

Dependency / merge order

This draft temporarily pins the Poseidon fork commit with a pseudo-version. Please merge and tag the linked Poseidon change as v0.0.19, bump the github.com/elliottech/poseidon_crypto requirement to v0.0.19, delete the temporary fork replace, and only then merge this PR.

The dependency pin is intentionally isolated in the final commit, so only the branch tip builds; the preceding implementation commit by itself still sees the old Poseidon API.

• Poseidon speedup: elliottech/poseidon_crypto#38
• Python wrapper: elliottech/lighter-python#165

Validation

• go test ./...
• go test -race ./...
• go vet ./...
• Linux c-shared build and generated-header/C++ consumer build
• Windows amd64 c-shared cross-build with MinGW and C11 atomics
• stack-cache idempotence and callback tests
• batch defaults, bounds, mixed-error, packing, and 2,000-order race/uniqueness tests