fix(keystore): fixed-width ECDSA r/s and X/Y encoding (leading-zero bug) + CI that actually runs - #17
Closed
amirradjou wants to merge 204 commits into
Closed
amirradjou wants to merge 204 commits into
amirradjou wants to merge 204 commits into
Conversation
Add CID Generation and Unit Tests for Entry Encoding
Refactor: Capitalize Log and Clock Structs for Exported Access
Added ID, clock, and Version to the entry and update the tests
Add Identity Support and Digital Signature to Entry Structure
Add orbit-sync
* Create go.yml * update go version format
big.Int.Bytes() strips leading zero bytes, so about 1 in 128 P-256
signatures (r || s) and 1 in 128 public keys (X || Y) came out 63 bytes
long. VerifyMessage split those at len/2 and ReconstructPublicKeyFromHex
rejected them outright, so ~1.5% of freshly created identities could not
verify their own signatures and every Log read path silently dropped the
affected entries ("Skipping entry with invalid signature", "Expected 3
Entries, got 2", "invalid public key length: 63").
- keystore: encode r/s and X/Y with big.Int.FillBytes into fixed
CoordinateSize (32-byte) halves; add EncodePublicKey as the single
encoder and make VerifyMessage / ReconstructPublicKeyFromHex require
the exact SignatureSize / PublicKeySize length instead of ">= 64,
split in the middle".
- identities, providers: use keystore.EncodePublicKey and
keystore.ReconstructPublicKeyFromHex instead of the ad-hoc
X.Bytes()||Y.Bytes() encoder and midpoint-split parsers.
- tests: 3000-iteration Create -> Sign -> Verify loops in keystore,
identities and providers, a 1000-identity oplog entry loop plus a Log
append/read-back loop, and deterministic vectors with a known
leading-zero X, Y, r and s (fixed-width verifies, legacy 63-byte form
is rejected). All of them fail against the old encoding.
Before: go test -count=50 ./keystore/ ./identities/... ./oplog/ -> FAIL
in identities, identities/providers and oplog (14 test failures).
After: all five packages ok, 0 failures.
- databases/database.go, syncutils/orbit-sync.go: drop the unused `mu`
mutex fields (U1000); the peerMap guard is tracked separately.
- databases/keyvalue.go: remove a no-op `.(interface{})` assertion (S1040).
- storage/ipfs_test.go: delete the unused encodeCBORMap helper (U1000).
`go run honnef.co/go/tools/cmd/staticcheck@latest ./...` is now clean so
it can gate in CI.
…, tidy
The workflow declared a three-OS matrix but ran everything on
`runs-on: ubuntu-latest`, ran `go mod tidy` as the install step (so a
dirty go.mod could never fail), used the deprecated upload-artifact@v3
and had no lint at all; the org repo has 0 recorded runs.
- lint job (Go stable): go mod verify, `go mod tidy -diff`, gofmt -l,
go vet, staticcheck and govulncheck via `go run ...@latest` so there is
no tool-install step to maintain.
- test job: `runs-on: ${{ matrix.os }}` on ubuntu/macos/windows at the
go.mod floor (1.23) plus ubuntu on stable, `go mod download`, build,
`go test -race -count=1 -coverprofile`, and a `-count=20` loop over
the keystore/identities/oplog packages so the fixed-width ECDSA
encoding cannot regress into a flake; upload-artifact@v4 with a
per-matrix artifact name.
- least-privilege `permissions: contents: read`, cancel superseded runs.
- .github/dependabot.yml for gomod (grouped) and github-actions, weekly.
Known red until follow-ups land: govulncheck reports GO-2026-5073 and
GO-2026-5684 (go-ipld-prime v0.21.0, reachable from oplog.Decode); the
fix, `go get github.com/ipld/go-ipld-prime@v0.23.0`, raises the go
directive to 1.25.7 and is left for a separate decision. The syncutils
TestSyncSendAndReceiveHead ordering flake under -race is tracked
separately as well.
amirradjou
force-pushed
the
hq/fix-leading-zero-encoding
branch
from
September 17, 2026 18:49
0180f05 to
4a5cd7f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the ~1.5% random signature / public-key verification failures that made the test suite flaky and silently dropped ~1-2% of log entries, then makes CI actually run and gate.
big.Int.Bytes()strips leading zero bytes, so about 1 in 128 P-256 signatures (r || s) and 1 in 128 public keys (X || Y) were encoded 63 bytes long.VerifyMessagesplit those atlen/2andReconstructPublicKeyFromHexrejected them (invalid public key length: 63), so freshly created identities could not verify their own signatures and everyLogread path (Get,Values,Traverse,JoinEntry) dropped the affected entries withSkipping entry with invalid signature.What changed
keystore/key-store.go): encoder/sandX/Ywithbig.Int.FillBytesinto fixed 32-byte halves (CoordinateSize,SignatureSize,PublicKeySize); newEncodePublicKeyis the single public-key encoder;VerifyMessageandReconstructPublicKeyFromHexnow require the exact length instead of "at least 64, split in the middle".CreateIdentityuseskeystore.EncodePublicKey;VerifyIdentityandIdentities.Verifyusekeystore.ReconstructPublicKeyFromHexinstead of their own midpoint-split parsers.keystore: 3000 fresh keys sign -> verify with fixed-width assertions; deterministic vectors with a known leading-zeroX,Y,rands(fixed-width verifies, the legacy 63-byte form is rejected); zero-padding unit test; malformed-length rejection.identities: 3000 identities Create -> VerifyIdentity -> Sign -> Verify; malformed public-key rejection.identities/providers: known short-X key throughCreateIdentity/VerifyIdentity; 3000-identity loop.oplog: 1000 fresh identitiesNewEntry->VerifyEntrySignature(+ CBOR round trip); 300Log.Append->Get/Valuesread-backs.mufields, an unused test helper, a no-op type assertion) so staticcheck can gate.runs-on: ${{ matrix.os }}(ubuntu/macos/windows at the go.mod floor 1.23, plus ubuntu on stable),go mod download+ build +go test -race -count=1 -coverprofile+ a-count=20loop over the crypto packages,upload-artifact@v4; a separate lint job withgo mod verify,go mod tidy -diff, gofmt,go vet, staticcheck and govulncheck viago run ...@latest;permissions: contents: read; concurrency cancel;.github/dependabot.yml(gomod grouped + github-actions).Verification (local, Go 1.26.8, linux/amd64)
Before (
upstream/main@ 1064195):After (this branch):
The
TestSignVerifyManyFreshKeyslog line from one run:3000 keys: 30 had a leading-zero coordinate, 25 signatures had a leading-zero r or s (all verified).Known red in the new CI until follow-ups land
go-ipld-prime v0.21.0, reachable fromoplog.Decode. The fix isgo get github.com/ipld/go-ipld-prime@v0.23.0 && go mod tidy, but v0.23.0 declaresgo 1.25.7, so it raises the module's Go floor from 1.23 (and the CI matrix version). Left out of this PR as a separate decision.TestSyncSendAndReceiveHeadunder-race: pre-existing ordering flake (peer-join event arrives onSyncedChbefore the Add payload), unrelated to this change.Note: the P-256
FillBytesfix is an explicit stopgap; the planned move to secp256k1 via go-libp2pcore/crypto(DER signatures, compressed keys) replaces this code entirely.Opened as a draft from the amirradjou fork; identical to amirradjou#1 (merged there). Happy to split the CI commit into its own PR if preferred.