report_auth_failure: optional consumer-supplied detail token for observation provenance #26

Activity

1. ckcred-alfonso commented on Aug 31, 2026

   

   ckcred-alfonsoBot

   on Aug 31, 2026

   More actions

   The need is real and the provenance gap is exactly as you describe: every consumer_report and consumer_report_stale row on my live store carries detail = NULL, so a report's origin is unrecoverable today. Two objections to the shape, one of which is a hole in the validation rather than a matter of taste.

   The regex does not buy the property it is there to buy

   ^[a-z0-9_]{1,64}$ admits 64 characters of lowercase hex, which is a canonical secret shape — session cookies, hex digests, and a good share of API keys are exactly that. The migration-3 rule is "never provider body text, because an error body can echo submitted parameters", and the regex defends against punctuation-bearing prose while passing the one shape that most looks like a token.

   So the validation stops the accident it was drawn against and not the accident that matters. A closed set does: if the accepted values are enumerated, token material is unwritable by construction rather than by a character-class argument.

   This also matters for the fallback you propose. Writing detail = Some("invalid_detail_rejected") is the right instinct — a refusal that records itself — but it means the rejected value has already been received and classified, and the reason to reject it is that it might be a secret. A closed set never holds one.

   detail is a vault-observed column, and the claim is consumer-asserted

   Live store, every non-null value:

   transport      refresh_failed        524
   invalid_grant  refresh_failed          2
   decode         refresh_failed          1
   no_grant       scoped_read_refusal     2
   

   All four are variant names from enums this repo owns, produced by RefreshError::variant_name() and the scoped-refusal reason. Since 12a1b4c each is pinned by an exhaustive match, so adding a variant is a compile error until it is documented.

   "direct_401" versus "relay_upstream_error_parse" is a different kind of statement. It is a consumer's claim about itself, which the vault cannot check — nothing in a report distinguishes an honest relay-parse label from a mistaken one. Both are useful; they are not the same evidence, and a reader of one column cannot tell which they are holding.

   That is the same defect twice over in this repo's recent history: account_id carrying either a live token parse or a stored login-time value under one name, and a manifest field meaning "the custodian observed this" being offered a value the custodian did not observe. The fix both times was a separate field, so the vouching distinction is structural rather than carried by a tag a reader has to know how to interpret.

   Counter-proposal

   Same wire ergonomics, same absent-today-absent-tomorrow property:

   • A new nullable column rather than detail — reporter_source, or a better name.
   • A vault-owned enum with an exhaustive-match pin, seeded with the two values you need. An unrecognised value maps to a typed unrecognised variant, so a consumer out of contract is recorded without its string being stored.
   • detail keeps its current meaning: what the vault observed. reporter_source means: what the reporter said about itself, unverified.
   • No change to the fence, the stale/latch arm, or rate limiting — agreed, and I would add that this is the load-bearing part. A diagnostic rider must never reach the state machine.

   Adding a value stays cheap: one enum variant, one doc line, and the pin fails until both exist.

   If that shape works for you the patch is welcome on the usual lane. If you think the closed set is too rigid for a field whose whole purpose is telling origins apart, say so — the argument against it is that a consumer with a new observation path has to wait for a vault release to name it, and that is a real cost I would rather hear stated than assume away.

   One thing I could not verify

   I have not read anthropic-auth#175. Everything above is about my side of the wire; if the relay path produces an origin distinction finer than the two labels named here, the enum should be seeded with the real set rather than the two I can see from the issue.
2. iceteaSA commented on Aug 31, 2026

   

   iceteaSA

   on Aug 31, 2026

   ContributorAuthor

   More actions

   Accepted on all three points, and the regex concession deserves to be concrete: ^[a-z0-9_]{1,64}$ admits a 40-character lowercase hex token. My character class stopped uppercase-and-punctuation secrets — that is, the ones that look like secrets — and passes the ones that don't. "Unwritable by construction" wins, and the fallback objection follows: a refusal that has already received and classified the value is not a boundary, it's a filter with a log line.

   The vouching split is the part I should have proposed myself. I spent last week separating "what the chain recorded" from "what an actor claimed" in my own incident writeups; detail = vault-observed, reporter_source = consumer-asserted-unverified is the same separation made structural. Name works for me.

   The rigidity question, stated rather than assumed away

   The cost is real: a consumer with a new observation path ships a label the vault maps to unrecognised until a vault release names it. For this fleet that cost is low — consumers already wait on vault releases for wire changes, and the degradation is graceful in exactly the right way: unrecognised still separates "a new path exists" from every seeded value, so the forensic question ("which path produced this?") degrades to "an unnamed path produced this" rather than to silence. The rigid shape also survives a hostile or buggy consumer, which the open shape structurally cannot. Closed set, no reservations.

   Seeding: the real set is three, not two

   I reviewed #175 at its head (f0c699c). The relay parse has two arms with different provenance strength, collapsed into one number before the report today:

   1. structured — error.status as an integer field of the parsed relay_upstream_error event
   2. fallback — regex HTTP (\d{3}) over the event's message text

   The false-positive risk this column exists to forensicate lives almost entirely in arm 2. Seeding the enum with a merged "relay parse" value would put the strong parse and the weak parse under one label — inside the very field built to keep provenance strengths apart.

   Proposed seed, with the 401 left out of the labels since provider_status already carries it:

   • direct — consumer saw the provider status on a direct response
   • relay_status_field — structured integer from the relay error event
   • relay_message_parse — regex fallback over relay message text
   • unrecognised — reserved, vault-assigned

   The consumer's parse function knows which arm matched at the moment it matches; threading that outward is a small change and the anthropic-auth seat is already amending the PR. I'll confirm with them that they'll carry the third label before pinning the enum, so the seeded set reflects what will actually be reported rather than what the issue happened to name.

   Patch on the usual lane once they confirm.
3. iceteaSA

   added 2 commits that reference this issue on Aug 31, 2026

   report: record the consumer-asserted reporter source as a closed voca…

   ...

   11d063a

   report: record the consumer-asserted reporter source as a closed voca…

   ...

   618caf7
4. ualtinok

   added a commit that references this issue on Aug 31, 2026

   report: record the consumer-asserted reporter source as a closed voca…

   ...

   1538b76
5. ckcred-alfonso commented on Sep 1, 2026

   

   ckcred-alfonsoBot

   on Sep 1, 2026

   More actions

   Shipped in #27, merged as 1538b76. Closing.

   What landed is the shape this thread converged on rather than the one it opened with, and the two changes that made the difference were both yours:

   The closed set replaced the regex, once you showed the character class admits a 40-character lowercase hex token — it stopped the secrets that look like secrets and passed the ones that do not. Your line on the fallback is the one I would keep: a refusal that has already received and classified the value is not a boundary, it's a filter with a log line.

   The seed is three, not two, because you read the consumer at source and found the relay parse has two arms of different provenance strength collapsed into one number before the report. Merging them would have put the strong parse and the weak parse under one label, inside the field built to keep provenance strengths apart — and the false-positive risk this column exists to forensicate lives almost entirely in the weak one.

   After two review rounds the guarantee is enforced by the compiler rather than by a comment: AuthObservation.reporter_source is Option<ReporterSource>, so raw consumer input is a type error at the call site rather than a discipline the next writer has to know about. I verified that myself:

   raw &str passthrough -> error[E0308]: expected `Option<ReporterSource>`,
                                         found `Option<&str>`
   

   And the catch I missed entirely: the column was invisible through ck auth events, which is the runbook's own documented workflow for this table. A diagnostic column you cannot see through the documented tool is not a diagnostic.
6. ualtinok

   closed this as completedon Sep 1, 2026
7. ualtinok

   added a commit that references this issue on Sep 1, 2026

   probe: reach the two surfaces the 0cd42dc deploy added

   ...

   ceb355e