feat: record AI coding agent involvement in chalk marks by nettrino · Pull Request #765 · crashappsec/chalk

nettrino · 2026-09-04T13:57:46Z

What this does

Adds two config-only mechanisms for recording AI coding agent involvement in a build. No Nim changes, no recompile — both ship as loadable .c4m.

A — agent env vars reach INJECTOR_ENV (configs/co/env_vars.c4m)

env_default_action defaults to "ignore", and ignore means dropped, not masked (src/plugins/system.nim:196-204). So today every AI-agent env var is silently absent from INJECTOR_ENV, which ships only the five allowlisted vars. This adds the agent vars to ~env_always_show, ordered to match gh's own detection list.

B — new X_AI_AUTHORSHIP key (configs/ai_provenance_key.c4m, configs/ai_provenance.c4m)

A ChalkTimeHost key collected by the conffile plugin via a con4m callback, so it lands in the chalk mark. Three independent signals, emitted as newline-separated field=value:

Field	Source	Needs
`build_agent=`	environment, detection order mirroring `gh`	nothing
`trailer=`	AI-authorship trailers on HEAD (repeatable)	`git` binary
`identity=`	HEAD author/committer matching a known agent identity	`git` binary

Modelled on the two in-tree config-only key precedents, configs/co/build_observables_key.c4m and configs/co/curiosity_logs_key.c4m.

Design notes

COMMIT_MESSAGE already carries the raw trailer. src/utils/git/chalk_git.c:1143 calls libgit2's git_commit_message() — the full message, not git_commit_summary() — trimmed only for whitespace (chalk_git.c:190-203). It's use = true in all four major mark templates and six report templates, so Co-Authored-By: Claude … has been shipping in chalk marks already. X_AI_AUTHORSHIP adds structure, not new data.
Trailer parsing is not done in con4m from COMMIT_MESSAGE. There is no con4m builtin to read another collected key, so the callback shells out to git instead. Chalk otherwise reads git through libgit2 and does not require the git binary, so this is guarded by find_exe("git", []) and skipped when unavailable rather than becoming a dependency. If first-class trailer keys are wanted later, the right home is packGitInfo() in src/plugins/vctlGit.nim:153-187, where info.commitMessage is already in hand — pure derivation, no subprocess.
Aider needs the identity= signal. Its attribute_co_authored_by defaults to false; by default it rewrites the git author/committer instead. A trailer-only check misses it entirely.
identity= matching is deliberately conservative — vendor addresses plus Aider's literal aider (<model>) name form, not bare product names. "Devin" is a common human first name.
Not subscribed to reproducable — the value derives from the environment and HEAD, so it isn't reproducible across builds. Also skipped: minimal, chalk_labels.
Key naming uses X_AI_AUTHORSHIP (underscore) to match in-tree practice (_X_BUILD_OBSERVABLES, _X_CURIOSITY_LOGS), rather than the X- spelling in the chalk.c42spec prose. Hyphens would need quoting in key.<NAME> attribute paths. No validator enforces either form.
Reporting only. These values are recorded, never acted on. PHP-CS-Fixer took backlash for making agent detection a hard runtime dependency and silently changing output format on AI_AGENT; reading the vars is uncontroversial, changing behaviour on them is not.

Reliability — read before depending on this

Every signal here is high-precision, low-and-unknowable recall. Presence is strong evidence an agent was involved. Absence is no evidence either way. Do not gate policy on a missing value.

Suppression paths, all user-controlled: Claude Code's attribution.{commit,pr,sessionUrl} (and legacy includeCoAuthoredBy: false), Aider's attribute_co_authored_by, CLAUDE_CODE_SUBPROCESS_ENV_SCRUB=1, CLAUDE_CODE_SUPPRESS_SESSION_ATTRIBUTION, or any commit-msg hook. Trailer stripping is widespread and well-tooled.

Separately worth fixing (not in this PR)

src/configs/base_keyspecs.c4m:389 documents env_default_action as defaulting to "redact". It actually defaults to "ignore" (src/configs/chalk.c42spec:4358-4366), and the two behave differently — redact emits <<redact>>, ignore drops the variable entirely. Same wording at :5581.

Testing

Validated end-to-end against crashappsec/chalk:latest, which is chalk 1.2.0 at commit 8543bf008665a4b86971bd7a121444baf1a58240 — exactly this branch's base.

chalk load configs/ai_provenance.c4m → "Configuration successfully validated." chalk help key X_AI_AUTHORSHIP then reports it as Chalk-Time, Host / string with the doc string, confirming the component takes effect rather than merely parsing.

Behaviour matrix, asserted on the mark actually written into the artifact (not the report):

Case	Result
Agent env + `Co-Authored-By:` + `Assisted-by:`	`build_agent=claude-code_2-1-220_agent`, plus both `trailer=` lines
AI trailer only, no agent env	`trailer=Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>`
Human-only commit, no agent env (message seeded with an em dash and curly quotes as bait)	key absent — no false positive
Aider-style rewritten git identity, no trailer	`identity=author aider (gpt-4o) <human@example.com>`
Non-git directory	key absent

Two incidental confirmations from that run: the published chalk image is distroless (no shell, no git), and there the find_exe("git", []) guard correctly degrades to emitting nothing rather than erroring — which is the intended behaviour in a minimal build container. And the keyspec and its template subscriptions must ship as one component: loading the subscriptions alone fails hard with "Chalk mark template 'mark_default' contains a key: 'X_AI_AUTHORSHIP', which does not exist".

No functional tests added to tests/functional/ — flagging as a reviewer decision, since test_git.py:164-184 is the model for git-key assertions if wanted.

Full research

The findings behind this PR follow in full, as requested. Method: local binary teardown of Claude Code 2.1.220, Codex 0.153.2 and Gemini CLI; chalk source audit; primary-source doc verification; community and policy survey.

AI-Authorship Provenance in Chalk — Research Findings

Research date: 2026-09-04. Chalk main @ 8543bf00 (v1.2.0).
Method: local binary teardown (Claude Code 2.1.220, Codex 0.153.2, Gemini CLI), chalk source
audit, primary-source doc verification, community/policy survey.

Confidence labels: [VERIFIED] = I reproduced it locally or read it in a primary source.
[PRIMARY] = quoted from vendor docs/spec. [SECONDARY] = credible third party.
[CLAIMED] = single-source or anecdote.

1. Bottom line

Three findings drive everything below.

There is no readable AI watermark in source code. Anthropic ships a real text
watermark, but it is statistical (server-side sampling), explicitly suppressed where code
correctness matters, detectable only via a private-preview API, and inactive on most
models in circulation. Nothing is embedded in the bytes.
Chalk already collects the highest-fidelity signal that exists — the git commit
trailer — inside COMMIT_MESSAGE. It is unparsed. This is a parsing problem, not a
collection problem.
The signal is high-precision, low-and-unknowable recall. Presence proves AI
involvement; absence proves nothing. Every mechanism is user-suppressible. Design for
advisory reporting, not enforcement.

2. What the vendors actually emit

2.1 Anthropic — statistical watermark, not steganographic

[PRIMARY] https://www.anthropic.com/news/claude-text-watermark (2026-08-14):

"Claude's text watermark is a version of the SynthID-Text approach published by Google
DeepMind in a Nature paper in 2024."
"Nothing is added to the text and there are no hidden characters."
"Where an exact output is required—where there isn't a choice, and something would be
factually wrong or a piece of code would break if a different term was chosen—the
watermark isn't applied."
"We are releasing a detection API in private preview… available to eligible organizations
as required under EU law (such as regulators, law enforcement, media, fact-checkers,
independent researchers, educational organizations, and EU civil society groups)."

[PRIMARY] https://support.claude.com/en/articles/16266773-how-claude-marks-ai-generated-content:

"Claude models launched on or after August 2, 2026 will support machine-readable marking at
launch." Supported today: Fable 5.1, Mythos 5.1 only.
"Claude markings cover output from supported models everywhere you use Claude, including
Claude Platform (API), Claude, Claude Code, Claude Cowork, and Claude Tag."
"A detected mark provides a signal that content was processed by Claude, but is not fully
conclusive."

Mechanism: SynthID-Text tournament sampling (Nature 634, 2024-10-24,
DOI 10.1038/s41586-024-08025-4; https://github.com/google-deepmind/synthid-text). Keyed
g-values over preceding context bias sampling. Not Kirchenbauer green-list.

Why chalk cannot use it — four independent blockers:

Code is the worst case for the scheme (low token-choice entropy → little to no mark).
No public detection endpoint, no published FPR.
claude-opus-5, claude-sonnet-5, Opus 4.x are all in the transition period → unmarked.
Detection requires the generation key. That is why no public detector can exist.

[VERIFIED] Local teardown of @anthropic-ai/claude-code/bin/claude.exe v2.1.220
(257 MB, 652k strings): zero hits for c2pa, synthid, steg. All 176 watermark hits are
Node's highWaterMark. Build constants: VERSION:"2.1.220",
BUILD_TIME:"2026-07-24T22:17:45Z", GIT_SHA:"4073f59596e272f39393db4f96abc5f4b10eff21".
Client-side absence is expected and is not evidence about the server-side watermark.

2.2 C2PA — architecturally excluded from source files

[PRIMARY] https://platform.claude.com/docs/en/agents-and-tools/tool-use/code-execution-tool
(line 893 of fetched doc):

"supported image, video, and audio files that Claude produces in the code execution sandbox
carry C2PA Content Credentials when you download them through the Files API. Supported
formats include PNG, JPEG, GIF, WebP, TIFF, HEIC, AVIF, SVG, MP4, MOV, MP3, WAV, FLAC, and
M4A… It identifies Anthropic as the issuer, carries a timestamp, and records the action
description "Claude provided this file at the request of a user and may have created or
modified the file contents.""
"Text files, PDFs, and office documents are not signed because they are not supported
formats for signing."
"Re-encoding, format conversion, screenshots, and tools that strip metadata remove the
credential"

→ Do not build a C2PA reader for source artifacts. Code is unsignable under C2PA, by
format, not by omission. (Chalk's existing model codecs handle GGUF/safetensors — C2PA on
media artifacts could matter there, separate scope.)

2.3 Git trailers — the real, usable signal

Tool	Commit trailer	PR body	How produced
Claude Code	`Co-Authored-By: <model> <noreply@anthropic.com>`	`🤖 Generated with [Claude Code](https://claude.com/claude-code)`	[VERIFIED] code-generated (`Rry()`), deterministic
Claude Code (3rd trailer)	`Claude-Session: <claude.ai session URL>`	matching PR-body link	[VERIFIED] `Ary()`; cloud/Remote-Control sessions only
Codex 0.153.2	`Co-authored-by: Codex <noreply@openai.com>`	`Generated with [Codex](https://openai.com/codex/).`	[VERIFIED] prompt-injected via `<git_attribution>` system-prompt section
GitHub Copilot	`Co-authored-by: Copilot <copilot@github.com>`	—	[SECONDARY] coding agent + VS Code `git.addAICoAuthor` (default `all`)
Aider	`Co-authored-by: aider ({model_name}) <aider@aider.chat>` — off by default	—	[PRIMARY] `attribute_co_authored_by` defaults `False`; rewrites git author/committer instead
Linux kernel policy	`Assisted-by: LLM [TOOL1] [TOOL2]`	—	[PRIMARY] human-written, mandated by `Documentation/process/coding-assistants.rst`
Fedora policy	`Assisted-by: <name of code assistant>`	—	[SECONDARY] human-written, Council-approved Oct 2025

⚠️ Aider is the exception that breaks a trailer-only parser. Its co-author trailer is
off by default; by default it rewrites git author/committer identity instead. So the
Aider signal lands in chalk's existing AUTHOR/COMMITTER keys
(base_keyspecs.c4m:772/:809), not COMMIT_MESSAGE. Any implementation must check both.

[VERIFIED] Claude Code trailer construction, deobfuscated from the bundle:

`Co-Authored-By: ${modelName} <noreply@anthropic.com>`
// settings: includeCoAuthoredBy:false kills both trailer+footer (legacy path)
//           attribution.{commit,pr,sessionUrl} = current path
// PR footer URL constant Tut = "https://claude.com/claude-code"

[VERIFIED] Codex is instructed, not coded: "Commit messages must end with Co-authored-by: Codex noreply@openai.com. Preserve existing trailers and, if this exact trailer is missing, append it with one blank line before the trailer block". Compliance-
dependent → less reliable than Claude's.

Reliability caveats that must shape the regex:

[VERIFIED] Case is inconsistent in the wild. This repo: Co-Authored-By: ×195,
Co-authored-by: ×28. Match case-insensitively.
[VERIFIED, this repo] Value leaks model + context tier: Claude Sonnet 4.6 ×166,
Claude Opus 4.8 ×22, Claude Opus 4.7 (1M context) ×7.
[PRIMARY, changelog 2.1.251] "Changed the default commit trailer to
Co-Authored-By: Claude Code when the active model isn't a recognized Claude model (e.g.
third-party models behind a custom ANTHROPIC_BASE_URL)." → bare Claude Code is a
legitimate value.
[SECONDARY] Older versions emit https://claude.ai/code, newer https://claude.com/claude-code.
Match both.
[SECONDARY] includeCoAuthoredBy is deprecated in favour of attribution.

Suggested trailer regex (superset, covers kernel/Fedora conventions too):

(?im)^\s*(?:co-authored-by|assisted-by|generated-by|signed-off-by)\s*:\s*([^<\n]+?)\s*(?:<([^>]*)>)?\s*$

2.4 Env vars — cross-vendor detection matrix

[VERIFIED] Measured inside a live Claude Code session:

AI_AGENT=claude-code_2-1-220_agent      # <tool>_<version>_<role>
CLAUDECODE=1
CLAUDE_CODE_ENTRYPOINT=cli
CLAUDE_CODE_SESSION_ID=<uuid>
CLAUDE_CODE_CHILD_SESSION=1
CLAUDE_CODE_EXECPATH=…
CLAUDE_PID=…
CLAUDE_EFFORT=xhigh
TRACEPARENT=…                            # W3C trace ctx, when propagation enabled

[VERIFIED] AI_AGENT originates in the Claude Code binary (7 occurrences), not the user's
shell or ~/.claude/settings*.json (both checked). Construction:

AI_AGENT = nbi("agent")
if (!process.env.AI_AGENT
    || process.env.AI_AGENT.startsWith("claude-code_")
    || process.env.AI_AGENT.startsWith("claude-code/"))
  process.env.AI_AGENT = nbi("harness")   // respects a foreign agent's value

AI_AGENT is a de facto convention with no spec.

[VERIFIED] Not set by Codex 0.153.2 (0 hits) or Gemini CLI (0 hits) — tested locally.
[SECONDARY] Origin is Vercel's detect-agent, not a standards body. The only
standardization attempt, agentsmd/agents.md issue Mynameismeerkat/key backup howto #136 (opened 2026-01-08, still open),
proposes AGENT, not AI_AGENT. Goose and Amp implement AGENT.
[SECONDARY] OpenAI explicitly declined the equivalent: openai/codex#13416 closed
2026-04-03 ("hasn't received enough upvotes").
[SECONDARY] Anthropic has never documented AI_AGENT; the docs bug
(anthropics/claude-code#53071) auto-closed unresolved 2026-08-13. Value format is not
contractually stable.
Purpose per changelog is gh telemetry attribution, not provenance.

[VERIFIED] cli/cli internal/agents/detect.go — authoritative cross-vendor ordering,
usable directly as chalk's detection matrix. AI_AGENT is checked first, validated with
^[a-zA-Z0-9_-]+$:

AI_AGENT → AGENT=="amp" → CODEX_SANDBOX|CODEX_CI|CODEX_THREAD_ID → GEMINI_CLI
→ COPILOT_CLI → OPENCODE → ANTIGRAVITY_AGENT → AUGMENT_AGENT → REPL_ID
→ CLAUDE_CODE_IS_COWORK → CLAUDECODE|CLAUDE_CODE → CURSOR_TRACE_ID
→ CURSOR_AGENT|CURSOR_EXTENSION_HOST_ROLE=="agent-exec" → TERM_PROGRAM=="kiro"
→ PATH contains /.pi/agent → GOOSE_PROVIDER

Suppression flags to document as known blind spots:
CLAUDE_CODE_SUBPROCESS_ENV_SCRUB=1, CLAUDE_CODE_SUPPRESS_SESSION_ATTRIBUTION,
CLAUDE_CODE_ATTRIBUTION_HEADER=0 (all three [VERIFIED] present in the 2.1.220 binary).

2.5 Cryptographic agent identity — where the industry is heading

[VERIFIED] Codex 0.153.2 binary contains:

CODEX_AGENT_IDENTITY_JWKS_BASE_URL
/agent-identities/jwks
/wham/agent-identities/jwks
struct JwkSet with 1 element
"agent identity registration attempt failed; retrying"
"agent identity bootstrap unavailable after …"

Public keys at a JWKS endpoint with a registration/bootstrap lifecycle → third-party-verifiable
signed agent claims. Categorically stronger than trailers or heuristics. Public documentation
for it: NOT FOUND as of research date.

Also [VERIFIED] in the Codex binary: pdf_c2pa_create_request, pdf_c2pa_reservation —
OpenAI applies C2PA to PDF uploads. Not code.

2.6 Prior steganography incident — relevant context

[VERIFIED via multiple independent sources] Claude Code v2.1.196 rewrote its own
system prompt steganographically: apostrophe in Today's → U+2019 / U+02BC / U+02B9, and
date separator - → /. Triggered by ANTHROPIC_BASE_URL set, plus timezone
Asia/Shanghai/Asia/Urumqi or hostname matches against base64+XOR(key=91)-obfuscated lists.
Anthropic confirmed it as an anti-distillation / unauthorized-reseller experiment and removed it
in 2.1.197 (2026-07-01). HN discussion id 48734373 (2,445 pts).
https://thereallo.dev/blog/claude-code-prompt-steganography

Scope: outbound requests, not artifacts on disk. Does not change artifact-side conclusions.
But it means "the vendor wouldn't embed hidden marks" is not a sound argument, and the removal
(not policy) explains a clean 2.1.220.

3. Detection accuracy — why heuristics are not evidence-grade

[SECONDARY] CodeMirage (Purdue, DL4C'25), across GLTR / Entropy / DetectGPT / GPTSniffer /
CodeXEmbed+RF / RAIDAR / BiScope:

F1 ≈ 0.45–0.75
TPR@FPR=1% ≈ 0.30–0.50 ← at a tolerable FPR you catch a third to half
Paraphrasing/refactoring degrades accuracy >30%; Java/C run 15–20% lower F1 than Python
Detectors trained on one LLM fail on unseen production models

Others: arXiv 2412.14611 — 84.1% ± 3.8% (10 languages, 121k snippets). SemEval-2026 Task 13
stylometric — macro-F1 67.35. Prose stylometry reaches ~97% but does not transfer to code.

[VERIFIED, this repo] Unicode heuristics produce real false positives on chalk itself:
src/configs/base_keyspecs.c4m has 11 U+00A0 hits — copy-pasted AWS documentation, not AI.
U+00A0 is category Zs, not Cf, so a category-based check correctly ignores it while a
loose "non-ASCII" check misfires.

[SECONDARY] Independent empirical disproof of Claude Unicode watermarking: 1,206 sessions,
7.2M prose characters audited pre/post-announcement — "no anomalous instances". Green/red-list
tests negative (p=0.677, p=0.556 Sonnet 5; p=0.886 Fable 5).
https://johnjwang.com/post/2026/08/12/how-claude-watermarking-probably-works/

[SECONDARY] The U+202F "ChatGPT watermark" (Rumi, Apr 2025) was debunked — OpenAI said
"not a watermark… a quirk of large-scale reinforcement learning"; Rumi retested and the
characters were gone.

Conclusion: statistical/stylistic AI detection is not evidence-grade for chalk. Trailers,
env vars, and signed attestations are.

4. Chalk integration — where the code is

4.1 The trailer is already collected [VERIFIED end-to-end]

Step	Location
libgit2 call	`src/utils/git/chalk_git.c:1143` — `git_commit_message(commit)`
whitespace trim only	`src/utils/git/chalk_git.c:190-203` — `trim_cstr`
C struct	`src/utils/git/chalk_git.h:9` — `char *commit_message;`
Nim FFI	`src/utils/git.nim:49`, `src/utils/git.nim:153`
keyspec emit	`src/plugins/vctlGit.nim:158`
declaration	`src/configs/base_keyspecs.c4m:846` (`COMMIT_MESSAGE`), `:1064` (`_COMMIT_MESSAGE`)
plugin manifest	`src/configs/base_plugins.c4m:127`, `:136`

git_commit_message() returns the full message — libgit2 has git_commit_summary() for
subject-only and chalk deliberately does not use it. Only transform is whitespace trim: no
truncation, no trailer stripping. Contrast: tag messages are signature-stripped
(chalk_git.c:297-311 trim_tag_message()), which shows the omission for commits is intent.

COMMIT_MESSAGE is use = true in all 4 major mark templates
(base_chalk_templates.c4m:66,183,299,388) and 6 report templates
(base_report_templates.c4m:69,765,1329,1878,2333,2437).

→ Co-Authored-By: Claude … ships in chalk marks and reports in production today.

Two limits:

Only HEAD. gitCollect() reads one commit; no history walk. Per-repo AI density over
history is out of scope without new libgit2 work.
AUTHOR/COMMITTER come from the git identity (chalk_git.c:1130-1139), not trailers.
The AI co-author signal exists only inside COMMIT_MESSAGE.

4.2 No existing AI-authorship key [VERIFIED]

All 534 keyspecs enumerated; zero hits for AI/LLM/Claude/Copilot/agent/generated_by.
Best existing homes: INJECTOR_ENV (:365), COMMIT_MESSAGE (:846),
INJECTOR_ARGV (:350), BUILD_TRIGGER (:1601), CODE_OWNERS (:1253), SAST (:2102).

4.3 `INJECTOR_ENV` drops agent vars today — and a real doc bug [VERIFIED]

env_default_action default is "ignore" (src/configs/chalk.c42spec:4358-4366), but
src/configs/base_keyspecs.c4m:389 says "This defaults to redact". And ignore means
dropped, not masked — src/plugins/system.nim:196-204:

if k in never:    continue
elif k in redact: envdict[k] = "<<redact>>"
elif k in always: envdict[k] = v
elif def == 'i':  continue          # <-- default path: silently dropped
elif def == 'r':  envdict[k] = "<<redact>>"
else: envdict[k] = v

Default env_always_show = ["PATH","PWD","XDG_SESSION_TYPE","USER","SSH_TTY"]
(chalk.c42spec:4333). Every agent env var is absent today, not redacted.

Worth fixing upstream independently: the env_default_action doc at
base_keyspecs.c4m:389 and :5581.

4.4 Config-only extension path [VERIFIED]

The conffile plugin (src/plugins/conffile.nim:18-49) evaluates keyspec … callback:,
enabling new keys with no Nim and no recompile. Two in-tree precedents:
configs/co/build_observables_key.c4m (_X_BUILD_OBSERVABLES) and
configs/co/curiosity_logs_key.c4m (_X_CURIOSITY_LOGS).

con4m builtins available in callbacks: read_file, is_file, find_exe, env, run,
system, parse_json, parse_jsonl, to_json, strip, split, starts_with, memoize,
command_name, binary_sha256, plus chalk-specific ones (src/con4mfuncs.nim:255-383).

Gotcha: use defaults true only once a key.<NAME> subsection exists.
registerKeys() (src/collect.nim:69-76) iterates existing subsections only, and
hasSubscribedKey() (:20-29) then skips the whole plugin. Declaring a keyspec without
touching a template is a silent no-op.

4.5 Chalk does not verify signatures — by documented design [VERIFIED]

docs/design-caller-attestation.md:258-261:

"No network calls, no signature verification (the attestation is informational, not
cryptographic). If signed attestations become interesting later, they slot in as
CALLER_ATTESTED_INFO.signature on the caller side and a downstream verifier — not
chalk's job."

:358-359: "The trust model is process-spawning, not cryptographic. Signed attestations are
out of scope."

Confirmed in code: src/plugins/callerAttestation.nim reads CHALK_CALLER_ATTESTATION (env or
file), parses JSON, checks a protocol version, warns on unknown top-level keys. isHex64
validates artifact-hash keys, not signatures. Zero JWT/JWKS/crypto verification.

→ Do not propose chalk verify Codex JWKS agent identity. Chalk should carry the claim;
a downstream verifier checks it. callerAttestation.nim:117-118 already passes through any
X--prefixed top-level key, so a caller can inject X-AI-Agent today, with zero chalk
changes.

4.6 Other relevant extension points

tool sections are the only user-instantiable object type (chalk.c42spec:446
user_def_ok: true). Driven by src/plugins/externalTool.nim. Precedents:
sastconfig.c4m (semgrep), sbomconfig.c4m (syft), secretscannerconfig.c4m (trufflehog).
⚠️ [VERIFIED] A novel tool kind has no off-switch. externalTool.nim:153-155
hardcodes gates for sbom/sast/secret_scanner only; an unrecognized kind matches no
branch and runs unconditionally. Declare AI-fingerprint tools as kind: "sast" to inherit
the run_sast_tools gate, the SAST key, and canonicalize_tools. Tools only run on
build/insert (externalTool.nim:146).
Source content is already in memory at src/plugins/codecSource.nim:283
(stream.readAll()), with detected language + extension — near-zero marginal I/O for a
content scanner. Emit via sourceGetChalkTimeArtifactInfo():377-382.
⚠️ chalk.c42spec:3349-3358 discourages marking source inside a repository ("Git does
that job well"). Embedding provenance into in-repo source cuts against documented intent;
metadata keys are grain-aligned.
No standalone metadata-key reference doc exists. Key docs are the doc:/shortdoc:
fields, surfaced by chalk help key <name> and chalk docgen.

5. Recommendation

Ranked by effort. A + B cover the practical ask with no Nim, no recompile, no upstream PR.

	What	Effort	Claim it supports	Weakness
A	Append agent vars to `~env_always_show` in `configs/co/env_vars.c4m`	4-line diff	"an agent invoked this build"	`CLAUDE_CODE_SUBPROCESS_ENV_SCRUB=1`
B	`_X_AI_AUTHORSHIP` derived key: con4m callback parsing trailers out of data chalk already has	XS, one `.c4m`	"an agent co-authored HEAD"; leaks model+tier	opt-out via `attribution.*`
C	Caller injects `X-AI-Agent` / agent-identity JWT into the attestation envelope	~0 in chalk	only tier that can back a policy gate	verification is explicitly downstream
D	Hidden-channel scanner as `tool` `kind:"sast"`	S	supply-chain integrity (not AI attribution)	see §6
E	First-class trailer keys in `vctlGit.nim` (`packGitInfo():153-187`)	M	same as B, but proper `doc:` in `chalk help`	recompile + PR + 8 template sites

A, concretely. configs/co/env_vars.c4m is a 12-line ~env_always_show = [...] overlay.
Add, ordered per the gh matrix (§2.4):

"AI_AGENT", "AGENT",
"CLAUDECODE", "CLAUDE_CODE_SESSION_ID", "CLAUDE_CODE_ENTRYPOINT", "CLAUDE_CODE_IS_COWORK",
"CODEX_SANDBOX", "CODEX_CI", "CODEX_THREAD_ID",
"GEMINI_CLI", "COPILOT_CLI", "OPENCODE", "CURSOR_TRACE_ID", "CURSOR_AGENT",
"TRACEPARENT",

⚠️ Consent caveat, learned the hard way by others: PHP-CS-Fixer made agent detection a hard
runtime dependency and silently changed output format on AI_AGENT; serious backlash on
consent/least-privilege grounds, walked back. Reading these vars is uncontroversial;
changing chalk behavior based on them would not be. Report only.

Do not build

A watermark detector — suppressed on code, gated detection, most models unmarked, requires
the generation key.
A C2PA reader for source artifacts — text is unsignable by format.
An AI-style/unicode heuristic as an AI signal — vendor confirms nothing is inserted, and it
false-positives on chalk's own source.

6. Separate, better-founded opportunity: hidden-channel detection

Not AI attribution — supply-chain integrity, squarely chalk's existing mission
(Trojan Source, CVE-2021-42574). There is a production-grade, adversarially-reviewed reference
implementation available: Anthropic's own, [VERIFIED] extracted from the 2.1.220 bundle
(FORBIDDEN_ANSWER_CHAR_RE / MAX_EXEMPT_INVISIBLE, src/frame/decisionBlocks.ts):

FORBIDDEN_RE = /[-��-�  ]|(?![‌‍︀-️\u{E0100}-\u{E01EF}])[\p{Cf}\p{Default_Ignorable_Code_Point}]/u
// carved-out invisibles (ZWNJ/ZWJ + variation selectors) capped at 8

Design properties worth copying, per their own comments:

Reject by Unicode category (\p{Cf}, \p{Default_Ignorable_Code_Point}), not an
enumerated codepoint list — covers "whatever the next Unicode version mints". Enumerated
blacklists rot.
Carve-outs for legitimate composed emoji, but capped — "unbounded they are a hidden
byte-per-codepoint channel".
C0/C1 + U+2028/U+2029 handled numerically (not in Cf).
Stated threat model: "an RLO could visually reverse a decided line; tag-encoded text is
invisible to humans but legible to models."

Nim's PCRE backend supports \p{Cf} in UTF mode → ports near-directly. Fits codecSource.nim
(contents already in memory) or a tool kind:"sast".

Prior art to study: untrace (Go; decodes payloads, context-aware per filetype, git clean
filter), boundaryguard (CI-shaped, fail-closed exit 0/1/2, SARIF output),
guillaumemeyer/watermarks-remover (widest codepoint list incl. U+E0000 block, noncharacters,
musical/format controls).

7. Policy landscape — why this matters commercially

Trailer-mandating (chalk could verify compliance, a sellable capability):

Fedora (Council-approved Oct 2025): "Contributors MUST disclose the use of AI tools when
the significant part of the contribution is taken from a tool without changes." Recommended
mechanism: Assisted-by: <name of code assistant>.
Linux kernel (Documentation/process/coding-assistants.rst): trailer
Assisted-by: LLM [TOOL1] [TOOL2]. And: "AI agents MUST NOT add Signed-off-by tags.
Only humans can legally certify the Developer Certificate of Origin (DCO)."
⚠️ [UNRESOLVED] third parties document Assisted-by: AGENT_NAME:MODEL_VERSION instead.
Diff the kernel tree before writing a parser.

Bans (chalk could detect violations):

QEMU: "Current QEMU project policy is to DECLINE any contributions which are believed to
include or derive from AI generated content."
Gentoo (Council 2024-04-14): "expressly forbidden to contribute… any content that has
been created with the assistance of Natural Language Processing artificial intelligence tools."
NetBSD: AI-generated code "is presumed to be tainted code, and must not be committed
without prior written approval by core."

Permissive: Debian GR vote_002 (Aug 2026) — "Responsible use of generative AI" won ~64/36,
425/1045 DDs voting. LLVM allows but treats unreviewed AI submissions as "extractive".

Regulatory: EU AI Act Art. 50(2) — synthetic text must be "marked in a machine-readable
format and detectable as artificially generated", marks "effective, reliable, robust and
interoperable". Live 2026-08-02. Carve-out for systems performing "an assistive function
for standard editing" plausibly covers much of agentic code editing. Art. 50(4) does not
apply to code (not "published to inform the public on matters of public interest").
NOT FOUND: any Anthropic statement addressing code generation under Art. 50 specifically.

Copyright pressure against Co-Authored-By:: U.S. Copyright Office, 88 Fed. Reg. 16190
(2023-03-16), 37 CFR Part 202 §IV.A — "Applicants should not list an AI technology or the
company that provided it as an author or co-author simply because they used it." This is the
strongest argument that Assisted-by: is the correct trailer and Co-Authored-By: is not.

[SECONDARY] SLSA 1.2 covers build/source provenance but explicitly not AI authorship.
That gap is unclaimed and sits directly adjacent to chalk.

8. Prior art on line-level AI provenance

git-ai (https://github.com/git-ai-project/git-ai) — Rust, Apache-2.0, per-line
attribution in Git Notes linked to agent/model/session; migrates attributions through
squash/merge/reset/rebase/stash/cherry-pick. ~2k stars, Thoughtworks Radar Assess.
whogitit — Git Notes, stores generating prompts, automatic redaction.
bcmyguest/assisted-by — PreToolUse hook intercepting git commit, exits 2 if
Assisted-by: missing; rejects Co-Authored-By: Claude and AI Signed-off-by:.
Grading scheme worth stealing: Assisted-by: ≤33% generated / Co-authored-by: 35–67% /
Generated-by: 67%+, each paired with a human Signed-off-by.

Git Notes is the interesting divergence from chalk's model: it survives history rewrites and
carries line granularity, but it is not embedded in the artifact. Chalk marks the artifact.
The two are complementary, not competing.

9. Trailer prevalence — order-of-magnitude only

[CLAIMED — GitHub commit search is token-based and approximate, public repos only]

Query	`total_count`
commits matching `"Co-Authored-By: Claude"`	~9,194,319
commits matching `"noreply@anthropic.com"`	~22,769,898
commits matching `"Generated with Claude Code"`	~3,377,150
commits matching `"Assisted-by:"`	~197,524
code files matching `"includeCoAuthoredBy"` + `false`	~8,800
commits matching `"Co-authored-by: Copilot"`	rate-limited, not obtained

The 22.8M noreply@anthropic.com figure exceeding the 9.2M "Co-Authored-By: Claude" figure is
most likely a tokenization artifact of matching the bare email, not evidence of 13M commits
carrying the address without the trailer. Do not cite it as a superset.

Design consequence — the policy-endorsed trailer has ~2% of the vendor default's volume
(197,524 vs 9,194,319). Assisted-by: is real but nowhere near displacing Co-Authored-By:.
A chalk implementation must read both trailer families, and must tolerate at least three
Assisted-by: value shapes (§7 unresolved item).

Stripping is widespread and well-tooled (anthropics/claude-code#617, 22 comments, circulates
a global core.hooksPath + commit-msg recipe; #45137 gives a one-line sed). Motivation
is often explicitly framed as avoiding a usage-telemetry trail.

10. Open items

Items 1–3 below were resolved after the first draft; see §10.1. Remaining gaps follow.

Not found: public documentation for Codex's agent-identity JWKS.
Coverage gap: Reddit was inaccessible by every route attempted (WebFetch domain block,
403 on api.reddit.com/oauth.reddit.com, 403 on redlib mirrors). Community signal came
from GitHub issues, HN, and blogs instead.
Not attempted: running chalk to confirm COMMIT_MESSAGE output empirically — the
in-repo chalk binary is Linux/aarch64 (2024-06-11) and won't execute on darwin/arm64.
The code path is unambiguous, but an end-to-end run in Docker would close the loop.

10.1 Resolved after first draft

Aider [PRIMARY, Aider-AI/aider/aider/repo.py] — trailer is
Co-authored-by: aider ({model_name}) <aider@aider.chat>, i.e. the model name is embedded,
same as Claude Code. Commit-message prefix "aider: ". Flags and defaults:

Flag	Default
`attribute_author`	`None` (treated as True)
`attribute_committer`	`None` (treated as True)
`attribute_commit_message_author`	`False`
`attribute_commit_message_committer`	`False`
`attribute_co_authored_by`	`False`

⚠️ Aider's co-author trailer is off by default — instead it rewrites git author/committer
identity by default. So for Aider the signal lands in AUTHOR/COMMITTER
(base_keyspecs.c4m:772/:809), not COMMIT_MESSAGE. A trailer-only parser misses Aider
entirely. This is the one case where chalk's existing AUTHOR/COMMITTER keys carry the AI
signal.

GitHub Copilot [SECONDARY] — Co-authored-by: Copilot <copilot@github.com>. Copilot coding
agent co-authors all commits for traceability. Separately, VS Code's Git extension setting
git.addAICoAuthor defaults to all, adding the co-author line whenever Copilot features touch
the code — so this trailer appears from the IDE, not just the cloud agent.
https://docs.github.com/en/copilot/concepts/coding-agent/about-copilot-coding-agent,
https://github.com/orgs/community/discussions/179983

Linux kernel Assisted-by: — discrepancy RESOLVED [PRIMARY,
https://docs.kernel.org/process/coding-assistants.html] — canonical format is
Assisted-by: LLM [TOOL1] [TOOL2], where the optional tools are specialized analysis
tools (coccinelle, sparse, smatch, clang-tidy). Basic tooling (git, gcc, make, editors) must not
be listed. The third-party Assisted-by: AGENT_NAME:MODEL_VERSION shape reported in §7 is
not the kernel format — do not implement it as such. Confirmed:
"AI agents MUST NOT add Signed-off-by tags. Only humans can legally certify the Developer
Certificate of Origin (DCO)." Co-developed-by is not discussed in the current doc.

SBOM standards — decisive negative [SECONDARY] — neither SPDX 3.0 nor CycloneDX has a
field for recording that an AI agent authored code. SPDX 3.0's AI Profile and CycloneDX 1.7's
ML-BOM describe AI models and datasets as SBOM components (training environment, energy
consumption, performance metrics, safety risk classification, intended use) — an entirely
different problem. SPDX 3.0's new Agent class (replacing 2.3 Creator/Supplier, covering
Person / Organization / software tool) is the closest structural fit, and CISA's crosswalk
maps "SBOM Author Name" → CycloneDX metadata.authors, but neither is specified for
AI-authorship-of-code.

Combined with SLSA 1.2 explicitly not covering AI authorship (§7), the conclusion is:
no SBOM or provenance standard currently carries "an AI agent authored this code." The gap
is real, unclaimed, and directly adjacent to chalk's existing metadata-key model. Chalk's own
X-/_X- user-key namespace is available to define it without waiting on a standards body.

🤖 Generated with Claude Code

env_default_action defaults to "ignore", which drops rather than redacts, so agent env vars never reached INJECTOR_ENV. Adds them to env_always_show, ordered to match gh's own agent detection so chalk and gh agree on a build. Adds X_AI_AUTHORSHIP (ChalkTimeHost, conffile callback): build_agent from the environment, plus AI-authorship trailers and agent git identities on HEAD. Trailer and identity collection shells out to git guarded by find_exe, since no con4m builtin can read COMMIT_MESSAGE and chalk otherwise needs only libgit2. Aider needs the identity path: it defaults to rewriting the git author rather than adding a trailer. Keyspec and template subscriptions ship as one component because loading the subscriptions alone is a hard validation error. All signals are advisory and user-suppressible: presence is evidence of agent involvement, absence is not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

nettrino requested a review from viega as a code owner September 4, 2026 13:57

nettrino requested a review from baron-crashoverride September 4, 2026 14:21

fix: harden AI provenance detection

780d61c

Navigation Menu

Uh oh!

feat: record AI coding agent involvement in chalk marks - #765

feat: record AI coding agent involvement in chalk marks#765
nettrino wants to merge 2 commits into
mainfrom
nettrino/ai-provenance-keys

nettrino commented Sep 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

nettrino commented Sep 4, 2026

What this does

Design notes

Reliability — read before depending on this

Separately worth fixing (not in this PR)

Testing

Full research

AI-Authorship Provenance in Chalk — Research Findings

1. Bottom line

2. What the vendors actually emit

2.1 Anthropic — statistical watermark, not steganographic

2.2 C2PA — architecturally excluded from source files

2.3 Git trailers — the real, usable signal

2.4 Env vars — cross-vendor detection matrix

2.5 Cryptographic agent identity — where the industry is heading

2.6 Prior steganography incident — relevant context

3. Detection accuracy — why heuristics are not evidence-grade

4. Chalk integration — where the code is

4.1 The trailer is already collected [VERIFIED end-to-end]

4.2 No existing AI-authorship key [VERIFIED]

4.3 INJECTOR_ENV drops agent vars today — and a real doc bug [VERIFIED]

4.4 Config-only extension path [VERIFIED]

4.5 Chalk does not verify signatures — by documented design [VERIFIED]

4.6 Other relevant extension points

5. Recommendation

Do not build

6. Separate, better-founded opportunity: hidden-channel detection

7. Policy landscape — why this matters commercially

8. Prior art on line-level AI provenance

9. Trailer prevalence — order-of-magnitude only

10. Open items

10.1 Resolved after first draft

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

4.3 `INJECTOR_ENV` drops agent vars today — and a real doc bug [VERIFIED]