Skip to content

fix(flags): reject an empty element in a multi-value flag - #1092

Merged
JonJagger merged 3 commits into
mainfrom
fix-empty-element-in-multi-value-flag
Aug 12, 2026
Merged

JonJagger merged 3 commits into
mainfrom
fix-empty-element-in-multi-value-flag

Conversation

@JonJagger

Copy link
Copy Markdown
Contributor

pflag's StringSlice splits each value with a CSV reader, and that reader yields
nothing for an empty string. An empty element is therefore appended as nothing
and leaves no trace: after parsing, --attachments "" --attachments file cannot
be told apart from --attachments file, so nothing downstream can report what
was lost.

Two flags where that matters now use a value type that refuses an empty element
at Set, the last point where it still exists. --attachments is the case this bug
was filed for: an attestation is recorded without evidence its author believed
was on it. --template is worse in kind, because it names the attestations a flow
requires, so a dropped element weakens that flow for every artifact passing
through it afterwards rather than spoiling one record.

Comma splitting is kept rather than switching to pflag's StringArray, which
stores values verbatim. An environment variable cannot be repeated, so a comma
list is the only way to give a multi-value flag more than one value from the
environment; a regression test pins that.

The type implements pflag.SliceValue as well as pflag.Value, so the refusal also
covers the config file, which bindFlags applies through Replace.

Checklist

@claude

claude Bot commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @JonJagger's task in 2m 2s —— View job


Review — all prior feedback addressed ✅

Re-reviewed after the latest push (2bfd5218, 9bc21f3e). The earlier review threads are now resolved and the change is in good shape.

Static checks

  • go build ./... — clean
  • go vet ./cmd/kosli/ — clean

Prior review points, now resolved

  • The doc comment in nonEmptyStringSlice.go:18-21 no longer claims config/env values arrive via Replace. It correctly states they flow through Set (stringified with %v), so the refusal covers them by the same path as argv. This matches bindFlags at root.go:641.
  • The comment now also honestly frames Append/Replace/GetSlice and the pflag.SliceValue assertion as future-proofing (:11-16), rather than load-bearing config-path code.
  • The "config/env only logs and continues" concern is genuinely closed: bindFlags joins the failed Set into bindErr (root.go:642) and initializeConfig returns it (root.go:544-546), so KOSLI_ATTACHMENTS="a,,b" is a hard failure, not a silent drop.

What's good here

  • The fix targets the right layer — Set is the last point where the empty element still exists, and refusing there is the minimal correct intervention. The extensive comments explain why (CSV reader eats empty strings) rather than what.
  • Test coverage is thorough and well-motivated: table-driven rejection cases ("", a,,b, ,a, a,), the comma-splitting regression guard that justifies not switching to StringArray, and — new in this push — end-to-end argv and env-path tests for both --attachments and --template. Using a real file (testdata/file1) as the surviving attachment so the test can't pass for the wrong reason is a nice touch.
  • String()/Type() deliberately mirror pflag's StringSlice so help text and defaults render identically after the flag switches type — no doc regression.

Minor, optional (non-blocking)

  • The literal "empty values are not allowed" appears in three places (Set, Append, Replace). A shared errEmptyElement var would keep them from drifting, but it's harmless as-is.
  • Replace assigns the caller's slice directly (*s.values = values) rather than copying, unlike pflag's StringSlice. No caller exercises it today, so this is theoretical — worth a note only if Replace ever becomes reachable.

Nothing blocking. LGTM.

  • Read changed files
  • Run static checks (build, vet)
  • Verify earlier review feedback is addressed
  • Post review

Comment thread cmd/kosli/nonEmptyStringSlice.go Outdated
Comment thread cmd/kosli/nonEmptyStringSlice.go Outdated
  pflag's StringSlice splits each value with a CSV reader, and that reader yields
  nothing for an empty string. An empty element is therefore appended as nothing
  and leaves no trace: after parsing, `--attachments "" --attachments file` cannot
  be told apart from `--attachments file`, so nothing downstream can report what
  was lost.

  Two flags where that matters now use a value type that refuses an empty element
  at Set, the last point where it still exists. --attachments is the case this bug
  was filed for: an attestation is recorded without evidence its author believed
  was on it. --template is worse in kind, because it names the attestations a flow
  requires, so a dropped element weakens that flow for every artifact passing
  through it afterwards rather than spoiling one record.

  Comma splitting is kept rather than switching to pflag's StringArray, which
  stores values verbatim. An environment variable cannot be repeated, so a comma
  list is the only way to give a multi-value flag more than one value from the
  environment; a regression test pins that.

  The type implements pflag.SliceValue as well as pflag.Value, so the refusal also
  covers the config file, which bindFlags applies through Replace.
@JonJagger
JonJagger force-pushed the fix-empty-element-in-multi-value-flag branch from 8c90ed1 to cffebae Compare August 12, 2026 14:41
Comment thread cmd/kosli/nonEmptyStringSlice.go Outdated
…fusal

docs(flags): say how config and env values reach the empty-element refusal
  The comments claimed bindFlags applies a config file list through Replace. It
  does not: it stringifies the value with %v and applies it through Set, so
  Append, Replace and GetSlice have no caller in this CLI at all. Left as written,
  the next maintainer would look for a config path that does not exist, and might
  take the SliceValue methods for load-bearing code.

  They now say what is true: the refusal reaches config and environment values
  through Set, by the same path as the command line, and pflag.SliceValue is
  implemented as future-proofing for the first caller that asserts to it.

  A test now pins the config/env guarantee rather than leaving it to be inferred
  from the two mechanisms lining up. Without the new flag type it fails on
  "lstat : no such file or directory" - the empty element surviving into a
  confusing error rather than a clear refusal, which is the symptom this work
  exists to remove.

 Please enter the commit message for your changes. Lines starting
  --template and --attachments share the value type, so refusing an empty element
  from KOSLI_TEMPLATE already held transitively from the attachments env test.
  Nothing said so where a reader of createFlow would look, leaving them to find
  the guarantee in another command's test file.

  An environment variable cannot be repeated, so a comma list is the only way to
  name several required attestations that way, which makes it the shape that
  silently drops one.
@JonJagger
JonJagger enabled auto-merge (squash) August 12, 2026 15:28
@JonJagger
JonJagger merged commit a839a0e into main Aug 12, 2026
20 checks passed
@JonJagger
JonJagger deleted the fix-empty-element-in-multi-value-flag branch August 12, 2026 16:00
JonJagger added a commit that referenced this pull request Aug 15, 2026
…ield

    The audit drives the CLI, so it can only answer half the question. When an
    empty value produces the same request as omitting the flag - which is 172 of
    the 189 the CLI does not refuse - the server never receives anything empty to
    reject, and no amount of running the CLI harder will make it.

    replay.py asks the other half by capturing a request and sending it again with
    one field emptied. It reads the same spec.json, reuses the audit's own
    invocations so it captures what the audit measures, and writes results-api.tsv.
    A separate script rather than a third mode of audit.py: --ci asks the same
    question in another environment and its rows line up with the plain run's,
    while these rows are about a captured request and do not.

    The mapping from a flag to the payload field it controls is measured, not
    written down: the payload of a run with the flag set, diffed against one with
    it omitted. The field is emptied here rather than by the CLI, which is why this
    keeps working after the CLI starts refusing empty values.

    requests.go now logs the method beside the URL. Without it the probe would have
    needed a hand-written table of which endpoint takes which verb, which is the
    thing this approach exists to avoid.

    Of 412 rows, 19 are an answer about the server: 11 fields it accepts empty and
    8 it refuses. The eight refusals are mostly types rather than emptiness - a
    boolean, an enum, two objects, a URL. Among the eleven, `create flow
    --template` is refused by the CLI by the type added in #1092 and accepted by
    the API, which stores a flow requiring an attestation whose name is empty. That
    is the decision document's "what a CLI rule cannot reach" argument with a
    measured instance behind it.

    Two things the control replay caught, and both are the point of having one. A
    captured request that was never valid gives a 400 beside an emptied 400, which
    reads as a refusal and is not: 35 rows say so rather than pretending. And
    chasing why those controls failed led to the CLI audit's own controls, where
    114 of 412 runs with a supposedly real value fail. The audit invents the values
    it gives flags. Written up, with the tempting remedy measured and rejected:
    the server publishes a schema, and it would fix 6 of the 114.

    Most rows are not an answer, and mostly for good reasons the file states rather
    than hides. The read commands put their flags in the query string and send no
    body, which is 85 rows and the largest thing still missing. Some flags never
    leave the machine. `kosli fingerprint` sends nothing at all.

    Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
social4hyq pushed a commit to social4hyq/homebrew-core that referenced this pull request Sep 20, 2026
kosli-cli 2.36.6

Created-by: HarmonybrewBot
Commit-by: HarmonybrewBot
Merged-by: HarmonybrewBot
Description: Created by `brew bump`

---

Created with `brew bump-formula-pr`.<details>
  <summary>release notes</summary>
  <pre># Bug fixes
- Empty `--attachments` or `--template` elements (e.g. from unset shell variables) are now rejected with a clear error instead of being silently dropped.
- An empty `KOSLI_CONFIG_FILE` environment variable no longer suppresses config file loading; it is treated as unset and falls back to the default.
- An empty `KOSLI_API_TOKEN` environment variable no longer skips decryption of a config-file token.
- A boolean flag given an empty value (e.g. `--compliant ""` or `--new-compliance-status ""`) is now rejected with a clear error naming the flag, preventing the opposite compliance verdict from being recorded silently.
- Config file or environment values that cannot be applied to a flag now produce an error naming the flag and its source, instead of silently failing.

## What's Changed
* fix(attest): always serialise commits in pull request attestations by @dangrondahl in kosli-dev/cli#1083
* fix(docs): correct invalid regex in snapshot ecs help examples by @dangrondahl in kosli-dev/cli#1080
* fix(config): treat empty KOSLI_* env vars as unset, not as set by @JonJagger in kosli-dev/cli#1089
* fix(config): report config values that cannot be applied to their flag by @JonJagger in kosli-dev/cli#1090
* fix(flags): reject an empty element in a multi-value flag by @JonJagger in kosli-dev/cli#1092
* fix(flags): reject an empty value written after a boolean flag by @JonJagger in kosli-dev/cli#1091
* chore(deps): bump the github-actions-dependencies group with 3 updates by @dependabot[bot] in kosli-dev/cli#1084
* chore(deps): bump the go-dependencies group with 9 updates by @dependabot[bot] in kosli-dev/cli#1085
* chore(deps): bump anthropics/claude-code-action from 1.0.189 to 1.0.191 in the github-actions-dependencies group by @dependabot[bot] in kosli-dev/cli#1094
* chore(deps): bump the go-dependencies group with 7 updates by @dependabot[bot] in kosli-dev/cli#1095
* chore(deps): bump google.golang.org/protobuf from 1.36.12-0.20260120151049-f2248ac996af to 1.36.12 by @dependabot[bot] in kosli-dev/cli#1096


**Full Changelog**: https://github.com/kosli-dev/cli/compare/v2.36.5...v2.36.6</pre>
  <p>View the full release notes at <a href="https://github.com/kosli-dev/cli/releases/tag/v2.36.6">https://github.com/kosli-dev/cli/releases/tag/v2.36.6</a>.</p>
</details>
<hr>

See merge request: Harmonybrew/homebrew-core!16566
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants