Skip to content

ci: add Spectral as a second lint gate - #129

Closed
LukasParke wants to merge 7 commits into
ci/pr-validationfrom
ci/spectral-gate
Closed

LukasParke wants to merge 7 commits into
ci/pr-validationfrom
ci/spectral-gate

Conversation

@LukasParke

Copy link
Copy Markdown
Owner

Stacked on #128. Evaluated three additional validation tools against the spec, per request; the winner joins CI.

Evaluation

Tool Result on the fixed spec Verdict
Scalar CLI (document validate) Passes; structural OpenAPI 3.1 validation only, no lint rules No added value over the existing gates
vacuum 1 error, ~1,400 warnings/infos — overwhelmingly style noise inapplicable here (paths-kebab-case on paths Plex defines, description-duplication, camel-case-properties) Fast, but poor default signal-to-noise; ruleset engine is Spectral-compatible anyway
Spectral (spectral:oas) 95 warnings: 89 false positives + 6 real findings Redocly's recommended set does not cover Winner — added to CI

Spectral's real findings, fixed here:

  • operation-tag-defined: the four plex.tv-hosted operations (/user, /users/signin, /users, /resources) used tags (Authentication, Users, Plex) never declared in the global tags list. Now declared with descriptions and grouped under a new Plex.tv tag group.
  • info-description / info-contact: info had neither. Added a concise description (JSON-via-Accept guidance, link to plexapi.dev) and a contact pointing at this repo.

The 89 false positives are oas3-operation-security-defined complaining about shared user / admin in security requirement arrays on the token apiKey scheme. That rule implements OpenAPI 3.0 semantics (array must be empty for non-OAuth2 schemes); this document is 3.1, where role names are explicitly allowed. Disabled in .spectral.yaml with a comment.

CI

Spectral runs in the existing lint job with --fail-severity=warn (any warning fails). Verified locally: clean spec passes; an op with an undeclared tag fails with exit 1.

Redocly stays as the primary gate rather than being replaced: it is the only tool of the four with per-pointer ignore-file baselining (the ratchet), and it caught whole classes the others missed (duplicate parameters, schema-invalid examples). Spectral covers the rule families Redocly's recommended set lacks. Both run in seconds.

- Remove duplicate inline X-Plex-* header parameters on the three
  universal transcode operations and /:/timeline (operation-parameters-unique),
  referencing the existing component parameters instead
- Fix resolution pattern ^\d[x:]\d$ -> ^\d+[x:]\d+$ so real values
  like 1080x1080 validate (videoResolution/photoResolution)
- Fix invalid examples: 'Burn' -> 'burn' (enum), string examples typed as
  numbers, array parameters with scalar examples, Guid.id multi-example
  map -> JSON Schema examples array, Policy.value type mismatch
- Change /library/file url parameter format from non-standard 'url' to 'uri'
- Remove unused components: Title, Type schemas and the 500 response

redocly lint: 16 errors / 148 warnings -> 0 errors / 118 warnings
- Add pr_validation.yaml: Redocly lint (strict config in redocly.yaml,
  pre-existing violations baselined in .redocly.lint-ignore.yaml so the
  count can only shrink) plus an oasdiff breaking-change gate that fails
  unless the PR carries the 'breaking-change' label
- Elevate operation-4xx-response, invalid-example, unused-component, and
  ambiguous-path rules to error severity for new violations
- Fix sdk_generation.yaml paths filter: './plex-api-spec.yaml' never
  matched, so spec pushes did not trigger generation (masked by the cron)
…ject

/library/sections/{sectionId}/autocomplete, /common, and /firstCharacters
declared inline 'type' (and 'sort' on firstCharacters) query parameters
that the form/explode mediaQuery object parameter already serializes,
so the same query key was documented twice per operation. The mediaQuery
versions are also better typed: 'type' is the MediaType enum and 'sort'
is a string supporting modifiers like duration:desc (the inline 'sort'
was implausibly typed integer).

This also made oasdiff's parameter matching nondeterministic (phantom
breaking changes on identical specs), which would have broken the CI
breaking-change gate.
- Declare the Authentication, Users, and Plex tags used by the four
  plex.tv-hosted operations (/user, /users/signin, /users, /resources)
  and group them under a new 'Plex.tv' tag group
  (operation-tag-defined)
- Add info.description and info.contact (info-description, info-contact)
- Add .spectral.yaml: spectral:oas ruleset with
  oas3-operation-security-defined disabled, since this document is
  OpenAPI 3.1 where role names in security requirement arrays are legal
  for non-OAuth2 schemes and Spectral implements the 3.0 semantics
Evaluated vacuum, Spectral, and Scalar CLI against the spec:
- Scalar: structural validation only, no lint rules; spec already passes
- vacuum: fast, but default ruleset is ~1,400 findings of mostly
  inapplicable style noise (kebab-case paths, description-duplication)
- Spectral: highest signal; caught undeclared tags and missing info
  fields that Redocly's recommended set does not cover

Spectral joins Redocly in the lint job with --fail-severity=warn, so any
warning fails. Redocly stays as the primary gate: it is the only tool
with per-pointer ignore-file baselining, and it catches classes the
others missed (duplicate parameters, invalid examples).
@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 17e57131-736d-4242-b2af-e0e7168268e5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@LukasParke LukasParke closed this Aug 13, 2026
@LukasParke
LukasParke deleted the ci/spectral-gate branch August 13, 2026 01:31
@LukasParke

Copy link
Copy Markdown
Owner Author

Agent: Retired after rebasing onto the post-#117 main. The two spec issues Spectral caught (undeclared tags, missing info fields) were fixed by the regeneration, and validate.yml now runs Redocly + Speakeasy + vacuum. On the regenerated spec, Spectral's remaining output is 306 invalid-example errors (already reported by Redocly, pending a generator-side fix), 167 OpenAPI 3.1 false positives (oas3-operation-security-defined implements 3.0 semantics), and 2 trailing-slash warnings already ratcheted in redocly.yaml. A fourth linter whose every rule would need disabling to go green adds maintenance cost with no signal, so it stays out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant