From 5d1d22a3ffc11a8d3a8eadade8b7c5d9dd130da6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?FabioLeit=C3=A3o?= Date: Tue, 25 Aug 2026 22:00:35 +0000 Subject: [PATCH] docs(ops): document CodeQL dual-source conflict and Zizmor every-PR enforce Align operator and contributor docs with live workflows: GitHub default setup still conflicts with .github/workflows/codeql.yml SARIF upload, and zizmor.yml runs on every push/PR with job failure unless ZIZMOR_ENFORCE=false. Issue #1757 stays open until default setup is actually disabled. --- .github/workflows/codeql.yml | 7 +++- CONTRIBUTING.md | 2 +- CONTRIBUTING.pt_BR.md | 2 +- docs/GLOSSARY.md | 4 +-- docs/GLOSSARY.pt_BR.md | 4 +-- docs/QUALITY_WORKFLOW_RECOMMENDATIONS.md | 6 ++-- .../QUALITY_WORKFLOW_RECOMMENDATIONS.pt_BR.md | 6 ++-- docs/TESTING.md | 17 +++++++++- docs/TESTING.pt_BR.md | 19 +++++++++-- docs/ops/BRANCH_PROTECTION.md | 32 +++++++++++++++---- docs/ops/BRANCH_PROTECTION.pt_BR.md | 32 +++++++++++++++---- 11 files changed, 103 insertions(+), 28 deletions(-) diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index 8a4009f6b..e9aa39c8f 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -1,7 +1,12 @@ # CodeQL: security and quality analysis for Python (OWASP, injection, etc.). # Results appear under Security > Code scanning; configure rulesets for merge protection. # https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning -# (verify: default-setup disabled 2026-08-25, this custom workflow is the sole CodeQL source of truth now, badge restored -- issue #1757) +# +# Conflict: GitHub default setup (path dynamic/github-code-scanning/codeql) and this +# advanced workflow cannot both upload SARIF. When default setup is enabled, Analyze +# fails with: "CodeQL analyses from advanced configurations cannot be processed when +# the default setup is enabled." Check live state with `gh workflow list --all` +# (two rows named CodeQL). Keep init/analyze action SHAs in lockstep. Issue #1757. # # Actions pinned to full commit SHAs (tag in comment); bump via Dependabot or after upstream review. # diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4cda5d20d..dc7702319 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -155,7 +155,7 @@ GitHub **rulesets** plus classic branch protection apply to **`main`**. This sec - **Required status checks** (merge-blocking, ruleset `main-gate-pii`): **Test (Python 3.12)**, **Test (Python 3.13)**, **Test (Python 3.14)**. - **Signed commits** are required (classic protection and ruleset `restriction baseline`). Dependabot cannot receive unsigned pushes from Actions — use the sync workflow handoff or a signed supersede PR ([#1419](https://github.com/DataBoar/data-boar/issues/1419); [docs/ops/DEPENDABOT_REQUIREMENTS_SYNC.md](docs/ops/DEPENDABOT_REQUIREMENTS_SYNC.md)). - **Approving reviews** are **not** required (`required_approving_review_count: 0`). [`.github/CODEOWNERS`](.github/CODEOWNERS) exists for PII/security gate paths; the ruleset does **not** currently require code-owner review. -- **Lint, Bandit, Windows pytest, Semgrep, CodeQL, Sonar, Zizmor** usually run on the PR. Treat a red job as a stop even when GitHub would still allow merge. **`ZIZMOR_ENFORCE=true`** fails the Zizmor job on workflow-path changes; that job is **not** a required check. +- **Lint, Bandit, Windows pytest, Semgrep, CodeQL, Sonar, Zizmor** usually run on the PR. Treat a red job as a stop even when GitHub would still allow merge. **Zizmor** runs on **every** push/PR (**no** path filter) and **fails** unless **`ZIZMOR_ENFORCE=false`**; that job is **not** a required check. **CodeQL** uses `.github/workflows/codeql.yml`; do not leave GitHub **default setup** enabled at the same time ([docs/TESTING.md](docs/TESTING.md) *CodeQL*). - **Before you push:** run **`./scripts/check-all.sh`** or **`.\scripts\check-all.ps1`**. Merge helpers: [docs/ops/COMMIT_AND_PR.md](docs/ops/COMMIT_AND_PR.md), `scripts/pr-merge-when-green.ps1`. Do **not** change GitHub protection settings from a docs PR. If the live UI disagrees with the runbook, re-query `gh api` and update the runbook. diff --git a/CONTRIBUTING.pt_BR.md b/CONTRIBUTING.pt_BR.md index ff8184cd4..257553b91 100644 --- a/CONTRIBUTING.pt_BR.md +++ b/CONTRIBUTING.pt_BR.md @@ -153,7 +153,7 @@ O Cursor registra isso em **`.cursor/rules/git-pr-sync-before-advice.mdc`**. Vej - **Status checks obrigatórios** (bloqueiam merge, ruleset `main-gate-pii`): **Test (Python 3.12)**, **Test (Python 3.13)**, **Test (Python 3.14)**. - **Commits assinados** são obrigatórios (proteção clássica e ruleset `restriction baseline`). O Dependabot não recebe push unsigned do Actions — use o handoff do workflow de sync ou um PR substituto assinado ([#1419](https://github.com/DataBoar/data-boar/issues/1419); [docs/ops/DEPENDABOT_REQUIREMENTS_SYNC.pt_BR.md](docs/ops/DEPENDABOT_REQUIREMENTS_SYNC.pt_BR.md)). - **Revisões aprovadoras** **não** são obrigatórias (`required_approving_review_count: 0`). [`.github/CODEOWNERS`](.github/CODEOWNERS) cobre os caminhos do gate de PII/segurança; o ruleset **ainda não** exige revisão de code owner. -- **Lint, Bandit, pytest Windows, Semgrep, CodeQL, Sonar, Zizmor** costumam rodar no PR. Trate um job vermelho como pare mesmo quando o GitHub ainda permitiria merge. **`ZIZMOR_ENFORCE=true`** falha o job Zizmor em mudanças de workflow; esse job **não** é check obrigatório. +- **Lint, Bandit, pytest Windows, Semgrep, CodeQL, Sonar, Zizmor** costumam rodar no PR. Trate um job vermelho como pare mesmo quando o GitHub ainda permitiria merge. O **Zizmor** roda em **todo** push/PR (**sem** filtro de path) e **falha** salvo **`ZIZMOR_ENFORCE=false`**; esse job **não** é check obrigatório. O **CodeQL** usa `.github/workflows/codeql.yml`; **não** deixe o **default setup** do GitHub ligado ao mesmo tempo ([docs/TESTING.pt_BR.md](docs/TESTING.pt_BR.md) *CodeQL*). - **Antes do push:** rode **`./scripts/check-all.sh`** ou **`.\scripts\check-all.ps1`**. Ajudas de merge: [docs/ops/COMMIT_AND_PR.pt_BR.md](docs/ops/COMMIT_AND_PR.pt_BR.md), `scripts/pr-merge-when-green.ps1`. **Não** altere as configurações de proteção do GitHub num PR só de docs. Se a UI ao vivo divergir do runbook, consulte o `gh api` de novo e atualize o runbook. diff --git a/docs/GLOSSARY.md b/docs/GLOSSARY.md index e63e7ca0a..1c6686650 100644 --- a/docs/GLOSSARY.md +++ b/docs/GLOSSARY.md @@ -139,9 +139,9 @@ Terms are grouped by **theme** (product, scanning workflow, compliance, APIs, et | **Sigstore** | Keyless artifact / attestation signing with a public transparency log (cosign, Fulcio, Rekor); pairs with **provenance** and **SBOM** for verifiable supply chains. **See also:** **attestation**, **SBOM**. | | **SAST** | **Static Application Security Testing**: analysis of **source code** (and sometimes bytecode) without running the full app—e.g. **CodeQL**, **Semgrep**, **Bandit** in **CI**. Complements manual review and runtime tests; not a guarantee of absence of bugs. This repo runs selected tools in pipelines; see **SECURITY.md** and `.github/workflows/`. **See also:** **CVE**, **shift-left security**. | | **Bandit** | Python **SAST** linter for common security issues (subprocess, eval, weak crypto). Data Boar's strict CI gate runs `bandit -r . -c pyproject.toml -ll -ii` (MEDIUM+ severity **and** confidence). **See also:** **SAST**, **CI/CD**. | -| **CodeQL** | GitHub's semantic code-analysis engine: queries code **as a database** to find security patterns. Runs on push / PR and on a weekly schedule. **See also:** **SAST**, **Semgrep**. | +| **CodeQL** | GitHub's semantic code-analysis engine: queries code **as a database** to find security patterns. This repo's advanced workflow is `.github/workflows/codeql.yml` (push / PR / weekly). GitHub **default setup** (`dynamic/github-code-scanning/codeql`) **cannot** run at the same time — GitHub rejects advanced SARIF while default setup is enabled. **See also:** **SAST**, **Semgrep**; [TESTING.md](TESTING.md). | | **Semgrep** | Fast, pattern-based **SAST** across many languages. Data Boar runs the OSS `p/python` ruleset in CI. **See also:** **SAST**, **CodeQL**. | -| **zizmor** | Security linter for **GitHub Actions** workflows (token scope, script injection, unpinned actions). An **enforced** CI gate (issue #732), with an optional local pre-commit stage via `scripts/workflow-security-lint`. **See also:** **SAST**, **CI/CD**, **least privilege**. | +| **zizmor** | Security linter for **GitHub Actions** workflows (token scope, script injection, unpinned actions). The CI **job fails by default** (issue #732; opt out: repo variable **`ZIZMOR_ENFORCE=false`**). It is **not** a required merge check. Runs on every push/PR (**no** path filter). Optional local pre-commit stage via `scripts/workflow-security-lint`. **See also:** **SAST**, **CI/CD**, **least privilege**. | | **Gitleaks** | Secret-scanning tool that flags committed credentials and keys across the working tree and Git history. Runs as a dedicated CI gate. **See also:** **SAST**, **CI/CD**. | | **SonarQube / SonarCloud** | Continuous code-quality and security-hotspot platform; a token-gated CI gate tracking maintainability, reliability, and security findings. **See also:** **SAST**, **CI/CD**. | | **pip-audit** | Audits installed Python dependencies against known vulnerability advisories (**CVE** / PyPI); the CI dependency-audit gate. **See also:** **CVE**, **SBOM**. | diff --git a/docs/GLOSSARY.pt_BR.md b/docs/GLOSSARY.pt_BR.md index c47662293..665806e14 100644 --- a/docs/GLOSSARY.pt_BR.md +++ b/docs/GLOSSARY.pt_BR.md @@ -140,9 +140,9 @@ ust/boar_fast_filter/. **Contrato de performance:** o delta medido entre tracks | **Sigstore** | Assinatura de artefatos / *attestations* sem chave persistente, com *log* público de transparência (cosign, Fulcio, Rekor); combina com **provenance** e **SBOM** para *supply chains* verificáveis. **Veja também:** **attestation**, **SBOM**. | | **SAST** | ***Static Application Security Testing*** (teste estático de segurança de aplicação): análise do **código-fonte** (e às vezes *bytecode*) sem executar o app inteiro—ex.: **CodeQL**, **Semgrep**, **Bandit** no **CI**. Complementa revisão manual e testes dinâmicos; **não** garante ausência de falhas. O repositório roda ferramentas selecionadas nos pipelines; veja **SECURITY.pt_BR.md** e `.github/workflows/`. **Veja também:** **CVE**, **shift-left security**. | | **Bandit** | Linter **SAST** de Python para problemas comuns de segurança (subprocess, eval, criptografia fraca). O gate estrito de CI do Data Boar roda `bandit -r . -c pyproject.toml -ll -ii` (severidade **e** confiança MEDIUM+). **Veja também:** **SAST**, **CI/CD**. | -| **CodeQL** | Motor de análise semântica de código do GitHub: consulta o código **como banco de dados** para achar padrões de segurança. Roda em push / PR e em agendamento semanal. **Veja também:** **SAST**, **Semgrep**. | +| **CodeQL** | Motor de análise semântica de código do GitHub: consulta o código **como banco de dados** para achar padrões de segurança. O workflow avançado deste repositório é `.github/workflows/codeql.yml` (push / PR / semanal). O **default setup** do GitHub (`dynamic/github-code-scanning/codeql`) **não** pode rodar ao mesmo tempo — o GitHub recusa o SARIF avançado com o default setup ligado. **Veja também:** **SAST**, **Semgrep**; [TESTING.pt_BR.md](TESTING.pt_BR.md). | | **Semgrep** | **SAST** rápido baseado em padrões, multilíngua. O Data Boar roda o conjunto OSS `p/python` na CI. **Veja também:** **SAST**, **CodeQL**. | -| **zizmor** | Linter de segurança para workflows do **GitHub Actions** (escopo de token, injeção de script, actions sem pin). Gate de CI **obrigatório** (issue #732), com etapa opcional local de pre-commit via `scripts/workflow-security-lint`. **Veja também:** **SAST**, **CI/CD**, **least privilege**. | +| **zizmor** | Linter de segurança para workflows do **GitHub Actions** (escopo de token, injeção de script, actions sem pin). O job de CI **falha por padrão** (issue #732; opt-out: variável **`ZIZMOR_ENFORCE=false`**). **Não** é check obrigatório de merge. Roda em todo push/PR (**sem** filtro de path). Etapa local opcional de pre-commit via `scripts/workflow-security-lint`. **Veja também:** **SAST**, **CI/CD**, **least privilege**. | | **Gitleaks** | Ferramenta de varredura de segredos que sinaliza credenciais e chaves commitadas na árvore de trabalho e no histórico Git. Roda como gate de CI dedicado. **Veja também:** **SAST**, **CI/CD**. | | **SonarQube / SonarCloud** | Plataforma contínua de qualidade de código e *hotspots* de segurança; gate de CI condicionado a token, acompanhando manutenibilidade, confiabilidade e achados de segurança. **Veja também:** **SAST**, **CI/CD**. | | **pip-audit** | Audita dependências Python instaladas contra avisos de vulnerabilidade conhecidos (**CVE** / PyPI); o gate de auditoria de dependências na CI. **Veja também:** **CVE**, **SBOM**. | diff --git a/docs/QUALITY_WORKFLOW_RECOMMENDATIONS.md b/docs/QUALITY_WORKFLOW_RECOMMENDATIONS.md index 068c1aca3..b50b19cd6 100644 --- a/docs/QUALITY_WORKFLOW_RECOMMENDATIONS.md +++ b/docs/QUALITY_WORKFLOW_RECOMMENDATIONS.md @@ -7,7 +7,7 @@ This document suggests **additional layers** (tools, habits, and workflow) to ke ## What you already have (baseline) - **Tests:** Full pytest suite with `-W error`; SonarQube-style tests (S3981, S3776, S4423, S5706, S1192, etc.), security tests (SQL injection, path traversal, safe_load), markdown lint (including `.cursor/`). -- **CI:** Tests + `pip-audit` on every push/PR; optional SonarQube when `SONAR_TOKEN` is set; CodeQL in a separate workflow. +- **CI:** Tests + `pip-audit` on every push/PR; optional SonarQube when `SONAR_TOKEN` is set; CodeQL in a separate workflow (advanced `codeql.yml` — do not run GitHub default setup at the same time; [TESTING.md](TESTING.md) *CodeQL*). - **Cursor:** Rule and skill so the agent avoids SonarQube/CodeQL violations and runs quality tests after editing Python/markdown. - **Docs:** CONTRIBUTING (release checklist, audit, min versions), SECURITY.md, TESTING.md, deploy hardening. @@ -80,7 +80,7 @@ This document suggests **additional layers** (tools, habits, and workflow) to ke ## What we do (Zizmor): -- **CI:** [`.github/workflows/zizmor.yml`](../.github/workflows/zizmor.yml) runs on push/PR via **`zizmorcore/zizmor-action`** (SHA-pinned). **Advisory** by default unless **`ZIZMOR_ENFORCE`** / repo **`ENFORCE_ZIZMOR`** is set — see [OPERATOR_NOTIFICATION_CHANNELS.md](ops/OPERATOR_NOTIFICATION_CHANNELS.md) and **`scripts/workflow-security-lint.sh`**. +- **CI:** [`.github/workflows/zizmor.yml`](../.github/workflows/zizmor.yml) runs on **every** push/PR to `main`/`master` via **`zizmorcore/zizmor-action`** (SHA-pinned, **no** `paths:` filter). Enforcement is **on** unless repository variable **`ZIZMOR_ENFORCE=false`**. The job is **not** a required merge check. PR checkout uses **`head.sha`**; do **not** add a second `upload-sarif` step. Snapshot: [BRANCH_PROTECTION.md](ops/BRANCH_PROTECTION.md). Wrappers: **`scripts/workflow-security-lint.sh`**. - **Local:** `uvx zizmor .github/workflows/` (same path as **`workflow-security-lint.sh`**). **`check-all`** runs Zizmor in the **default** security tier and **fails** on findings (shift-left mirror of a strict posture before merge). **Prevents:** Workflow misconfigurations that Bandit/Semgrep do not see. @@ -118,7 +118,7 @@ Running **`check-all`** locally (default tier) trades **~2–5 minutes** on the | Finding class | Without local mirror | With `check-all` default tier | | ------------- | ------------------- | ----------------------------- | | Bandit on connectors | Fails in CI **Bandit** job after lint+test already ran | Surfaces in one local pass | -| Workflow permission drift | Fails in **Zizmor** workflow (or stays advisory until enforced) | Surfaces before push | +| Workflow permission drift | Fails in **Zizmor** job unless **`ZIZMOR_ENFORCE=false`** | Surfaces before push | | Semgrep SQLAlchemy FP path | Only when **`--enforced`** or CI Semgrep | Operator/agent opt-in — matches container command | **Token-aware habit:** Use **`./scripts/check-all.sh --skip-pre-commit`** only when iterating on tests; run **full** `check-all` before opening the PR. Use **`--enforced`** when the diff touches **security surfaces** listed in Wave 3 — not on every docs-only slice. diff --git a/docs/QUALITY_WORKFLOW_RECOMMENDATIONS.pt_BR.md b/docs/QUALITY_WORKFLOW_RECOMMENDATIONS.pt_BR.md index e4b186702..d8de8d78c 100644 --- a/docs/QUALITY_WORKFLOW_RECOMMENDATIONS.pt_BR.md +++ b/docs/QUALITY_WORKFLOW_RECOMMENDATIONS.pt_BR.md @@ -9,7 +9,7 @@ Este documento sugere **camadas adicionais** (ferramentas, hábitos e fluxo de t ## O que você já tem (linha de base) - **Testes:** Suite completa de pytest com `-W error`; testes estilo SonarQube (S3981, S3776, S4423, S5706, S1192, etc.), testes de segurança (SQL injection, path traversal, safe_load), lint de markdown (incluindo `.cursor/`). -- **CI:** Testes + `pip-audit` a cada push/PR; SonarQube opcional quando `SONAR_TOKEN` está definido; CodeQL em workflow separado. +- **CI:** Testes + `pip-audit` a cada push/PR; SonarQube opcional quando `SONAR_TOKEN` está definido; CodeQL em workflow separado (`codeql.yml` avançado — **não** rode o default setup do GitHub ao mesmo tempo; [TESTING.pt_BR.md](TESTING.pt_BR.md) *CodeQL*). - **Cursor:** Regra e skill para que o agente evite violações SonarQube/CodeQL e execute testes de qualidade após editar Python/markdown. - **Docs:** CONTRIBUTING (checklist de release, auditoria, versões mínimas), SECURITY.md, TESTING.md, hardening de deploy. @@ -82,7 +82,7 @@ Este documento sugere **camadas adicionais** (ferramentas, hábitos e fluxo de t #### O que fazemos (Zizmor): -- **CI:** [`.github/workflows/zizmor.yml`](../.github/workflows/zizmor.yml) roda em push/PR via **`zizmorcore/zizmor-action`** (SHA fixo). **Avisório** por padrão, salvo **`ZIZMOR_ENFORCE`** / **`ENFORCE_ZIZMOR`** no repo — veja [OPERATOR_NOTIFICATION_CHANNELS.md](ops/OPERATOR_NOTIFICATION_CHANNELS.md) e **`scripts/workflow-security-lint.sh`**. +- **CI:** [`.github/workflows/zizmor.yml`](../.github/workflows/zizmor.yml) roda em **todo** push/PR para `main`/`master` via **`zizmorcore/zizmor-action`** (SHA fixo, **sem** filtro `paths:`). O job **falha por padrão**, salvo a variável do repositório **`ZIZMOR_ENFORCE=false`**. O job **não** é check obrigatório de merge. Em PR o checkout usa **`head.sha`**; **não** adicione um segundo passo `upload-sarif`. Instantâneo: [BRANCH_PROTECTION.pt_BR.md](ops/BRANCH_PROTECTION.pt_BR.md). Wrappers: **`scripts/workflow-security-lint.sh`**. - **Local:** `uvx zizmor .github/workflows/` (mesmo caminho que **`workflow-security-lint.sh`**). O **`check-all`** executa Zizmor no tier **padrão** de security scans e **falha** em achados (postura shift-left antes do merge). **Previne:** Misconfigurações de workflow que Bandit/Semgrep não veem. @@ -120,7 +120,7 @@ Rodar **`check-all`** localmente (tier padrão) troca **~2–5 minutos** no PC d | Classe de achado | Sem espelho local | Com tier padrão do `check-all` | | ---------------- | ----------------- | ------------------------------ | | Bandit em connectors | Falha no job **Bandit** depois de lint+test | Surge num passe local | -| Drift de permissão em workflow | Falha no workflow **Zizmor** (ou fica avisório) | Surge antes do push | +| Drift de permissão em workflow | Falha no job **Zizmor** salvo **`ZIZMOR_ENFORCE=false`** | Surge antes do push | | Caminho FP SQLAlchemy no Semgrep | Só com **`--enforced`** ou Semgrep no CI | Opt-in operador/agente — mesmo comando do container | **Hábito token-aware:** Use **`./scripts/check-all.sh --skip-pre-commit`** só ao iterar testes; rode **`check-all` completo** antes de abrir o PR. Use **`--enforced`** quando o diff tocar **superfícies de segurança** da Onda 3 — não em todo slice só de docs. diff --git a/docs/TESTING.md b/docs/TESTING.md index 2bab9e655..fa39af3e0 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -136,7 +136,22 @@ GitHub Actions (`.github/workflows/ci.yml`) runs: 1. **Dependency audit** – `uv run pip-audit` after `uv sync` (Python 3.12). 1. **SonarQube / SonarCloud** – Code quality and security analysis when `SONAR_TOKEN` is set; uses Python 3.12 after tests pass. See [SonarQube / SonarCloud](#sonarqube--sonarcloud) below. -Code scanning (CodeQL) is in `.github/workflows/codeql.yml` and analyzes Python for security and quality findings. See the repository **Security** tab for CodeQL alerts. +### CodeQL (advanced workflow vs GitHub default setup) + +Tracked workflow: [`.github/workflows/codeql.yml`](../.github/workflows/codeql.yml). It analyzes **Python** with `queries: security-and-quality` on **push** / **pull_request** to `main`/`master` and a **weekly** schedule (`0 6 * * 1` UTC). Results land under **Security → Code scanning**. The README **CodeQL** badge points at this workflow file. Keep `github/codeql-action/init` and `.../analyze` on the **same** commit SHA (Dependabot may bump them separately). + +**Pitfall — two CodeQL sources:** GitHub **default setup** is a second workflow at path `dynamic/github-code-scanning/codeql` (not in the git tree). GitHub **rejects** SARIF from this advanced workflow while default setup is enabled (`CodeQL analyses from advanced configurations cannot be processed when the default setup is enabled` — that was the 2026-08-12 `main` failure). Check both: + +```bash +gh workflow list --all +# Two rows named "CodeQL" means default setup is still on alongside codeql.yml. +``` + +Do **not** re-enable default setup while `codeql.yml` is the intended source. CodeQL is **not** a required merge check ([BRANCH_PROTECTION.md](ops/BRANCH_PROTECTION.md)). Issue **#1757** tracks the badge vs dual-source migration. + +### Zizmor (workflow lint — every PR) + +[`.github/workflows/zizmor.yml`](../.github/workflows/zizmor.yml) runs on **every** `pull_request` / `push` to `main`/`master` (**no** `paths:` filter). A `code_scanning` ruleset needs a zizmor SARIF for **this** commit; filtering to `.github/workflows/**` deadlocks merge on docs-only PRs. The job **fails** unless repository variable **`ZIZMOR_ENFORCE=false`**. It is **not** in `required_status_checks`. On PRs, checkout is **`head.sha`** (not the merge commit) so the built-in SARIF upload hits `refs/pull//head`. Do **not** add a second `upload-sarif` step (duplicate category `zizmor`). Local: `uvx zizmor .github/workflows/` via **`check-all`**. Operator snapshot: [BRANCH_PROTECTION.md](ops/BRANCH_PROTECTION.md). ### Operator Slack workflows (not live-tested in pytest) diff --git a/docs/TESTING.pt_BR.md b/docs/TESTING.pt_BR.md index 496f122bb..f92febdd0 100644 --- a/docs/TESTING.pt_BR.md +++ b/docs/TESTING.pt_BR.md @@ -110,9 +110,24 @@ O GitHub Actions (`.github/workflows/ci.yml`) executa: - **Testes** — `uv run pytest -v -W error` no Ubuntu para **Python 3.12 e 3.13** (matriz, `fail-fast: false`). A instalação padrão é `uv sync --extra shares --group dev`. - **Testes com extras opcionais** — job **`test-extras`** (só Python **3.13**) instala extras SQL **exceto** `mariadb`, mais `nosql` + `compressed` + `dataformats` (+ `shares`) para os testes de conectores opcionais rodarem em vez de pular; um teto de skips (**60**) falha o job se os pulos silenciosos crescerem (issue **#1638**). Os guardrails consumer-side do Maestro (`tests/test_maestro_scripts.py` e os casos gateados por `MAESTRO_ROOT` em `tests/test_issue_dev_license_qa.py` / `tests/test_security.py`) são **deselecionados** neste job: o CI público e os forks não devem clonar o **DataBoar/maestro** privado, e esses testes pulam de propósito quando o clone está ausente (spinout maestro#8, “typical public CI”). Eles ainda rodam na matriz Test padrão (pulam se não houver clone irmão). Um job **opt-in** futuro pode executar os guards do Maestro de verdade; não pendure isso no **`test-extras`**. O extra `mariadb` fica de fora neste job 3.13: PyPI **1.1.14** (estável mais recente) levanta `SyntaxError` no import (`connectionpool.py` sem raw-string); **2.0.0** ainda é só rc. Restaure `sql-all` / `--extra mariadb` quando um conector estável importar no 3.13. - **Auditoria de dependências** — `uv run pip-audit` após `uv sync` (Python 3.12). -- **SonarQube/SonarCloud** — quando `SONAR_TOKEN` está definido; usa Python 3.12 após os testes passarem. +- **SonarQube/SonarCloud** — quando `SONAR_TOKEN` está definido; usa Python 3.12 após os testes passarem. O mesmo `sonar-project.properties` na raiz do repositório é usado pelo extension no IDE e pelo scanner no CI. -O CodeQL está em `.github/workflows/codeql.yml`. O mesmo `sonar-project.properties` na raiz do repositório é usado pelo extension no IDE e pelo scanner no CI. +### CodeQL (workflow avançado vs default setup do GitHub) + +Workflow versionado: [`.github/workflows/codeql.yml`](../.github/workflows/codeql.yml). Analisa **Python** com `queries: security-and-quality` em **push** / **pull_request** para `main`/`master` e numa agenda **semanal** (`0 6 * * 1` UTC). Os resultados aparecem em **Security → Code scanning**. O badge **CodeQL** do README aponta para este arquivo. Mantenha `github/codeql-action/init` e `.../analyze` no **mesmo** SHA de commit (o Dependabot pode subir os dois pins em PRs separados). + +**Armadilha — duas fontes CodeQL:** o **default setup** do GitHub é um segundo workflow no caminho `dynamic/github-code-scanning/codeql` (não está no git). O GitHub **recusa** SARIF deste workflow avançado enquanto o default setup está ligado (`CodeQL analyses from advanced configurations cannot be processed when the default setup is enabled` — essa foi a falha em `main` em 2026-08-12). Confira os dois: + +```bash +gh workflow list --all +# Duas linhas com o nome "CodeQL" significam que o default setup ainda está ligado junto com o codeql.yml. +``` + +**Não** religue o default setup enquanto o `codeql.yml` for a fonte pretendida. O CodeQL **não** é check obrigatório de merge ([BRANCH_PROTECTION.pt_BR.md](ops/BRANCH_PROTECTION.pt_BR.md)). A issue **#1757** acompanha o badge e a migração das duas fontes. + +### Zizmor (lint de workflow — todo PR) + +[`.github/workflows/zizmor.yml`](../.github/workflows/zizmor.yml) roda em **todo** `pull_request` / `push` para `main`/`master` (**sem** filtro `paths:`). Um ruleset `code_scanning` precisa de SARIF do zizmor para **este** commit; filtrar só `.github/workflows/**` trava o merge em PRs só de docs. O job **falha** salvo a variável do repositório **`ZIZMOR_ENFORCE=false`**. Ele **não** está em `required_status_checks`. Em PRs, o checkout é **`head.sha`** (não o merge commit) para o upload SARIF interno cair em `refs/pull//head`. **Não** adicione um segundo passo `upload-sarif` (categoria `zizmor` duplicada). Local: `uvx zizmor .github/workflows/` via **`check-all`**. Instantâneo do operador: [BRANCH_PROTECTION.pt_BR.md](ops/BRANCH_PROTECTION.pt_BR.md). ### Workflows Slack do operador (sem teste “vivo” no pytest) diff --git a/docs/ops/BRANCH_PROTECTION.md b/docs/ops/BRANCH_PROTECTION.md index aa6f4f7fa..27940e3d6 100644 --- a/docs/ops/BRANCH_PROTECTION.md +++ b/docs/ops/BRANCH_PROTECTION.md @@ -60,18 +60,38 @@ From ruleset `main-gate-pii` `pull_request` parameters (2026-08-19): ## `ZIZMOR_ENFORCE` -| Fact | Value (2026-08-19) | -| --- | --- | -| Repo Actions variable | **`ZIZMOR_ENFORCE=true`** | -| Workflow | [`.github/workflows/zizmor.yml`](../../.github/workflows/zizmor.yml) | -| When it runs | Every `pull_request` / `push` to `main`/`master` (**no** `paths:` filter — a `code_scanning` ruleset that requires zizmor needs a result for that commit and ref); weekly schedule; `workflow_dispatch` | +| Fact | Value (2026-08-19) | +| --- | --- | +| Repo Actions variable | **`ZIZMOR_ENFORCE=true`** | +| Workflow | [`.github/workflows/zizmor.yml`](../../.github/workflows/zizmor.yml) | +| When it runs | Every `pull_request` / `push` to `main`/`master` (**no** `paths:` filter — a `code_scanning` ruleset that requires zizmor needs a result for that commit and ref); weekly schedule; `workflow_dispatch` | | Job behaviour | With the variable **not** `false`, a zizmor finding **fails the job** (`ENFORCE_ZIZMOR`). That job is **not** in `required_status_checks`. On `pull_request`, checkout **`head.sha`** so `zizmor-action` **`advanced-security`** uploads once to **`refs/pull//head`** (ruleset **21118976**). Do **not** add a second `upload-sarif` in the same job (duplicate category `zizmor`). | -| Local / `check-all` | Advisory unless `DATA_BOAR_ENFORCE_ZIZMOR` / `-Enforce` — see [WORKFLOW_DEFERRED_FOLLOWUPS.md](WORKFLOW_DEFERRED_FOLLOWUPS.md) and `scripts/workflow-security-lint.*`. | +| Local / `check-all` | **`./scripts/check-all.sh`** default security tier **fails** on zizmor findings (`scripts/check-all-security-scans.sh`). Standalone **`scripts/workflow-security-lint.sh`** stays advisory unless `--enforce` / `DATA_BOAR_ENFORCE_ZIZMOR=true`. | **Re-check:** `gh api repos/DataBoar/data-boar/actions/variables/ZIZMOR_ENFORCE` Do not turn the variable **off** to silence a finding. Fix the workflow YAML (or use a documented, operator-approved exception). Same posture as other security gates. +## CodeQL sources (advanced vs default setup) + +**Verified:** `gh workflow list --all` on **2026-08-25** against `DataBoar/data-boar`. + +| Fact | Value (2026-08-25) | +| --- | --- | +| Advanced (in git) | [`.github/workflows/codeql.yml`](../../.github/workflows/codeql.yml) — `state: active`. README badge points here. | +| Default setup (not in git) | Path `dynamic/github-code-scanning/codeql` — still `state: active` when last checked. | +| Conflict | GitHub **does not process** advanced-config SARIF while default setup is enabled. Symptom: job **Analyze (Python)** fails with *CodeQL analyses from advanced configurations cannot be processed when the default setup is enabled* (seen on `main` **2026-08-12**). | +| Required merge check? | **No** — CodeQL stays advisory in `required_status_checks` (table above). | + +## Re-check CodeQL workflows: + +```bash +gh workflow list --all +gh run list --workflow=codeql.yml --limit 5 +``` + +If two workflows named **CodeQL** are `active`, disable **default setup** in the GitHub **Settings → Code security → CodeQL** UI (or re-disable the dynamic workflow) **before** treating `codeql.yml` as the only scanner. Issue **#1757**. Contributor write-up: [TESTING.md](../TESTING.md) *CodeQL*. + ## Heat model (cold → warm → hot) Same tokens as [WORKFLOW_DEFERRED_FOLLOWUPS.md](WORKFLOW_DEFERRED_FOLLOWUPS.md): diff --git a/docs/ops/BRANCH_PROTECTION.pt_BR.md b/docs/ops/BRANCH_PROTECTION.pt_BR.md index 744fccff8..b37adeff7 100644 --- a/docs/ops/BRANCH_PROTECTION.pt_BR.md +++ b/docs/ops/BRANCH_PROTECTION.pt_BR.md @@ -60,18 +60,38 @@ Parâmetros `pull_request` do ruleset `main-gate-pii` (2026-08-19): ## `ZIZMOR_ENFORCE` -| Fato | Valor (2026-08-19) | -| --- | --- | -| Variável Actions do repositório | **`ZIZMOR_ENFORCE=true`** | -| Workflow | [`.github/workflows/zizmor.yml`](../../.github/workflows/zizmor.yml) | -| Quando roda | Todo `pull_request` / `push` em `main`/`master` (**sem** filtro `paths:` — um ruleset `code_scanning` que exige zizmor precisa de resultado para aquele commit e ref); agenda semanal; `workflow_dispatch` | +| Fato | Valor (2026-08-19) | +| --- | --- | +| Variável Actions do repositório | **`ZIZMOR_ENFORCE=true`** | +| Workflow | [`.github/workflows/zizmor.yml`](../../.github/workflows/zizmor.yml) | +| Quando roda | Todo `pull_request` / `push` em `main`/`master` (**sem** filtro `paths:` — um ruleset `code_scanning` que exige zizmor precisa de resultado para aquele commit e ref); agenda semanal; `workflow_dispatch` | | Comportamento do job | Com a variável **diferente** de `false`, um achado do zizmor **falha o job** (`ENFORCE_ZIZMOR`). Esse job **não** está em `required_status_checks`. Em `pull_request`, checkout em **`head.sha`** para o upload **`advanced-security`** do `zizmor-action` cair em **`refs/pull//head`** (ruleset **21118976**). **Não** adicionar segundo `upload-sarif` no mesmo job (categoria `zizmor` duplicada). | -| Local / `check-all` | Consultivo salvo `DATA_BOAR_ENFORCE_ZIZMOR` / `-Enforce` — veja [WORKFLOW_DEFERRED_FOLLOWUPS.pt_BR.md](WORKFLOW_DEFERRED_FOLLOWUPS.pt_BR.md) e `scripts/workflow-security-lint.*`. | +| Local / `check-all` | O tier padrão de segurança do **`./scripts/check-all.sh`** **falha** em achados do zizmor (`scripts/check-all-security-scans.sh`). O **`scripts/workflow-security-lint.sh`** isolado permanece consultivo salvo `--enforce` / `DATA_BOAR_ENFORCE_ZIZMOR=true`. | **Conferir de novo:** `gh api repos/DataBoar/data-boar/actions/variables/ZIZMOR_ENFORCE` Não desligue a variável para silenciar um achado. Corrija o YAML do workflow (ou use exceção documentada e aprovada pelo operador). Mesma postura dos outros gates de segurança. +## Fontes CodeQL (avançado vs default setup) + +**Verificado:** `gh workflow list --all` em **2026-08-25** contra `DataBoar/data-boar`. + +| Fato | Valor (2026-08-25) | +| --- | --- | +| Avançado (no git) | [`.github/workflows/codeql.yml`](../../.github/workflows/codeql.yml) — `state: active`. O badge do README aponta para cá. | +| Default setup (fora do git) | Caminho `dynamic/github-code-scanning/codeql` — ainda `state: active` na última checagem. | +| Conflito | O GitHub **não processa** SARIF de configuração avançada enquanto o default setup está ligado. Sintoma: o job **Analyze (Python)** falha com *CodeQL analyses from advanced configurations cannot be processed when the default setup is enabled* (visto em `main` em **2026-08-12**). | +| Check obrigatório de merge? | **Não** — o CodeQL permanece consultivo em `required_status_checks` (tabela acima). | + +## Conferir de novo os workflows CodeQL: + +```bash +gh workflow list --all +gh run list --workflow=codeql.yml --limit 5 +``` + +Se dois workflows com o nome **CodeQL** estiverem `active`, desligue o **default setup** em **Settings → Code security → CodeQL** no GitHub (ou desative o workflow dinâmico) **antes** de tratar o `codeql.yml` como o único scanner. Issue **#1757**. Texto para quem contribui: [TESTING.pt_BR.md](../TESTING.pt_BR.md) *CodeQL*. + ## Modelo de calor (cold → warm → hot) Os mesmos tokens de [WORKFLOW_DEFERRED_FOLLOWUPS.pt_BR.md](WORKFLOW_DEFERRED_FOLLOWUPS.pt_BR.md):