Add detected requirements for secrets, vars, and environments

Implement sync-time workflow reference discovery and dispatch-time missing-configuration detection. Workflows now record which secrets, variables, and environments they reference via YAML metadata. Two new read endpoints compute "referenced but not configured" sets at query time, and UI cards offer one-click create flows to satisfy them. Names only—never values. Adds database queries, handlers, enhanced workflow parsing, frontend components, and types for the full feature flow including preset dialogs and bound-workflow listings.

Author: BotCoder254

CLAUDE.md

@@ -177,6 +177,34 @@ catalog (summary strip + URL-synced filters + keyset infinite scroll + detail pa
 audit history) plus a create/replace dialog that never echoes values and a security
 posture card that warns when `SECRETS_MASTER_KEY` is unset.
 
+**Detected requirements (sync-time reference discovery).** Workflow parsing records what
+each workflow *expects* as structured metadata in `workflows.metadata` JSONB — no
+migration: `secretRefs` (`${{ secrets.NAME }}`, dot AND bracket `secrets['NAME']` forms,
+word-boundary checked so `mysecrets.` never matches, `GITHUB_TOKEN` excluded),
+`varRefs` (`${{ vars.NAME }}` — detect/display only; the platform doesn't manage plain
+variables), and `environments` (distinct job `environment:` names, case-insensitively
+deduped, slug-filtered so dynamic `${{ … }}` names are never stored). All scanners share
+`workflow_parse::scan_context_refs` (identifier `[A-Za-z_][A-Za-z0-9_]*` ≤200 B, dedupe,
+cap 100, sort). Two read endpoints compute "referenced but not configured" as a set
+difference at QUERY time (`db/workflows.rs::missing_secret_refs/workspace_var_refs/
+missing_environment_refs` — `jsonb_array_elements_text` LATERAL with a `jsonb_typeof`
+guard so pre-feature/failed-parse metadata reads as empty; anti-join semantics: a secret
+name counts as configured via workspace scope, same-repo repository scope, or any
+environment scope): `GET …/secrets/requirements` (`secrets.read`) and
+`GET …/environments/requirements` (`content.read`). Handlers re-filter names at read
+time against the configurable-name allow-lists (defense-in-depth against hand-edited
+metadata) and cap output (200 names, 20 refs/name, true `referenceCount`). YAML never
+mints rows — the UI offers one-click create through the ordinary RBAC'd endpoints:
+`DetectedRequirementsCard` (secrets page right rail; "Add" pre-fills `SecretFormDialog`
+via `presetName`/`presetRepository`, repo scope only when every reference shares one
+repo) and `DetectedEnvironmentsCard` (environments page; "Create" pre-fills
+`EnvironmentFormDialog` via `presetName`). Both requirements queries live under their
+feature's react-query prefix, so every mutation's existing invalidation clears satisfied
+warnings automatically. The environment detail page lists `boundWorkflows` (from
+`list_binding_environment`, case-insensitive) linking to the workflow detail pages.
+Detection refreshes on every repo sync; workflows synced before this feature light up
+after their next push/manual re-sync.
+
 **Notification Center (operational inbox).** Notifications are a per-user, actionable
 PROJECTION of the immutable `audit_logs` ledger — the Activity Feed keeps the complete
 history, notifications hold only what a user should act on (OWASP's audit-vs-messaging
@@ -265,14 +293,18 @@ overup/
 │   │   ├── pipelines/          # execution ledger + live detail workspace: PipelineGraph,
 │   │   │                       #   ExecutionTimeline, LogViewer (xterm), tab panels,
 │   │   │                       #   usePipelineStream (WS), stores/logStore (zustand)
-│   │   ├── artifacts/          # workspace artifact catalog: summary strip, URL-synced
-│   │   │                       #   filters, keyset infinite scroll, provenance detail page
+│   │   ├── artifacts/          # workspace artifact catalog: summary strip (4 KPI cells —
+│   │   │                       #   the house norm), URL-synced filters, keyset infinite
+│   │   │                       #   scroll, provenance detail page
 │   │   ├── secrets/            # write-only encrypted secrets: catalog + posture column,
-│   │   │                       #   create/replace dialog (value never echoed), detail
-│   │   │                       #   page with audit history
+│   │   │                       #   detected-requirements card (missing secretRefs +
+│   │   │                       #   vars, one-click Add), create/replace dialog (value
+│   │   │                       #   never echoed), detail page with audit history
 │   │   ├── environments/       # deployment environments: catalog (summary strip +
-│   │   │                       #   URL-synced search + keyset infinite scroll), detail
-│   │   │                       #   page with scoped secrets + audit, create/edit dialog
+│   │   │                       #   URL-synced search + keyset infinite scroll +
+│   │   │                       #   detected-environments card w/ one-click create),
+│   │   │                       #   detail page with scoped secrets + bound workflows +
+│   │   │                       #   audit, create/edit dialog
 │   │   ├── notifications/      # operational inbox: NotificationBell (badge, popover/
 │   │   │                       #   bottom-sheet switch), NotificationPanel, history page,
 │   │   │                       #   PreferencesDialog, useNotificationStream (per-user WS)
@@ -558,6 +590,13 @@ into a traversal-safe tar streamed into the container via the Docker archive API
   `environment.created/updated/deleted` audit rows in-transaction, and a delete cascade
   audits every removed secret. A secret create with an `environmentId` from another
   workspace gets the same flat error as a bad `repositoryId` — no existence oracle
+- **Detected requirements are read-only projections of parser output**: sync-time
+  scanners only store allow-listed identifiers (charset + length + caps) into
+  `workflows.metadata`; the requirements endpoints re-filter names at read time against
+  the same configurable-name rules, cap output (200 names / 20 refs each), ride the
+  sibling read permissions (`secrets.read` / `content.read`), and expose names only —
+  never values. YAML never mints secrets or environments: "Add"/"Create" go through the
+  ordinary RBAC'd mutation endpoints
 - **Notifications are self-scoped by construction**: every read/mutation predicate pins
   `user_id = caller` in SQL (a forged notification id 404s flat), the live hub is keyed by
   the AUTHENTICATED user id (routing, not filtering, is the isolation), titles/bodies are

backend/src/db/workflows.rs

@@ -228,6 +228,140 @@ pub struct PushRunnableWorkflow {
     pub raw_content: String,
 }
 
+/// One workflow-YAML reference to a secret/var/environment name, with the
+/// repository and workflow it came from. Names only — never values.
+#[derive(Debug, sqlx::FromRow)]
+pub struct RequirementRefRow {
+    pub name: String,
+    pub repository_id: Uuid,
+    pub repository_name: String,
+    pub workflow_id: Uuid,
+    pub workflow_path: String,
+}
+
+/// CTE expanding one metadata ref array across a workspace's workflows.
+/// `key` is always a compile-time literal (`secretRefs`/`varRefs`/
+/// `environments`) — never user input; every user-facing value stays a bind.
+/// The `jsonb_typeof` guard makes workflows synced before a key existed (and
+/// failed parses, whose metadata is `{}`) read as empty rather than erroring.
+fn refs_cte(key: &str) -> String {
+    format!(
+        r#"
+        WITH refs AS (
+            SELECT ref.name AS name,
+                   r.id AS repository_id,
+                   r.full_name AS repository_name,
+                   w.id AS workflow_id,
+                   w.path AS workflow_path
+            FROM workflows w
+            JOIN repositories r ON r.id = w.repository_id
+            CROSS JOIN LATERAL jsonb_array_elements_text(
+                CASE WHEN jsonb_typeof(w.metadata->'{key}') = 'array'
+                     THEN w.metadata->'{key}' ELSE '[]'::jsonb END
+            ) AS ref(name)
+            WHERE r.workspace_id = $1
+        )
+        "#
+    )
+}
+
+/// Secret names referenced by workflow YAML with no configured secret that
+/// could satisfy them: a name counts as configured when a workspace-scoped
+/// secret, a repository-scoped secret on the referencing repo, or an
+/// environment-scoped secret in any environment carries it (which environment
+/// applies is a dispatch-time question — "any environment" keeps the rule
+/// simple and matches how precedence is explained in the UI).
+pub async fn missing_secret_refs(
+    pool: &PgPool,
+    workspace_id: Uuid,
+) -> sqlx::Result<Vec<RequirementRefRow>> {
+    let sql = refs_cte("secretRefs")
+        + r#"
+        SELECT name, repository_id, repository_name, workflow_id, workflow_path
+        FROM refs
+        WHERE NOT EXISTS (
+            SELECT 1 FROM secrets s
+            WHERE s.workspace_id = $1
+              AND s.name = refs.name
+              AND (
+                   (s.repository_id IS NULL AND s.environment_id IS NULL)
+                OR s.repository_id = refs.repository_id
+                OR s.environment_id IS NOT NULL
+              )
+        )
+        ORDER BY name, repository_name, workflow_path
+        LIMIT 1000
+        "#;
+    sqlx::query_as::<_, RequirementRefRow>(&sql)
+        .bind(workspace_id)
+        .fetch_all(pool)
+        .await
+}
+
+/// Every `${{ vars.NAME }}` reference across the workspace — informational
+/// only (the platform doesn't manage plain variables), so no anti-join.
+pub async fn workspace_var_refs(
+    pool: &PgPool,
+    workspace_id: Uuid,
+) -> sqlx::Result<Vec<RequirementRefRow>> {
+    let sql = refs_cte("varRefs")
+        + r#"
+        SELECT name, repository_id, repository_name, workflow_id, workflow_path
+        FROM refs
+        ORDER BY name, repository_name, workflow_path
+        LIMIT 1000
+        "#;
+    sqlx::query_as::<_, RequirementRefRow>(&sql)
+        .bind(workspace_id)
+        .fetch_all(pool)
+        .await
+}
+
+/// Environment names bound by workflow YAML with no matching environment row
+/// (case-insensitive, like dispatch resolution).
+pub async fn missing_environment_refs(
+    pool: &PgPool,
+    workspace_id: Uuid,
+) -> sqlx::Result<Vec<RequirementRefRow>> {
+    let sql = refs_cte("environments")
+        + r#"
+        SELECT name, repository_id, repository_name, workflow_id, workflow_path
+        FROM refs
+        WHERE NOT EXISTS (
+            SELECT 1 FROM environments e
+            WHERE e.workspace_id = $1 AND lower(e.name) = lower(refs.name)
+        )
+        ORDER BY name, repository_name, workflow_path
+        LIMIT 1000
+        "#;
+    sqlx::query_as::<_, RequirementRefRow>(&sql)
+        .bind(workspace_id)
+        .fetch_all(pool)
+        .await
+}
+
+/// Workflows whose YAML binds the given environment name (case-insensitive)
+/// — feeds the environment detail page's "bound workflows" list.
+pub async fn list_binding_environment(
+    pool: &PgPool,
+    workspace_id: Uuid,
+    name: &str,
+) -> sqlx::Result<Vec<RequirementRefRow>> {
+    let sql = refs_cte("environments")
+        + r#"
+        SELECT name, repository_id, repository_name, workflow_id, workflow_path
+        FROM refs
+        WHERE lower(refs.name) = lower($2)
+        ORDER BY repository_name, workflow_path
+        LIMIT 50
+        "#;
+    sqlx::query_as::<_, RequirementRefRow>(&sql)
+        .bind(workspace_id)
+        .bind(name)
+        .fetch_all(pool)
+        .await
+}
+
 pub async fn list_jobs(pool: &PgPool, workflow_id: Uuid) -> sqlx::Result<Vec<WorkflowJobRow>> {
     sqlx::query_as::<_, WorkflowJobRow>(
         r#"

backend/src/handlers/environments.rs

@@ -142,6 +142,30 @@ pub async fn summary(
     })))
 }
 
+/// GET /api/workspaces/{workspace_id}/environments/requirements
+///
+/// Environment names bound by workflow YAML (`environment:`) with no
+/// matching environment row — detected at sync time, computed as a set
+/// difference at query time. Deliberately never auto-created: YAML must not
+/// mint workspace resources past RBAC; the UI offers a one-click create
+/// through the ordinary RBAC'd endpoint instead.
+pub async fn requirements(
+    State(state): State<AppState>,
+    CurrentUser(user): CurrentUser,
+    Path(workspace_id): Path<Uuid>,
+) -> AppResult<Json<serde_json::Value>> {
+    authz::require_permission(&state.pool, user.id, workspace_id, authz::CONTENT_READ).await?;
+
+    let missing = db::workflows::missing_environment_refs(&state.pool, workspace_id).await?;
+    Ok(Json(json!({
+        // Read-time re-filter (the secrets requirements pattern): only names
+        // an environment row could actually take surface as creatable.
+        "environments": super::secrets::group_requirements(missing, |name| {
+            validate_name(name).is_ok()
+        }),
+    })))
+}
+
 #[derive(Debug, Deserialize)]
 #[serde(rename_all = "camelCase")]
 pub struct AuditQuery {
@@ -175,9 +199,24 @@ pub async fn detail(
         .ok_or(AppError::NotFound)?;
     let audit =
         db::environments::list_audit(&state.pool, workspace_id, Some(environment_id), 20).await?;
+    // Workflows whose YAML binds this environment (case-insensitive, like
+    // dispatch resolution) — sync-time metadata, names/paths only.
+    let bound = db::workflows::list_binding_environment(&state.pool, workspace_id, &meta.name)
+        .await?
+        .into_iter()
+        .map(|row| {
+            json!({
+                "repositoryId": row.repository_id,
+                "repositoryName": row.repository_name,
+                "workflowId": row.workflow_id,
+                "workflowPath": row.workflow_path,
+            })
+        })
+        .collect::<Vec<_>>();
     Ok(Json(json!({
         "environment": EnvironmentResponse::from(meta),
         "audit": audit_events(audit),
+        "boundWorkflows": bound,
     })))
 }
 

backend/src/handlers/secrets.rs

@@ -217,6 +217,80 @@ pub async fn summary(
     })))
 }
 
+/// A well-formed `${{ … }}` reference identifier (`[A-Za-z_][A-Za-z0-9_]*`,
+/// ≤200 bytes). The parser only writes such names, but requirements are
+/// re-filtered at read time so legacy or hand-edited metadata can never
+/// smuggle arbitrary strings into a response.
+pub(crate) fn valid_ref_ident(name: &str) -> bool {
+    let mut chars = name.chars();
+    matches!(chars.next(), Some(c) if c.is_ascii_alphabetic() || c == '_')
+        && name.len() <= 200
+        && chars.all(|c| c.is_ascii_alphanumeric() || c == '_')
+}
+
+/// Fold requirement rows into per-name entries: names sorted (BTreeMap),
+/// capped at 200 names and 20 references each — `referenceCount` carries the
+/// true total. `keep` is the read-time allow-list filter.
+pub(crate) fn group_requirements(
+    rows: Vec<db::workflows::RequirementRefRow>,
+    keep: impl Fn(&str) -> bool,
+) -> Vec<serde_json::Value> {
+    const MAX_NAMES: usize = 200;
+    const MAX_REFS_PER_NAME: usize = 20;
+
+    let mut grouped: std::collections::BTreeMap<String, (i64, Vec<serde_json::Value>)> =
+        std::collections::BTreeMap::new();
+    for row in rows {
+        if !keep(&row.name) {
+            continue;
+        }
+        if !grouped.contains_key(&row.name) && grouped.len() >= MAX_NAMES {
+            continue;
+        }
+        let entry = grouped.entry(row.name).or_default();
+        entry.0 += 1;
+        if entry.1.len() < MAX_REFS_PER_NAME {
+            entry.1.push(json!({
+                "repositoryId": row.repository_id,
+                "repositoryName": row.repository_name,
+                "workflowId": row.workflow_id,
+                "workflowPath": row.workflow_path,
+            }));
+        }
+    }
+    grouped
+        .into_iter()
+        .map(|(name, (count, references))| {
+            json!({ "name": name, "referenceCount": count, "references": references })
+        })
+        .collect()
+}
+
+/// GET /api/workspaces/{workspace_id}/secrets/requirements
+///
+/// Workflow-declared requirements detected at sync time: secret names the
+/// YAML references (`${{ secrets.X }}`) with no configured secret that could
+/// satisfy them, plus `${{ vars.X }}` references (informational — the
+/// platform doesn't manage plain variables). Names only, never values.
+pub async fn requirements(
+    State(state): State<AppState>,
+    CurrentUser(user): CurrentUser,
+    Path(workspace_id): Path<Uuid>,
+) -> AppResult<Json<serde_json::Value>> {
+    authz::require_permission(&state.pool, user.id, workspace_id, authz::SECRETS_READ).await?;
+
+    let missing = db::workflows::missing_secret_refs(&state.pool, workspace_id).await?;
+    let vars = db::workflows::workspace_var_refs(&state.pool, workspace_id).await?;
+
+    Ok(Json(json!({
+        // Only names that could actually become overup secrets surface as
+        // missing — a lowercase or GITHUB_*-reserved ref is unconfigurable
+        // here (GitHub folds case; we don't) and would be a dead-end button.
+        "secrets": group_requirements(missing, |name| validate_name(name).is_ok()),
+        "vars": group_requirements(vars, valid_ref_ident),
+    })))
+}
+
 #[derive(Debug, Deserialize)]
 #[serde(rename_all = "camelCase")]
 pub struct AuditQuery {

backend/src/routes/mod.rs

@@ -324,6 +324,10 @@ pub fn build_router(state: AppState) -> anyhow::Result<Router> {
             "/workspaces/{workspace_id}/secrets/summary",
             get(secrets::summary),
         )
+        .route(
+            "/workspaces/{workspace_id}/secrets/requirements",
+            get(secrets::requirements),
+        )
         .route(
             "/workspaces/{workspace_id}/secrets/audit",
             get(secrets::audit),
@@ -344,6 +348,10 @@ pub fn build_router(state: AppState) -> anyhow::Result<Router> {
             "/workspaces/{workspace_id}/environments/summary",
             get(environments::summary),
         )
+        .route(
+            "/workspaces/{workspace_id}/environments/requirements",
+            get(environments::requirements),
+        )
         .route(
             "/workspaces/{workspace_id}/environments/audit",
             get(environments::audit),