Skip to content

lisa/feat/workflow-import-route#413

Merged
gusfcarvalho merged 7 commits into
mainfrom
lisa/feat/workflow-import-route
Jun 4, 2026
Merged

lisa/feat/workflow-import-route#413
gusfcarvalho merged 7 commits into
mainfrom
lisa/feat/workflow-import-route

Conversation

@ccf-lisa
Copy link
Copy Markdown
Contributor

@ccf-lisa ccf-lisa Bot commented Jun 3, 2026

automated implementation by lisa.

Summary by CodeRabbit

  • New Features

    • POST /workflows/import endpoint to upload/import up to 10 workflow seed JSON files (5MB per file). Returns per-file results and aggregated summary; responds 200/400/413 and requires admin access. Idempotent re-imports supported.
  • Documentation

    • OpenAPI/Swagger updated with the new endpoint and request/response schemas for per-file and aggregate import results.
  • Tests

    • Expanded unit and integration tests covering multipart parsing, body limits, payload-too-large handling, auth, error reporting, aggregation, idempotency, and import validation.
  • Bug Fixes / Validation

    • Improved input validation, duplicate-name checks, and preservation of existing scheduling and audit fields during imports.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jun 3, 2026

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: d82225ac-5a0b-4658-8dfb-5c830162ee07

📥 Commits

Reviewing files that changed from the base of the PR and between 8cdcbfe and d294c43.

📒 Files selected for processing (4)
  • internal/api/handler/workflows/import.go
  • internal/api/handler/workflows/import_test.go
  • internal/service/relational/workflows/seedimport.go
  • internal/service/relational/workflows/seedimport_test.go

📝 Walkthrough

Walkthrough

Delegates CLI workflow seeding to a new shared relational importer, adds a multipart POST /workflows/import HTTP handler with per-file and body size limits, updates OpenAPI docs/schemas, and adds unit and integration tests for the service and handler.

Changes

Workflow Seed Import Feature

Layer / File(s) Summary
Seed Import Service Implementation
internal/service/relational/workflows/seedimport.go
DecodeSeedDefinitions and ImportSeedDefinitions decode JSON seed files, validate/normalize inputs, generate deterministic UUIDs, and perform transactional upserts for definitions, steps, control relationships, instances, and role assignments. Preserves schedules and returns aggregated SeedSummary.
Seed Import Service Tests
internal/service/relational/workflows/seedimport_test.go
Database-backed tests covering import/re-import idempotency, duplicate-step/instance validation, deterministic ID trimming, grace-period updates, schedule preservation for soft-deleted instances, and preservation of audit/soft-delete fields on upserts; includes helpers that assert seeded graph and counts.
HTTP Handler for Workflow Import
internal/api/handler/workflows/import.go
WorkflowImportHandler accepts multipart form-data (max 10 files, 5MB/file), enforces body and per-file limits (including a limited reader for decoding), decodes each file, imports via service, aggregates per-file results and merged SeedSummary, and returns 200, 400, or 413.
HTTP Handler Tests
internal/api/handler/workflows/import_test.go, internal/api/handler/workflows/import_integration_test.go
Unit tests for file-count/size and body-limit propagation; integration tests for single/multi-file imports, mixed malformed+valid files, empty uploads, idempotent re-import behavior, and admin-group authorization.
API Registration and OpenAPI Documentation
internal/api/handler/api.go, docs/docs.go, docs/swagger.json, docs/swagger.yaml
Registers POST /workflows/import guarded by admin-group middleware and body-limit middleware; adds OpenAPI schemas workflows.SeedSummary, workflows.WorkflowImportFileResult, and workflows.WorkflowImportResponse and documents multipart upload and error responses.
Command-line Delegation
cmd/seed/workflows.go, cmd/seed/workflows_test.go
CLI seed command now opens the JSON file, defers close (logging close errors), and delegates decoding/import to workflows.DecodeSeedDefinitions and workflows.ImportSeedDefinitions; tests reduced to a single shared-importer validation.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant HTTPHandler as WorkflowImportHandler
  participant Decoder as DecodeSeedDefinitions
  participant Service as ImportSeedDefinitions
  participant DB as Database

  Client->>HTTPHandler: POST /workflows/import (multipart files)
  HTTPHandler->>HTTPHandler: validate file count & sizes
  loop per file
    HTTPHandler->>Decoder: decode JSON (limited reader)
    Decoder->>Service: send decoded definitions
    Service->>DB: transactional upserts (deterministic IDs)
    Service->>HTTPHandler: per-file SeedSummary
  end
  HTTPHandler->>Client: aggregated WorkflowImportDataResponse (200/400/413)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • compliance-framework/api#410: Prior work implementing the original CLI workflow seed parsing and tests that this change delegates to the shared service.

Suggested labels

ready-for-e2e

Suggested reviewers

  • gusfcarvalho

Poem

🐰 I found the seeds and hopped away,
Files parsed neatly, no disarray.
UUIDs steady, imports idempotent too,
Docs sing loud and tests prove true.
A carrot for CI — code fresh as dew.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 2.08% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title is vague and does not clearly convey the scope or nature of the changes; 'lisa/feat/workflow-import-route' is a branch name convention rather than a descriptive pull request summary. Use a descriptive title that summarizes the main change, e.g., 'Add workflow import endpoint with multipart form support' or similar.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@gusfcarvalho gusfcarvalho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings inline (5 threads). No blockers — all Low/nit. Clean, well-tested PR; the seed-engine de-duplication and the extra resource hardening on this endpoint (per-file + file-count + BodyLimit caps, admin-only) are nice. Build/vet/gofmt/unit tests pass locally; the previously self-reviewed BodyLimit sizing bug is genuinely fixed and covered by a route-level test.

Comment thread internal/api/handler/workflows/import.go Outdated
Comment thread internal/api/handler/workflows/import.go
Comment thread internal/api/handler/workflows/import.go
Comment thread internal/api/handler/workflows/import.go
Comment thread internal/api/handler/workflows/import_test.go
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/docs.go`:
- Around line 25286-25293: The OpenAPI entry for the request parameter "files"
incorrectly models a single file while WorkflowImportHandler.Import (method
Import) actually reads ctx.MultipartForm() and handles multiple files; update
the Swagger/Swagger-annotated parameter for "files" to represent a multi-file
form field by changing its schema to an array of files (e.g., type: "array",
items: { "type": "file" }, and use collectionFormat: "multi" or the equivalent
for the generator in use) so clients understand multiple files under the "files"
field are accepted; locate the parameter definition for "files" in the docs (the
parameter with name "files", in "formData") and replace the single-file spec
with the array-of-file spec.

In `@docs/swagger.json`:
- Around line 40686-40717: Update the workflows.SeedSummary OpenAPI schema to
declare which counters are always returned by adding a required array (e.g.,
include
"control_relationships","definitions_created","definitions_updated","dependencies","failed","instances","role_assignments","skipped","steps")
and add a description string for each property in the properties object
explaining what that counter represents (for example, describe what
"definitions_created" and "definitions_updated" count and what "failed"
indicates). Ensure the required array references the exact property names in
workflows.SeedSummary and keep descriptions concise and consumer-facing to
improve the API contract clarity.
- Around line 25280-25287: Update the Swagger parameter for "files" (parameter
name "files", type "file", in "formData") to explicitly state that multiple
files are supported by repeating the "files" form field in a multipart/form-data
request (i.e., send multiple parts with the same name), so API consumers know to
upload one or more files by repeating the "files" parameter rather than trying a
single multi-file value.
- Around line 41334-41381: Update the three OpenAPI definitions
workflows.WorkflowImportDataResponse, workflows.WorkflowImportFileResult, and
workflows.WorkflowImportResponse to include "required" arrays and add property
"description" strings for each property; specifically mark
WorkflowImportDataResponse.data as required, mark
WorkflowImportFileResult.filename and WorkflowImportFileResult.success as
required, and mark WorkflowImportResponse.total_files,
WorkflowImportResponse.successful_files, WorkflowImportResponse.failed_files,
and WorkflowImportResponse.results as required, and add clear descriptions for
each property (e.g., what filename, message, success, summary, total_files,
successful_files, failed_files, results and data represent and whether they can
be null) so consumers know guaranteed fields and semantics.

In `@docs/swagger.yaml`:
- Around line 9391-9421: The new OpenAPI schemas
(workflows.WorkflowImportFileResult, workflows.WorkflowImportResponse, and
workflows.WorkflowImportDataResponse) lack required arrays so every property is
treated as optional; add explicit "required" arrays to each schema listing the
always-present fields (e.g., for workflows.WorkflowImportFileResult include
"filename" and "success", for workflows.WorkflowImportResponse include
"total_files", "successful_files", "failed_files" and likely "results" and
"summary" as applicable, and for workflows.WorkflowImportDataResponse include
"data") to make the contract explicit for consumers.
- Around line 26046-26051: The Swagger spec currently models the form-data
parameter "files" as a single type: file which only supports one file; update
the parameter "files" in docs/swagger.yaml to represent multiple files by
changing it to an array formData parameter (e.g., type: array, items: { type:
string, format: binary } and collectionFormat: multi) so the endpoint that
accepts "one or more seed JSON files" is accurately described; locate the
parameter named "files" in the parameters block and replace its single-file
definition with the array/collectionFormat workaround (or alternatively note
migration to OpenAPI 3 for native multipart file arrays).

In `@internal/api/handler/workflows/import.go`:
- Around line 73-113: After calling ctx.MultipartForm() in
WorkflowImportHandler.Import, ensure the multipart temporary files are removed
by deferring form.RemoveAll(); specifically, once form, err :=
ctx.MultipartForm() succeeds, add a defer that checks form != nil and calls
form.RemoveAll() (discarding any error) so temp files on disk are cleaned up
after processing; reference the Import method on WorkflowImportHandler and the
form variable returned by ctx.MultipartForm().

In `@internal/service/relational/workflows/seedimport.go`:
- Around line 84-91: The DecodeSeedDefinitions function currently allows
trailing JSON after the array because json.Decoder.Decode stops at the first
value; after successfully decoding into definitions, try to decode one more
value into a temporary variable (e.g., var extra interface{}) and if that Decode
does not return io.EOF, return an error indicating unexpected trailing JSON;
update DecodeSeedDefinitions to perform this extra decode check so malformed
inputs like `[...]{...}` are rejected.
- Around line 316-331: The loop over seedDef.Instances builds a deterministic ID
with deterministicSeedUUID("workflow-instance", seedDef.Key, seedInstance.Name)
without checking for duplicate instance names, causing upserts to collide;
before deriving instanceID (i.e., before calling deterministicSeedUUID) add a
guard like importSeedSteps does: maintain a local map[string]struct{} of seen
instance names for this seedDef, check seedInstance.Name against it, and return
an error if duplicated so duplicate names are rejected rather than merged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ab174d22-0c17-4d7e-8038-2779b5f0ba99

📥 Commits

Reviewing files that changed from the base of the PR and between 4e9fe14 and 7750e04.

📒 Files selected for processing (11)
  • cmd/seed/workflows.go
  • cmd/seed/workflows_test.go
  • docs/docs.go
  • docs/swagger.json
  • docs/swagger.yaml
  • internal/api/handler/api.go
  • internal/api/handler/workflows/import.go
  • internal/api/handler/workflows/import_integration_test.go
  • internal/api/handler/workflows/import_test.go
  • internal/service/relational/workflows/seedimport.go
  • internal/service/relational/workflows/seedimport_test.go

Comment thread docs/docs.go
Comment thread docs/swagger.json
Comment thread docs/swagger.json
Comment thread docs/swagger.json
Comment thread docs/swagger.yaml
Comment thread docs/swagger.yaml
Comment thread internal/api/handler/workflows/import.go
Comment thread internal/service/relational/workflows/seedimport.go
Comment thread internal/service/relational/workflows/seedimport.go
Copy link
Copy Markdown
Contributor

@gusfcarvalho gusfcarvalho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. All substantive findings are resolved and verified at 8cdcbfe.

My 5 original findings — 4 fixed (omitempty→json:"summary"; 413 propagation for chunked over-limit bodies + test; defer form.RemoveAll() for multipart temp files; full JWT+admin+BodyLimit middleware-chain test), 1 was informational (documented continue-on-error contract).

CodeRabbit's 2 real source findings — both fixed with tests: trailing-JSON rejection in DecodeSeedDefinitions (+TestDecodeSeedDefinitionsRejectsTrailingJSON) and the duplicate-instance-name guard mirroring importSeedSteps (+TestImportWorkflowSeedDefinitionRejectsDuplicateInstanceNames, asserting rollback). Multi-file files param doc clarified and swagger regenerated.

Local build / go vet / gofmt / unit tests clean; CI green on check-diff, lint, unit-tests, and integration-tests.

The 3 remaining open threads are 🔵 Trivial swaggo-generated-schema required/description nits — non-blocking, fine to defer or resolve as won't-fix. Nice refactor consolidating the seed engine; solid test coverage.

@ccf-lisa
Copy link
Copy Markdown
Contributor Author

ccf-lisa Bot commented Jun 3, 2026

PR approved. Marking as ready for e2e.

Copy link
Copy Markdown
Contributor

@gusfcarvalho gusfcarvalho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested the import end-to-end against the running stack. One blocking issue (re-import/reseed of a previously soft-deleted definition fails — details inline) plus a result-message accuracy nit. See inline threads.

Comment thread internal/service/relational/workflows/seedimport.go
Comment thread internal/service/relational/workflows/seedimport.go
Comment thread internal/api/handler/workflows/import.go Outdated
@ccf-lisa ccf-lisa Bot removed the ready-for-e2e label Jun 4, 2026
Copy link
Copy Markdown
Contributor

@gusfcarvalho gusfcarvalho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. The soft-delete re-import failure and the misleading import-count message are both fixed and verified (resurrection regression tests pass locally; lint/unit/check-diff CI green). Two non-blocking Low observations left inline (instance schedule can stay NULL on re-import; upsert-only import doesn't prune removed children) — surface them for a future pass, not required here.

Comment thread internal/service/relational/workflows/seedimport.go
Comment thread internal/service/relational/workflows/seedimport.go
@ccf-lisa
Copy link
Copy Markdown
Contributor Author

ccf-lisa Bot commented Jun 4, 2026

PR approved. Marking as ready for e2e.

@compliance-framework compliance-framework deleted a comment from ccf-lisa Bot Jun 4, 2026
@gusfcarvalho gusfcarvalho merged commit 484bc05 into main Jun 4, 2026
5 checks passed
@gusfcarvalho gusfcarvalho deleted the lisa/feat/workflow-import-route branch June 4, 2026 00:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant