Skip to content

test(acceptance): give every worker its own UCP test data - #249

Merged
Dominik Grothaus (dgrothaus-sw) merged 11 commits into
mainfrom
test-ucp-fixture-for-acceptance
Sep 30, 2026
Merged

Dominik Grothaus (dgrothaus-sw) merged 11 commits into
mainfrom
test-ucp-fixture-for-acceptance

Conversation

@dgrothaus-sw

Copy link
Copy Markdown
Contributor

Each Playwright worker owns its UCP test data.

UcpTestDataService extends the ATS TestDataService with a second storefront channel, a headless channel and a product-feed channel, each on a path-prefixed domain, plus activateUcp() and a cleanup that resets the UCP rows it wrote and deletes the channels it created.
UcpAgentProfileHost writes a test agent's profile with an ES256 public key into the shop's public/ directory, where the web container fetches it from localhost. It never falls back to the shop's own profile.
UcpAclUsers creates a non-admin user per UCP role from the privilege sets in the Administration ACL file and logs it in.
UcpConsole runs ucp:signing-keys:* through bin/console.

Each fixture is proven by its own spec, tagged setup so it runs in the project the journeys already depend on. Two specs that need a path-prefixed domain to resolve to its own channel run in UcpKnownBlocked as D8: core strips the prefix from the request URI before the SDK builds its request, and the plugin resolves the channel from that URI alone. Under a prefixed domain /.well-known/ucp serves the root channel or the global configuration.

Closes #189.

`UcpTestDataService` extends the ATS data service with storefront,
headless and feed channels, UCP activation and UCP row cleanup.
`UcpAgentProfileHost` publishes a test agent's ES256 profile under the
shop's `public/` directory. `UcpAclUsers` logs one Administration user
per UCP role into a separate page. `UcpConsole` wraps
`ucp:signing-keys:*`. Known blocker D8 records that path-prefixed
domains resolve to the wrong channel.
@dgrothaus-sw Dominik Grothaus (dgrothaus-sw) added the testing Automated tests and test infrastructure: unit, integration, acceptance, CI test wiring label Sep 22, 2026

@BrocksiNet Björn Meyer (BrocksiNet) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two fixture issues need fixing: repeated ACL fixture use fails within the same worker, and the published agent profile cannot negotiate any shopping capability. Details inline.

TypeScript checking passed. The ACL loader failure was reproduced locally; the negotiation finding was verified against SDK 0.0.7. Full acceptance journeys were not run against a live shop.

Comment thread tests/acceptance/services/pluginSource.ts
Comment thread tests/acceptance/fixtures/UcpAgentProfileHost.ts Outdated
The privilege mapping is evaluated once per worker and reused, since
Node runs an imported module only once. Test agents publish every
shopping capability at the protocol version, and `publish()` accepts an
override. Workers delete only their own profile files, never the shared
directory. New `Setup` specs cover the repeated read and the
negotiation.
The ATS registry cleanup now runs even when the UCP cleanup throws. D1
describes what SDK 0.0.7 still does: verification requires `created`,
and its own signing still covers `@target-uri`. A doc comment no longer
names an issue number.

@BrocksiNet Björn Meyer (BrocksiNet) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both earlier findings are fixed (the cached mapping, and the capabilities with their override). typecheck and lint are clean. On trunk, 6 of 7 Setup specs pass with a working console; the details are inline. Nits:

  • README:108 says the worker channel's /.well-known/ucp "is its own". D8 says the opposite.
  • resolvePluginDir() falls back to custom/plugins/agentic-commerce, but lanes mount the plugin as SwagAgenticCommerce.
  • readAdminPrivilegeMapping() also caches a failed read. Reset privilegeMapping on failure so one error doesn't fail every later caller in the worker.
  • The D5 description says "typed array payload". #224 made the MCP payload an object schema. Rewording D1 and D5 is also outside this PR's scope.
  • The README fixture table has cells of about 700 characters. A short paragraph per fixture would read better than Prettier's padded table.
  • The default docker compose exec -T web prefix silently skips key cleanup on podman-only machines, where docker is only a shell alias that spawnSync can't see. That's worth one line in the README.

Comment thread tests/acceptance/services/UcpTestDataService.ts Outdated
Comment thread tests/acceptance/known-blockers.ts
Comment thread tests/acceptance/services/UcpTestDataService.ts Outdated
Comment thread tests/acceptance/services/UcpConsole.ts Outdated
Delete signing keys only for the channels a test activated, and continue
past a failed deletion. The created channels are now always deleted, and
every cleanup failure is reported together. Restore each surviving
channel's UCP config from a snapshot taken before the first write; the
fixed default emptied its capabilities. Probe the console with the real
command, and resolve the Shopware project only when a fixture first
needs it, which lets HTTP-only runners use the data service. Find the
plugin under any `custom/plugins` directory name, and retry an ACL read
that failed. Reword D5 and D8, and describe each fixture in its own
README paragraph.

Retry creating an ACL user up to three times. Core revokes refresh
tokens on every user insert, and MariaDB 11.6 and newer rejects that
with error 1020 while other workers log in, which failed about one
`Setup` run in three. Remove the retry once core skips the revocation
for new users.
@dgrothaus-sw

Copy link
Copy Markdown
Contributor Author

All six addressed in ddbda70: the README no longer claims the worker channel's profile is its own; resolvePluginDir() finds the plugin by its Composer name under any custom/plugins directory; a failed ACL read is no longer cached; D5 says "structured object payload"; each fixture has its own README paragraph; and the README notes that UCP_CONSOLE runs without a shell, so a docker alias for podman is invisible and key cleanup is skipped. D1 and D5 stay in this PR on purpose, to keep the registry in line with the epic table, which was updated at the same time.

One more change you did not ask for: the ACL fixture now retries user creation up to three times. Creating a user failed with a 500 in about one full Setup run in three, because core revokes refresh tokens on every user insert and MariaDB 11.6 and newer rejects that with error 1020 while other workers log in. The core fix is tracked separately; the retry goes once it lands.

@BrocksiNet Björn Meyer (BrocksiNet) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All four findings and the six nits hold up on trunk:

  • An HTTP-only runner passes the data-service specs.
  • A console prefix that can't reach the lane counts as unavailable, and nothing leaks.
  • With a working console, Setup passes three times in a row, the feed-channel spec included, with no channels left behind.
  • After a D8 run, every stored UCP config row is byte-identical.

One non-blocking note on the ACL retry is inline.

One more nit: when the console is unavailable, key cleanup is skipped without a word. Three such runs left 6 signing keys with private keys in ucp_signing_keys for channels that no longer exist. A console.warn or a test annotation naming the channels would make that visible.

Comment thread tests/acceptance/fixtures/UcpAclUsers.ts Outdated
Move the user-creation retry from the ACL fixture into
`UcpTestDataService::createUserRetryingTokenConflicts()`. Each attempt
presets the user id. When an attempt fails but the user exists, the user
is registered for cleanup and returned. Before Shopware 6.7.13.1 core
commits the user row before the token revocation that fails with MariaDB
error 1020, so the retry left one user behind per failed attempt.

Warn when the UCP console is unavailable at cleanup, naming the sales
channels whose signing keys stay in `ucp_signing_keys`. The skip was
silent before.
@dgrothaus-sw

Copy link
Copy Markdown
Contributor Author

Done: when the console is unavailable, cleanup warns with the reason and the ids of the sales channels whose keys stay in ucp_signing_keys, for example UCP console unavailable (docker compose exec -T web php bin/console (in /shopware)), signing keys left in ucp_signing_keys for sales channels: 7fec33574ee140ab98361869f1f281e9. A Setup run from the Playwright service printed it for the three channels it activated, which were exactly the three orphaned keys in the table afterwards. The README's UcpConsole paragraph mentions the warning.

@dgrothaus-sw
Dominik Grothaus (dgrothaus-sw) merged commit d50ea99 into main Sep 30, 2026
29 checks passed
@dgrothaus-sw
Dominik Grothaus (dgrothaus-sw) deleted the test-ucp-fixture-for-acceptance branch September 30, 2026 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Automated tests and test infrastructure: unit, integration, acceptance, CI test wiring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Isolated test data: UcpTestDataService, per-worker sales channel, agent profile host

2 participants