From ca87f350ecec8f4719ef3eda959a8d6a8603d393 Mon Sep 17 00:00:00 2001 From: Adi El-Sammak <71227923+aelsammak@users.noreply.github.com> Date: Mon, 28 Sep 2026 09:51:35 -0400 Subject: [PATCH 01/12] EBP-3103: Add best-practice Python pub/sub reference samples (#8) * EBP-3103: Add best-practice Python pub/sub reference samples - Add python-solace-connection-config.py plus guaranteed and direct publisher and subscriber samples under references/python, derived from the Solace Python samples and elevated to the Python developer guide and C API best practices - Register service listeners before connect(), use a 3000 ms reconnect interval, pair reject back pressure with a readiness listener, check publish receipts, subscribe the CREATE_ON_START queue to the topic, client-ack after processing, and shut down gracefully on SIGINT and SIGTERM - Read config.json or command-line arguments through the shared helper, mirroring jcsmp-solace-connection-config.java, with no version pin - Reword JCSMP sample comments to drop the legacy product name and the comparisons to upstream samples - Verify locally against solace-pubsubplus resolved from PyPI with a no-broker smoke test and live round trips per pair * EBP-3103: Remove PEP 563 future import from Python samples - Drop `from __future__ import annotations` and its explanatory comment from the five Python reference samples - The built-in generics in the signatures run natively on every Python release that Solace supports, so the shim added nothing - Match the upstream SolaceSamples/solace-samples-python style, which carries no future import - Verified: all five modules import with solace-pubsubplus installed * EBP-3103: Condense the HA failover comment in the Python samples - Replace the ten-line HA and replication comment with a five-line HA-only comment that mirrors the JCSMP sample - List the C API Best Practices values as Python API calls: connection and reconnection retry strategies plus the CONNECTION_RETRIES_PER_HOST transport property - Drop the replication guidance, the forever_retry remark, and the comma-separated host list - Shorten the back pressure comment above PUBLISH_BUFFER_CAPACITY to one line in both publisher samples * EBP-3103: Condense the Python connection-config helper docstring - Shorten the module docstring of python-solace-connection-config.py while keeping every contract point (key names, pass-through, override order, gitignore rule, empty-password handling) - Move the AI-assisted code header block to the end of the docstring to match the other Python samples * EBP-3103: Disable the trigger-evals CI job - Add a job-level `if: false` to trigger-evals: no policy permits an API key on this public repo, so the job could only self-skip and asserted nothing - Keep the job definition so it can be re-enabled by removing the `if` once a key is allowed; trigger evals continue to run locally * EBP-3103: Ignore Python bytecode caches - Add __pycache__/ to .gitignore so byte-compiling the Python reference samples leaves no untracked files * EBP-3103: Fix shutdown and exit-code handling in the Python samples - Register the SIGINT/SIGTERM handlers before setup_solace(), so a SIGTERM during the blocking connect() still reaches teardown instead of killing the process - Return exit code 1 from every failure exit (failed queue bind, service interruption, receiver termination, publish() failure); main() now returns the code and __main__ passes it to sys.exit() - Correct the guaranteed subscriber's terminate() comments: the receiver turns TERMINATED once its buffer is empty, so the ack() of the message still in the handler can fail and the message is redelivered - Verified against a live broker: both round trips exit 0, and the bind failure, a service interruption (reconnect exhausted), and SIGTERM during connect each run teardown and exit 1 * EBP-3103: Explain the terminate() grace period in the Python samples - Describe why the samples bound terminate(): a predictable shutdown time, for example within a process supervisor's stop timeout, so the disconnect still runs - State the API default (600000 ms) as a fact and invite adjusting the value to suit the application * EBP-3103: Fix review findings in the Python samples and CI docs - Name MessageDestinationDoesNotExistError for the no-subscription-match NACK, matching the library's subcode mapping - Drop "a broker rejection" from the direct publisher failure listener comment; broker events never reach that listener - Split terminate() handling into IncompleteMessageDeliveryError and other client errors, and exit non-zero on both - Update the exit_code comments to match, and drop the inaccurate "failed bind" case in the guaranteed subscriber - Describe the disabled trigger-evals CI job in the evals README and point to local runs - Include *.py in the link check so the Python sample header URLs are swept * EBP-3103: Stop naming specific model versions in the evals README - Use the claude CLI aliases (sonnet, opus) in the run examples so they always select the latest model in each family - Describe the runner defaults, the CI matrix, and the judge by model family instead of pinned IDs - Remove the stale claude-opus-5 reference * EBP-3103: Restructure the Python samples as functions and bound buffers - Replace the Java-style app classes with module-level functions (main, setup_solace, run_publish_loop or await_messages, teardown_solace) that share one dataclass state - Keep the listener classes, since the Python API requires listener subclasses; they now take the shared state - Replace the SolaceConnectionConfig class with one load_service_properties() function that returns the from_properties() dict - Use X | None type hints for the Python 3.10+ floor - Publishers: on_back_pressure_wait(50) replaces on_back_pressure_reject(1000), so the readiness listener and the PublisherOverflowError retry are removed - Direct subscriber: bound the receive buffer with on_back_pressure_drop_latest(50) - Verified with the smoke test, both live round trips, and a frozen-proxy test of a blocked publish() (returns on drain, or raises after the reconnect budget) * EBP-3103: Build Solace resources before connect in the Python samples - setup_solace() now builds the service and the publisher or receiver before any connection exists, and returns a fully populated state, so no state object is created empty and filled in later (no | None fields) - New connect_solace() connects the service and starts the endpoint; main() calls setup_solace() before its try and runs teardown in finally - Remove the reconnect pause from both publishers: a guaranteed publish waits in the buffer during a reconnect, and the direct publisher now documents that messages published during a reconnect are lost and reported through the publish failure listener, pointing to guaranteed messaging for lossless delivery - Raise the back pressure buffer from 50 to 1000, the value the Python developer guide uses in its examples (the API has no default capacity), and note that it counts messages and is tunable to deployment memory * EBP-3103: Publish direct messages builderless and template guaranteed ones - Direct publisher passes the payload straight to publish() with no OutboundMessageBuilder, and shows additional_message_properties for per-message headers; a comment notes the builder suits "templated" messages with many fixed headers - Guaranteed publisher uses the builder as a template: the time-to-live (new MESSAGE_TTL_MS, 60 s) is set once, since build() copies the template into each message, and the per-message application message id goes through build()'s additional_message_properties so it never changes the template * EBP-3103: Remove the publish rate throttle from the samples - Remove APPROX_MSG_RATE_PER_SEC and the delay after each send from the Python and JCSMP direct and guaranteed publisher samples, so the loop publishes as fast as send() or publish() returns, the way an application publishes each message as soon as it has one - Update the demo-harness rule in implement-mode.md: the continuous publish loop replaces the pacing Thread.sleep in the list, and generated applications never add a delay or rate limit between sends that the application does not ask for - Output evals not run for the implement-mode.md change * EBP-3103: Let configuration override the reconnect defaults in Python - Set the reconnection retry strategy before from_properties() in all four samples: the builder merges calls in order and a later call wins, so the code's strategy is now a default that reconnection keys in the properties (for example config.json) override; update the connection config helper docstring to match - Load the Solace properties in main() and pass them to setup_solace(), so converting the application's input (config.json or argv) stays out of the Solace setup - Log a buffered-message loss at direct publisher shutdown as a warning and keep exit code 0, since direct (at-most-once) messaging allows it * EBP-3103: Set the message handler before starting the Python receivers - Call receive_async() before receiver.start() in both subscriber samples, so the handler is in place when delivery begins and messages do not wait in the API buffer - Keep receive_async() after connect(), since _can_receive_message() requires a connected service - Address PR #8 review comment on the guaranteed subscriber sample * EBP-3103: Add an async suffix to the Python sample names - Rename the four Python samples to python---async-sample.py - Update the generated project file names to match, for example guaranteed_subscriber_async.py - Use the async/sync terms from the Python developer guide, so later sync samples (publish_await_acknowledgement, receive_message) fit the scheme - Add guidance to the guaranteed publisher on how to choose the message TTL: at least the longest consumer recovery time (restart plus reconnect), or a multiple of it --- .github/workflows/ci.yml | 6 +- .gitignore | 3 + .../solace-messaging-skills/evals/README.md | 12 +- .../references/jcsmp/implement-mode.md | 2 +- .../jcsmp/jcsmp-direct-publisher-sample.java | 12 +- .../jcsmp/jcsmp-direct-requestor-sample.java | 3 +- .../jcsmp-guaranteed-publisher-sample.java | 12 +- .../jcsmp-guaranteed-replier-sample.java | 7 +- .../jcsmp-guaranteed-subscriber-sample.java | 3 +- .../python-direct-publisher-async-sample.py | 329 ++++++++++++++++ .../python-direct-subscriber-async-sample.py | 309 +++++++++++++++ ...ython-guaranteed-publisher-async-sample.py | 353 +++++++++++++++++ ...thon-guaranteed-subscriber-async-sample.py | 372 ++++++++++++++++++ .../python/python-solace-connection-config.py | 124 ++++++ tools/check-links.sh | 8 +- 15 files changed, 1517 insertions(+), 38 deletions(-) create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-publisher-async-sample.py create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-subscriber-async-sample.py create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-publisher-async-sample.py create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-subscriber-async-sample.py create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-solace-connection-config.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 5b89401..ca19207 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -7,7 +7,7 @@ name: ci # + post-merge version backstop on pushes to main # link-check: CI-02: the tools/check-links.sh sweep # compile-check: EVAL-02 compile the baked JCSMP samples verbatim -# trigger-evals: lightweight skill-triggering check; self-skips green without a key +# trigger-evals: lightweight skill-triggering check; disabled until CI may hold an API key on: # Deliberately no `edited` type: running on title/body edits either re-runs # the paid eval matrix for a no-code change or, if jobs are gated, lets the @@ -133,7 +133,11 @@ jobs: # run summary. The secret is legal in the job env but illegal in a job-level # if, so the guard reads HAS_KEY at step level and the real key is scoped to # the run step only. + # Disabled: no policy permits an API key on this public repo, so the job can + # only self-skip. Run the trigger evals locally instead; remove this `if` to + # re-enable the job once a key is allowed. trigger-evals: + if: false runs-on: ubuntu-latest timeout-minutes: 120 # per model leg; 57 cases x 3 runs (~50 min observed) strategy: diff --git a/.gitignore b/.gitignore index 7a26224..34d846d 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,5 @@ # GSD planning directory (local-only) .planning/ + +# Python bytecode cache +__pycache__/ diff --git a/plugins/solace-messaging-skills/evals/README.md b/plugins/solace-messaging-skills/evals/README.md index e8b4661..f02ec65 100644 --- a/plugins/solace-messaging-skills/evals/README.md +++ b/plugins/solace-messaging-skills/evals/README.md @@ -8,15 +8,15 @@ To run the evals manually, you need the `claude` CLI and `jq` on your PATH, plus ```shell export ANTHROPIC_API_KEY= # or CLAUDE_CODE_OAUTH_TOKEN -./tools/run-trigger-evals.sh # defaults to claude-haiku-4-5 -./tools/run-trigger-evals.sh --model claude-sonnet-5 +./tools/run-trigger-evals.sh # defaults to a Haiku model +./tools/run-trigger-evals.sh --model sonnet # the latest Sonnet ``` Each prompt runs three times and the verdict is the majority result. Set `TRIGGER_EVAL_RUNS=1` for a quicker, cheaper local check, and `TRIGGER_EVAL_MODEL` to change the default model. A run passes when at least 90% of cases match their expectation, and any infrastructure failure (such as a missing credential, a malformed corpus, or zero discovered cases) fails the run regardless of the rate. A case may also set `"must_pass": true`, and any must-pass failure fails the run regardless of the pooled rate. Note that a negative case asserts only that the named skill stays silent, so another skill in the plugin may legitimately fire on the same prompt. ## Continuous integration -In GitHub Actions, the `trigger-evals` job in `.github/workflows/ci.yml` runs this same script on every pull request as a two model matrix (`claude-haiku-4-5` and `claude-sonnet-5`), each model an independent check, and self-skips green when the `ANTHROPIC_API_KEY` secret is absent. +The `trigger-evals` job in `.github/workflows/ci.yml` is disabled (`if: false`), because no policy permits an API key on this public repository. Run the trigger evals locally as shown above. To re-enable the job once a key is allowed, remove the `if`. It then runs this same script on every pull request as a two model matrix (Haiku and Sonnet), each model an independent check, and self-skips green when the `ANTHROPIC_API_KEY` secret is absent. # Output evals @@ -60,12 +60,12 @@ Run from the repository root: ```shell export ANTHROPIC_API_KEY= # or CLAUDE_CODE_OAUTH_TOKEN -./tools/run-output-evals.sh # defaults to claude-sonnet-5 -./tools/run-output-evals.sh --model claude-opus-5 +./tools/run-output-evals.sh # defaults to a Sonnet model +./tools/run-output-evals.sh --model opus # the latest Opus ./tools/run-output-evals.sh --case appdev-quickstart-implement-full ``` -Run the suite for both `claude-sonnet-5` and `claude-opus-5` before a PR that affects skill content, and record both results in the PR description, including whether live verify ran or skipped (the runner prints this in its summary). A full leg is a long run; keep the machine awake for it (on macOS, prefix the command with `caffeinate -i`), because a sleep mid-run surfaces as INFRA failures. A permission denial of a tool the runner grants (Bash, Write, and the rest of its allowlist) is also INFRA: managed settings, hooks, or local command shims can deny a command, the model then improvises, and the result no longer measures the skill. The case line names the denied tool and command; adjust the policy for the run instead of reading the result as a skill failure. Each case runs once by default (`OUTPUT_EVAL_RUNS=1`), because a full implement-flow case is expensive; raise it for a majority-vote stability study. `OUTPUT_EVAL_JUDGE_MODEL` (default `claude-sonnet-5`) stays fixed across subject models so leg differences are attributable to the subject. `OUTPUT_EVAL_WORKDIR` receives transcripts and generated projects, and the work directory is kept and printed when the run fails. +Run the suite for both the latest Sonnet and the latest Opus (`--model sonnet` and `--model opus`) before a PR that affects skill content, and record both results in the PR description, including whether live verify ran or skipped (the runner prints this in its summary). A full leg is a long run; keep the machine awake for it (on macOS, prefix the command with `caffeinate -i`), because a sleep mid-run surfaces as INFRA failures. A permission denial of a tool the runner grants (Bash, Write, and the rest of its allowlist) is also INFRA: managed settings, hooks, or local command shims can deny a command, the model then improvises, and the result no longer measures the skill. The case line names the denied tool and command; adjust the policy for the run instead of reading the result as a skill failure. Each case runs once by default (`OUTPUT_EVAL_RUNS=1`), because a full implement-flow case is expensive; raise it for a majority-vote stability study. `OUTPUT_EVAL_JUDGE_MODEL` (default a pinned Sonnet model) stays fixed across subject models so leg differences are attributable to the subject. `OUTPUT_EVAL_WORKDIR` receives transcripts and generated projects, and the work directory is kept and printed when the run fails. The gate matches the trigger evals: a run passes when at least 90% of cases pass, any infrastructure failure fails the run, and any `must_pass` failure fails the run regardless of the pooled rate. The forbidden-behavior negatives and the compile case are `must_pass`. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/implement-mode.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/implement-mode.md index 76b78ee..a13aa14 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/implement-mode.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/implement-mode.md @@ -126,7 +126,7 @@ Hold every invariant while adapting: keep the shared connection helper, the sess **Preserve the marker contract.** A variant adapts the SHAPE, never the marker strings. It MUST still emit its leaf's `VERIFY:` markers, character for character, so the fixed `verify.sh` stages drive it unchanged (this is exactly why a durable-reply-queue requestor still prints `VERIFY: REPLY_RECEIVED`). Never rename or drop a marker to fit a variant. -**Demo harness is not application logic.** The samples carry demo-harness elements that exist only to make a standalone run observable: the ENTER-to-quit `System.in` loop, the once-per-second stats-printing thread, the pacing `Thread.sleep` between sends, and the rotating example payload. When the developer's request describes a real application domain, do NOT carry these into the generated classes: replace the example payload and send cadence with the application's real messages and triggers, and keep only the lifecycle pieces the leaf's steps call for (the SIGINT hook on the long-running role, the self-exit on the foreground role). When the request IS a demo or a try-it run, keeping the harness is fine. Two things are NOT demo harness: the `trace(...)` narration method (keep it; applications replace its single body to route narration to their logging or reporting system) and the `VERIFY:` markers, which are emitted through `trace(...)` like every other status line. Because `verify.sh` greps captured stdout for the markers and for the shutdown hook's `Shutdown signal received` proof line (both flow through `trace(...)`), the generated `trace(...)` body MUST stay `System.out.println` through the Step 5 verification stages; rerouting that body to a logger or reporting system is the developer's step AFTER the verify stages pass, never part of generation. Never treat the logging backend (Step 3) as demo harness: the backend and its config file ship in every generated app. +**Demo harness is not application logic.** The samples carry demo-harness elements that exist only to make a standalone run observable: the ENTER-to-quit `System.in` loop, the once-per-second stats-printing thread, the continuous publish loop, and the rotating example payload. When the developer's request describes a real application domain, do NOT carry these into the generated classes: replace the example payload and the publish loop with the application's real messages and triggers, and never add a delay or rate limit between sends that the application does not ask for, and keep only the lifecycle pieces the leaf's steps call for (the SIGINT hook on the long-running role, the self-exit on the foreground role). When the request IS a demo or a try-it run, keeping the harness is fine. Two things are NOT demo harness: the `trace(...)` narration method (keep it; applications replace its single body to route narration to their logging or reporting system) and the `VERIFY:` markers, which are emitted through `trace(...)` like every other status line. Because `verify.sh` greps captured stdout for the markers and for the shutdown hook's `Shutdown signal received` proof line (both flow through `trace(...)`), the generated `trace(...)` body MUST stay `System.out.println` through the Step 5 verification stages; rerouting that body to a logger or reporting system is the developer's step AFTER the verify stages pass, never part of generation. Never treat the logging backend (Step 3) as demo harness: the backend and its config file ship in every generated app. **Comments follow their code.** The samples' comments are part of the reference, not decoration: they carry the documented best practices (the session-independent `createMessage`, the binary payload and the legacy `XML` naming, `getData()` versus `getBytes()`, registering a session event handler at session creation, the ACK/NACK handling options, the correlation-key rule) into the code a developer reads long after this session. Generation carries them under ONE contract, on every door and every app shape: diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-direct-publisher-sample.java b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-direct-publisher-sample.java index 93c618f..69beb5f 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-direct-publisher-sample.java +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-direct-publisher-sample.java @@ -57,7 +57,6 @@ public class DirectPublisher { private static final String APP_NAME = DirectPublisher.class.getSimpleName(); static final String TOPIC_PREFIX = "solace/samples/"; // used as the topic "root" private static final String API = "JCSMP"; - private static final int APPROX_MSG_RATE_PER_SEC = 100; private static final int PAYLOAD_SIZE = 512; // remember to add log4j2.xml to your classpath @@ -189,15 +188,10 @@ private static void runPublishLoop() throws IOException { if (e instanceof JCSMPTransportException) { // all reconnect attempts failed isShutdown = true; // let's quit; or, could initiate a new connection attempt } - } finally { // add a delay between messages - try { - Thread.sleep(1000 / APPROX_MSG_RATE_PER_SEC); // do Thread.sleep(0) for max speed - // Note: STANDARD Edition Solace PubSub+ broker is limited to 10k msg/s max ingress - } catch (InterruptedException e) { - Thread.currentThread().interrupt(); // restore the interrupt status, do not swallow it - isShutdown = true; - } } + // no delay between messages: the loop publishes as fast as send() returns, the way an + // application publishes each message as soon as it has one + // Note: STANDARD Edition Solace broker is limited to 10k msg/s max ingress } isShutdown = true; } diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-direct-requestor-sample.java b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-direct-requestor-sample.java index 3bfe9a2..a564530 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-direct-requestor-sample.java +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-direct-requestor-sample.java @@ -60,8 +60,7 @@ public class DirectRequestor { private static final String API = "JCSMP"; // POSITIVE blocking-request timeout in milliseconds: requestor.request(...) blocks up // to this long for the correlated reply before throwing JCSMPRequestTimeoutException. - // A positive timeout is the direct reliability knob (NOT a burst); 3000 ms matches the - // upstream DirectRequestorBlocking sample. + // A positive timeout is the direct reliability knob (NOT a burst). private static final int REQUEST_TIMEOUT_MS = 3000; private static JCSMPSession session; diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-guaranteed-publisher-sample.java b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-guaranteed-publisher-sample.java index a4d0ef1..e0e8a5f 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-guaranteed-publisher-sample.java +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-guaranteed-publisher-sample.java @@ -58,7 +58,6 @@ public class GuaranteedPublisher { static final String TOPIC_PREFIX = "solace/samples/"; // used as the topic "root" private static final String API = "JCSMP"; private static final int PUBLISH_WINDOW_SIZE = 255; // max guaranteed-publish window (range 1-255) - private static final int APPROX_MSG_RATE_PER_SEC = 100; private static final int PAYLOAD_SIZE = 512; // remember to add log4j2.xml to your classpath @@ -220,15 +219,10 @@ private static void runPublishLoop() throws IOException, InterruptedException { if (e instanceof JCSMPTransportException) { // all reconnect attempts failed isShutdown = true; // let's quit; or, could initiate a new connection attempt } - } finally { // add a delay between messages - try { - Thread.sleep(1000 / APPROX_MSG_RATE_PER_SEC); // do Thread.sleep(0) for max speed - // Note: STANDARD Edition Solace PubSub+ broker is limited to 10k msg/s max ingress - } catch (InterruptedException e) { - Thread.currentThread().interrupt(); // restore the interrupt status, do not swallow it - isShutdown = true; - } } + // no delay between messages: the loop publishes as fast as send() returns, the way an + // application publishes each message as soon as it has one + // Note: STANDARD Edition Solace broker is limited to 10k msg/s max ingress } isShutdown = true; } diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-guaranteed-replier-sample.java b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-guaranteed-replier-sample.java index 471a19c..bfbacd2 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-guaranteed-replier-sample.java +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-guaranteed-replier-sample.java @@ -157,8 +157,7 @@ public void handleEvent(SessionEventArgs event) { final Queue requestQueue = JCSMPFactory.onlyInstance().createQueue(REQUEST_QUEUE_NAME); // ELEVATION: provision the durable request queue in-process and map the request - // topic onto it at startup, so a fresh broker works out of the box (the upstream - // sample provisions with flag 0, which throws "already exists" on a re-run). + // topic onto it at startup, so a fresh broker works out of the box. // best practice: confirm the broker allows client-side endpoint management first if (!session.isCapable(CapabilityType.ENDPOINT_MANAGEMENT)) { System.err.println("This client/broker does not allow client-side endpoint management; " @@ -168,8 +167,8 @@ public void handleEvent(SessionEventArgs event) { EndpointProperties endpointProps = new EndpointProperties(); endpointProps.setAccessType(EndpointProperties.ACCESSTYPE_EXCLUSIVE); // single-replier sample endpointProps.setPermission(EndpointProperties.PERMISSION_CONSUME); - // provision the durable request queue IDEMPOTENTLY so a re-run is safe (NOT the - // upstream flag 0, which throws on the second run) + // provision the durable request queue IDEMPOTENTLY so a re-run is safe (flag 0 would + // throw "already exists" on the second run) session.provision(requestQueue, endpointProps, JCSMPSession.FLAG_IGNORE_ALREADY_EXISTS); // map the request topic onto the queue with a subscription (PERSISTENT pub/sub onto // a queue); WAIT_FOR_CONFIRM blocks until the broker confirms the subscription diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-guaranteed-subscriber-sample.java b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-guaranteed-subscriber-sample.java index 6f13cbd..b2471c2 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-guaranteed-subscriber-sample.java +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/jcsmp-guaranteed-subscriber-sample.java @@ -153,8 +153,7 @@ public void handleEvent(SessionEventArgs event) { final Queue queue = JCSMPFactory.onlyInstance().createQueue(QUEUE_NAME); // ELEVATION: provision the durable queue in-process and map the topic onto it - // at startup, so a fresh broker works out of the box (the upstream sample expects - // the queue to pre-exist and errors out otherwise). + // at startup, so a fresh broker works out of the box. // best practice: confirm the broker allows client-side endpoint management first if (!session.isCapable(CapabilityType.ENDPOINT_MANAGEMENT)) { System.err.println("This client/broker does not allow client-side endpoint management; " diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-publisher-async-sample.py b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-publisher-async-sample.py new file mode 100644 index 0000000..1ea90bd --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-publisher-async-sample.py @@ -0,0 +1,329 @@ +""" +Derived from patterns/direct_publisher.py + SolaceSamples/solace-samples-python + https://github.com/SolaceSamples/solace-samples-python + +Elevated to documented best practices per the Python API developer guide: + Messaging Service: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md + Publishing Direct Messages: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-DM-Publish.md + the Python API reference (units, defaults, listener contracts): + https://docs.solace.com/API-Developer-Online-Ref-Documentation/python/index.html + and the C API Best Practices (the Python API wraps the C API, so its callback, + reconnect, acknowledgement, and time-to-live practices apply): + https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md + +Reference sample. Wired for a basic direct (at-most-once) pub/sub journey: +basic-auth connect with a reconnection retry strategy, a continuous DIRECT binary +publish loop with a rotating payload to a topic, a publish failure listener for the +asynchronous failures the API reports, capacity-bounded back pressure that blocks +publish() while the buffer is full, reconnection and service interruption listeners +registered BEFORE connect(), and a graceful SIGINT/SIGTERM shutdown. Direct +messaging is at-most-once: there is no broker receipt, so there is NO publish +receipt listener and NO user context here (the inverse of the guaranteed +publisher, which acts on each ACK/NACK), and there are no outstanding receipts to +drain at shutdown. Messages published while the API reconnects are lost, and each one +is reported through the publish failure listener. Publishes the payload directly, with +no OutboundMessageBuilder. Structured as module-level functions, setup_solace(...), +connect_solace(...), run_publish_loop(...), and teardown_solace(...), that share one +PublisherState; setup_solace() creates the service and the publisher before any +connection exists, and main() runs teardown from its finally on every exit path after that. + +Only practices documented in canonical Solace sources are encoded here. + +Copied into a generated project as direct_publisher_async.py, next to the shared helper +python-solace-connection-config.py copied as solace_connection_config.py. + +Any generated adaptation of this sample MUST begin with the exact line: + AI-assisted code. Review before production use. +(This reference sample itself carries no such header by design.) +""" + +import logging +import signal +import sys +import threading +from dataclasses import dataclass + +from solace.messaging.config.retry_strategy import RetryStrategy +from solace.messaging.errors.pubsubplus_client_error import IncompleteMessageDeliveryError, PubSubPlusClientError +from solace.messaging.messaging_service import ( + MessagingService, + ReconnectionAttemptListener, + ReconnectionListener, + ServiceEvent, + ServiceInterruptionListener, +) +from solace.messaging.publisher.direct_message_publisher import ( + DirectMessagePublisher, + FailedPublishEvent, + PublishFailureListener, +) +from solace.messaging.resources.topic import Topic + +from solace_connection_config import load_service_properties + +APP_NAME = "DirectPublisher" +TOPIC_PREFIX = "solace/samples/" # used as the topic "root" +API = "Python" +PAYLOAD_SIZE = 512 +# reconnect budget: 20 attempts 3 s apart, a minute of reconnect attempts, a reasonable +# default when the design says nothing about high availability. The interval is in +# MILLISECONDS per the RetryStrategy API reference. +RECONNECT_RETRIES = 20 +RECONNECT_RETRY_INTERVAL_MS = 3000 +# For HA failover resilience, use the C API Best Practices values +# (https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md): +# HA failover: with_connection_retry_strategy(RetryStrategy.parametrized_retry(1, 3000)), +# with_reconnection_retry_strategy(RetryStrategy.parametrized_retry(20, 3000)), +# properties[transport_layer_properties.CONNECTION_RETRIES_PER_HOST] = 5 +# back pressure: publish() blocks once this many messages wait unsent, until there is room +# The capacity is a count of messages, not a size in bytes: the memory it holds depends on +# the size of each message. Tune it to the memory the application's deployment can give +# the buffer. The API has no default capacity (its default, on_back_pressure_elastic, +# buffers without bound); 1000 is the value the Python developer guide uses in its back +# pressure examples. +PUBLISH_BUFFER_CAPACITY = 1000 +# terminate(grace_period): how long to wait for buffered sends to leave before the +# publisher stops; terminate() raises IncompleteMessageDeliveryError when messages are +# still buffered after that. A bounded grace period keeps shutdown time predictable, for +# example within the stop timeout of a process supervisor, so the disconnect still runs. +# The API default is 600000 ms (10 minutes). Adjust the value to suit the application: +# longer gives buffered sends more time to leave, shorter exits sooner. +TERMINATE_GRACE_PERIOD_MS = 10_000 + +# API events go through standard logging; the trace() narration below is a separate channel +logger = logging.getLogger(APP_NAME) + + +@dataclass +class PublisherState: + """What the publish loop, the signal handler, and the API listener threads share: the + service, the publisher, the shutdown event, and the counters. The service and the + publisher are required, so every PublisherState holds both (built, though not + necessarily connected or started).""" + messaging_service: MessagingService + publisher: DirectMessagePublisher + shutdown: threading.Event + connect_attempted: bool = False + exit_code: int = 0 # set to 1 on a failure exit (service interruption, a publish() failure, or a failed terminate()) + msg_sent_counter: int = 0 # num messages sent + + +def main(args: list[str]) -> int: + trace(f"{API} {APP_NAME} initializing...") + shutdown = threading.Event() + + def on_shutdown_signal(signum: int, _frame) -> None: + trace(f"Shutdown signal received ({signal.Signals(signum).name}), stopping publisher...") + shutdown.set() + + # graceful shutdown: SIGINT (Ctrl-C) or SIGTERM sets the shutdown event, the publish + # loop exits, and teardown_solace() in the finally below stops the publisher and + # disconnects. Python runs signal handlers on the main thread, so the handler only + # flags; the cleanup runs on the main thread's normal exit path. Registered before + # connect, so a SIGTERM during the blocking connect() still reaches teardown. + signal.signal(signal.SIGINT, on_shutdown_signal) + signal.signal(signal.SIGTERM, on_shutdown_signal) + # basic username/password connection details, built by the shared connection-config + # helper: read from a config.json in the working directory if present, else from the + # command line ( [password]). Turning the + # application's input into Solace properties is the application's job, so it stays out + # of setup_solace(), which works from the properties alone. + properties = load_service_properties(args, APP_NAME) + # setup_solace() raises before any connection exists, so there is nothing to tear down + # if it fails + state = setup_solace(properties, shutdown) + try: + connect_solace(state) + run_publish_loop(state) + finally: + teardown_solace(state) + trace("Main thread quitting.") + return state.exit_code # non-zero after a failure, so scripts and supervisors see it + + +def setup_solace(properties: dict, shutdown: threading.Event) -> PublisherState: + # build() creates the native session and resolves the host, so an unresolvable host + # fails here, before connect() + messaging_service = ( + MessagingService.builder() + # the builder merges each call in order and a later call wins, so the reconnection + # strategy comes first: it is the default, and reconnection keys in the properties + # (for example from config.json) override it + .with_reconnection_retry_strategy( + RetryStrategy.parametrized_retry(RECONNECT_RETRIES, RECONNECT_RETRY_INTERVAL_MS)) + .from_properties(properties) + .build() + ) + # DIRECT publisher with capacity-bounded back pressure: publish() blocks once + # PUBLISH_BUFFER_CAPACITY messages wait unsent, and returns when there is room again. + # The API default (on_back_pressure_elastic) buffers without bound and the developer + # guide cautions it can exhaust memory; on_back_pressure_reject(n) raises + # PublisherOverflowError instead of blocking. Building the publisher needs only the + # built service; start() is what requires the connection. One service can serve + # several publishers. + publisher = ( + messaging_service.create_direct_message_publisher_builder() + .on_back_pressure_wait(PUBLISH_BUFFER_CAPACITY) + .build() + ) + # direct messaging has no broker receipt, so failures the API detects after publish() + # returned (a buffered message that could not be sent) arrive asynchronously through + # this listener; without it they are silent + publisher.set_publish_failure_listener(PublishFailureHandler()) + state = PublisherState(messaging_service=messaging_service, publisher=publisher, shutdown=shutdown) + # best practice: register the service event listeners BEFORE connect(), so no + # reconnection or interruption event raised during or right after the connect is lost, + # and handle each event appropriately rather than only logging it + service_events = ServiceEventHandler(state) + messaging_service.add_reconnection_attempt_listener(service_events) + messaging_service.add_reconnection_listener(service_events) + messaging_service.add_service_interruption_listener(service_events) + return state + + +def connect_solace(state: PublisherState) -> None: + state.connect_attempted = True + state.messaging_service.connect() # blocking connect + state.publisher.start() # raises IllegalStateError unless the service is connected + + +def run_publish_loop(state: PublisherState) -> None: + threading.Thread(target=print_stats, args=(state,), name="stats", daemon=True).start() + + trace(f"{API} {APP_NAME} connected, and running. Press Ctrl-C to quit.") + trace(f"Publishing to topic '{TOPIC_PREFIX}{API.lower()}/direct/pub/...', " + "at-most-once (no broker receipt, no redelivery).") + while not state.shutdown.is_set(): + # each loop, change the payload as an example + chosen_character = chr(state.msg_sent_counter % 26 + 65) # choose a "random" letter [A-Z] + # a bytearray payload travels as the binary attachment; consumers read it back with + # get_payload_as_bytes() (a str payload would arrive via get_payload_as_string()) + payload = bytearray(chosen_character.encode("ascii") * PAYLOAD_SIZE) + # NO user context: direct is at-most-once with no broker receipt, so there is + # nothing to correlate (the inverse of the guaranteed publisher, which passes the + # message id as user_context for local ACK/NACK correlation) + topic = Topic.of(f"{TOPIC_PREFIX}{API.lower()}/direct/pub/{chosen_character}") + try: + # blocks while the buffer is full (back pressure) on a connected transport; during + # a reconnect it does not block (see ServiceEventHandler.on_reconnecting). A + # shutdown signal that arrives during that wait takes effect once publish() + # returns: when the buffer drains, or when the service goes down and the API + # releases the wait. + # builderless publish: the payload goes to publish() directly, and the API builds the + # message. Set a per-message header with additional_message_properties, for example + # additional_message_properties={message_properties.APPLICATION_MESSAGE_ID: msg_id} + # with `from solace.messaging.config.solace_properties import message_properties`. + # Note: an OutboundMessageBuilder (messaging_service.message_builder()) is very + # useful for "templated" messages: set the headers that are the same on every + # message once on the builder, and build each message with only its few + # differences (see the guaranteed publisher sample). + state.publisher.publish(payload, topic) + state.msg_sent_counter += 1 + except PubSubPlusClientError as error: + # publish() raises when the message cannot be sent and retrying would not help + logger.warning("publish() failed, quitting: %s", error) + state.exit_code = 1 + break # let's quit; or, could initiate a new connection attempt + # no delay between messages: the loop publishes as fast as publish() returns, the way + # an application publishes each message as soon as it has one + # Note: STANDARD Edition Solace broker is limited to 10k msg/s max ingress + state.shutdown.set() + + +def print_stats(state: PublisherState) -> None: + # simple way of calculating message rates; wait() returns True (ending the loop) on shutdown + while not state.shutdown.wait(1.0): + trace(f"{API} {APP_NAME} Published msgs/s: {state.msg_sent_counter:,}") + state.msg_sent_counter = 0 + + +def teardown_solace(state: PublisherState) -> None: + # Application cleanup belongs here: once setup_solace() returns, teardown_solace() runs + # in main's finally on EVERY exit path (normal quit, SIGINT/SIGTERM, service + # interruption, or an exception), so Solace teardown and application-side cleanup are + # never skipped. + state.shutdown.set() + if state.publisher.is_running(): + # direct is at-most-once: there are no broker receipts to drain, so terminate() only + # gives the messages still in the publisher's buffer up to the grace period to leave. + # It raises IncompleteMessageDeliveryError when messages are still buffered after + # that; catch it so the disconnect below still runs. Those messages are lost, which + # direct (at-most-once) messaging allows, so this is a warning and the exit code stays + # 0. An application that must not lose messages at shutdown can raise the level (or + # use guaranteed messaging). + try: + state.publisher.terminate(TERMINATE_GRACE_PERIOD_MS) + except IncompleteMessageDeliveryError as error: + logger.warning("Publisher stopped with messages still buffered after %d ms: %s", + TERMINATE_GRACE_PERIOD_MS, error) + except PubSubPlusClientError as error: + logger.error("Publisher terminate() failed: %s", error) + state.exit_code = 1 + if state.connect_attempted: + # disconnect() raises IllegalStateError on a service that never attempted to connect, + # hence the flag; on a service that is already down it returns quietly + state.messaging_service.disconnect() + + +def trace(message: str) -> None: + """Demo narration sink: every status line in this sample funnels through this one + function. An application replaces this single body to route narration to its + logger or reporting system. The logger calls for API events (service, publish + failure) are a separate channel and stay as they are.""" + print(message, flush=True) + + +############################################################################ + + +class ServiceEventHandler(ReconnectionAttemptListener, ReconnectionListener, ServiceInterruptionListener): + """The three service event listeners in one class. Callbacks run on an API thread: + return promptly and do not block in them, since events and delivery stall while a + callback runs (C API Best Practices).""" + + def __init__(self, state: PublisherState) -> None: + self.state = state + + def on_reconnecting(self, event: ServiceEvent) -> None: + # connection lost, automatic reconnect attempt in progress. Direct messaging is + # at-most-once: while the transport is down, publish() does not block, and the API + # fails each message it cannot send and reports it through the publish failure + # listener (PublishFailureHandler below). An application that must not lose messages + # uses guaranteed messaging instead (see the guaranteed publisher sample); one that + # must not produce messages during an outage can pause its publish loop between this + # event and on_reconnected(). + logger.warning("Service reconnecting: %s (%s)", event.get_message(), event.get_cause()) + + def on_reconnected(self, event: ServiceEvent) -> None: + # automatic reconnect succeeded: messages published from now on are sent again + logger.info("Service reconnected to %s", event.get_broker_uri()) + + def on_service_interrupted(self, event: ServiceEvent) -> None: + # the connection went down and cannot be restored (reconnect attempts exhausted) + logger.error("Service interrupted and cannot be restored, quitting: %s (%s)", + event.get_message(), event.get_cause()) + # Application cleanup signal: the service will not recover. Trigger application-side + # cleanup from here. This sample sets shutdown, so the publish loop exits and + # teardown_solace() runs in main's finally. + self.state.exit_code = 1 + self.state.shutdown.set() + + +class PublishFailureHandler(PublishFailureListener): + """Very simple class, used for the asynchronous publish failures the API reports for + direct messages (there is no broker receipt to act on). Runs on an API thread: + return promptly and do not block in it (C API Best Practices).""" + + def on_failed_publish(self, failed_publish_event: FailedPublishEvent) -> None: + # a buffered message could not be sent (the developer guide names an invalid topic or a + # termination of the service as causes); direct has no NACK, so this never fires for one + logger.warning("Failed to publish to %s: %s", + failed_publish_event.get_destination().get_name(), failed_publish_event.get_exception()) + + +if __name__ == "__main__": + logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s") + sys.exit(main(sys.argv[1:])) diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-subscriber-async-sample.py b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-subscriber-async-sample.py new file mode 100644 index 0000000..4f1319c --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-subscriber-async-sample.py @@ -0,0 +1,309 @@ +""" +Derived from patterns/direct_receiver.py + SolaceSamples/solace-samples-python + https://github.com/SolaceSamples/solace-samples-python + +Elevated to documented best practices per the Python API developer guide: + Messaging Service: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md + Consuming Direct Messages: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-DM-Receive.md + the Python API reference (units, defaults, listener contracts): + https://docs.solace.com/API-Developer-Online-Ref-Documentation/python/index.html + and the C API Best Practices (the Python API wraps the C API, so its callback, + reconnect, acknowledgement, and time-to-live practices apply): + https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md + +Reference sample. Wired for a basic direct (at-most-once) pub/sub journey: +basic-auth connect with a reconnection retry strategy, a plain topic subscription +(no queue, no provisioning), an async DIRECT receiver with a capacity-bounded +buffer that detects discards, reconnection and service interruption listeners +registered BEFORE connect(), and a graceful SIGINT/SIGTERM shutdown. Direct +messaging is at-most-once: there is no broker ACK and no redelivery, so there is +NO client acknowledgement here (the inverse of the guaranteed subscriber). +Structured as module-level functions, setup_solace(...), connect_solace(...), +await_messages(...), and teardown_solace(...), that share one SubscriberState; +setup_solace() creates the service and the receiver before any connection exists, +and main() runs teardown from its finally on every exit path after that. + +Only practices documented in canonical Solace sources are encoded here. + +Copied into a generated project as direct_subscriber_async.py, next to the shared helper +python-solace-connection-config.py copied as solace_connection_config.py. + +Any generated adaptation of this sample MUST begin with the exact line: + AI-assisted code. Review before production use. +(This reference sample itself carries no such header by design.) +""" + +import logging +import signal +import sys +import threading +from dataclasses import dataclass + +from solace.messaging.config.retry_strategy import RetryStrategy +from solace.messaging.errors.pubsubplus_client_error import IncompleteMessageDeliveryError, PubSubPlusClientError +from solace.messaging.messaging_service import ( + MessagingService, + ReconnectionAttemptListener, + ReconnectionListener, + ServiceEvent, + ServiceInterruptionListener, +) +from solace.messaging.receiver.direct_message_receiver import DirectMessageReceiver +from solace.messaging.receiver.inbound_message import InboundMessage +from solace.messaging.receiver.message_receiver import MessageHandler +from solace.messaging.resources.topic_subscription import TopicSubscription + +from solace_connection_config import load_service_properties + +APP_NAME = "DirectSubscriber" +# topic to subscribe to directly; matches the direct publisher sample's topic root +TOPIC_NAME = "solace/samples/python/direct/pub/>" +API = "Python" +# reconnect budget: 20 attempts 3 s apart, a minute of reconnect attempts, a reasonable +# default when the design says nothing about high availability. The interval is in +# MILLISECONDS per the RetryStrategy API reference. +RECONNECT_RETRIES = 20 +RECONNECT_RETRY_INTERVAL_MS = 3000 +# For HA failover resilience, use the C API Best Practices values +# (https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md): +# HA failover: with_connection_retry_strategy(RetryStrategy.parametrized_retry(1, 3000)), +# with_reconnection_retry_strategy(RetryStrategy.parametrized_retry(20, 3000)), +# properties[transport_layer_properties.CONNECTION_RETRIES_PER_HOST] = 5 +# back pressure: once this many received messages wait for the handler, the receiver +# discards each new incoming message until there is room +# The capacity is a count of messages, not a size in bytes: the memory it holds depends on +# the size of each message. Tune it to the memory the application's deployment can give +# the buffer. The API has no default capacity (its default, on_back_pressure_elastic, +# buffers without bound); 1000 is the value the Python developer guide uses in its back +# pressure examples. +RECEIVE_BUFFER_CAPACITY = 1000 +# terminate(grace_period): how long to wait for the handler to drain the messages the API +# has already received before the receiver stops; terminate() raises +# IncompleteMessageDeliveryError when messages remain after that. A bounded grace period +# keeps shutdown time predictable, for example within the stop timeout of a process +# supervisor, so the disconnect still runs. The API default is 600000 ms (10 minutes). +# Adjust the value to suit the application: longer lets the handler finish more of the +# received messages, shorter exits sooner. +TERMINATE_GRACE_PERIOD_MS = 10_000 + +# API events go through standard logging; the trace() narration below is a separate channel +logger = logging.getLogger(APP_NAME) + + +@dataclass +class SubscriberState: + """What the main loop, the signal handler, and the API listener threads share: the + service, the receiver, the shutdown event, the flags the listeners flip, and the + counters. The service and the receiver are required, so every SubscriberState holds + both (built, though not necessarily connected or started). Use this type of app for + receiving Direct (at-most-once) messages from a topic.""" + messaging_service: MessagingService + receiver: DirectMessageReceiver + shutdown: threading.Event + connect_attempted: bool = False + exit_code: int = 0 # set to 1 on a failure exit (service interruption or a failed terminate()) + msg_recv_counter: int = 0 # num messages received + has_detected_discard: bool = False # any discards seen? + + +def main(args: list[str]) -> int: + trace(f"{API} {APP_NAME} initializing...") + shutdown = threading.Event() + + def on_shutdown_signal(signum: int, _frame) -> None: + trace(f"Shutdown signal received ({signal.Signals(signum).name}), stopping subscriber...") + shutdown.set() + + # graceful shutdown: SIGINT (Ctrl-C) or SIGTERM sets the shutdown event, the main + # loop exits, and teardown_solace() in the finally below stops the receiver and + # disconnects. Python runs signal handlers on the main thread, so the handler only + # flags; the cleanup runs on the main thread's normal exit path. Registered before + # connect, so a SIGTERM during the blocking connect() still reaches teardown. + signal.signal(signal.SIGINT, on_shutdown_signal) + signal.signal(signal.SIGTERM, on_shutdown_signal) + # basic username/password connection details, built by the shared connection-config + # helper: read from a config.json in the working directory if present, else from the + # command line ( [password]). Turning the + # application's input into Solace properties is the application's job, so it stays out + # of setup_solace(), which works from the properties alone. + properties = load_service_properties(args, APP_NAME) + # setup_solace() raises before any connection exists, so there is nothing to tear down + # if it fails + state = setup_solace(properties, shutdown) + try: + connect_solace(state) + await_messages(state) + finally: + teardown_solace(state) + trace("Main thread quitting.") + return state.exit_code # non-zero after a failure, so scripts and supervisors see it + + +def setup_solace(properties: dict, shutdown: threading.Event) -> SubscriberState: + # build() creates the native session and resolves the host, so an unresolvable host + # fails here, before connect() + messaging_service = ( + MessagingService.builder() + # the builder merges each call in order and a later call wins, so the reconnection + # strategy comes first: it is the default, and reconnection keys in the properties + # (for example from config.json) override it + .with_reconnection_retry_strategy( + RetryStrategy.parametrized_retry(RECONNECT_RETRIES, RECONNECT_RETRY_INTERVAL_MS)) + .from_properties(properties) + .build() + ) + # DIRECT receiver: a topic subscription on the receiver (no queue, no provisioning). + # Direct messaging is at-most-once: messages flow straight to the subscriber with no + # broker ACK and no redelivery. start() applies the subscription and blocks until the + # broker confirms it, so the route is live before this subscriber reports ready. + # Capacity-bounded back pressure: once RECEIVE_BUFFER_CAPACITY messages wait for the + # handler, on_back_pressure_drop_latest(n) discards each new incoming message, and the + # next message delivered carries an internal discard indication. The API default + # (on_back_pressure_elastic) buffers without bound; on_back_pressure_drop_oldest(n) + # discards the oldest buffered message instead. Building the receiver needs only the + # built service; start() is what requires the connection. + receiver = ( + messaging_service.create_direct_message_receiver_builder() + .with_subscriptions([TopicSubscription.of(TOPIC_NAME)]) + .on_back_pressure_drop_latest(RECEIVE_BUFFER_CAPACITY) + .build() + ) + state = SubscriberState(messaging_service=messaging_service, receiver=receiver, shutdown=shutdown) + # best practice: register the service event listeners BEFORE connect(), so no + # reconnection or interruption event raised during or right after the connect is lost, + # and handle each event appropriately rather than only logging it + service_events = ServiceEventHandler(state) + messaging_service.add_reconnection_attempt_listener(service_events) + messaging_service.add_reconnection_listener(service_events) + messaging_service.add_service_interruption_listener(service_events) + return state + + +def connect_solace(state: SubscriberState) -> None: + state.connect_attempted = True + state.messaging_service.connect() # blocking connect + # set the message handler BEFORE start(), so the handler is in place when delivery + # begins and received messages do not wait in the API's buffer for it. receive_async() + # needs a connected service, so it runs here after connect(), not in setup_solace(). + # see bottom of file for DirectMessageHandler, which receives the messages from the topic + state.receiver.receive_async(DirectMessageHandler(state)) + trace(f"Adding direct topic subscription '{TOPIC_NAME}'.") + state.receiver.start() + trace(f"{APP_NAME} subscribed and consuming. Press Ctrl-C to quit.") + + +def await_messages(state: SubscriberState) -> None: + # async direct receive working now, so time to wait until done... + while not state.shutdown.wait(1.0): # wait 1 second; the wait returns True once shutdown is requested + trace(f"{API} {APP_NAME} Received msgs/s: {state.msg_recv_counter:,}") # simple way of calculating message rates + state.msg_recv_counter = 0 + if state.has_detected_discard: # at least one direct message was dropped before this subscriber + trace("*** Discard detected (at-most-once: a direct message was dropped) ***") + state.has_detected_discard = False # only show the warning once per second + + +def teardown_solace(state: SubscriberState) -> None: + # Application cleanup belongs here: once setup_solace() returns, teardown_solace() runs + # in main's finally on EVERY exit path (normal quit, SIGINT/SIGTERM, service + # interruption, or an exception), so Solace teardown and application-side cleanup are + # never skipped. + state.shutdown.set() + if state.receiver.is_running(): + # direct is at-most-once: there are no acknowledgements to drain before exit, so + # terminate() stops delivery and gives the handler up to the grace period to drain + # the messages the API has already received. It raises IncompleteMessageDeliveryError + # when messages remain after that; catch it so the disconnect below still runs + try: + state.receiver.terminate(TERMINATE_GRACE_PERIOD_MS) + except IncompleteMessageDeliveryError as error: + logger.error("Receiver stopped with messages still undelivered to the handler after %d ms: %s", + TERMINATE_GRACE_PERIOD_MS, error) + state.exit_code = 1 + except PubSubPlusClientError as error: + logger.error("Receiver terminate() failed: %s", error) + state.exit_code = 1 + if state.connect_attempted: + # disconnect() raises IllegalStateError on a service that never attempted to connect, + # hence the flag; on a service that is already down it returns quietly + state.messaging_service.disconnect() # will also release the receiver + + +def trace(message: str) -> None: + """Demo narration sink: every status line in this sample funnels through this one + function. An application replaces this single body to route narration to its + logger or reporting system. The logger calls for API events (service) are a + separate channel and stay as they are.""" + print(message, flush=True) + + +############################################################################ + + +class ServiceEventHandler(ReconnectionAttemptListener, ReconnectionListener, ServiceInterruptionListener): + """The three service event listeners in one class. Callbacks run on an API thread: + return promptly and do not block in them, since events and delivery stall while a + callback runs (C API Best Practices).""" + + def __init__(self, state: SubscriberState) -> None: + self.state = state + + def on_reconnecting(self, event: ServiceEvent) -> None: + # connection lost, automatic reconnect attempt in progress: direct delivery is paused, + # and the broker does not store direct messages for this subscriber while it is + # disconnected (at-most-once) + logger.warning("Service reconnecting; direct delivery is paused until re-established: %s (%s)", + event.get_message(), event.get_cause()) + + def on_reconnected(self, event: ServiceEvent) -> None: + # automatic reconnect succeeded; the API re-applies the topic subscription and delivery + # resumes (unlike JCSMP, no reapply-subscriptions property is needed) + logger.info("Service reconnected to %s; the subscription is restored and delivery resumes", + event.get_broker_uri()) + + def on_service_interrupted(self, event: ServiceEvent) -> None: + # the connection went down and cannot be restored (reconnect attempts exhausted) + logger.error("Service interrupted and cannot be restored, quitting: %s (%s)", + event.get_message(), event.get_cause()) + # Application cleanup signal: the service will not recover. Trigger application-side + # cleanup from here. This sample sets shutdown, so the main loop exits and + # teardown_solace() runs in main's finally. + self.state.exit_code = 1 + self.state.shutdown.set() + + +class DirectMessageHandler(MessageHandler): + """Very simple class, used for receiving direct messages from the topic. on_message runs + on an API-owned receiver thread, not the main thread: return promptly and do not block in + it (hand heavy work to the application's own thread or queue), since delivery stalls while + it runs (C API Best Practices).""" + + def __init__(self, state: SubscriberState) -> None: + self.state = state + + def on_message(self, message: InboundMessage) -> None: + self.state.msg_recv_counter += 1 + # the publisher sample sends a bytearray payload (the binary attachment), read back with + # get_payload_as_bytes(); a str payload arrives via get_payload_as_string() instead + # best practice: handle an unexpected message format without raising (C API Best + # Practices); errors="replace" keeps a non-UTF-8 payload from raising in the callback + payload = message.get_payload_as_bytes() + text = payload.decode("utf-8", errors="replace") if payload is not None else message.get_payload_as_string() + trace(f"{API} {APP_NAME} received: {text}") + # at-most-once discard evidence (API reference): the broker sets the broker discard + # indication when it discarded direct messages destined for this subscriber for any + # reason, and the API sets the internal one when its own receiver buffer dropped + # messages. There is NO redelivery in direct messaging. + discard = message.get_message_discard_notification() + if discard.has_broker_discard_indication() or discard.has_internal_discard_indication(): + self.state.has_detected_discard = True + # Direct messaging is at-most-once: there is no consumer-side acknowledgement here (the + # inverse of the guaranteed CLIENT-ack subscriber). The message is consumed as it + # arrives; the broker holds no copy and expects no ack. + + +if __name__ == "__main__": + logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s") + sys.exit(main(sys.argv[1:])) diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-publisher-async-sample.py b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-publisher-async-sample.py new file mode 100644 index 0000000..a94e4b5 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-publisher-async-sample.py @@ -0,0 +1,353 @@ +""" +Derived from patterns/guaranteed_publisher.py + SolaceSamples/solace-samples-python + https://github.com/SolaceSamples/solace-samples-python + +Elevated to documented best practices per the Python API developer guide: + Messaging Service: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md + Publishing Persistent Messages: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md + the Python API reference (units, defaults, listener contracts): + https://docs.solace.com/API-Developer-Online-Ref-Documentation/python/index.html + and the C API Best Practices (the Python API wraps the C API, so its callback, + reconnect, acknowledgement, and time-to-live practices apply): + https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md + +Reference sample. Wired for a basic publish-subscribe journey: basic-auth connect +with a reconnection retry strategy, PERSISTENT binary publish to a topic from a +templated message builder that sets the time-to-live once, publish receipt (ACK/NACK) +handling correlated by user context, capacity-bounded back pressure that blocks +publish() while the buffer is full, reconnection and service interruption listeners +registered BEFORE connect(), and a graceful SIGINT/SIGTERM shutdown that drains +outstanding receipts before disconnecting. Structured as module-level functions, +setup_solace(...), connect_solace(...), run_publish_loop(...), and teardown_solace(...), +that share one PublisherState; setup_solace() creates the service and the publisher +before any connection exists, and main() runs teardown from its finally on every exit +path after that. + +Only practices documented in canonical Solace sources are encoded here. + +Copied into a generated project as guaranteed_publisher_async.py, next to the shared +helper python-solace-connection-config.py copied as solace_connection_config.py. + +Any generated adaptation of this sample MUST begin with the exact line: + AI-assisted code. Review before production use. +(This reference sample itself carries no such header by design.) +""" + +import logging +import signal +import sys +import threading +import uuid +from dataclasses import dataclass + +from solace.messaging.config.retry_strategy import RetryStrategy +from solace.messaging.config.solace_properties import message_properties +from solace.messaging.errors.pubsubplus_client_error import IncompleteMessageDeliveryError, PubSubPlusClientError +from solace.messaging.messaging_service import ( + MessagingService, + ReconnectionAttemptListener, + ReconnectionListener, + ServiceEvent, + ServiceInterruptionListener, +) +from solace.messaging.publisher.persistent_message_publisher import ( + MessagePublishReceiptListener, + PersistentMessagePublisher, + PublishReceipt, +) +from solace.messaging.resources.topic import Topic + +from solace_connection_config import load_service_properties + +APP_NAME = "GuaranteedPublisher" +TOPIC_PREFIX = "solace/samples/" # used as the topic "root" +API = "Python" +PAYLOAD_SIZE = 512 +# reconnect budget: 20 attempts 3 s apart, a minute of reconnect attempts, a reasonable +# default when the design says nothing about high availability. The interval is in +# MILLISECONDS per the RetryStrategy API reference. +RECONNECT_RETRIES = 20 +RECONNECT_RETRY_INTERVAL_MS = 3000 +# For HA failover resilience, use the C API Best Practices values +# (https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md): +# HA failover: with_connection_retry_strategy(RetryStrategy.parametrized_retry(1, 3000)), +# with_reconnection_retry_strategy(RetryStrategy.parametrized_retry(20, 3000)), +# properties[transport_layer_properties.CONNECTION_RETRIES_PER_HOST] = 5 +# back pressure: publish() blocks once this many messages wait unsent, until there is room +# The capacity is a count of messages, not a size in bytes: the memory it holds depends on +# the size of each message. Tune it to the memory the application's deployment can give +# the buffer. The API has no default capacity (its default, on_back_pressure_elastic, +# buffers without bound); 1000 is the value the Python developer guide uses in its back +# pressure examples. +PUBLISH_BUFFER_CAPACITY = 1000 +# terminate(grace_period): how long to wait for buffered sends and their broker receipts +# to complete before the publisher stops; terminate() raises IncompleteMessageDeliveryError +# when sends or receipts are still outstanding after that. A bounded grace period keeps +# shutdown time predictable, for example within the stop timeout of a process supervisor, +# so the disconnect still runs. The API default is 600000 ms (10 minutes). Adjust the value +# to suit the application: longer gives sends and receipts more time to complete, shorter +# exits sooner. +TERMINATE_GRACE_PERIOD_MS = 10_000 +# best practice for Guaranteed publishing (C API Best Practices): set a time-to-live so an +# unconsumed message does not sit on a queue forever. In MILLISECONDS; 0, the API default, +# never expires. The queue must have respect-ttl enabled (off by default), or the broker +# ignores the TTL. Adjust the value to how long a message stays useful to its consumers. +# For most applications, make it at least the longest time a consumer can take to come +# back (restart plus reconnect), or a multiple of that time for a safety margin, so a +# message does not expire while its consumer recovers; use a shorter or longer TTL only +# for a specific need. +MESSAGE_TTL_MS = 60_000 + +# API events go through standard logging; the trace() narration below is a separate channel +logger = logging.getLogger(APP_NAME) + + +@dataclass +class PublisherState: + """What the publish loop, the signal handler, and the API listener threads share: the + service, the publisher, the shutdown event, and the counters. The service and the + publisher are required, so every PublisherState holds both (built, though not + necessarily connected or started).""" + messaging_service: MessagingService + publisher: PersistentMessagePublisher + shutdown: threading.Event + connect_attempted: bool = False + exit_code: int = 0 # set to 1 on a failure exit (service interruption, a publish() failure, or a failed terminate()) + msg_sent_counter: int = 0 # num messages sent + + +def main(args: list[str]) -> int: + trace(f"{API} {APP_NAME} initializing...") + shutdown = threading.Event() + + def on_shutdown_signal(signum: int, _frame) -> None: + trace(f"Shutdown signal received ({signal.Signals(signum).name}), stopping publisher...") + shutdown.set() + + # graceful shutdown: SIGINT (Ctrl-C) or SIGTERM sets the shutdown event, the publish + # loop exits, and teardown_solace() in the finally below drains the outstanding + # receipts and disconnects. Python runs signal handlers on the main thread, so the + # handler only flags; the cleanup runs on the main thread's normal exit path. + # Registered before connect, so a SIGTERM during the blocking connect() still reaches + # teardown. + signal.signal(signal.SIGINT, on_shutdown_signal) + signal.signal(signal.SIGTERM, on_shutdown_signal) + # basic username/password connection details, built by the shared connection-config + # helper: read from a config.json in the working directory if present, else from the + # command line ( [password]). Turning the + # application's input into Solace properties is the application's job, so it stays out + # of setup_solace(), which works from the properties alone. + properties = load_service_properties(args, APP_NAME) + # setup_solace() raises before any connection exists, so there is nothing to tear down + # if it fails + state = setup_solace(properties, shutdown) + try: + connect_solace(state) + run_publish_loop(state) + finally: + teardown_solace(state) + trace("Main thread quitting.") + return state.exit_code # non-zero after a failure, so scripts and supervisors see it + + +def setup_solace(properties: dict, shutdown: threading.Event) -> PublisherState: + # build() creates the native session and resolves the host, so an unresolvable host + # fails here, before connect() + messaging_service = ( + MessagingService.builder() + # the builder merges each call in order and a later call wins, so the reconnection + # strategy comes first: it is the default, and reconnection keys in the properties + # (for example from config.json) override it + .with_reconnection_retry_strategy( + RetryStrategy.parametrized_retry(RECONNECT_RETRIES, RECONNECT_RETRY_INTERVAL_MS)) + .from_properties(properties) + .build() + ) + # PERSISTENT publisher with capacity-bounded back pressure: publish() blocks once + # PUBLISH_BUFFER_CAPACITY messages wait unsent, and returns when there is room again. + # The API default (on_back_pressure_elastic) buffers without bound and the developer + # guide cautions it can exhaust memory; on_back_pressure_reject(n) raises + # PublisherOverflowError instead of blocking. Building the publisher needs only the + # built service; start() is what requires the connection. One service can serve + # several publishers. + publisher = ( + messaging_service.create_persistent_message_publisher_builder() + .on_back_pressure_wait(PUBLISH_BUFFER_CAPACITY) + .build() + ) + state = PublisherState(messaging_service=messaging_service, publisher=publisher, shutdown=shutdown) + # best practice: register the service event listeners BEFORE connect(), so no + # reconnection or interruption event raised during or right after the connect is lost, + # and handle each event appropriately rather than only logging it + service_events = ServiceEventHandler(state) + messaging_service.add_reconnection_attempt_listener(service_events) + messaging_service.add_reconnection_listener(service_events) + messaging_service.add_service_interruption_listener(service_events) + return state + + +def connect_solace(state: PublisherState) -> None: + state.connect_attempted = True + state.messaging_service.connect() # blocking connect + state.publisher.start() # raises IllegalStateError unless the service is connected + # best practice: check every publish receipt; a receipt that carries an exception is a + # NACK or a failure to persist (see PublishReceiptHandler at the bottom of the file). + # The listener is set AFTER start(): the API raises IllegalStateError on a publisher + # that is not started yet. + state.publisher.set_message_publish_receipt_listener(PublishReceiptHandler()) + + +def run_publish_loop(state: PublisherState) -> None: + threading.Thread(target=print_stats, args=(state,), name="stats", daemon=True).start() + # one OutboundMessageBuilder as a message "template", reused for every message: set the + # headers that are the same on every message (here the TTL) once on the builder. + # build() starts each message as a copy of the template, so there is no set call per + # message for these headers + message_builder = ( + state.messaging_service.message_builder() + .with_property(message_properties.PERSISTENT_TIME_TO_LIVE, MESSAGE_TTL_MS) + ) + + trace(f"{API} {APP_NAME} connected, and running. Press Ctrl-C to quit.") + trace(f"Publishing to topic '{TOPIC_PREFIX}{API.lower()}/pers/pub/...', " + "please ensure queue has matching subscription.") + while not state.shutdown.is_set(): + # each loop, change the payload as an example + chosen_character = chr(state.msg_sent_counter % 26 + 65) # choose a "random" letter [A-Z] + # a bytearray payload travels as the binary attachment; consumers read it back with + # get_payload_as_bytes() (a str payload would arrive via get_payload_as_string()) + payload = bytearray(chosen_character.encode("ascii") * PAYLOAD_SIZE) + msg_id = str(uuid.uuid4()) + # the per-message difference goes in build()'s additional_message_properties, which + # applies to this message only; a with_*() call on the builder would change the + # template, so every later message would carry it too + message = message_builder.build( + payload, additional_message_properties={message_properties.APPLICATION_MESSAGE_ID: msg_id}) + # NOTE: publishing to topic, so make sure the consumer's queue is subscribed to the same topic, + # or enable "Reject Message to Sender on No Subscription Match" in the client-profile + topic = Topic.of(f"{TOPIC_PREFIX}{API.lower()}/pers/pub/{chosen_character}") + try: + # user_context for local ACK/NACK correlation: the API hands it back in the + # PublishReceipt, so an immutable per-send identifier (the message id) lets the + # receipt handler name the exact message the broker accepted or rejected. + # publish() blocks while the buffer is full (back pressure). During an automatic + # reconnect the messages wait in the buffer for the transport and go out after + # it, and publish() blocks only once the buffer fills; the loop needs no pause of + # its own (the direct publisher, whose messages fail instead of waiting, does need + # one). A shutdown signal that + # arrives during that wait takes effect once publish() returns: when the buffer + # drains, or when the service goes down and the API releases the wait. + state.publisher.publish(message, topic, user_context=msg_id) + state.msg_sent_counter += 1 + except PubSubPlusClientError as error: + # publish() raises when the message cannot be sent and retrying would not help + logger.warning("publish() failed, quitting: %s", error) + state.exit_code = 1 + break # let's quit; or, could initiate a new connection attempt + # no delay between messages: the loop publishes as fast as publish() returns, the way + # an application publishes each message as soon as it has one + # Note: STANDARD Edition Solace broker is limited to 10k msg/s max ingress + state.shutdown.set() + + +def print_stats(state: PublisherState) -> None: + # simple way of calculating message rates; wait() returns True (ending the loop) on shutdown + while not state.shutdown.wait(1.0): + trace(f"{API} {APP_NAME} Published msgs/s: {state.msg_sent_counter:,}") + state.msg_sent_counter = 0 + + +def teardown_solace(state: PublisherState) -> None: + # Application cleanup belongs here: once setup_solace() returns, teardown_solace() runs + # in main's finally on EVERY exit path (normal quit, SIGINT/SIGTERM, service + # interruption, or an exception), so Solace teardown and application-side cleanup are + # never skipped. + state.shutdown.set() + if state.publisher.is_running(): + # gracefully finish outstanding receipts before exit: terminate(grace_period) first + # sends what is still buffered, then waits up to the grace period for the broker's + # receipt of every published message, so no in-flight PERSISTENT message is + # abandoned unacknowledged. This is the explicit-condition shutdown; no fixed sleep. + # It raises IncompleteMessageDeliveryError when sends or receipts are still + # outstanding after the grace period; catch it so the disconnect below still runs + try: + state.publisher.terminate(TERMINATE_GRACE_PERIOD_MS) + except IncompleteMessageDeliveryError as error: + logger.error("Publisher stopped with sends or receipts still outstanding after %d ms: %s", + TERMINATE_GRACE_PERIOD_MS, error) + state.exit_code = 1 + except PubSubPlusClientError as error: + logger.error("Publisher terminate() failed: %s", error) + state.exit_code = 1 + if state.connect_attempted: + # disconnect() raises IllegalStateError on a service that never attempted to connect, + # hence the flag; on a service that is already down it returns quietly + state.messaging_service.disconnect() + + +def trace(message: str) -> None: + """Demo narration sink: every status line in this sample funnels through this one + function. An application replaces this single body to route narration to its + logger or reporting system. The logger calls for API events (service, receipt) + are a separate channel and stay as they are.""" + print(message, flush=True) + + +############################################################################ + + +class ServiceEventHandler(ReconnectionAttemptListener, ReconnectionListener, ServiceInterruptionListener): + """The three service event listeners in one class. Callbacks run on an API thread: + return promptly and do not block in them, since events and delivery stall while a + callback runs (C API Best Practices).""" + + def __init__(self, state: PublisherState) -> None: + self.state = state + + def on_reconnecting(self, event: ServiceEvent) -> None: + # connection lost, automatic reconnect attempt in progress: new messages wait in the + # buffer (publish() blocks once it is full) and go out after the reconnect, so no + # guaranteed message is lost + logger.warning("Service reconnecting: %s (%s)", event.get_message(), event.get_cause()) + + def on_reconnected(self, event: ServiceEvent) -> None: + # automatic reconnect succeeded: the waiting messages go out + logger.info("Service reconnected to %s", event.get_broker_uri()) + + def on_service_interrupted(self, event: ServiceEvent) -> None: + # the connection went down and cannot be restored (reconnect attempts exhausted) + logger.error("Service interrupted and cannot be restored, quitting: %s (%s)", + event.get_message(), event.get_cause()) + # Application cleanup signal: the service will not recover. Trigger application-side + # cleanup from here. This sample sets shutdown, so the publish loop exits and + # teardown_solace() runs in main's finally. + self.state.exit_code = 1 + self.state.shutdown.set() + + +class PublishReceiptHandler(MessagePublishReceiptListener): + """Very simple class, used for handling publish receipts (ACKs/NACKs) from the broker. + Runs on an API thread: + return promptly and do not block in it (C API Best Practices).""" + + def on_publish_receipt(self, publish_receipt: PublishReceipt) -> None: + msg_id = publish_receipt.user_context # the message id passed to publish() + if publish_receipt.exception is None: # ACK + logger.debug("ACK for Message ID %s", msg_id) # good enough, the broker has it now + return + # NACK: the exception says why (for example MessageDestinationDoesNotExistError when no + # queue subscribes to the topic and the client-profile rejects on no subscription match) + logger.warning("NACK for Message ID %s - %s", msg_id, publish_receipt.exception) + # probably want to do something here. some error handling possibilities: + # - look the message up by this ID in your application's outbound store and send it again + # - send it somewhere else (error handling queue?) + # - log and continue + # - pause and retry (backoff) - maybe set a flag to slow down the publisher + + +if __name__ == "__main__": + logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s") + sys.exit(main(sys.argv[1:])) diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-subscriber-async-sample.py b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-subscriber-async-sample.py new file mode 100644 index 0000000..dd0936e --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-subscriber-async-sample.py @@ -0,0 +1,372 @@ +""" +Derived from patterns/guaranteed_receiver.py + SolaceSamples/solace-samples-python + https://github.com/SolaceSamples/solace-samples-python + +Elevated to documented best practices per the Python API developer guide: + Messaging Service: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md + Consuming Persistent Messages: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md + Creating Queues with the Solace Python API: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md + the Python API reference (units, defaults, listener contracts): + https://docs.solace.com/API-Developer-Online-Ref-Documentation/python/index.html + and the C API Best Practices (the Python API wraps the C API, so its callback, + reconnect, acknowledgement, and time-to-live practices apply): + https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md + +Reference sample. Wired for a basic publish-subscribe journey: basic-auth connect +with a reconnection retry strategy, in-process provisioning of a durable exclusive +queue plus a topic subscription so a fresh broker works out of the box, a +CLIENT-ack receiver that ACKs only after processing, receiver active/passive state +and termination handling, reconnection and service interruption listeners +registered BEFORE connect(), and a graceful SIGINT/SIGTERM shutdown. Structured as +module-level functions, setup_solace(...), connect_solace(...), await_messages(...), +and teardown_solace(...), that share one SubscriberState; setup_solace() creates the +service and the receiver before any connection exists, and main() runs teardown from +its finally on every exit path after that. + +Only practices documented in canonical Solace sources are encoded here. + +Copied into a generated project as guaranteed_subscriber_async.py, next to the shared +helper python-solace-connection-config.py copied as solace_connection_config.py. + +Any generated adaptation of this sample MUST begin with the exact line: + AI-assisted code. Review before production use. +(This reference sample itself carries no such header by design.) +""" + +import logging +import signal +import sys +import threading +from dataclasses import dataclass + +from solace.messaging.config.missing_resources_creation_configuration import MissingResourcesCreationStrategy +from solace.messaging.config.receiver_activation_passivation_configuration import ( + ReceiverState, + ReceiverStateChangeListener, +) +from solace.messaging.config.retry_strategy import RetryStrategy +from solace.messaging.errors.pubsubplus_client_error import IncompleteMessageDeliveryError, PubSubPlusClientError +from solace.messaging.messaging_service import ( + MessagingService, + ReconnectionAttemptListener, + ReconnectionListener, + ServiceEvent, + ServiceInterruptionListener, +) +from solace.messaging.receiver.inbound_message import InboundMessage +from solace.messaging.receiver.message_receiver import MessageHandler +from solace.messaging.receiver.persistent_message_receiver import PersistentMessageReceiver +from solace.messaging.resources.queue import Queue +from solace.messaging.resources.topic_subscription import TopicSubscription +from solace.messaging.utils.life_cycle_control import TerminationEvent, TerminationNotificationListener + +from solace_connection_config import load_service_properties + +APP_NAME = "GuaranteedSubscriber" +QUEUE_NAME = "q_python_sub" +# topic to map onto the queue; matches the publisher sample's topic root +TOPIC_NAME = "solace/samples/python/pers/pub/>" +API = "Python" +# reconnect budget: 20 attempts 3 s apart, a minute of reconnect attempts, a reasonable +# default when the design says nothing about high availability. The interval is in +# MILLISECONDS per the RetryStrategy API reference. +RECONNECT_RETRIES = 20 +RECONNECT_RETRY_INTERVAL_MS = 3000 +# For HA failover resilience, use the C API Best Practices values +# (https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md): +# HA failover: with_connection_retry_strategy(RetryStrategy.parametrized_retry(1, 3000)), +# with_reconnection_retry_strategy(RetryStrategy.parametrized_retry(20, 3000)), +# properties[transport_layer_properties.CONNECTION_RETRIES_PER_HOST] = 5 +# terminate(grace_period): how long to wait for the handler to drain the messages the API +# has already received before the receiver stops; terminate() raises +# IncompleteMessageDeliveryError when messages remain after that. A bounded grace period +# keeps shutdown time predictable, for example within the stop timeout of a process +# supervisor, so the disconnect still runs. The API default is 600000 ms (10 minutes). +# Adjust the value to suit the application: longer lets the handler finish more of the +# received messages, shorter exits sooner. +TERMINATE_GRACE_PERIOD_MS = 10_000 + +# API events go through standard logging; the trace() narration below is a separate channel +logger = logging.getLogger(APP_NAME) + + +@dataclass +class SubscriberState: + """What the main loop, the signal handler, and the API listener threads share: the + service, the receiver, the shutdown event, the flags the listeners flip, and the + counters. The service and the receiver are required, so every SubscriberState holds + both (built, though not necessarily connected or started). Use this type of app for + receiving Guaranteed messages (e.g. via a queue endpoint).""" + messaging_service: MessagingService + receiver: PersistentMessageReceiver + shutdown: threading.Event + connect_attempted: bool = False + exit_code: int = 0 # set to 1 on a failure exit (service interruption, receiver termination, or a failed terminate()) + msg_recv_counter: int = 0 # num messages received + has_detected_redelivery: bool = False # detected any messages being redelivered? + + +def main(args: list[str]) -> int: + trace(f"{API} {APP_NAME} initializing...") + shutdown = threading.Event() + + def on_shutdown_signal(signum: int, _frame) -> None: + trace(f"Shutdown signal received ({signal.Signals(signum).name}), stopping consumer...") + shutdown.set() + + # graceful shutdown: SIGINT (Ctrl-C) or SIGTERM sets the shutdown event, the main + # loop exits, and teardown_solace() in the finally below stops the receiver and + # disconnects. Python runs signal handlers on the main thread, so the handler only + # flags; the cleanup runs on the main thread's normal exit path. Registered before + # connect, so a SIGTERM during the blocking connect() still reaches teardown. + signal.signal(signal.SIGINT, on_shutdown_signal) + signal.signal(signal.SIGTERM, on_shutdown_signal) + # basic username/password connection details, built by the shared connection-config + # helper: read from a config.json in the working directory if present, else from the + # command line ( [password]). Turning the + # application's input into Solace properties is the application's job, so it stays out + # of setup_solace(), which works from the properties alone. + properties = load_service_properties(args, APP_NAME) + # setup_solace() raises before any connection exists, so there is nothing to tear down + # if it fails + state = setup_solace(properties, shutdown) + try: + if not connect_solace(state): + return 1 # the bind failed; the finally below still runs teardown + await_messages(state) + finally: + teardown_solace(state) + trace("Main thread quitting.") + return state.exit_code # non-zero after a failure, so scripts and supervisors see it + + +def setup_solace(properties: dict, shutdown: threading.Event) -> SubscriberState: + # build() creates the native session and resolves the host, so an unresolvable host + # fails here, before connect() + messaging_service = ( + MessagingService.builder() + # the builder merges each call in order and a later call wins, so the reconnection + # strategy comes first: it is the default, and reconnection keys in the properties + # (for example from config.json) override it + .with_reconnection_retry_strategy( + RetryStrategy.parametrized_retry(RECONNECT_RETRIES, RECONNECT_RETRY_INTERVAL_MS)) + .from_properties(properties) + .build() + ) + # configure the queue API object locally: durable, exclusive (single-consumer sample) + queue = Queue.durable_exclusive_queue(QUEUE_NAME) + + # ELEVATION: provision the durable queue in-process and map the topic onto it at + # start, so a fresh broker works out of the box. CREATE_ON_START creates the queue + # passed to build() when start() runs, as long as the client has permission; the + # default, DO_NOT_CREATE, disables provisioning so an administrator provisions the + # queue instead. with_subscriptions() on a durable queue adds the topic subscription + # to the queue at start (pub/sub onto a queue). A re-run reuses the existing queue + # and subscription. + receiver_builder = ( + messaging_service.create_persistent_message_receiver_builder() + .with_missing_resources_creation_strategy(MissingResourcesCreationStrategy.CREATE_ON_START) + .with_subscriptions([TopicSubscription.of(TOPIC_NAME)]) + # best practice: client acknowledgement (also the API default), so a message leaves + # the queue only after the handler has processed it and called ack() + .with_message_client_acknowledgement() + # request active/passive state changes: on an exclusive queue only one bound receiver + # is ACTIVE; any other is PASSIVE (bound, not receiving) until the active one goes away + .with_activation_passivation_support(ReceiverStateHandler()) + ) + # building the receiver needs only the built service; start() is what requires the + # connection + receiver = receiver_builder.build(queue) + state = SubscriberState(messaging_service=messaging_service, receiver=receiver, shutdown=shutdown) + # a broker-initiated termination (for example the queue was deleted or shut down) + # surfaces here; the handler ends the main loop (see ReceiverTerminationHandler below) + receiver.set_termination_notification_listener(ReceiverTerminationHandler(state)) + # best practice: register the service event listeners BEFORE connect(), so no + # reconnection or interruption event raised during or right after the connect is lost, + # and handle each event appropriately rather than only logging it + service_events = ServiceEventHandler(state) + messaging_service.add_reconnection_attempt_listener(service_events) + messaging_service.add_reconnection_listener(service_events) + messaging_service.add_service_interruption_listener(service_events) + return state + + +def connect_solace(state: SubscriberState) -> bool: + state.connect_attempted = True + state.messaging_service.connect() # blocking connect + # set the message handler BEFORE start(), so the handler is in place when delivery + # begins and received messages do not wait in the API's buffer for it. receive_async() + # needs a connected service, so it runs here after connect(), not in setup_solace(). + # see bottom of file for QueueMessageHandler, which receives the messages from the queue + state.receiver.receive_async(QueueMessageHandler(state, state.receiver)) + trace(f"Attempting to bind to queue '{QUEUE_NAME}' on the broker.") + try: + # start() provisions the queue plus its subscription (CREATE_ON_START) and binds + # to the queue; the broker then starts sending messages on this receiver + state.receiver.start() + except PubSubPlusClientError as error: + logger.error("Could not bind to queue '%s': %s. If this client may not create endpoints, " + "provision the queue and its topic subscription out-of-band or grant the " + "capability. Exiting.", QUEUE_NAME, error) + return False # teardown_solace() in main's finally disconnects + return True + + +def await_messages(state: SubscriberState) -> None: + # async queue receive working now, so time to wait until done... + trace(f"{APP_NAME} connected, and running. Press Ctrl-C to quit.") + while not state.shutdown.wait(1.0): # wait 1 second; the wait returns True once shutdown is requested + trace(f"{API} {APP_NAME} Received msgs/s: {state.msg_recv_counter:,}") # simple way of calculating message rates + state.msg_recv_counter = 0 + if state.has_detected_redelivery: # try shutting -> enabling the queue on the broker to see this + trace("*** Redelivery detected ***") + state.has_detected_redelivery = False # only show the error once per second + + +def teardown_solace(state: SubscriberState) -> None: + # Application cleanup belongs here: once setup_solace() returns, teardown_solace() runs + # in main's finally on EVERY exit path (normal quit, SIGINT/SIGTERM, service + # interruption, receiver termination, or an exception), so Solace teardown and + # application-side cleanup are never skipped. + state.shutdown.set() + if state.receiver.is_running(): + # gracefully stop delivery before exit: terminate(grace_period) stops the receiver + # and gives the handler up to the grace period to drain (and ACK) the messages the + # API has already received. The receiver turns TERMINATED once its buffer is empty, + # while the handler may still hold the last message, so that message's ack() can + # fail (the API logs a warning). A message received but not yet ACKed stays on the + # queue and is redelivered. terminate() raises IncompleteMessageDeliveryError when + # messages remain after the grace period; catch it so the disconnect below still runs + try: + state.receiver.terminate(TERMINATE_GRACE_PERIOD_MS) + except IncompleteMessageDeliveryError as error: + logger.error("Receiver stopped with messages still undelivered to the handler after %d ms: %s", + TERMINATE_GRACE_PERIOD_MS, error) + state.exit_code = 1 + except PubSubPlusClientError as error: + logger.error("Receiver terminate() failed: %s", error) + state.exit_code = 1 + if state.connect_attempted: + # disconnect() raises IllegalStateError on a service that never attempted to connect, + # hence the flag; on a service that is already down it returns quietly + state.messaging_service.disconnect() # will also release the receiver + + +def trace(message: str) -> None: + """Demo narration sink: every status line in this sample funnels through this one + function. An application replaces this single body to route narration to its + logger or reporting system. The logger calls for API events (service, receiver + state, termination) are a separate channel and stay as they are.""" + print(message, flush=True) + + +############################################################################ + + +class ServiceEventHandler(ReconnectionAttemptListener, ReconnectionListener, ServiceInterruptionListener): + """The three service event listeners in one class. Callbacks run on an API thread: + return promptly and do not block in them, since events and delivery stall while a + callback runs (C API Best Practices).""" + + def __init__(self, state: SubscriberState) -> None: + self.state = state + + def on_reconnecting(self, event: ServiceEvent) -> None: + # connection lost, automatic reconnect attempt in progress + logger.warning("Service reconnecting; message delivery is paused until re-established: %s (%s)", + event.get_message(), event.get_cause()) + + def on_reconnected(self, event: ServiceEvent) -> None: + # automatic reconnect succeeded; the receiver rebinds to the queue and delivery resumes + logger.info("Service reconnected to %s; the receiver rebinds and delivery resumes", event.get_broker_uri()) + + def on_service_interrupted(self, event: ServiceEvent) -> None: + # the connection went down and cannot be restored (reconnect attempts exhausted) + logger.error("Service interrupted and cannot be restored, quitting: %s (%s)", + event.get_message(), event.get_cause()) + # Application cleanup signal: the service will not recover. Trigger application-side + # cleanup from here. This sample sets shutdown, so the main loop exits and + # teardown_solace() runs in main's finally. + self.state.exit_code = 1 + self.state.shutdown.set() + + +class ReceiverStateHandler(ReceiverStateChangeListener): + """Reports whether this receiver is the one the broker delivers to. Runs on an API thread: + return promptly and do not block in it (C API Best Practices).""" + + def on_change(self, old_state: ReceiverState, new_state: ReceiverState, change_time_stamp: float) -> None: + if new_state == ReceiverState.ACTIVE: # bound and actively receiving from the queue + logger.info("Receiver active: consuming from queue %s", QUEUE_NAME) + else: # PASSIVE: bound but not receiving (another consumer is active on this exclusive queue) + logger.warning("Receiver passive: bound but not receiving (another consumer is active on " + "exclusive queue %s)", QUEUE_NAME) + + +class ReceiverTerminationHandler(TerminationNotificationListener): + """Handles a broker-initiated termination of the receiver. Runs on an API thread: + return promptly and do not block in it (C API Best Practices).""" + + def __init__(self, state: SubscriberState) -> None: + self.state = state + + def on_termination(self, event: TerminationEvent) -> None: + # the receiver was terminated by the broker: delivery from the queue has stopped + logger.error("Receiver terminated, delivery from queue %s has stopped: %s (%s)", + QUEUE_NAME, event.message, event.cause) + # Application cleanup signal: decide here whether to recreate the receiver or shut + # down. This sample shuts down: the main loop exits and teardown_solace() runs in + # main's finally. + self.state.exit_code = 1 + self.state.shutdown.set() + + +class QueueMessageHandler(MessageHandler): + """Very simple class, used for receiving messages from the queue. on_message runs on an + API-owned receiver thread, not the main thread: return promptly and do not block in it + (hand heavy work to the application's own thread or queue), since delivery from the queue + stalls while it runs (C API Best Practices).""" + + def __init__(self, state: SubscriberState, receiver: PersistentMessageReceiver) -> None: + self.state = state + self.receiver = receiver + + def on_message(self, message: InboundMessage) -> None: + self.state.msg_recv_counter += 1 + try: + # the publisher sample sends a bytearray payload (the binary attachment), read back with + # get_payload_as_bytes(); a str payload arrives via get_payload_as_string() instead. + # errors="replace" keeps a non-UTF-8 payload from raising, but the payload read or the + # trace can still raise, hence the try/finally around the processing + payload = message.get_payload_as_bytes() + text = payload.decode("utf-8", errors="replace") if payload is not None else message.get_payload_as_string() + trace(f"{API} {APP_NAME} received: {text}") + if message.is_redelivered(): # useful check + # this is the broker telling the consumer that this message has been sent and not ACKed before. + # this can happen if an exception is thrown, or the broker restarts, or the network disconnects + # perhaps an error in processing? Should do extra checks to avoid duplicate processing + self.state.has_detected_redelivery = True + except Exception: + # best practice: handle an unexpected message format without crashing, log it, and + # still ACK below (C API Best Practices). Without this, the API swallows the exception + # with one warning and the message stays unacknowledged on the flow. + logger.exception("Processing failed for message id %s; acknowledging anyway", + message.get_application_message_id()) + finally: + # Messages are removed from the broker queue when the ACK is received. + # ACK only after processing is complete: DO NOT ACK until all processing/storing + # of this message is done. ACK even an unexpectedly formatted message (C API Best + # Practices): a message left unacknowledged stays on the queue and is redelivered. To + # NACK explicitly instead, build the receiver with + # with_required_message_outcome_support(Outcome.FAILED, Outcome.REJECTED) and call + # receiver.settle(message, outcome). + self.receiver.ack(message) # ACKs are asynchronous + + +if __name__ == "__main__": + logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s") + sys.exit(main(sys.argv[1:])) diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-solace-connection-config.py b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-solace-connection-config.py new file mode 100644 index 0000000..a0e4343 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-solace-connection-config.py @@ -0,0 +1,124 @@ +""" +Shared connection-config helper for the Python reference samples. + +Loads connection properties into the dict that +MessagingService.builder().from_properties(...) takes. Source precedence: + 1. config.json in the working directory, when present. Its keys are Solace + Python API property names (the values of the solace_properties constants): + "solace.messaging.transport.host" + "solace.messaging.service.vpn-name" + "solace.messaging.authentication.basic.username" + "solace.messaging.authentication.basic.password" + Every key passes through to from_properties() unchanged, so other service + properties need no parser change. A later builder call overrides the same + key, and each sample sets its reconnection strategy before from_properties(), + so reconnection keys in the file override the sample's default. The native library rejects an invalid value at build() or + connect(); the API silently ignores an unknown key. config.json holds broker + credentials, so it MUST be gitignored. + 2. otherwise the command-line arguments: + [password] + +The password is optional. The API requires the basic-auth password key at build(), +so an absent or blank password is sent as an empty string. config.json must be a +single flat JSON object; the standard-library json module reads it. + +Grounding: the "load a JSON file, then from_properties()" idiom and the required +host and vpn-name keys: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md + +Copied into a generated project as solace_connection_config.py (a module name +needs underscores); every sample imports it with + from solace_connection_config import load_service_properties + +Any generated adaptation of this sample MUST begin with the exact line: + AI-assisted code. Review before production use. +(This reference sample itself carries no such header by design.) +""" + +import json +import os +import sys + +from solace.messaging.config.solace_properties import ( + authentication_properties, + service_properties, + transport_layer_properties, +) + +CONFIG_FILE = "config.json" + +# the three connection values every sample needs; the password is optional +REQUIRED_KEYS = ( + transport_layer_properties.HOST, + service_properties.VPN_NAME, + authentication_properties.SCHEME_BASIC_USER_NAME, +) + + +def load_service_properties(args: list[str], app_name: str) -> dict[str, object]: + """ + Load connection details from config.json when present in the working directory, + otherwise from the command-line arguments, log which source was used, and return the + from_properties() dict with every loaded property (generic pass-through). Prints a + usage line and exits when neither source supplies host, message-vpn, and + client-username; rejects a blank host, message-vpn, or client-username from either + source. The basic-auth password key is always present: the API requires it at build(), + so an absent or blank password is passed as an empty string. + + Args: + args: the app's command-line arguments without the script name (the fallback source) + app_name: the application name, used only in the usage message + """ + properties = _load_config_file() + if properties is not None: + source = CONFIG_FILE + elif len(args) >= 3: + source = "command-line arguments" + properties = { + transport_layer_properties.HOST: args[0], + service_properties.VPN_NAME: args[1], + authentication_properties.SCHEME_BASIC_USER_NAME: args[2], + } + if len(args) > 3: + properties[authentication_properties.SCHEME_BASIC_PASSWORD] = args[3] + else: + trace(f"No {CONFIG_FILE} in the working directory; provide connection details on the command line.") + trace(f"Usage: {app_name} [password]\n") + sys.exit(1) + for key in REQUIRED_KEYS: + _require_non_blank(properties.get(key), key) + # password is optional: absent or blank means an empty password is sent + if not properties.get(authentication_properties.SCHEME_BASIC_PASSWORD): + properties[authentication_properties.SCHEME_BASIC_PASSWORD] = "" + host = properties[transport_layer_properties.HOST] + trace(f"{app_name}: using connection details from {source} (host={host}).") + return properties + + +def _load_config_file() -> dict[str, object] | None: + if not os.path.exists(CONFIG_FILE): + return None # no config file: fall back to the command-line arguments + # present but unreadable (for example a permission-locked credentials file): open() + # raises PermissionError, so this fails loud rather than silently falling back to + # different command-line connection args. A malformed file raises JSONDecodeError. + with open(CONFIG_FILE, encoding="utf-8") as config_file: + parsed = json.load(config_file) + if not isinstance(parsed, dict): + raise ValueError(f"{CONFIG_FILE} must hold a single JSON object") + for required in REQUIRED_KEYS: + if required not in parsed: + raise ValueError(f'{CONFIG_FILE} is missing required key "{required}"') + return parsed + + +def _require_non_blank(value: object, key: str) -> None: + """Reject a missing, non-string, or blank required connection value, whatever its source.""" + if not isinstance(value, str) or not value.strip(): + raise ValueError(f'connection value for "{key}" must be a non-blank string') + + +def trace(message: str) -> None: + """Demo narration sink: every status line in this helper funnels through this one + function. An application replaces this single body to route narration to its + logger or reporting system.""" + print(message, flush=True) diff --git a/tools/check-links.sh b/tools/check-links.sh index 9bccbc5..e0dd888 100755 --- a/tools/check-links.sh +++ b/tools/check-links.sh @@ -6,8 +6,8 @@ # Complements the CI validate job (gh skill publish --dry-run), which checks the # Agent Skills spec but not links. # This sweep additionally: -# * covers the WHOLE skills/ tree — SKILL.md, references/*.md, the .java sample -# headers, and scripts/*.sh (not just *.md) — while excluding the repo-root +# * covers the WHOLE skills/ tree — SKILL.md, references/*.md, the .java and .py +# sample headers, and scripts/*.sh (not just *.md) — while excluding the repo-root # README (D-11: the sweep globs the skills/ tree only, never the repo root), # * detects Solace "soft 404s" — pages that return HTTP 200/30x but actually land # on a not-found page (docs.solace.com redirects bad paths to Not-Found.htm; @@ -74,7 +74,7 @@ should_skip() { return 1 } -# Collect unique http(s) links from the whole skills/ tree (*.md, *.java, *.sh). +# Collect unique http(s) links from the whole skills/ tree (*.md, *.java, *.py, *.sh). # README at the repo root is excluded because ROOT is the skills/ tree, never the # repo root (D-11). Doc links are now public docs.solace.com .md URLs, checked inline. # @@ -89,7 +89,7 @@ while IFS= read -r line; do links+=("$line") done < <( grep -rhoE 'https?://[^ )"`'"'"'>]+' "$ROOT" \ - --include='*.md' --include='*.java' --include='*.sh' \ + --include='*.md' --include='*.java' --include='*.py' --include='*.sh' \ --exclude-dir='docs' \ | sed -E 's/[.,;:*]+$//' \ | sort -u From 33363212d9cc797ae84cdb4ce56b6b7278d203ec Mon Sep 17 00:00:00 2001 From: Adi El-Sammak <71227923+aelsammak@users.noreply.github.com> Date: Thu, 1 Oct 2026 08:21:17 -0400 Subject: [PATCH 02/12] EBP-3104: Add best-practice Python request-reply reference samples (#9) - Add python-direct-requestor-blocking-sample.py (publish_await_response with a positive reply timeout and one retry on PubSubTimeoutError), python-direct-requestor-sample.py (non-blocking publish() returning a Future per request, then collected), and python-direct-replier-sample.py (receive_async with a RequestMessageHandler, a None-replier guard, and reply()) - Direct only: the Python API has no guaranteed request-reply, so there is no counterpart to the JCSMP guaranteed requestor and replier - Follow the EBP-3103 conventions: shared connection-config helper, listeners registered before connect(), 3000 ms reconnect interval, SIGINT/SIGTERM shutdown, bounded terminate(), exit code 1 on failure and 0 on a shutdown signal - Document that the request-reply builders have no back-pressure setting and that their from_properties() applies no property - Rename the four EBP-3103 samples to drop the -async suffix: a -blocking suffix now marks only a variant that waits for the broker's answer, matching the SolaceSamples naming - Correct a stale guaranteed publisher comment that said the direct publisher pauses during a reconnect - Verified against solace-pubsubplus 1.11.0: pyflakes clean, a no-broker smoke test, and live round trips from both requestors to the replier --- ...e.py => python-direct-publisher-sample.py} | 2 +- .../python/python-direct-replier-sample.py | 316 +++++++++++++++++ ...python-direct-requestor-blocking-sample.py | 310 +++++++++++++++++ .../python/python-direct-requestor-sample.py | 324 ++++++++++++++++++ ....py => python-direct-subscriber-sample.py} | 2 +- ... => python-guaranteed-publisher-sample.py} | 10 +- ...=> python-guaranteed-subscriber-sample.py} | 2 +- 7 files changed, 958 insertions(+), 8 deletions(-) rename plugins/solace-messaging-skills/skills/solace-application-development/references/python/{python-direct-publisher-async-sample.py => python-direct-publisher-sample.py} (99%) create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-replier-sample.py create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-requestor-blocking-sample.py create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-requestor-sample.py rename plugins/solace-messaging-skills/skills/solace-application-development/references/python/{python-direct-subscriber-async-sample.py => python-direct-subscriber-sample.py} (99%) rename plugins/solace-messaging-skills/skills/solace-application-development/references/python/{python-guaranteed-publisher-async-sample.py => python-guaranteed-publisher-sample.py} (97%) rename plugins/solace-messaging-skills/skills/solace-application-development/references/python/{python-guaranteed-subscriber-async-sample.py => python-guaranteed-subscriber-sample.py} (99%) diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-publisher-async-sample.py b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-publisher-sample.py similarity index 99% rename from plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-publisher-async-sample.py rename to plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-publisher-sample.py index 1ea90bd..8a5d262 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-publisher-async-sample.py +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-publisher-sample.py @@ -32,7 +32,7 @@ Only practices documented in canonical Solace sources are encoded here. -Copied into a generated project as direct_publisher_async.py, next to the shared helper +Copied into a generated project as direct_publisher.py, next to the shared helper python-solace-connection-config.py copied as solace_connection_config.py. Any generated adaptation of this sample MUST begin with the exact line: diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-replier-sample.py b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-replier-sample.py new file mode 100644 index 0000000..b5e1c11 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-replier-sample.py @@ -0,0 +1,316 @@ +""" +Derived from patterns/direct_replier.py + SolaceSamples/solace-samples-python + https://github.com/SolaceSamples/solace-samples-python + +Elevated to documented best practices per the Python API developer guide: + Messaging Service: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md + Request-Reply Messaging: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Request-Reply.md + the Python API reference (units, defaults, listener contracts): + https://docs.solace.com/API-Developer-Online-Ref-Documentation/python/index.html + and the C API Best Practices (the Python API wraps the C API, so its callback, + reconnect, acknowledgement, and time-to-live practices apply): + https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md + +Reference sample. Wired for a basic direct request-reply journey on the replier +side: basic-auth connect with a reconnection retry strategy, a +RequestReplyMessageReceiver built with the request topic subscription (no queue, no +provisioning), an async RequestMessageHandler that guards on a replier (a request +without a reply-to cannot be answered) and answers each request with +replier.reply(), which routes to the request's reply-to and copies the correlation, +reconnection and service interruption listeners registered BEFORE connect(), and a +graceful SIGINT/SIGTERM shutdown. Request-reply in the Python API is direct only, so +it is at-most-once on both legs. Answers both requestor samples, +python-direct-requestor-blocking-sample.py and python-direct-requestor-sample.py. +Structured as module-level functions, setup_solace(...), connect_solace(...), +await_requests(...), and teardown_solace(...), that share one ReplierState; +setup_solace() creates the service and the replier before any connection exists, and +main() runs teardown from its finally on every exit path after that. + +Only practices documented in canonical Solace sources are encoded here. + +Copied into a generated project as direct_replier.py, next to the shared helper +python-solace-connection-config.py copied as solace_connection_config.py. + +Any generated adaptation of this sample MUST begin with the exact line: + AI-assisted code. Review before production use. +(This reference sample itself carries no such header by design.) +""" + +import logging +import signal +import sys +import threading +from dataclasses import dataclass + +from solace.messaging.config.retry_strategy import RetryStrategy +from solace.messaging.config.solace_properties import message_properties +from solace.messaging.errors.pubsubplus_client_error import IncompleteMessageDeliveryError, PubSubPlusClientError +from solace.messaging.messaging_service import ( + MessagingService, + ReconnectionAttemptListener, + ReconnectionListener, + ServiceEvent, + ServiceInterruptionListener, +) +from solace.messaging.receiver.inbound_message import InboundMessage +from solace.messaging.receiver.request_reply_message_receiver import ( + Replier, + RequestMessageHandler, + RequestReplyMessageReceiver, +) +from solace.messaging.resources.topic_subscription import TopicSubscription + +from solace_connection_config import load_service_properties + +APP_NAME = "DirectReplier" +# request topic to subscribe to; the requestor samples send to this topic +REQUEST_TOPIC = "solace/samples/python/direct/request" +API = "Python" +# reconnect budget: 20 attempts 3 s apart, a minute of reconnect attempts, a reasonable +# default when the design says nothing about high availability. The interval is in +# MILLISECONDS per the RetryStrategy API reference. +RECONNECT_RETRIES = 20 +RECONNECT_RETRY_INTERVAL_MS = 3000 +# For HA failover resilience, use the C API Best Practices values +# (https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md): +# HA failover: with_connection_retry_strategy(RetryStrategy.parametrized_retry(1, 3000)), +# with_reconnection_retry_strategy(RetryStrategy.parametrized_retry(20, 3000)), +# properties[transport_layer_properties.CONNECTION_RETRIES_PER_HOST] = 5 +# terminate(grace_period): how long to wait for the handler to answer the requests the API +# has already received before the replier stops; terminate() raises +# IncompleteMessageDeliveryError when requests remain after that. A bounded grace period +# keeps shutdown time predictable, for example within the stop timeout of a process +# supervisor, so the disconnect still runs. The API default is 600000 ms (10 minutes). +# Adjust the value to suit the application: longer lets the handler answer more of the +# received requests, shorter exits sooner. +TERMINATE_GRACE_PERIOD_MS = 10_000 + +# API events go through standard logging; the trace() narration below is a separate channel +logger = logging.getLogger(APP_NAME) + + +@dataclass +class ReplierState: + """What the main loop, the signal handler, and the API listener threads share: the + service, the replier, the shutdown event, and the counters. The service and the + replier are required, so every ReplierState holds both (built, though not necessarily + connected or started). Use this type of app to answer direct (at-most-once) requests + on a topic.""" + messaging_service: MessagingService + receiver: RequestReplyMessageReceiver + shutdown: threading.Event + connect_attempted: bool = False + exit_code: int = 0 # set to 1 on a failure exit (service interruption or a failed terminate()) + msg_recv_counter: int = 0 # num requests received + + +def main(args: list[str]) -> int: + trace(f"{API} {APP_NAME} initializing...") + shutdown = threading.Event() + + def on_shutdown_signal(signum: int, _frame) -> None: + trace(f"Shutdown signal received ({signal.Signals(signum).name}), stopping replier...") + shutdown.set() + + # graceful shutdown: SIGINT (Ctrl-C) or SIGTERM sets the shutdown event, the main + # loop exits, and teardown_solace() in the finally below stops the replier and + # disconnects. Python runs signal handlers on the main thread, so the handler only + # flags; the cleanup runs on the main thread's normal exit path. Registered before + # connect, so a SIGTERM during the blocking connect() still reaches teardown. + signal.signal(signal.SIGINT, on_shutdown_signal) + signal.signal(signal.SIGTERM, on_shutdown_signal) + # basic username/password connection details, built by the shared connection-config + # helper: read from a config.json in the working directory if present, else from the + # command line ( [password]). Turning the + # application's input into Solace properties is the application's job, so it stays out + # of setup_solace(), which works from the properties alone. + properties = load_service_properties(args, APP_NAME) + # setup_solace() raises before any connection exists, so there is nothing to tear down + # if it fails + state = setup_solace(properties, shutdown) + try: + connect_solace(state) + await_requests(state) + finally: + teardown_solace(state) + trace("Main thread quitting.") + return state.exit_code # non-zero after a failure, so scripts and supervisors see it + + +def setup_solace(properties: dict, shutdown: threading.Event) -> ReplierState: + # build() creates the native session and resolves the host, so an unresolvable host + # fails here, before connect() + messaging_service = ( + MessagingService.builder() + # the builder merges each call in order and a later call wins, so the reconnection + # strategy comes first: it is the default, and reconnection keys in the properties + # (for example from config.json) override it + .with_reconnection_retry_strategy( + RetryStrategy.parametrized_retry(RECONNECT_RETRIES, RECONNECT_RETRY_INTERVAL_MS)) + .from_properties(properties) + .build() + ) + # request-reply receiver: the request topic subscription goes to build() (no queue, no + # provisioning). start() applies the subscription and blocks until the broker confirms + # it, so the route is live before this replier reports ready. Its builder has no + # back-pressure setting (unlike the direct receiver), and its from_properties() applies + # no property, so there is nothing more to configure. To spread requests across several + # replier instances, pass a share name: build(subscription, ShareName.of("name")), and + # each request then goes to one member of the group. Building the receiver needs only + # the built service; start() is what requires the connection. + receiver = ( + messaging_service.request_reply() + .create_request_reply_message_receiver_builder() + .build(TopicSubscription.of(REQUEST_TOPIC)) + ) + state = ReplierState(messaging_service=messaging_service, receiver=receiver, shutdown=shutdown) + # best practice: register the service event listeners BEFORE connect(), so no + # reconnection or interruption event raised during or right after the connect is lost, + # and handle each event appropriately rather than only logging it + service_events = ServiceEventHandler(state) + messaging_service.add_reconnection_attempt_listener(service_events) + messaging_service.add_reconnection_listener(service_events) + messaging_service.add_service_interruption_listener(service_events) + return state + + +def connect_solace(state: ReplierState) -> None: + state.connect_attempted = True + state.messaging_service.connect() # blocking connect + # set the request handler BEFORE start(), so the handler is in place when delivery + # begins and received requests do not wait in the API's buffer for it. receive_async() + # needs a connected service, so it runs here after connect(), not in setup_solace(). + # see bottom of file for RequestHandler, which answers the requests from the topic + state.receiver.receive_async(RequestHandler(state)) + trace(f"Adding direct request-topic subscription '{REQUEST_TOPIC}'.") + state.receiver.start() + trace(f"{APP_NAME} subscribed and ready to reply. Press Ctrl-C to quit.") + + +def await_requests(state: ReplierState) -> None: + # async request receive working now, so time to wait until done... + while not state.shutdown.wait(1.0): # wait 1 second; the wait returns True once shutdown is requested + trace(f"{API} {APP_NAME} Requests/s: {state.msg_recv_counter:,}") # simple way of calculating request rates + state.msg_recv_counter = 0 + + +def teardown_solace(state: ReplierState) -> None: + # Application cleanup belongs here: once setup_solace() returns, teardown_solace() runs + # in main's finally on EVERY exit path (normal quit, SIGINT/SIGTERM, service + # interruption, or an exception), so Solace teardown and application-side cleanup are + # never skipped. + state.shutdown.set() + if state.receiver.is_running(): + # request-reply is at-most-once: there are no acknowledgements to drain before exit, + # so terminate() stops delivery and gives the handler up to the grace period to answer + # the requests the API has already received. It raises IncompleteMessageDeliveryError + # when requests remain after that; catch it so the disconnect below still runs + try: + state.receiver.terminate(TERMINATE_GRACE_PERIOD_MS) + except IncompleteMessageDeliveryError as error: + logger.error("Replier stopped with requests still unanswered after %d ms: %s", + TERMINATE_GRACE_PERIOD_MS, error) + state.exit_code = 1 + except PubSubPlusClientError as error: + logger.error("Replier terminate() failed: %s", error) + state.exit_code = 1 + if state.connect_attempted: + # disconnect() raises IllegalStateError on a service that never attempted to connect, + # hence the flag; on a service that is already down it returns quietly + state.messaging_service.disconnect() # will also release the replier + + +def trace(message: str) -> None: + """Demo narration sink: every status line in this sample funnels through this one + function. An application replaces this single body to route narration to its + logger or reporting system. The logger calls for API events (service, reply) + are a separate channel and stay as they are.""" + print(message, flush=True) + + +############################################################################ + + +class ServiceEventHandler(ReconnectionAttemptListener, ReconnectionListener, ServiceInterruptionListener): + """The three service event listeners in one class. Callbacks run on an API thread: + return promptly and do not block in them, since events and delivery stall while a + callback runs (C API Best Practices).""" + + def __init__(self, state: ReplierState) -> None: + self.state = state + + def on_reconnecting(self, event: ServiceEvent) -> None: + # connection lost, automatic reconnect attempt in progress: request delivery is + # paused, and the broker does not store direct requests for this replier while it + # is disconnected (at-most-once) + logger.warning("Service reconnecting; request delivery is paused until re-established: %s (%s)", + event.get_message(), event.get_cause()) + + def on_reconnected(self, event: ServiceEvent) -> None: + # automatic reconnect succeeded; the API re-applies the request topic subscription and + # delivery resumes (unlike JCSMP, no reapply-subscriptions property is needed) + logger.info("Service reconnected to %s; the subscription is restored and replies resume", + event.get_broker_uri()) + + def on_service_interrupted(self, event: ServiceEvent) -> None: + # the connection went down and cannot be restored (reconnect attempts exhausted) + logger.error("Service interrupted and cannot be restored, quitting: %s (%s)", + event.get_message(), event.get_cause()) + # Application cleanup signal: the service will not recover. Trigger application-side + # cleanup from here. This sample sets shutdown, so the main loop exits and + # teardown_solace() runs in main's finally. + self.state.exit_code = 1 + self.state.shutdown.set() + + +class RequestHandler(RequestMessageHandler): + """Very simple class, used for answering the direct requests from the topic. on_message + runs on an API-owned receiver thread, not the main thread: return promptly and do not + block in it (hand heavy work to the application's own thread or queue), since delivery + stalls while it runs (C API Best Practices).""" + + def __init__(self, state: ReplierState) -> None: + self.state = state + # one OutboundMessageBuilder for every reply; the per-reply difference goes in + # build()'s additional_message_properties. Replier.reply() takes an OutboundMessage, + # not a bare payload, so the builder is required here + self.message_builder = state.messaging_service.message_builder() + + def on_message(self, message: InboundMessage, replier: Replier) -> None: + self.state.msg_recv_counter += 1 + # guard on the replier: the API passes None when the request has no reply-to + # destination, and such a message cannot be answered. The requestor samples set the + # reply-to automatically; a message arriving here without one is not a request. + if replier is None: + logger.warning("Received a message on the request topic with no reply-to; ignoring.") + return + try: + # the requestor samples send a bytearray payload (the binary attachment), read back + # with get_payload_as_bytes(); a str payload arrives via get_payload_as_string() + # instead. best practice: handle an unexpected message format without raising (C + # API Best Practices); errors="replace" keeps a non-UTF-8 payload from raising + payload = message.get_payload_as_bytes() + text = payload.decode("utf-8", errors="replace") if payload is not None else message.get_payload_as_string() + # echo the request's application message id onto the reply for traceability; the + # API copies the correlation itself, so the requestor matches the reply without it + msg_id = message.get_application_message_id() + reply_properties = {message_properties.APPLICATION_MESSAGE_ID: msg_id} if msg_id is not None else None + reply = self.message_builder.build( + bytearray(f"Reply from {APP_NAME} to: {text}".encode("utf-8")), + additional_message_properties=reply_properties) + # reply() routes to the request's reply-to and copies the correlation id, so the + # requestor's publish_await_response() or publish() Future receives it + replier.reply(reply) + except PubSubPlusClientError as error: + # the reply could not be sent (for example the service is down); the requestor + # times out. Without this catch, the API logs the exception as one warning. + logger.warning("Could not send the reply for message id %s: %s", + message.get_application_message_id(), error) + + +if __name__ == "__main__": + logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s") + sys.exit(main(sys.argv[1:])) diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-requestor-blocking-sample.py b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-requestor-blocking-sample.py new file mode 100644 index 0000000..7ad5d69 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-requestor-blocking-sample.py @@ -0,0 +1,310 @@ +""" +Derived from patterns/direct_requestor_blocking.py + SolaceSamples/solace-samples-python + https://github.com/SolaceSamples/solace-samples-python + +Elevated to documented best practices per the Python API developer guide: + Messaging Service: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md + Request-Reply Messaging: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Request-Reply.md + the Python API reference (units, defaults, listener contracts): + https://docs.solace.com/API-Developer-Online-Ref-Documentation/python/index.html + and the C API Best Practices (the Python API wraps the C API, so its callback, + reconnect, acknowledgement, and time-to-live practices apply): + https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md + +Reference sample. Wired for a basic direct request-reply journey on the requestor +side, BLOCKING variant: basic-auth connect with a reconnection retry strategy, a +RequestReplyMessagePublisher, one blocking publish_await_response() with a POSITIVE +reply timeout, a PubSubTimeoutError catch with a single retry to absorb a cold-start +race, reconnection and service interruption listeners registered BEFORE connect(), +and a graceful SIGINT/SIGTERM shutdown. The blocking request IS the synchronization +point: the requestor sends one request, waits for the correlated reply, and exits +(0 on a reply or a shutdown signal, 1 without a reply). Request-reply in the Python API is direct only, so +it is at-most-once on both legs. The non-blocking variant is +python-direct-requestor-sample.py. Structured as module-level functions, +setup_solace(...), connect_solace(...), send_request(...), and teardown_solace(...), +that share one RequestorState; setup_solace() creates the service and the requestor +before any connection exists, and main() runs teardown from its finally on every +exit path after that. + +Only practices documented in canonical Solace sources are encoded here. + +Copied into a generated project as direct_requestor_blocking.py, next to the shared +helper python-solace-connection-config.py copied as solace_connection_config.py. + +Any generated adaptation of this sample MUST begin with the exact line: + AI-assisted code. Review before production use. +(This reference sample itself carries no such header by design.) +""" + +import logging +import signal +import sys +import threading +import uuid +from dataclasses import dataclass + +from solace.messaging.config.retry_strategy import RetryStrategy +from solace.messaging.config.solace_properties import message_properties +from solace.messaging.errors.pubsubplus_client_error import ( + IncompleteMessageDeliveryError, + PubSubPlusClientError, + PubSubTimeoutError, +) +from solace.messaging.messaging_service import ( + MessagingService, + ReconnectionAttemptListener, + ReconnectionListener, + ServiceEvent, + ServiceInterruptionListener, +) +from solace.messaging.publisher.request_reply_message_publisher import RequestReplyMessagePublisher +from solace.messaging.resources.topic import Topic + +from solace_connection_config import load_service_properties + +APP_NAME = "DirectRequestorBlocking" +# request topic the replier subscribes to; matches the direct replier sample +REQUEST_TOPIC = "solace/samples/python/direct/request" +API = "Python" +# POSITIVE reply timeout in MILLISECONDS: publish_await_response() blocks up to this long +# for the correlated reply, then raises PubSubTimeoutError. The API has no default and +# requires a positive value. +REQUEST_TIMEOUT_MS = 3000 +# one retry after a timeout absorbs a cold-start race: the replier may not yet have its +# subscription in place when the first request goes out +REQUEST_ATTEMPTS = 2 +# reconnect budget: 20 attempts 3 s apart, a minute of reconnect attempts, a reasonable +# default when the design says nothing about high availability. The interval is in +# MILLISECONDS per the RetryStrategy API reference. +RECONNECT_RETRIES = 20 +RECONNECT_RETRY_INTERVAL_MS = 3000 +# For HA failover resilience, use the C API Best Practices values +# (https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md): +# HA failover: with_connection_retry_strategy(RetryStrategy.parametrized_retry(1, 3000)), +# with_reconnection_retry_strategy(RetryStrategy.parametrized_retry(20, 3000)), +# properties[transport_layer_properties.CONNECTION_RETRIES_PER_HOST] = 5 +# terminate(grace_period): how long to wait for a request still waiting on its reply before +# the requestor stops; terminate() raises IncompleteMessageDeliveryError when a request is +# still outstanding after that. A bounded grace period keeps shutdown time predictable, for +# example within the stop timeout of a process supervisor, so the disconnect still runs. +# The API default is 600000 ms (10 minutes). Adjust the value to suit the application: +# longer gives an outstanding request more time to get its reply, shorter exits sooner. +TERMINATE_GRACE_PERIOD_MS = 10_000 + +# API events go through standard logging; the trace() narration below is a separate channel +logger = logging.getLogger(APP_NAME) + + +@dataclass +class RequestorState: + """What the main thread, the signal handler, and the API listener threads share: the + service, the requestor, and the shutdown event. The service and the requestor are + required, so every RequestorState holds both (built, though not necessarily connected + or started).""" + messaging_service: MessagingService + requestor: RequestReplyMessagePublisher + shutdown: threading.Event + connect_attempted: bool = False + exit_code: int = 0 # set to 1 on a failure exit (no reply, service interruption, or a failed terminate()) + + +def main(args: list[str]) -> int: + trace(f"{API} {APP_NAME} initializing...") + shutdown = threading.Event() + + def on_shutdown_signal(signum: int, _frame) -> None: + trace(f"Shutdown signal received ({signal.Signals(signum).name}), stopping requestor...") + shutdown.set() + + # graceful shutdown: SIGINT (Ctrl-C) or SIGTERM sets the shutdown event, no further + # request goes out, and teardown_solace() in the finally below stops the requestor and + # disconnects. Python runs signal handlers on the main thread, so the handler only + # flags; the cleanup runs on the main thread's normal exit path. Registered before + # connect, so a SIGTERM during the blocking connect() still reaches teardown. + signal.signal(signal.SIGINT, on_shutdown_signal) + signal.signal(signal.SIGTERM, on_shutdown_signal) + # basic username/password connection details, built by the shared connection-config + # helper: read from a config.json in the working directory if present, else from the + # command line ( [password]). Turning the + # application's input into Solace properties is the application's job, so it stays out + # of setup_solace(), which works from the properties alone. + properties = load_service_properties(args, APP_NAME) + # setup_solace() raises before any connection exists, so there is nothing to tear down + # if it fails + state = setup_solace(properties, shutdown) + try: + connect_solace(state) + send_request(state) + finally: + teardown_solace(state) + trace("Main thread quitting.") + return state.exit_code # non-zero after a failure, so scripts and supervisors see it + + +def setup_solace(properties: dict, shutdown: threading.Event) -> RequestorState: + # build() creates the native session and resolves the host, so an unresolvable host + # fails here, before connect() + messaging_service = ( + MessagingService.builder() + # the builder merges each call in order and a later call wins, so the reconnection + # strategy comes first: it is the default, and reconnection keys in the properties + # (for example from config.json) override it + .with_reconnection_retry_strategy( + RetryStrategy.parametrized_retry(RECONNECT_RETRIES, RECONNECT_RETRY_INTERVAL_MS)) + .from_properties(properties) + .build() + ) + # request-reply publisher: the API sets a unique reply-to topic on each request and + # correlates the reply, so the requestor needs no reply subscription and no correlation + # id of its own. Its builder has no back-pressure setting (unlike the direct publisher), + # and its from_properties() applies no property, so there is nothing more to configure. + # Building the requestor needs only the built service; start() is what requires the + # connection. + requestor = ( + messaging_service.request_reply() + .create_request_reply_message_publisher_builder() + .build() + ) + state = RequestorState(messaging_service=messaging_service, requestor=requestor, shutdown=shutdown) + # best practice: register the service event listeners BEFORE connect(), so no + # reconnection or interruption event raised during or right after the connect is lost, + # and handle each event appropriately rather than only logging it + service_events = ServiceEventHandler(state) + messaging_service.add_reconnection_attempt_listener(service_events) + messaging_service.add_reconnection_listener(service_events) + messaging_service.add_service_interruption_listener(service_events) + return state + + +def connect_solace(state: RequestorState) -> None: + state.connect_attempted = True + state.messaging_service.connect() # blocking connect + state.requestor.start() # raises IllegalStateError unless the service is connected + + +def send_request(state: RequestorState) -> None: + # the request is an OutboundMessage: publish_await_response() takes no bare payload, so + # the message builder is required here. A bytearray payload travels as the binary + # attachment; the replier reads it back with get_payload_as_bytes() + request = state.messaging_service.message_builder().build( + bytearray(f"Request from {APP_NAME}".encode("utf-8"))) + # an application message id lets the replier echo it onto the reply for traceability; + # the API does the request-reply correlation itself and does not need it + msg_id = str(uuid.uuid4()) + request_topic = Topic.of(REQUEST_TOPIC) + trace(f"{APP_NAME} sending a direct request to topic '{REQUEST_TOPIC}' (timeout {REQUEST_TIMEOUT_MS} ms).") + reply = None + for attempt in range(1, REQUEST_ATTEMPTS + 1): + if state.shutdown.is_set(): + break # a shutdown signal or a service interruption: send no further request + try: + # blocks until the correlated reply arrives or REQUEST_TIMEOUT_MS passes. A + # shutdown signal that arrives during the wait takes effect once the call + # returns, within REQUEST_TIMEOUT_MS. + reply = state.requestor.publish_await_response( + request_message=request, + request_destination=request_topic, + reply_timeout=REQUEST_TIMEOUT_MS, + additional_message_properties={message_properties.APPLICATION_MESSAGE_ID: msg_id}, + ) + break + except PubSubTimeoutError: + if attempt < REQUEST_ATTEMPTS: + logger.warning("Request timed out after %d ms; retrying once.", REQUEST_TIMEOUT_MS) + else: + logger.warning("Request timed out after %d ms on the retry; giving up.", REQUEST_TIMEOUT_MS) + except PubSubPlusClientError as error: + # the request could not be sent (for example the service is down); retrying would + # not help + logger.error("Request failed, quitting: %s", error) + break + + if reply is None: + if state.shutdown.is_set(): + # a shutdown signal is a clean stop (exit 0); a service interruption has already + # set exit_code + trace(f"{API} {APP_NAME} stopped before a reply arrived.") + else: + trace(f"{API} {APP_NAME} received no reply.") + state.exit_code = 1 + return + # the replier sample sends a bytearray payload (the binary attachment), read back with + # get_payload_as_bytes(); a str payload arrives via get_payload_as_string() instead. + # errors="replace" keeps a non-UTF-8 payload from raising + payload = reply.get_payload_as_bytes() + text = payload.decode("utf-8", errors="replace") if payload is not None else reply.get_payload_as_string() + trace(f"{API} {APP_NAME} received a correlated reply (message id {reply.get_application_message_id()}): {text}") + + +def teardown_solace(state: RequestorState) -> None: + # Application cleanup belongs here: once setup_solace() returns, teardown_solace() runs + # in main's finally on EVERY exit path (a reply, no reply, SIGINT/SIGTERM, service + # interruption, or an exception), so Solace teardown and application-side cleanup are + # never skipped. + state.shutdown.set() + if state.requestor.is_running(): + # terminate() gives a request still waiting on its reply up to the grace period. It + # raises IncompleteMessageDeliveryError when a request is still outstanding after + # that; catch it so the disconnect below still runs + try: + state.requestor.terminate(TERMINATE_GRACE_PERIOD_MS) + except IncompleteMessageDeliveryError as error: + logger.error("Requestor stopped with a request still outstanding after %d ms: %s", + TERMINATE_GRACE_PERIOD_MS, error) + state.exit_code = 1 + except PubSubPlusClientError as error: + logger.error("Requestor terminate() failed: %s", error) + state.exit_code = 1 + if state.connect_attempted: + # disconnect() raises IllegalStateError on a service that never attempted to connect, + # hence the flag; on a service that is already down it returns quietly + state.messaging_service.disconnect() + + +def trace(message: str) -> None: + """Demo narration sink: every status line in this sample funnels through this one + function. An application replaces this single body to route narration to its + logger or reporting system. The logger calls for API events (service, request) + are a separate channel and stay as they are.""" + print(message, flush=True) + + +############################################################################ + + +class ServiceEventHandler(ReconnectionAttemptListener, ReconnectionListener, ServiceInterruptionListener): + """The three service event listeners in one class. Callbacks run on an API thread: + return promptly and do not block in them, since events and delivery stall while a + callback runs (C API Best Practices).""" + + def __init__(self, state: RequestorState) -> None: + self.state = state + + def on_reconnecting(self, event: ServiceEvent) -> None: + # connection lost, automatic reconnect attempt in progress. Request-reply is direct + # (at-most-once): a request or reply in flight during the outage can be lost, and the + # request then times out + logger.warning("Service reconnecting; a request in flight can time out: %s (%s)", + event.get_message(), event.get_cause()) + + def on_reconnected(self, event: ServiceEvent) -> None: + # automatic reconnect succeeded: requests can go out again + logger.info("Service reconnected to %s", event.get_broker_uri()) + + def on_service_interrupted(self, event: ServiceEvent) -> None: + # the connection went down and cannot be restored (reconnect attempts exhausted) + logger.error("Service interrupted and cannot be restored, quitting: %s (%s)", + event.get_message(), event.get_cause()) + # Application cleanup signal: the service will not recover. Trigger application-side + # cleanup from here. This sample sets shutdown, so no further request goes out, the + # request in flight fails or times out, and teardown_solace() runs in main's finally. + self.state.exit_code = 1 + self.state.shutdown.set() + + +if __name__ == "__main__": + logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s") + sys.exit(main(sys.argv[1:])) diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-requestor-sample.py b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-requestor-sample.py new file mode 100644 index 0000000..a9807a7 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-requestor-sample.py @@ -0,0 +1,324 @@ +""" +Derived from patterns/direct_requestor.py + SolaceSamples/solace-samples-python + https://github.com/SolaceSamples/solace-samples-python + +Elevated to documented best practices per the Python API developer guide: + Messaging Service: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md + Request-Reply Messaging: + https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Request-Reply.md + the Python API reference (units, defaults, listener contracts): + https://docs.solace.com/API-Developer-Online-Ref-Documentation/python/index.html + and the C API Best Practices (the Python API wraps the C API, so its callback, + reconnect, acknowledgement, and time-to-live practices apply): + https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md + +Reference sample. Wired for a basic direct request-reply journey on the requestor +side, NON-BLOCKING variant: basic-auth connect with a reconnection retry strategy, a +RequestReplyMessagePublisher, several non-blocking publish() calls with a POSITIVE +reply timeout that each return a concurrent.futures.Future at once, so the requests +are in flight together, then each Future collected for its correlated reply or its +PubSubTimeoutError, reconnection and service interruption listeners registered +BEFORE connect(), and a graceful SIGINT/SIGTERM shutdown. The requestor sends its +requests, collects every reply, and exits (0 when every request sent got a reply, 1 +otherwise). Request-reply in the Python API is direct only, so it is at-most-once on +both legs. The blocking variant, one request at a time, is +python-direct-requestor-blocking-sample.py. Structured as module-level functions, +setup_solace(...), connect_solace(...), send_requests(...), and teardown_solace(...), +that share one RequestorState; setup_solace() creates the service and the requestor +before any connection exists, and main() runs teardown from its finally on every +exit path after that. + +Only practices documented in canonical Solace sources are encoded here. + +Copied into a generated project as direct_requestor.py, next to the shared helper +python-solace-connection-config.py copied as solace_connection_config.py. + +Any generated adaptation of this sample MUST begin with the exact line: + AI-assisted code. Review before production use. +(This reference sample itself carries no such header by design.) +""" + +import concurrent.futures +import logging +import signal +import sys +import threading +import uuid +from dataclasses import dataclass + +from solace.messaging.config.retry_strategy import RetryStrategy +from solace.messaging.config.solace_properties import message_properties +from solace.messaging.errors.pubsubplus_client_error import ( + IncompleteMessageDeliveryError, + PubSubPlusClientError, + PubSubTimeoutError, +) +from solace.messaging.messaging_service import ( + MessagingService, + ReconnectionAttemptListener, + ReconnectionListener, + ServiceEvent, + ServiceInterruptionListener, +) +from solace.messaging.publisher.request_reply_message_publisher import RequestReplyMessagePublisher +from solace.messaging.resources.topic import Topic + +from solace_connection_config import load_service_properties + +APP_NAME = "DirectRequestor" +# request topic the replier subscribes to; matches the direct replier sample +REQUEST_TOPIC = "solace/samples/python/direct/request" +API = "Python" +# how many requests go out before the first reply is collected +REQUEST_COUNT = 5 +# POSITIVE reply timeout in MILLISECONDS, per request: each Future completes with the +# correlated reply, or with PubSubTimeoutError once this long passes without one. The API +# has no default and requires a positive value. +REQUEST_TIMEOUT_MS = 3000 +# reconnect budget: 20 attempts 3 s apart, a minute of reconnect attempts, a reasonable +# default when the design says nothing about high availability. The interval is in +# MILLISECONDS per the RetryStrategy API reference. +RECONNECT_RETRIES = 20 +RECONNECT_RETRY_INTERVAL_MS = 3000 +# For HA failover resilience, use the C API Best Practices values +# (https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md): +# HA failover: with_connection_retry_strategy(RetryStrategy.parametrized_retry(1, 3000)), +# with_reconnection_retry_strategy(RetryStrategy.parametrized_retry(20, 3000)), +# properties[transport_layer_properties.CONNECTION_RETRIES_PER_HOST] = 5 +# terminate(grace_period): how long to wait for requests still waiting on their replies +# before the requestor stops; terminate() raises IncompleteMessageDeliveryError when +# requests are still outstanding after that. A bounded grace period keeps shutdown time +# predictable, for example within the stop timeout of a process supervisor, so the +# disconnect still runs. The API default is 600000 ms (10 minutes). Adjust the value to +# suit the application: longer gives outstanding requests more time to get their replies, +# shorter exits sooner. +TERMINATE_GRACE_PERIOD_MS = 10_000 + +# API events go through standard logging; the trace() narration below is a separate channel +logger = logging.getLogger(APP_NAME) + + +@dataclass +class RequestorState: + """What the main thread, the signal handler, and the API listener threads share: the + service, the requestor, and the shutdown event. The service and the requestor are + required, so every RequestorState holds both (built, though not necessarily connected + or started).""" + messaging_service: MessagingService + requestor: RequestReplyMessagePublisher + shutdown: threading.Event + connect_attempted: bool = False + exit_code: int = 0 # set to 1 on a failure exit (a missing reply, service interruption, or a failed terminate()) + + +def main(args: list[str]) -> int: + trace(f"{API} {APP_NAME} initializing...") + shutdown = threading.Event() + + def on_shutdown_signal(signum: int, _frame) -> None: + trace(f"Shutdown signal received ({signal.Signals(signum).name}), stopping requestor...") + shutdown.set() + + # graceful shutdown: SIGINT (Ctrl-C) or SIGTERM sets the shutdown event, no further + # request goes out, and teardown_solace() in the finally below stops the requestor and + # disconnects. Python runs signal handlers on the main thread, so the handler only + # flags; the cleanup runs on the main thread's normal exit path. Registered before + # connect, so a SIGTERM during the blocking connect() still reaches teardown. + signal.signal(signal.SIGINT, on_shutdown_signal) + signal.signal(signal.SIGTERM, on_shutdown_signal) + # basic username/password connection details, built by the shared connection-config + # helper: read from a config.json in the working directory if present, else from the + # command line ( [password]). Turning the + # application's input into Solace properties is the application's job, so it stays out + # of setup_solace(), which works from the properties alone. + properties = load_service_properties(args, APP_NAME) + # setup_solace() raises before any connection exists, so there is nothing to tear down + # if it fails + state = setup_solace(properties, shutdown) + try: + connect_solace(state) + send_requests(state) + finally: + teardown_solace(state) + trace("Main thread quitting.") + return state.exit_code # non-zero after a failure, so scripts and supervisors see it + + +def setup_solace(properties: dict, shutdown: threading.Event) -> RequestorState: + # build() creates the native session and resolves the host, so an unresolvable host + # fails here, before connect() + messaging_service = ( + MessagingService.builder() + # the builder merges each call in order and a later call wins, so the reconnection + # strategy comes first: it is the default, and reconnection keys in the properties + # (for example from config.json) override it + .with_reconnection_retry_strategy( + RetryStrategy.parametrized_retry(RECONNECT_RETRIES, RECONNECT_RETRY_INTERVAL_MS)) + .from_properties(properties) + .build() + ) + # request-reply publisher: the API sets a unique reply-to topic on each request and + # correlates each reply to its request, so the requestor needs no reply subscription + # and no correlation id of its own. Its builder has no back-pressure setting (unlike + # the direct publisher), and its from_properties() applies no property, so there is + # nothing more to configure. Building the requestor needs only the built service; + # start() is what requires the connection. + requestor = ( + messaging_service.request_reply() + .create_request_reply_message_publisher_builder() + .build() + ) + state = RequestorState(messaging_service=messaging_service, requestor=requestor, shutdown=shutdown) + # best practice: register the service event listeners BEFORE connect(), so no + # reconnection or interruption event raised during or right after the connect is lost, + # and handle each event appropriately rather than only logging it + service_events = ServiceEventHandler(state) + messaging_service.add_reconnection_attempt_listener(service_events) + messaging_service.add_reconnection_listener(service_events) + messaging_service.add_service_interruption_listener(service_events) + return state + + +def connect_solace(state: RequestorState) -> None: + state.connect_attempted = True + state.messaging_service.connect() # blocking connect + state.requestor.start() # raises IllegalStateError unless the service is connected + + +def send_requests(state: RequestorState) -> None: + # the requests are OutboundMessages: publish() takes no bare payload, so the message + # builder is required here. One builder serves every request; the per-request + # difference goes in publish()'s additional_message_properties + message_builder = state.messaging_service.message_builder() + request_topic = Topic.of(REQUEST_TOPIC) + trace(f"{APP_NAME} sending {REQUEST_COUNT} direct requests to topic '{REQUEST_TOPIC}' " + f"(timeout {REQUEST_TIMEOUT_MS} ms each).") + # send phase: publish() returns a Future at once and the API waits for the reply on + # its own thread, so every request is in flight before the first reply is collected + pending: list[tuple[str, concurrent.futures.Future]] = [] + for count in range(1, REQUEST_COUNT + 1): + if state.shutdown.is_set(): + break # a shutdown signal or a service interruption: send no further request + # an application message id lets the replier echo it onto the reply for + # traceability; the API does the request-reply correlation itself and does not + # need it + msg_id = str(uuid.uuid4()) + # a bytearray payload travels as the binary attachment; the replier reads it back + # with get_payload_as_bytes() + request = message_builder.build(bytearray(f"Request {count} from {APP_NAME}".encode("utf-8"))) + try: + future = state.requestor.publish( + request_message=request, + request_destination=request_topic, + reply_timeout=REQUEST_TIMEOUT_MS, + additional_message_properties={message_properties.APPLICATION_MESSAGE_ID: msg_id}, + ) + except PubSubPlusClientError as error: + # the requestor cannot take the request (for example it is terminating); + # retrying would not help + logger.error("Request %s could not be sent, quitting: %s", msg_id, error) + state.exit_code = 1 + break + pending.append((msg_id, future)) + + # collect phase: each Future completes within REQUEST_TIMEOUT_MS of its send, with the + # reply or with the exception that ended the request. A Future can also take a + # callback, future.add_done_callback(fn), which runs on the API thread that completes + # it; the same do-not-block rule as the listeners applies there. + replies = 0 + for msg_id, future in pending: + try: + reply = future.result() + except PubSubTimeoutError: + logger.warning("Request %s got no reply within %d ms.", msg_id, REQUEST_TIMEOUT_MS) + continue + except PubSubPlusClientError as error: + # the request could not be sent (for example the service is down) + logger.error("Request %s failed: %s", msg_id, error) + continue + replies += 1 + # the replier sample sends a bytearray payload (the binary attachment), read back + # with get_payload_as_bytes(); a str payload arrives via get_payload_as_string() + # instead. errors="replace" keeps a non-UTF-8 payload from raising + payload = reply.get_payload_as_bytes() + text = payload.decode("utf-8", errors="replace") if payload is not None else reply.get_payload_as_string() + trace(f"{API} {APP_NAME} received a correlated reply (message id {reply.get_application_message_id()}): {text}") + + trace(f"{API} {APP_NAME} received {replies} of {len(pending)} replies.") + # a request without its reply is a failure; a shutdown signal that stopped the send + # phase early is a clean stop, so only the requests actually sent count + if replies < len(pending): + state.exit_code = 1 + + +def teardown_solace(state: RequestorState) -> None: + # Application cleanup belongs here: once setup_solace() returns, teardown_solace() runs + # in main's finally on EVERY exit path (all replies, a missing reply, SIGINT/SIGTERM, + # service interruption, or an exception), so Solace teardown and application-side + # cleanup are never skipped. + state.shutdown.set() + if state.requestor.is_running(): + # terminate() gives requests still waiting on their replies up to the grace period. + # It raises IncompleteMessageDeliveryError when requests are still outstanding after + # that; catch it so the disconnect below still runs + try: + state.requestor.terminate(TERMINATE_GRACE_PERIOD_MS) + except IncompleteMessageDeliveryError as error: + logger.error("Requestor stopped with requests still outstanding after %d ms: %s", + TERMINATE_GRACE_PERIOD_MS, error) + state.exit_code = 1 + except PubSubPlusClientError as error: + logger.error("Requestor terminate() failed: %s", error) + state.exit_code = 1 + if state.connect_attempted: + # disconnect() raises IllegalStateError on a service that never attempted to connect, + # hence the flag; on a service that is already down it returns quietly + state.messaging_service.disconnect() + + +def trace(message: str) -> None: + """Demo narration sink: every status line in this sample funnels through this one + function. An application replaces this single body to route narration to its + logger or reporting system. The logger calls for API events (service, request) + are a separate channel and stay as they are.""" + print(message, flush=True) + + +############################################################################ + + +class ServiceEventHandler(ReconnectionAttemptListener, ReconnectionListener, ServiceInterruptionListener): + """The three service event listeners in one class. Callbacks run on an API thread: + return promptly and do not block in them, since events and delivery stall while a + callback runs (C API Best Practices).""" + + def __init__(self, state: RequestorState) -> None: + self.state = state + + def on_reconnecting(self, event: ServiceEvent) -> None: + # connection lost, automatic reconnect attempt in progress. Request-reply is direct + # (at-most-once): a request or reply in flight during the outage can be lost, and + # that request then times out + logger.warning("Service reconnecting; requests in flight can time out: %s (%s)", + event.get_message(), event.get_cause()) + + def on_reconnected(self, event: ServiceEvent) -> None: + # automatic reconnect succeeded: requests can go out again + logger.info("Service reconnected to %s", event.get_broker_uri()) + + def on_service_interrupted(self, event: ServiceEvent) -> None: + # the connection went down and cannot be restored (reconnect attempts exhausted) + logger.error("Service interrupted and cannot be restored, quitting: %s (%s)", + event.get_message(), event.get_cause()) + # Application cleanup signal: the service will not recover. Trigger application-side + # cleanup from here. This sample sets shutdown, so no further request goes out, the + # requests in flight fail or time out, and teardown_solace() runs in main's finally. + self.state.exit_code = 1 + self.state.shutdown.set() + + +if __name__ == "__main__": + logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s") + sys.exit(main(sys.argv[1:])) diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-subscriber-async-sample.py b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-subscriber-sample.py similarity index 99% rename from plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-subscriber-async-sample.py rename to plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-subscriber-sample.py index 4f1319c..3da08ee 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-subscriber-async-sample.py +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-direct-subscriber-sample.py @@ -28,7 +28,7 @@ Only practices documented in canonical Solace sources are encoded here. -Copied into a generated project as direct_subscriber_async.py, next to the shared helper +Copied into a generated project as direct_subscriber.py, next to the shared helper python-solace-connection-config.py copied as solace_connection_config.py. Any generated adaptation of this sample MUST begin with the exact line: diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-publisher-async-sample.py b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-publisher-sample.py similarity index 97% rename from plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-publisher-async-sample.py rename to plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-publisher-sample.py index a94e4b5..429babf 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-publisher-async-sample.py +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-publisher-sample.py @@ -28,7 +28,7 @@ Only practices documented in canonical Solace sources are encoded here. -Copied into a generated project as guaranteed_publisher_async.py, next to the shared +Copied into a generated project as guaranteed_publisher.py, next to the shared helper python-solace-connection-config.py copied as solace_connection_config.py. Any generated adaptation of this sample MUST begin with the exact line: @@ -236,10 +236,10 @@ def run_publish_loop(state: PublisherState) -> None: # publish() blocks while the buffer is full (back pressure). During an automatic # reconnect the messages wait in the buffer for the transport and go out after # it, and publish() blocks only once the buffer fills; the loop needs no pause of - # its own (the direct publisher, whose messages fail instead of waiting, does need - # one). A shutdown signal that - # arrives during that wait takes effect once publish() returns: when the buffer - # drains, or when the service goes down and the API releases the wait. + # its own (in the direct publisher, those messages fail instead of waiting). A + # shutdown signal that arrives during that wait takes effect once publish() + # returns: when the buffer drains, or when the service goes down and the API + # releases the wait. state.publisher.publish(message, topic, user_context=msg_id) state.msg_sent_counter += 1 except PubSubPlusClientError as error: diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-subscriber-async-sample.py b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-subscriber-sample.py similarity index 99% rename from plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-subscriber-async-sample.py rename to plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-subscriber-sample.py index dd0936e..0c67bde 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-subscriber-async-sample.py +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/python-guaranteed-subscriber-sample.py @@ -29,7 +29,7 @@ Only practices documented in canonical Solace sources are encoded here. -Copied into a generated project as guaranteed_subscriber_async.py, next to the shared +Copied into a generated project as guaranteed_subscriber.py, next to the shared helper python-solace-connection-config.py copied as solace_connection_config.py. Any generated adaptation of this sample MUST begin with the exact line: From c21a78beb62e266aee9c21555fb5603b7cffb34a Mon Sep 17 00:00:00 2001 From: Adi El-Sammak <71227923+aelsammak@users.noreply.github.com> Date: Mon, 5 Oct 2026 13:12:07 -0400 Subject: [PATCH 03/12] EBP-3105: Add Python API routing and the guaranteed pub/sub Quickstart (#10) * EBP-3105: Add Python API routing and the guaranteed pub/sub Quickstart - Route Solace Python API requests (solace-pubsubplus, MessagingService, PersistentMessageReceiver) to a new references/python.md entry file; remove Python from the description's exclusions and keep JMS, Go, .NET, and JavaScript excluded - Ask "JCSMP (Java) or Python?" when a Solace build request names no language; workspace files are facts to report, not the answer - Generalize Invariants 3 and 5: the registry per API (Maven metadata for JCSMP, PyPI info.version for Python) and the API's own bundled verify script, with the compile-only fallback for an API that has none yet - Add python/prerequisites.md, design-mode.md (five leaves; no guaranteed request-reply in the Python API), implement-mode.md, and implement-guaranteed-pubsub.md, mirroring the JCSMP files - Quickstart generates a venv project with requirements.txt pinned to the version PyPI reports at generation time, a gitignored config.json, and the four JCSMP VERIFY: markers; it ends in py_compile plus an import smoke and handed-back run commands until the Python verify.sh lands - Document the concurrency rules: receive_async() is not asyncio and calls the handler on a new thread per message, and the API cannot be used with multiprocessing - The direct pub/sub and direct request-reply leaves, and the Solace Suggested and Custom doors, stop with "not yet available" for Python until EBP-3106 - Flip the two Python trigger cases to positives (the guaranteed pub/sub one must_pass) and add Go, JavaScript, asyncio, and Kafka Python near-miss negatives * EBP-3105: Apply review fixes to the Python routing and Quickstart path - Scope the JCSMP routing row to Java, so a request with no language gets the JCSMP or Python question - Let a saved solace-design.md title select the API, and restore the input contract sentence in Python design mode - Exclude the Solace Messaging API for Java, which shares MessagingService and PersistentMessageReceiver with Python; add a negative trigger case - Add the environment failure rule to the manual run, so a broker or credential failure never leads to code changes - Scale the live broker heads-up, run commands, and round trip items to every fan-out subscriber; require an identifier for - Keep the requirements.txt pin when PyPI is unreachable - Add header, checklist, and version drift checks to Step 5 - Align the interim checklist with the JCSMP master: doc link per item, command line credentials under Your responsibility, stronger auth and grounding items - Ground the asyncio ack rule in the Consuming Persistent Messages page, and add traps for the parametrized_retry doc comment and PUBLISH_ACKED without delivery - Name the missing verify script and the Request-Reply page in python.md - Fix the Try Broker description in both prerequisites files to match the page --- .../evals/trigger-evals.json | 9 +- .../solace-application-development/SKILL.md | 20 +- .../references/jcsmp/prerequisites.md | 2 +- .../references/python.md | 74 +++++ .../references/python/design-mode.md | 108 +++++++ .../python/implement-guaranteed-pubsub.md | 53 ++++ .../references/python/implement-mode.md | 269 ++++++++++++++++++ .../references/python/prerequisites.md | 29 ++ 8 files changed, 553 insertions(+), 11 deletions(-) create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python.md create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/design-mode.md create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-guaranteed-pubsub.md create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-mode.md create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/prerequisites.md diff --git a/plugins/solace-messaging-skills/evals/trigger-evals.json b/plugins/solace-messaging-skills/evals/trigger-evals.json index 750d510..bcc5107 100644 --- a/plugins/solace-messaging-skills/evals/trigger-evals.json +++ b/plugins/solace-messaging-skills/evals/trigger-evals.json @@ -12,9 +12,14 @@ {"skill":"solace-application-development","prompt":"For the logistics platform we run in Java on our Solace event broker, a ShipmentTracker service publishes a DeliveryException event whenever a package misses its checkpoint. Three separate services each need to act on it independently: a CustomerNotifier emails the customer, a ReroutePlanner recalculates the route, and an AnalyticsIngest service records it for reporting. Each is owned by a different team, each needs its own full copy of every event, and none of them can miss one even if that service is down for maintenance. How should I structure the messaging so every one of these services reliably receives every event on its own?","should_trigger":true,"must_pass":true}, {"skill":"solace-application-development","prompt":"We've standardized on the Solace JMS API. How do I create a JMS ConnectionFactory and a MessageProducer to send to a queue in Java?","should_trigger":false}, - {"skill":"solace-application-development","prompt":"I'm writing my Solace client in Python with the solace-pubsubplus library. How do I set up the messaging service and a persistent receiver?","should_trigger":false}, - {"skill":"solace-application-development","prompt":"We've got a warehouse-robotics system where two Python services communicate through our Solace event broker using the solace-pubsubplus library. A RobotDispatcher publishes task assignments and a RobotController consumes them and must not lose any while a robot reboots. How should I structure the messaging between these two Python services?","should_trigger":false}, + {"skill":"solace-application-development","prompt":"I'm writing my Solace client in Python with the solace-pubsubplus library. How do I set up the messaging service and a persistent receiver?","should_trigger":true}, + {"skill":"solace-application-development","prompt":"We've got a warehouse-robotics system where two Python services communicate through our Solace event broker using the solace-pubsubplus library. A RobotDispatcher publishes task assignments and a RobotController consumes them and must not lose any while a robot reboots. How should I structure the messaging between these two Python services?","should_trigger":true,"must_pass":true}, {"skill":"solace-application-development","prompt":"Show me how to publish to a Solace topic from C# using the .NET messaging API.","should_trigger":false}, + {"skill":"solace-application-development","prompt":"I'm writing a Go service with the Solace PubSub+ Messaging API for Go. How do I build a persistent message publisher and handle publish receipts?","should_trigger":false}, + {"skill":"solace-application-development","prompt":"I'm on the Solace Messaging API for Java (com.solace:solace-messaging-client). How do I set up a MessagingService and a PersistentMessageReceiver in my Java service to consume from a durable queue?","should_trigger":false}, + {"skill":"solace-application-development","prompt":"In our browser dashboard we use the solclientjs library. How do I subscribe to a Solace topic from JavaScript and update the page when messages arrive?","should_trigger":false}, + {"skill":"solace-application-development","prompt":"I have a Python asyncio service that calls a few REST APIs concurrently. How do I limit it to five requests in flight at once with a semaphore?","should_trigger":false}, + {"skill":"solace-application-development","prompt":"I'm using confluent-kafka in Python. How do I write a consumer that commits offsets only after each message has been processed?","should_trigger":false}, {"skill":"solace-application-development","prompt":"I need a Kafka producer and consumer in Java for our clickstream pipeline. How do I configure the KafkaProducer with the right key and value serializers?","should_trigger":false}, {"skill":"solace-application-development","prompt":"How do I provision a durable queue named q.orders.intake and set its max-spool-usage on our Solace broker through the SEMP API and PubSub+ Manager? I'm configuring the broker, not writing a client.","should_trigger":false}, {"skill":"solace-application-development","prompt":"How do I model our order and shipment events and the relationships between them in Solace Event Portal?","should_trigger":false}, diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/SKILL.md b/plugins/solace-messaging-skills/skills/solace-application-development/SKILL.md index a291599..b2c0cf5 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/SKILL.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/SKILL.md @@ -1,6 +1,6 @@ --- name: solace-application-development -description: "Use when the user mentions JCSMP, the com.solacesystems:sol-jcsmp library, JCSMPSession, publishing to a Solace topic, consuming from a Solace queue, or guaranteed messaging, OR asks broadly to build or design a Solace messaging app in Java (publisher, consumer, pub/sub). This is the single front door for Solace API development: it grounds the work in canonical Solace docs and routes the request to the right API (JCSMP today), helping choose a messaging pattern or generate a runnable Maven app. Do NOT trigger for an explicitly named non JCSMP API, which beats the generic Java signal: Solace JMS, or the Solace Python, Go, .NET, or JavaScript APIs. Also do NOT trigger for Apache Kafka, broker administration and configuration, Solace Event Portal event modeling, Schema Registry, OpenTelemetry tracing, or topic-architecture and topic-hierarchy design questions (use solace-topic-best-practices)." +description: "Use when the user mentions JCSMP, the com.solacesystems:sol-jcsmp library, JCSMPSession, the Solace Python API (solace-pubsubplus), MessagingService, PersistentMessageReceiver, publishing to a Solace topic, consuming from a Solace queue, or guaranteed messaging, OR asks broadly to build or design a Solace messaging app in Java or Python (publisher, consumer, pub/sub). This is the single front door for Solace API development: it grounds the work in canonical Solace docs and routes to the right API, helping choose a messaging pattern or generate a runnable app. Do NOT trigger for an explicitly named other Solace API, which beats the generic Java signal: Solace JMS, the Solace Messaging API for Java, or the Solace Go, .NET, or JavaScript APIs. Also do NOT trigger for Apache Kafka (any language), broker administration and configuration, Solace Event Portal event modeling, Schema Registry, OpenTelemetry tracing, or topic-architecture and topic-hierarchy design questions (use solace-topic-best-practices)." allowed-tools: Read Write Edit Bash Grep Glob WebFetch --- @@ -13,6 +13,7 @@ This skill is the front door for building Solace messaging applications grounded **Do NOT read all reference files upfront. Read ONLY what you need, when you need it.** - User wants a JCSMP application (design help or code generation) then read `references/jcsmp.md`, which selects the right mode within the JCSMP API. +- User wants a Solace Python API application (design help or code generation) then read `references/python.md`, which selects the right mode within the Python API. - Read a mode file only at the moment a step needs it. - Most requests need the API's entry file plus one mode file, not the whole tree. @@ -30,22 +31,25 @@ Determine which Solace API the developer is building against, then read that API | User intent | API | What to do | |---|---|---| -| Anything about JCSMP, the `com.solacesystems:sol-jcsmp` library, a Java publisher or consumer, publishing to a topic, or consuming from a queue with guaranteed messaging | **JCSMP** | Read `references/jcsmp.md` | +| Anything about JCSMP, the `com.solacesystems:sol-jcsmp` library, `JCSMPSession`, or a Java publisher or consumer (publishing to a topic, or consuming from a queue with guaranteed messaging) | **JCSMP** | Read `references/jcsmp.md` | +| Anything about the Solace Python API, the `solace-pubsubplus` library, `MessagingService`, `PersistentMessageReceiver`, or a Python publisher or consumer | **Python** | Read `references/python.md` | -If the API is unclear but the request is clearly about Solace messaging in Java, default to **JCSMP**. Mode detection (Design, Implement) then happens in that API's entry file. +If the API is unclear but the request is clearly about Solace messaging in Java, default to **JCSMP**. If the API is unclear but the request is clearly about Solace messaging in Python, default to **Python**. If the request names no language at all (for example "build me a Solace publisher"), ask one question before anything else: JCSMP (Java) or Python? Files in the workspace (a `pom.xml`, `.py` files, a `requirements.txt`) are facts to report while you ask, never the answer. The one exception is a saved `solace-design.md` that the developer asks you to build from: its title (`Solace JCSMP Design Summary` or `Solace Python Design Summary`) names the API, so read that API's entry file, whose design-contract gate restates the summary and takes the developer's confirm. The Solace Messaging API for Java (`com.solace:solace-messaging-client`) shares class names such as `MessagingService` and `PersistentMessageReceiver` with the Python API, but it is neither JCSMP nor Python: when a Java request names that API or its `com.solace` package, say in one line that this skill does not cover it yet, and stop. Mode detection (Design, Implement) then happens in that API's entry file. ## Invariants Non-negotiable rules that apply across every API. Apply all. 1. **Content sourcing**: Every named entity (API, parameter, configuration value) must be traceable to a canonical Solace source. Do not assert behavior, defaults, or best practices that are not in the grounding documentation. -2. **WebFetch-on-demand doc grounding**: This skill bundles no documentation. When a step needs documentation content to ground a generation or design decision, WebFetch the live canonical page on demand (its `docs.solace.com` `.md` URL, the same link the reference files carry), then quote or summarize the fetched page. Do not answer from memory and do not paraphrase guidance the page does not contain. Do not restate doc content in the skill; link the live `.md` URL. Never substitute another channel for that WebFetch: grounding taken from a docs chatbot, a search tool, or decompiling a jar does not satisfy this invariant, and a page never fetched in the session must never be cited as grounding. A short list of references stays live for the same reason it always has (Javadoc HTML, the JCSMP API Release Notes, Maven Central, the tutorials, and the GitHub samples) because those have no `docs.solace.com` `.md` form. -3. **No hardcoded versions**: Never pin a library version anywhere in skill content. The generated build resolves the latest release of every dependency at generation time from the AUTHORITATIVE Maven repository metadata; each API's entry file names the exact coordinate and carries the exact lookup command. Never read a version from the legacy `search.maven.org/solrsearch` index: it lags the repository metadata and reports stale versions. Security floors (a minimum version below which known CVEs live) are enforced as checks on the resolved result, never as pins. -4. **AI-assisted disclaimer header**: Every source file generated under this skill starts with this exact line: `AI-assisted code. Review before production use.` Directly below it: `See the verification checklist: solace-verification-checklist.md`. This applies to EVERY generated source file, including custom builds, web apps, and embedded shapes that adapt the reference samples outside the canonical generator. The pom and other non-source artifacts are exempt; the bundled reference samples themselves carry no header. -5. **Verification artifacts, always**: Any session that generates Solace code from this skill writes a tailored `solace-verification-checklist.md` beside the code as generation output, copies the bundled `verify.sh` (plus its generated `verify-hooks.sh`) into the project, and ends in a real verify run or the compile-only fallback with the exact handed-back commands. The `VERIFY:` markers are non-optional in generated Solace messaging code, whatever the app shape. An improvised check (for example curl against the app's own HTTP API) may TRIGGER traffic but never renders the verdict; the verdict comes from the markers and the verify exit code. -6. **Live-broker heads-up**: Immediately before starting ANY process that will connect to a live broker — verify.sh, `java -jar`, a `mvn` run, a run script, anything — print one line that names the broker host and states the actual broker-side effects of this run (the queues it provisions, the connections it opens, the messages it publishes), scaled to the real app. Environment discovery is never consent and never an answer: a running local broker, an existing config file, or found credentials are facts to report, not answers to consume; report them, then still ask which broker the developer wants to target. +2. **WebFetch-on-demand doc grounding**: This skill bundles no documentation. When a step needs documentation content to ground a generation or design decision, WebFetch the live canonical page on demand (its `docs.solace.com` `.md` URL, the same link the reference files carry), then quote or summarize the fetched page. Do not answer from memory and do not paraphrase guidance the page does not contain. Do not restate doc content in the skill; link the live `.md` URL. Never substitute another channel for that WebFetch: grounding taken from a docs chatbot, a search tool, or decompiling a jar does not satisfy this invariant, and a page never fetched in the session must never be cited as grounding. A short list of references stays live for the same reason it always has (the JCSMP Javadoc and the Python API reference, the JCSMP and Python API Release Notes, Maven Central and PyPI, the tutorials, and the GitHub samples) because those have no `docs.solace.com` `.md` form. Each API's entry file lists its own live exceptions. +3. **No hardcoded versions**: Never pin a library version anywhere in skill content. The generated build resolves the latest release of every dependency at generation time from the API's AUTHORITATIVE registry: the Maven repository metadata for JCSMP, the PyPI JSON `info.version` for Python. Each API's entry file names the exact coordinate and carries the exact lookup command. Never read a JCSMP version from the legacy `search.maven.org/solrsearch` index: it lags the repository metadata and reports stale versions. Security floors (a minimum version below which known CVEs live) are enforced as checks on the resolved result, never as pins. +4. **AI-assisted disclaimer header**: Every source file generated under this skill starts with this exact line: `AI-assisted code. Review before production use.` Directly below it: `See the verification checklist: solace-verification-checklist.md`. This applies to EVERY generated source file, including custom builds, web apps, and embedded shapes that adapt the reference samples outside the canonical generator. Build files (the pom, `requirements.txt`) and other non-source artifacts are exempt; the bundled reference samples themselves carry no header. +5. **Verification artifacts, always**: Any session that generates Solace code from this skill writes a tailored `solace-verification-checklist.md` beside the code as generation output, copies the API's own bundled verify script (plus its generated `verify-hooks.sh`) into the project, and ends in a real verify run or the compile-only fallback with the exact handed-back commands. The API's entry file names its verify script. An API that does not bundle a verify script yet (Python, until its `verify.sh` lands) ends in the compile-only fallback every time, and its implement-mode file defines that fallback; never claim a verified run without the script. The `VERIFY:` markers are non-optional in generated Solace messaging code, whatever the app shape and whatever the API. An improvised check (for example curl against the app's own HTTP API) may TRIGGER traffic but never renders the verdict; the verdict comes from the markers and the verify exit code. +6. **Live-broker heads-up**: Immediately before starting ANY process that will connect to a live broker — verify.sh, `java -jar`, a `mvn` run, a `python` run, a run script, anything — print one line that names the broker host and states the actual broker-side effects of this run (the queues it provisions, the connections it opens, the messages it publishes), scaled to the real app. Environment discovery is never consent and never an answer: a running local broker, an existing config file, or found credentials are facts to report, not answers to consume; report them, then still ask which broker the developer wants to target. ## Reference Files (read on-demand only) - `references/jcsmp.md`: the JCSMP entry file (mode detection, the JCSMP coordinate, and the live JCSMP doc links); read this for any JCSMP request. - `references/jcsmp/`: all JCSMP API content (mode files, the reference samples, the verify script, and the compile fixture), reached through `references/jcsmp.md`. +- `references/python.md`: the Python entry file (mode detection, the `solace-pubsubplus` coordinate, and the live Python doc links); read this for any Solace Python API request. +- `references/python/`: all Python API content (mode files and the reference samples), reached through `references/python.md`. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/prerequisites.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/prerequisites.md index f817f62..0902c14 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/prerequisites.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/prerequisites.md @@ -11,7 +11,7 @@ Read this when a developer does not yet have a broker to build against. Both Des You need one reachable broker. The primary, recommended choice is [Solace Cloud](https://docs.solace.com/Get-Started/Getting-Started-Try-Broker.md): it is the simplest path to a running broker for a greenfield app and for the run-and-observe round-trip. Default to it unless the developer has a reason not to. -As brief alternatives, the same page also documents a self-hosted Software Broker (run locally via container or VM) and an Appliance (existing hardware the developer already operates) for developers who cannot use Solace Cloud. +As a brief alternative, the same page also covers setting up a free broker locally, for developers who cannot use Solace Cloud. The skill does not provision or configure the broker. It assumes the broker is reachable and that the developer has connection details (host, message VPN, client username, password). A broker discovered running in the environment (for example a local container) is a fact to report, never an answer: still ask which broker the developer wants to target. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python.md new file mode 100644 index 0000000..d22ee78 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python.md @@ -0,0 +1,74 @@ +# Python API + +The Python entry file of the Solace API development skill. The Solace Messaging API for Python is published on PyPI as `solace-pubsubplus`. This file detects the developer's intent and selects the right Python mode; the detailed guidance lives in the lazy-loaded mode files under `python/`. + +The cross-cutting invariants (content sourcing, WebFetch-on-demand doc grounding, and no hardcoded versions) are owned by the umbrella `SKILL.md`. Apply them here; this file does not restate them. The Python-specific points below are the package to depend on and the canonical pages to ground in. + +## Mode Detection + +Determine the user's intent and enter the appropriate mode. Routing is keyed on the design contract, not on the phrasing of the request: Implement mode never starts without a confirmed design (the design-contract gate below), so a "build me..." prompt is NOT an Implement signal by itself. + +| User intent | Mode | What to do | +|---|---|---| +| "Help me choose the Solace messaging pattern for a new Python app" / "Topic or queue?" / "What delivery semantics should I use?" | **Design** | Read `python/design-mode.md` | +| Build or generate a NEW app or messaging component with NO valid design contract yet ("Build me a Solace Python publisher/consumer", "Generate a Python project with solace-pubsubplus", a web app / dashboard / service, or any embedded shape where the Python messaging layer lives inside a larger application) | **Design first** | Read `python/design-mode.md`. A prompt that already states the design resolves on its fully-specified path in ONE confirm; a bare prompt walks the tree. The confirmed summary then feeds Implement. | +| A build request WITH a valid design contract (see the design-contract gate below) | **Implement** | Read `python/implement-mode.md` | +| An edit to an EXISTING app that changes the messaging topology (the topology rule below) | **Design first** | Re-enter `python/design-mode.md` to re-confirm the affected summary fields, then Implement applies the change | +| A mechanical edit to an EXISTING app with no topology change (reconnect handling, payload format, logging, renames, ack tuning) | **Implement** | Read `python/implement-mode.md`; no design pass | +| "My Python app is throwing on connect" / "Why is my receiver not binding the queue?" | **Debug** | Debug mode is not yet available in this release. Redirect the user to the canonical Solace Python API documentation: [Python API Home](https://docs.solace.com/API/Messaging-APIs/Python-API/python-home.md). The redirect is the ENTIRE debug answer: acknowledge the problem in one line, link the page, and stop. Do not generate debugging guidance from memory. That prohibition covers root-cause hypotheses, likely causes, interim checks or "things worth checking in the meantime", workarounds, and configuration or code suggestions. Restating this rule does not license adding such guidance after the redirect. | + +If unclear, default to **Design**. Understand the messaging problem before generating code. + +**The topology rule** decides between the two existing-app rows. An edit changes the messaging topology when it changes the structure the design summary records: a new app or messaging component (a new publisher, receiver, requestor, or replier, including a leaf change such as single-service growing into fan-out), a pattern change, a delivery-mode change, a consumption-endpoint change, or a queue access-type change. That list is the core; also treat an unlisted change as topology when your judgment says it alters the message flow between the apps and the broker. Everything else is a mechanical edit. + +Three gates hold on every path: + +- **The design-contract gate.** Implement mode never starts without a design contract, and never invents design values. Exactly three sources satisfy it: (1) a summary Design mode confirmed in this session: use it as-is, with no re-confirm; (2) an explicit summary the developer supplied in chat (the eight fields or an equivalent statement of them): use it directly; (3) a saved `solace-design.md` in the project: restate it in one line and proceed on confirm. A prose build request that merely mentions pattern details is NOT a contract: route it through Design mode, whose fully-specified path derives the summary, echoes it, and takes one confirm. The one carve-out is the mechanical-edit row above. An experienced developer therefore never has to walk the design tree, but never skips the contract either. +- **Implement mode opens with a mandatory door question.** Ask Quickstart, Solace Suggested, or Custom, before you request or accept ANY broker details (`python/implement-mode.md` Step 0). Ask it even when a `config.json` with credentials already exists, even when a local broker is already running, and even when the requested app is bigger than the canonical generated shape; none of those answers the question. A confirmed design summary does not answer it either: the door question always follows the design contract, never merges into it. +- **Environment discovery never answers a question.** A running broker container, an existing config file, or found credentials are facts to report, not answers to consume. Report what you found, then still ask which broker the developer wants to target. + +## Python package + +The single Solace dependency for this API is the PyPI package `solace-pubsubplus`, imported as `solace.messaging`. Resolve the latest release at generation time (the no-hardcoded-versions invariant); never pin a `solace-pubsubplus` version in skill content. Resolve it from the authoritative PyPI JSON API with exactly this command, and read `info.version`: + +```bash +curl -s https://pypi.org/pypi/solace-pubsubplus/json \ + | python3 -c 'import json, sys; print(json.load(sys.stdin)["info"]["version"])' +``` + +The package's `requires_python` value on PyPI is an install floor, not a support statement. Solace supports only the Python versions that are in active support and also have security update support (see `python/prerequisites.md`). + +## Verify script + +The Python API does not bundle a verify script yet. Every Implement run ends in the compile-only fallback that `python/implement-mode.md` Step 5 defines, and never claims a verified run (SKILL.md Invariant 5). + +## Canonical doc links + +Python-specific grounding. Each page below is a live `docs.solace.com` `.md` URL, WebFetched on demand; the live exceptions are listed separately because they have no `docs.solace.com` `.md` form. + +- [Python API Home](https://docs.solace.com/API/Messaging-APIs/Python-API/python-home.md): entry point for the Python API. +- [Python API Developer Guide](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Dev-Guide.md): the index of the developer guide pages. +- [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md): building and connecting the `MessagingService`. +- [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) and [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md): the guaranteed pub/sub pages. +- [Request-Reply Messaging in the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Request-Reply.md): request-reply in the Python API, which uses direct messages only. +- [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md): queue provisioning from the client. +- [Supported Environments](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-supported-Environments.md): Python versions, platforms, and OpenSSL. +- [Solace C API Best Practices](https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md): the Python API has no best practices page of its own. It wraps the C API, so the reference samples ground their callback, reconnect, acknowledgement, and time-to-live practices here. +- For the broader Solace grounding, see the shared [Solace Core Concepts](https://docs.solace.com/Get-Started/event-mesh-basics.md) page. + +Live exceptions (these have no `docs.solace.com` `.md` form; use the live URL directly): + +- [Python API reference](https://docs.solace.com/API-Developer-Online-Ref-Documentation/python/index.html) (Sphinx HTML, one large page per sub-package) +- [Python API Release Notes](https://products.solace.com/download/PYTHON_API_RN) +- [solace-pubsubplus on PyPI](https://pypi.org/project/solace-pubsubplus/) (its project description carries the `multiprocessing` restriction, which no `docs.solace.com` page states) + +## Mode and reference files (read on-demand only) + +- `python/prerequisites.md`: broker acquisition and the supported environments (route here first if the developer has no reachable broker). +- `python/design-mode.md`: choose and confirm the messaging pattern and topology before any code is generated; every new build without a design contract routes here first. +- `python/implement-mode.md`: generate a runnable Python project from a confirmed design; its Step 4 dispatches onto the per-leaf wiring files below. +- `python/implement-guaranteed-pubsub.md`: the Guaranteed Pub/Sub leaf wiring (read from implement-mode Step 4). +- `python/python-guaranteed-publisher-sample.py`: best-practices publisher sample (basic-auth connect, PERSISTENT publish to a topic, publish receipt handling, graceful shutdown). +- `python/python-guaranteed-subscriber-sample.py`: best-practices receiver sample (provisions a durable queue plus a topic subscription, CLIENT-ack receiver). +- `python/python-solace-connection-config.py`: the shared connection-config helper every generated app uses. +- `python/python-direct-*-sample.py`: the direct pub/sub and direct request-reply samples. Implement mode does not generate those leaves yet (see `python/implement-mode.md` Step 4). diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/design-mode.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/design-mode.md new file mode 100644 index 0000000..32ac8ff --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/design-mode.md @@ -0,0 +1,108 @@ +# Design Mode + +Help the developer choose the right Solace messaging pattern and topology for a Python application before any code is generated, grounded in canonical Solace docs. Work through the steps in order. Read this file's grounding links only as each step needs them; link the canonical doc and state the decision in one line, never paraphrase doc content. + +The output of this mode is a filled-in design summary the developer can carry into Implement mode. + +## Step 0: Confirm broker access and grounding + +Before triaging anything, confirm the developer has access to a reachable broker. Confirm that they HAVE one; Design mode only records the broker TYPE (for example, Solace Cloud) in the summary, so do NOT ask them to paste connection details (host, message VPN, client username, password) into the chat. Those values are handled later in Implement mode, where they go into a gitignored `config.json` or, on Quickstart, the command line. A broker discovered running in the environment (a local container, an existing config file) is a fact to report, never an answer: still confirm which broker the developer wants to target. + +If the developer has no reachable broker, route through `prerequisites.md` first (broker acquisition, Solace Cloud recommended), then return here. + +## Step 1: Read the prompt for stated intent before asking anything + +Design mode is ask-or-recommend, not interrogate-first. Before walking the tree, read what the developer already told you and try to resolve a pattern up front. Match the prompt against this signal table first; it is the deterministic path. + +| Stated intent in the prompt | Recommended leaf (or branch to lock if partial) | +|---|---| +| "request a reply", "needs a correlated response", "ask and wait for an answer" | Request-Reply (Direct) (the delivery question still applies; see the no-guaranteed-request-reply rule below) | +| "fan out to many services", "broadcast to several independent apps", "every team gets its own copy" | Guaranteed Pub/Sub (fan-out) | +| "competing workers", "share the load across consumers", "scale out processing of one stream" | Guaranteed Pub/Sub (single service, non-exclusive) | +| "one active consumer", "strict ordering", "only one worker at a time" | Guaranteed Pub/Sub (single service, exclusive) | +| "fire-and-forget telemetry", "high-rate metrics, occasional loss is fine", "live ticks, no persistence" | Direct Pub/Sub | +| "must not lose messages", "survive consumer downtime", "persistent delivery" | a Guaranteed leaf (let the remaining questions pick which) | + +If a row matches, or if your own judgment maps an unmatched prompt cleanly to one of the five leaves, take the recommend path. Do not default to a full interrogation just because the prompt is not a verbatim table row; fall back to your own judgment to pick the closest leaf, then confirm. + +**Recommend path (intent is clear).** Recommend the leaf in ONE line and confirm, for example "Sounds like Guaranteed Pub/Sub (fan-out), confirm?", then proceed on the developer's confirm. One line plus one confirm, not a rationale paragraph and not a silent zero-confirm jump. + +**Fully-specified path (the prompt answers every applicable tree question).** When the prompt resolves the leaf AND answers every Step 2 question that applies to that leaf, skip the separate leaf confirm. Derive the full summary, present it (Step 4), and let the close-out question carry the single confirm: one confirm covers the leaf and the design. Do not ask a leaf confirm and then a close-out confirm; that is two questions where the prompt earned one. + +**Partial-intent path (intent is partial).** When the prompt answers some branches but not all, lock the branches the prompt already answers and ask ONLY the remaining tree questions, in the Step 2 order, skipping every question the prompt already settled. Reach a leaf, then confirm it the same one-line way. + +**Unclear path (no stated intent).** When the prompt states no usable intent, walk all four tree questions in Step 2 order to reach a leaf. + +**Variant of a leaf (the prompt states a requirement the base shape does not meet).** A leaf is a pattern FAMILY, not a fixed topology. Alongside the intent match, read the prompt for reliability or topology requirements that the chosen leaf's DEFAULT shape does not satisfy (for example "the consumer must dedupe redeliveries"). When you find one, the leaf still resolves to one of the five strings verbatim, but the design is a VARIANT of that leaf: its default sample shape is adapted to meet the requirement. Name the deviation in the same one-line confirm and carry the actual topology into the Step 4 summary. Keep a variant within the same pattern family and to topology or reliability knobs the canonical docs ground; if a requirement leaves the Solace topic and queue publish and subscribe model, or needs behavior the docs cannot ground, say so and treat it as off-catalog rather than inventing it. + +## Step 2: Walk the decision tree to a leaf + +Ask only the questions the prompt left open, in this order. Before you answer each question you do walk, WebFetch its grounding page first (even when the prompt pre-answers the question), then link the canonical doc and state the decision in one line, quoting one line from the fetched page; never paraphrase guidance the page does not contain. A page you did not fetch in this session must not appear in the summary's Grounding docs field. + +1. **Interaction shape: request-reply or publish/subscribe?** Grounded in [Message Exchange Patterns](https://docs.solace.com/Get-Started/message-exchange-patterns.md). Decision: a request that expects a correlated reply is Request-Reply; an event fanned out to one or many independent consumers is publish/subscribe. +2. **Delivery guarantee: direct or guaranteed?** This ONE question applies to both interaction shapes, so ask it next regardless of the answer to question 1. Grounded in [Message Delivery Modes](https://docs.solace.com/Get-Started/message-delivery-modes.md). Decision: direct is fire-and-forget with no broker-side persistence, for high-rate flows that tolerate occasional loss; guaranteed (persistent) is persisted and survives consumer downtime. +3. **Number of consuming services (guaranteed publish/subscribe branch only).** Decision: a single consuming service is one logical consumer of the stream; fan-out is several independent services that each need their own full copy of every event. +4. **Competing consumers (guaranteed publish/subscribe, single service branch only).** Grounded in [Topic Endpoints and Queues](https://docs.solace.com/Get-Started/topic-endpoints-queues.md). Decision: exclusive gives one active consumer at a time (strict ordering, hot standby); non-exclusive lets multiple competing workers share the queue and split the load. + +Questions 3 and 4 apply ONLY on the guaranteed publish/subscribe branch. Request-reply and direct publish/subscribe resolve at the delivery-guarantee answer and never reach them. + +**No guaranteed request-reply in the Python API.** When question 1 resolves to Request-Reply and question 2 resolves to guaranteed, there is no Python leaf for it. WebFetch [Request-Reply Messaging in the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Request-Reply.md) and quote its note that request-reply messaging can only be used with direct messages in the Python API. Then give the developer the choice in one question: Request-Reply (Direct), which keeps the request-reply shape and accepts direct delivery, or a Guaranteed Pub/Sub leaf, which keeps persistence and leaves the reply correlation to the application. Never invent a guaranteed request-reply leaf and never record `Request-Reply (Guaranteed)` in a Python summary. + +## Step 3: Resolve to one of the five leaves + +The tree resolves to exactly five leaves. Spell the chosen leaf in the summary exactly as written here; these strings are the contract Implement mode reads. + +- **Request-Reply (Direct)**. Request-reply over direct messaging. +- **Direct Pub/Sub**. Publish/subscribe over direct messaging, fire-and-forget. +- **Guaranteed Pub/Sub (single service, exclusive)** GENERATED. One durable queue, exclusive access, one active consumer at a time. +- **Guaranteed Pub/Sub (single service, non-exclusive)** GENERATED. One durable queue, non-exclusive access, competing workers share the load. +- **Guaranteed Pub/Sub (fan-out)** SNIPPET. One durable queue plus a topic subscription per independent consuming service. Implement mode reuses the single-service guaranteed generator, so there is no dedicated fan-out sample; the leaf is realized by repeating the single-service queue-and-subscription setup once per consuming service. + +The two direct leaves are valid designs, and Design mode confirms and saves them like any other leaf, but Implement mode does not generate them in this release (see `implement-mode.md` Step 4). When the confirmed leaf is one of them, say so in the Step 4 close-out instead of offering to move into Implement mode. + +Inside the guaranteed publish/subscribe single-service leaf, the exclusive versus non-exclusive choice is realized as a queue-factory SNIPPET, not a separate sample: the same generated app builds an exclusive or a non-exclusive durable queue based on the developer's answer to question 4. + +**A leaf can carry a variant.** The five strings name the pattern family; a design may adapt a leaf's default topology to a stated requirement (the variant path in Step 1). In every case the `Pattern` string stays one of the five verbatim, and the deviation is recorded in the topology fields of the Step 4 summary (for example the `Consumption endpoint`), never by inventing a sixth leaf string. + +**Topic hierarchy and queue name.** Always publish to topics; never address a queue directly. Do NOT ask the developer to choose or design the topic string. Instead, apply the co-installed solace-topic-best-practices skill (which reads the canonical Topic Architecture Best Practices page live) to the developer's use case, derive the recommended topic hierarchy, and recommend it in ONE line; use that recommended topic without making the developer pick it. Do not design the hierarchy here by hand and do not restate that skill's guidance. Likewise, for any leaf that consumes from a durable queue, derive the queue name automatically rather than asking: follow the `q.`-prefixed dotted convention (for example the topic `acme/orders/new` maps to the queue `q.acme.orders`), recommend it in one line, and record it. The developer overrides the recommended topic or queue name ONLY if they explicitly ask to. + +**Traps to avoid:** + +- **Publishing direct-to-queue instead of pub/sub.** Publish to a topic; let the durable queue carry a topic subscription. ([Topic Endpoints and Queues](https://docs.solace.com/Get-Started/topic-endpoints-queues.md).) +- **Text payload instead of binary.** Default to a binary (`bytearray`) payload; the receiver reads it back with `get_payload_as_bytes()`. +- **Direct where guaranteed is required.** Use persistent delivery whenever the design cannot tolerate message loss or must survive consumer downtime. ([Message Delivery Modes](https://docs.solace.com/Get-Started/message-delivery-modes.md).) +- **A topic endpoint instead of a queue-with-topic-subscription.** Promote the durable queue. ([Topic Endpoints and Queues](https://docs.solace.com/Get-Started/topic-endpoints-queues.md).) +- **A guaranteed request-reply leaf.** The Python API has none; follow the rule at the end of Step 2. + +## Step 4: Emit the design summary + +Fill in this ONE unified design summary every run, in chat, with the confirmed values. The same eight fields appear every run regardless of which leaf the tree reached, so Implement mode reads one stable structure. Use explicit `n/a` for any field the chosen pattern does not apply. The title names the API, so a saved `solace-design.md` routes back to the Python entry file (the saved-design exception in SKILL.md API Selection). + +``` +Solace Python Design Summary +- Pattern: +- Delivery: +- Access type: +- Topic: +- Consumption endpoint: +- Auth: Basic username/password +- Broker: Solace Cloud +- Grounding docs: +``` + +The eight fields are fixed: Pattern, Delivery, Access type, Topic, Consumption endpoint, Auth, Broker, Grounding docs. Formatting is at your discretion, but always present all eight, every run, with explicit `n/a` where a field does not apply. The `Pattern` value stays one of the five leaf strings verbatim even for a variant; the deviation lives in the topology fields, never in a new leaf string. + +Field semantics: + +- **Pattern.** Exactly one of the five leaf strings, spelled verbatim: `Request-Reply (Direct)`, `Direct Pub/Sub`, `Guaranteed Pub/Sub (single service, exclusive)`, `Guaranteed Pub/Sub (single service, non-exclusive)`, `Guaranteed Pub/Sub (fan-out)`. This is the contract Implement mode reads, so the spelling is not cosmetic. +- **Delivery.** `Direct` for the two direct leaves; `Guaranteed (persistent)` for the three guaranteed leaves. +- **Access type.** `Exclusive` or `Non-exclusive` for the two guaranteed single-service leaves; `n/a` for every other leaf. Surface it explicitly even though it is also encoded in the Pattern string. +- **Consumption endpoint.** The pattern-specific place the consuming side reads from. Record the ACTUAL topology the design requires; the defaults are `direct topic subscription` for Direct Pub/Sub, `durable queue + topic subscription` for the three guaranteed leaves, and `API-managed reply (RequestReplyMessagePublisher)` for Request-Reply (Direct). For the durable-queue leaves, include the auto-derived queue name (for example `durable queue q.acme.orders + topic subscription`) so Implement mode reads the concrete queue name here. For a variant, record the variant topology here with a short inline reason. +- **Topic.** The recommended topic string or hierarchy, derived by applying the solace-topic-best-practices skill to the use case. +- **Auth.** Basic username/password. +- **Broker.** Default to `Solace Cloud` and record it without asking. Switch to another type (Software Broker or Appliance) ONLY if the developer explicitly insists. +- **Grounding docs.** ONLY the pages this session actually WebFetched. Never list a page you did not fetch. Write `none fetched` when no page was fetched. + +After presenting the summary in chat, close Design mode explicitly so the developer knows exactly what happens next; do NOT just display the summary and stop. On the fully-specified path (Step 1) this close-out is the ONLY confirm of the run; it carries the leaf confirmation too. If the design is a variant of its leaf, state the deviation and its rationale in one line here too. Ask them directly, in one step, both whether they are happy with this design AND whether to save it to `solace-design.md` in their project (for example: "Happy with this design? If so, I can save it to `solace-design.md` and move into Implement mode to generate the runnable Python project."). Write the file only on their OK; never write it unprompted. Once they confirm, state the next step plainly: the work moves into Implement mode, which generates the runnable Python project from this summary. Implement mode treats this summary, whether it lives in the chat or in the saved `solace-design.md`, as its input contract. Implement mode OPENS with its own Step 0 door question (Quickstart, Solace Suggested, or Custom), asked before any broker details are requested or accepted; an existing `config.json` or a running broker does not answer it, so ask it on the way in. + +For a direct leaf (Direct Pub/Sub or Request-Reply (Direct)), the close-out differs: offer to save the summary to `solace-design.md`, and say in one line that generating a Python project for this leaf is not yet available in this release, so the saved design is ready for when it is. Do not offer to move into Implement mode, do not generate code from the direct samples by hand, and do not offer to write the code another way (from scratch, "outside the generator", or as unverified code). When the developer asks for the code anyway, repeat that generation for this leaf is not yet available and point them at the [Python tutorials](https://tutorials.solace.dev/python/); write no code. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-guaranteed-pubsub.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-guaranteed-pubsub.md new file mode 100644 index 0000000..2eed182 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-guaranteed-pubsub.md @@ -0,0 +1,53 @@ +# Implement: Guaranteed Pub/Sub leaves + +The per-leaf wiring for the three Guaranteed Pub/Sub leaves of the Python API. Read this file from `implement-mode.md` Step 4 when the `Pattern` field is `Guaranteed Pub/Sub (single service, exclusive)`, `Guaranteed Pub/Sub (single service, non-exclusive)`, or `Guaranteed Pub/Sub (fan-out)`. The shared Step 4 rules in `implement-mode.md` (the disclaimer header, the shared connection helper, the fixed file names, the concurrency rules, the variant discipline, the demo-harness rule, the comments rule, and the Quickstart fail-fast rule) apply here unchanged. + +## Guaranteed pub/sub flow + +The skill ships two best-practices reference samples for this flow. Read them by their exact relative paths and adapt each into its own generated script. There is no merge and no third skeleton: `guaranteed_publisher.py` is a near-verbatim adaptation of the publisher sample, and `guaranteed_subscriber.py` is a near-verbatim adaptation of the subscriber sample. + +- `python-guaranteed-publisher-sample.py`: basic-auth connect, persistent binary publish to a topic from a message template that sets the time-to-live once, publish receipt (ACK/NACK) handling, capacity-bounded back pressure, service event listeners registered before `connect()`, and a graceful SIGINT/SIGTERM shutdown that drains outstanding receipts. +- `python-guaranteed-subscriber-sample.py`: basic-auth connect, durable queue plus topic subscription provisioned at start, a client-ack receiver that ACKs only after processing, receiver state and termination handling, service event listeners registered before `connect()`, and a graceful SIGINT/SIGTERM shutdown. + +Each script builds its OWN `MessagingService` (two services, one per script, matching the samples). Keep the samples' structure: the shared state dataclass, `setup_solace(...)` that builds the service and the receiver or publisher before any connection exists, `connect_solace(...)`, the main loop, and `teardown_solace(...)` in `main()`'s `finally`. Keep every listener class; they are part of the best-practices contract and must not be stripped during adaptation. + +**Ground before you write.** Before writing either script, WebFetch [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) and [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md) in this session (SKILL.md Invariant 2). The second page also grounds the `receive_async()` and acknowledgement rules in `implement-mode.md` Step 4's concurrency section. + +The three guaranteed pub/sub leaves generate the same publisher and subscriber. The exclusive versus non-exclusive choice is one queue-factory line (read the `Access type` summary field). The fan-out leaf reuses the single-service subscriber once per consuming service (see Fan-out below). + +## Subscriber script (`guaranteed_subscriber.py`) + +The subscriber provisions the durable queue and the topic subscription, and it is the long-running process the developer runs FIRST. The publisher does NOT provision anything. Start the file with the disclaimer header, then wire the subscriber in this sequence, each step grounded in the pages linked in `implement-mode.md`'s Grounding references: + +1. Build the connection properties with `load_service_properties(args, APP_NAME)`, then the service with `MessagingService.builder().from_properties(properties).build()` (on Quickstart without the reconnection strategy call, per the fail-fast rule). +2. Build the queue from the design summary's `Consumption endpoint` queue name and `Access type` field (see the access-type snippet below). +3. Build the persistent receiver as the sample does: `create_persistent_message_receiver_builder()` with `with_missing_resources_creation_strategy(MissingResourcesCreationStrategy.CREATE_ON_START)`, `with_subscriptions([TopicSubscription.of(topic)])` for the design summary's topic, `with_message_client_acknowledgement()`, and `with_activation_passivation_support(...)`, then `build(queue)`. Keep the receiver termination listener and register the three service event listeners BEFORE `connect()`. See [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md). +4. `messaging_service.connect()`. Immediately after `connect()` returns, emit the connect marker: `trace("VERIFY: CONNECTED")`. +5. `receiver.receive_async(handler)` BEFORE `receiver.start()`, as the sample orders them, then `receiver.start()`. Immediately after `start()` returns without raising, emit the queue-bound marker: `trace("VERIFY: QUEUE_BOUND")`. Keep the sample's `PubSubPlusClientError` branch around `start()`: a client that may not create endpoints fails there, and the error message tells the developer what to do. +6. In the handler's `on_message`, keep the sample's try/except/finally and call `receiver.ack(message)` only after processing. Immediately after `ack()` in the `finally`, emit the round-trip marker: `trace("VERIFY: MESSAGE_RECEIVED")`. See [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md). +7. KEEP the long-running wait loop, the SIGINT/SIGTERM handler that only sets the shutdown event, and `teardown_solace(...)` with `terminate(grace_period)` and `disconnect()`. The subscriber is the SIGINT target, and its shutdown line (`Shutdown signal received ...`) stays a `trace(...)` call. + +**Access type (exclusive vs non-exclusive): one line, not a separate sample.** The `Access type` field picks the queue factory: + +```python +queue = Queue.durable_exclusive_queue(queue_name) # one active consumer / ordered failover +# queue = Queue.durable_non_exclusive_queue(queue_name) # competing consumers share the load +``` + +When the `Access type` field reads exclusive, keep the `durable_exclusive_queue` line: only one bound receiver is active at a time, and the receiver state listener reports the passive standby. When it reads non-exclusive, use the `durable_non_exclusive_queue` line instead, so several subscriber processes bound to the same queue share the messages. Emit exactly one of the two lines. Ground the choice in [Topic Endpoints and Queues](https://docs.solace.com/Get-Started/topic-endpoints-queues.md). + +**Fan-out.** On the fan-out leaf, every consuming service gets its own durable queue with the same topic subscription, so each service receives its own full copy of every event. Generate one subscriber script per consuming service: `guaranteed_subscriber.py` for the first service, and `guaranteed_subscriber_.py` for each further service, where `` is a lowercase Python identifier (letters, digits, and underscores, for example `billing` or `audit_log`) so the import smoke can import the module, each with its own queue name from the design summary and an exclusive durable queue. Hand back one run command per subscriber in Step 5, and add each script to the import smoke. + +## Publisher script (`guaranteed_publisher.py`) + +The publisher only connects and publishes; it does NOT provision the queue or the subscription. It must exit on its own after the broker acknowledges its messages, so the developer does not have to interrupt it. Start the file with the disclaimer header, then wire the publisher in this sequence: + +1. Build the connection properties and the service exactly as the subscriber does (step 1 above), then the publisher with `create_persistent_message_publisher_builder().on_back_pressure_wait(PUBLISH_BUFFER_CAPACITY).build()`. Register the three service event listeners BEFORE `connect()`. +2. `messaging_service.connect()`. Immediately after `connect()` returns, emit the connect marker: `trace("VERIFY: CONNECTED")`. +3. `publisher.start()`, then `publisher.set_message_publish_receipt_listener(...)` AFTER `start()`, as the sample orders them. In the receipt handler's ACK branch, emit the publish-ACK marker: `trace("VERIFY: PUBLISH_ACKED")`. CRITICAL: the sample logs the ACK with `logger.debug`, which the INFO-level logging never shows, so the marker would never appear. Emit it through `trace(...)`, not through the logger. In the NACK branch, keep the sample's warning and also set the shared state's `exit_code` to 1, so a rejected message makes the publisher exit non-zero. Pass the shared state to the handler to do this. +4. Publish persistent messages to the design summary's topic as the sample does: one `message_builder()` template with `PERSISTENT_TIME_TO_LIVE` set once, `build(payload, additional_message_properties={APPLICATION_MESSAGE_ID: msg_id})` per message, and `publish(message, topic, user_context=msg_id)`. See [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md). +5. EXIT on its own: replace the sample's continuous publish loop with the application's own messages, or with one message on a try-it run, then return from the publish step. `teardown_solace(...)` in `main()`'s `finally` calls `terminate(grace_period)`, which sends what is buffered and waits for the broker's receipt of every published message, so the `VERIFY: PUBLISH_ACKED` marker prints before the script exits. Drop the stats thread with the loop. Keep the SIGINT/SIGTERM handler, so Ctrl-C during a back pressure wait still reaches teardown. + +## The marker contract + +These four `VERIFY:` marker strings (`VERIFY: CONNECTED`, `VERIFY: PUBLISH_ACKED`, `VERIFY: QUEUE_BOUND`, `VERIFY: MESSAGE_RECEIVED`) are the same strings the JCSMP guaranteed pub/sub leaf emits, character for character. They are read PER PROCESS: the subscriber output carries `QUEUE_BOUND` and `MESSAGE_RECEIVED`, the publisher output carries `PUBLISH_ACKED`, and both print `CONNECTED`. Until the Python verify script lands, the developer reads them from the two terminals (`implement-mode.md` Step 5); the verify script will read the same strings, so never change one. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-mode.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-mode.md new file mode 100644 index 0000000..5b9eff4 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-mode.md @@ -0,0 +1,269 @@ +# Implement Mode + +Generate a complete, runnable Python project from a confirmed design summary. The canonical output is TWO scripts (a publisher and a subscriber) plus the shared connection-config helper in ONE project directory with its own virtual environment. Each script is a near-verbatim adaptation of its reference sample. Near-verbatim adaptation of two samples is more reliable than merging them into one program, and a real deployment lifts each script into its own service or project. + +Implement mode is NOT only for the canonical shape. A web app, a dashboard, a service, or any embedded shape where the Python messaging layer lives inside a larger application follows the SAME steps: the same door question, the same leaf wiring for the messaging layer, the same generation outputs, and the same verification fallback. Building an off-catalog shape never waives a step in this file. + +Work through the steps in order. Read this file's grounding links only as each step needs them. Link the canonical doc and state the decision in one line, never paraphrase doc content. Resolve every dependency version at generation time; pin nothing in skill content (SKILL.md Invariant 3). + +**Coverage in this release.** Implement mode generates the three Guaranteed Pub/Sub leaves on the Quickstart door. The Direct Pub/Sub and Request-Reply (Direct) leaves (Step 4) and the Solace Suggested and Custom doors (Step 0) are not yet available for Python. The Python API does not bundle a verify script yet, so every run ends in the compile-only fallback of Step 5. + +## Generation outputs: the file contract + +Every Implement run writes ALL of these as generation output, on every app shape. None of them is gated on consent, and none of them waits for the end of the session: + +1. The generated scripts for the chosen leaf, plus the shared `solace_connection_config.py` helper. Every source file starts with the AI-assisted disclaimer header (Step 4). +2. `requirements.txt` with the `solace-pubsubplus` version resolved per Step 3. +3. `config.example.json` (placeholders) and a `.gitignore` that ignores `config.json` and the virtual environment (Step 4). +4. A tailored `solace-verification-checklist.md` at the project root (tailoring rules in Step 6; the WRITE happens here at generation time, like the scripts). + +There is no `verify.sh` and no `verify-hooks.sh` in a Python project yet. Do not copy the JCSMP `verify.sh` into a Python project: it drives Maven and cannot run Python scripts. + +## Step 0: Ask Quickstart, Solace Suggested, or Custom, before any broker details + +This is the FIRST question, and you ask it before you request or accept ANY broker connection details. Ask the developer plainly which of three doors they want: Quickstart, Solace Suggested, or Custom. Ask it as a direct question and wait for the answer. Frame the three doors honestly: Quickstart is a learning and try-it path to see a round-trip work, NOT a deployment target; Solace Suggested is the hardened baseline Solace suggests as a starting point (TLS, dead message queue eligibility, HA failover, separate projects), and Solace suggests it without guaranteeing it is ready for deployment; Custom lets the developer tick exactly which hardening knobs to include. State in the same question that Solace Suggested and Custom are not yet available for the Python API in this release. + +Three things do NOT answer this question, and none of them waives it: + +- **An existing `config.json`**, even one that already holds real credentials. Ask anyway. +- **A running local broker discovered in the environment.** Report what you found, then ask both this question and which broker to target. +- **An app shape larger than the generated two-script project** (a web app, a dashboard, a service). The doors still govern the messaging layer and the credential placement. Ask anyway. + +The doors: + +- **Quickstart.** Each script builds its connection through the shared `solace_connection_config.py` helper, which reads the connection properties from a `config.json` in the working directory when present and otherwise falls back to the command-line arguments (` [password]`). Quickstart adds no TLS hardening and has no TLS requirement: it connects with whatever scheme the developer's host carries, and a plaintext `tcp://` dev broker is a normal Quickstart target. Generate the scripts without per-stage preview gates. Quickstart output fails fast (the fail-fast rule in Step 4). +- **Solace Suggested or Custom.** Not yet available for Python. Say so plainly, and do not describe the hardened doors beyond the one-line framing in the Step 0 question. Offer the developer two honest choices: Quickstart now, framed as a learning run, or stop here with the design saved to `solace-design.md` for when the hardened doors are available. Never generate a Quickstart project under the Solace Suggested or Custom name. + +State plainly that a real deployment lifts each script into its own service or project. Basic username/password is the only auth this journey generates; ground it in the authentication section of [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md). + +## Step 1: Confirm broker access and grounding + +With the door chosen, confirm the developer has access to a reachable broker, and confirm WHICH broker they want to target. Confirm that they HAVE one; do NOT ask them to paste connection details into the chat. When Design mode already confirmed broker access in this session (its Step 0), do not re-ask whether a broker exists; ask only WHICH broker to target. A broker discovered in the environment is a fact to report while asking, never an answer. The credential VALUES reach the app through a `config.json` when present, otherwise as command-line arguments at run time; the skill itself never needs the values typed into the chat. + +If the developer has no reachable broker, route through `prerequisites.md` first, then return here. + +Grounding catch-up: if the design summary names a page in its Grounding docs field that this session has not WebFetched, fetch it now before generating against it (SKILL.md Invariant 2). + +## Step 2: Establish the design as the input contract + +Implement mode works from the unified design summary of eight fields that `design-mode.md` defines: Pattern, Delivery, Access type, Topic, Consumption endpoint, Auth, Broker, Grounding docs. The `Pattern` field is one of the five Python leaf strings. + +The summary is a HARD precondition (the design-contract gate in `python.md`): Implement mode never proceeds past this step without one, and never invents design values. Exactly three sources satisfy it: + +- A summary Design mode confirmed in this session: use it as-is, with no re-confirm. +- An explicit summary the developer supplied in chat (the eight fields or an equivalent statement of them): use it directly. If it omits a derivable field, derive it rather than asking (the topic via the solace-topic-best-practices skill, the queue name via the `q.`-prefixed convention), the same way Design mode does. +- A saved `solace-design.md` in the developer's project: read it, restate it in one line, and proceed on their confirm. A saved summary titled `Solace JCSMP Design Summary` belongs to the JCSMP API: say so and ask whether the developer wants a Python project from it, then re-confirm it through Design mode, because a JCSMP leaf such as `Request-Reply (Guaranteed)` has no Python equivalent. + +If none of the three exists, route through Design mode (`design-mode.md`) and return with its confirmed summary. The only carve-out is the topology rule in `python.md`: a mechanical edit to an existing app proceeds without a summary. + +## Step 3: Resolve the version, then write requirements.txt and the virtual environment + +### Resolve the latest solace-pubsubplus version at generation time + +Never hardcode a `solace-pubsubplus` version in this skill or carry one in your memory. Resolve the latest release from the AUTHORITATIVE PyPI JSON API at generation time: + +```bash +curl -s https://pypi.org/pypi/solace-pubsubplus/json \ + | python3 -c 'import json, sys; print(json.load(sys.stdin)["info"]["version"])' +``` + +If the network query fails, do not guess a version and do not write an unpinned requirement. Ask the developer for the version they want, pin that version, and tell the developer that PyPI was unreachable, so the pin is their choice and not a resolved release. + +### Write requirements.txt and create the virtual environment + +- **Single Solace dependency: `solace-pubsubplus`.** Write it as `solace-pubsubplus==` (or the developer's version when PyPI was unreachable), so the generated project installs exactly the release it was generated against. That pin is generation output resolved at generation time, the same way the JCSMP pom carries a resolved `sol-jcsmp` version; it is never a version written in skill content. Add no other dependency: the samples need only the standard library besides the Solace API. +- **Python version.** Check `python3 --version` in the developer's environment. When it reports a Python that is no longer in active or security support, tell the developer, link `prerequisites.md`'s Supported environments section, and do not choose an older interpreter for them. +- **Virtual environment.** Create it at the project root and install into it: `python3 -m venv .venv`, then `.venv/bin/pip install -r requirements.txt` (on Windows, `.venv\Scripts\pip`). Every run command this mode hands back uses the virtual environment's interpreter (`.venv/bin/python`), never the system `python3`. +- **Logging.** The samples log API events through the standard `logging` module, configured once with `logging.basicConfig(level=logging.INFO, ...)` in the `__main__` block. Keep that configuration in every generated script. Never raise the level above INFO, because the API's own connect and reconnect diagnostics travel the same channel. + +## Step 4: Dispatch on the Pattern leaf, then generate the scripts + +Generation is keyed on the `Pattern` leaf read in Step 2. Read ONLY the leaf file the design chose, and do not read samples for a leaf the design did not choose. + +| `Pattern` leaf (verbatim) | Leaf wiring file | Samples it reads | +|---------------------------|------------------|------------------| +| `Guaranteed Pub/Sub (single service, exclusive)` | `implement-guaranteed-pubsub.md` | `python-guaranteed-publisher-sample.py` + `python-guaranteed-subscriber-sample.py` | +| `Guaranteed Pub/Sub (single service, non-exclusive)` | `implement-guaranteed-pubsub.md` | `python-guaranteed-publisher-sample.py` + `python-guaranteed-subscriber-sample.py` | +| `Guaranteed Pub/Sub (fan-out)` | `implement-guaranteed-pubsub.md` | `python-guaranteed-publisher-sample.py` + `python-guaranteed-subscriber-sample.py` | +| `Direct Pub/Sub` | not yet available | none | +| `Request-Reply (Direct)` | not yet available | none | + +**The two direct leaves stop here.** Say in one line that generating a Python project for this leaf is not yet available in this release. Offer to save the confirmed summary to `solace-design.md` if it is not saved yet, and stop. Do not read the direct samples and do not adapt them by hand: without a wiring file there is no marker contract and no conformance check for the result. Do not offer to write the code another way (from scratch, "outside the generator", or as unverified code), and do not write it when the developer asks again: the stop is the ENTIRE answer, the same way the Debug redirect is. Point the developer at the saved design and at the [Python tutorials](https://tutorials.solace.dev/python/) instead. + +**Every leaf also generates the shared connection helper.** Read `python-solace-connection-config.py` and generate it as `solace_connection_config.py` (a module name needs underscores). Each script builds its connection dict by calling `load_service_properties(args, APP_NAME)` from that module and passes it to `MessagingService.builder().from_properties(...)`, rather than setting the host, VPN, username, or password inline. + +**Fixed file names.** Generate the scripts as `guaranteed_publisher.py` and `guaranteed_subscriber.py`, the names each sample's docstring gives, next to `solace_connection_config.py` (the fan-out leaf adds one subscriber script per further consuming service, named in `implement-guaranteed-pubsub.md`). The run commands in Step 5 and the checklist address these names; the names of functions, constants, and the `APP_NAME` value inside each script stay your discretion. + +**Disclaimer header on EVERY generated source file.** Stamp the top of each generated `.py` file with these two comment lines, before the module docstring and before any import: + +```python +# AI-assisted code. Review before production use. +# See the verification checklist: solace-verification-checklist.md +``` + +Stamp the header AT WRITE TIME, as the first content of each new file. `requirements.txt`, `config.example.json`, and `.gitignore` are not source and carry no header. The reference samples themselves carry NO disclaimer. This rule is SKILL.md Invariant 4. + +**Embedded and web-app shapes.** When the Python messaging layer sits inside a larger application, the chosen leaf's wiring still governs the messaging layer: the same connection helper, the same service, receiver, and publisher listeners, the same `VERIFY:` markers emitted through a stdout-reaching `trace(...)`, and the disclaimer header on every generated file, messaging or not. The `VERIFY:` markers are NOT demo harness. + +**Write the verification artifacts in this same step.** Alongside the scripts, write the tailored `solace-verification-checklist.md` at the project root (Step 6), `config.example.json`, and `.gitignore`. A session that ends early still leaves the checklist in the project. + +### Concurrency: threads, not asyncio, and no multiprocessing + +The Python API calls application code on its own threads, and the generated code must respect that model. WebFetch [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md) before generating a receiver. That page grounds the `receive_async()` rule and the acknowledgement rule; the handler rule comes from the samples' handler docstrings and the C API Best Practices they cite, and the `multiprocessing` rule comes from the PyPI page: + +- **`receive_async()` is not asyncio.** The page states that `receive_async()` is not an asynchronous coroutine or generator, is not `asyncio` compatible, returns immediately, and invokes the callback on a new Python thread for every message. Never `await` the API and never call it from inside a coroutine. In an asyncio application, keep the Solace objects on their own threads and hand each received message to the event loop with the standard library's `loop.call_soon_threadsafe(...)` (a standard library call, not a Solace API). Acknowledge the message only after the application has processed it and before the receiver terminates: the page states that any routine can acknowledge messages at any time as long as the receiver has not been terminated. +- **Handlers return promptly.** Every listener and message handler runs on an API thread. Return promptly and hand heavy work to the application's own thread or queue, as the samples' handler docstrings state. +- **No `multiprocessing`.** The Solace Python API cannot be used in an application that uses the Python `multiprocessing` module (the [solace-pubsubplus PyPI page](https://pypi.org/project/solace-pubsubplus/) states this). To scale consumption, run more independent processes of the subscriber on a non-exclusive queue, each started on its own; never fork workers from one process with `multiprocessing`. + +### Adapting a sample to a variant of its leaf + +The samples are grounded REFERENCE for the API idioms, not templates that must be reproduced line for line. Near-verbatim adaptation is the default WHEN the design summary matches the leaf's base shape. When the summary describes a variant, ADAPT the sample's idioms to honor the approved design; do NOT snap the variant back to the sample's default shape, and do NOT frame the deviation as a "conflict" with the skill. + +Hold every invariant while adapting: keep the shared connection helper, the service event listeners registered before `connect()`, the receipt, termination, and state listeners, the disclaimer header, and the leaf's `VERIFY:` markers. Any NEW named entity the variant introduces must be traceable to a canonical Solace source per Invariant 1: WebFetch the grounding page for it. + +**Guardrail: variant versus off-catalog.** Adapt freely WITHIN the chosen pattern family when the change is a topology or reliability knob that the canonical docs ground and that still emits the leaf's `VERIFY:` markers. If a requirement leaves the Solace topic and queue publish and subscribe model, or needs behavior the canonical docs cannot ground, STOP and tell the developer it is off-catalog rather than inventing it. + +**Preserve the marker contract.** A variant adapts the SHAPE, never the marker strings. It MUST still emit its leaf's `VERIFY:` markers, character for character. Never rename or drop a marker to fit a variant. + +**Demo harness is not application logic.** The samples carry demo-harness elements that exist only to make a standalone run observable: the once-per-second stats loop or thread, the continuous publish loop, and the rotating example payload. When the developer's request describes a real application domain, do NOT carry these into the generated scripts: replace the example payload and the publish loop with the application's real messages and triggers, never add a delay or rate limit between sends that the application does not ask for, and keep only the lifecycle pieces the leaf's steps call for (the SIGINT/SIGTERM shutdown on the long-running role, the self-exit on the foreground role). When the request IS a demo or a try-it run, keeping the harness is fine. Two things are NOT demo harness: the `trace(...)` narration function (keep it; applications replace its single body to route narration elsewhere) and the `VERIFY:` markers, which are emitted through `trace(...)`. The generated `trace(...)` body MUST stay `print(message, flush=True)`, so the markers reach stdout without buffering delays; rerouting it is the developer's step after verification. + +**Comments follow their code.** The samples' comments carry the documented best practices into the code a developer reads long after this session. A generated script that carries a construct from a sample carries that construct's comment too, with only the names adapted. A dropped construct takes its comment with it. Fresh messaging code with no sample twin that applies a documented practice gets a SHORT comment naming the practice, in the samples' comment style. The one-line comments this skill mandates elsewhere (the Quickstart fail-fast posture below) always apply on their paths. + +### Quickstart fail-fast reconnect defaults + +On the Quickstart door, do not carry the samples' reconnect-tuning block into the generated scripts. The samples set a reconnection budget (`with_reconnection_retry_strategy(RetryStrategy.parametrized_retry(...))` with the `RECONNECT_RETRIES` and `RECONNECT_RETRY_INTERVAL_MS` constants) and carry an HA failover comment block; Quickstart generation OMITS the builder call, the two constants, the HA comment block, and the then-unused `RetryStrategy` import, leaving connect and reconnect behavior at the API defaults, so a wrong host, port, or credential in this learning setup fails immediately instead of retrying. Drop ONE short comment at the `MessagingService.builder()` site in each generated script stating that posture: connect and reconnect settings are at the API defaults so failures surface immediately; see the [Solace C API Best Practices](https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md) for reconnect tuning beyond Quickstart. + +### Scaffold the config files + +Generate two small project files next to the scripts: + +- A committed `config.example.json` that carries placeholder values for the four keys the shared helper requires, spelled as the Solace Python API property names the helper documents. The placeholder values must be obviously fake: + + ```json + { + "solace.messaging.transport.host": "tcp://HOST:55555", + "solace.messaging.service.vpn-name": "YOUR_VPN", + "solace.messaging.authentication.basic.username": "YOUR_USERNAME", + "solace.messaging.authentication.basic.password": "YOUR_PASSWORD" + } + ``` + +- A `.gitignore` at the project root that ignores `config.json`, `.venv/`, and `__pycache__/`, so the real credential-bearing file and the local environment are never committed. + +Tell the developer they opt in to the config file by copying `config.example.json` to `config.json` and filling in their real broker values. Both scripts then read it from the working directory, so they run from the project root. + +## Step 5: Check the project, then hand back the run commands + +The Python API does not bundle a verify script yet, so Implement mode ends in the compile-only fallback on every run, whatever the contents of `config.json` (SKILL.md Invariant 5). Never fabricate a run result and never call a check below a verified round trip. + +1. **Compile check.** Run `.venv/bin/python -m py_compile` on every generated `.py` file. A failure is a code failure: fix it and re-run, up to 3 attempts, then report the evidence and hand the decision to the developer. +2. **Import smoke.** From the project root, run `.venv/bin/python -c "import solace_connection_config, guaranteed_publisher, guaranteed_subscriber"`, adding every further generated module (for example each fan-out subscriber). Each script keeps its `if __name__ == "__main__":` guard, so the import resolves every `solace.messaging` name the scripts use without connecting to a broker. A failure (for example an `ImportError` on a misspelled module path) is a code failure under the same fix loop. +3. **Conformance checks.** From the project root, run these three checks and record each result in the checklist. A missing header or a missing checklist is a code failure under the same fix loop; a version mismatch is reported to the developer, not fixed silently. + + ```bash + # every generated .py file starts with the disclaimer header + for f in *.py; do head -n 1 "$f" | grep -qxF '# AI-assisted code. Review before production use.' || echo "MISSING HEADER: $f"; done + # the tailored checklist exists + test -f solace-verification-checklist.md || echo "MISSING CHECKLIST" + # the pin matches the release PyPI reports now (skip and record it when PyPI is unreachable) + grep -qxF "solace-pubsubplus==$(curl -s https://pypi.org/pypi/solace-pubsubplus/json | python3 -c 'import json, sys; print(json.load(sys.stdin)["info"]["version"])')" requirements.txt || echo "VERSION DRIFT" + ``` + +4. **Report and hand back.** Report the compile, import, and conformance results, then hand the developer the exact commands to run against their own broker from the project root, subscriber first so the durable queue and its subscription exist before the first publish: + + ``` + # terminal 1: provisions the queue, binds, and waits (Ctrl-C stops it) + .venv/bin/python guaranteed_subscriber.py [ [password]] + # terminal 2: publishes, waits for the broker's acknowledgement, and exits + .venv/bin/python guaranteed_publisher.py [ [password]] + ``` + + On the fan-out leaf, hand back one subscriber command per subscriber script (`guaranteed_subscriber.py` and each `guaranteed_subscriber_.py`), each in its own terminal and all started before the publisher; every subscriber prints its own markers. + + The arguments are needed only when there is no `config.json`. Tell the developer what a passing run prints: the subscriber prints `VERIFY: CONNECTED` and then `VERIFY: QUEUE_BOUND`; the publisher prints `VERIFY: CONNECTED` and `VERIFY: PUBLISH_ACKED` and exits with status 0; the subscriber then prints `VERIFY: MESSAGE_RECEIVED`; Ctrl-C on the subscriber prints `Shutdown signal received` and it exits with status 0. + +**When the developer asks you to run it.** If the developer explicitly asks you to run the two scripts against their broker, print the live-broker heads-up first (SKILL.md Invariant 6: name the host, every durable queue the subscribers provision, one connection per script, and the messages the publisher sends, scaled to the generated scripts). Then start each subscriber in the background with its output captured, wait for `VERIFY: QUEUE_BOUND` from each, run the publisher in the foreground, wait for `VERIFY: MESSAGE_RECEIVED` in each subscriber's output, and stop the subscribers with SIGINT. Report the markers you observed as evidence of a manual run. It is not a verify verdict: record it in the checklist as a manual run, and never describe it as a passed verify stage. + +**Environment failure on a manual run: do NOT modify the code.** When the captured output shows that the broker is unreachable, the credentials or VPN are wrong, or the client may not create the queue (the subscriber's `Could not bind to queue` error), the code is not the problem. Report the matched output line as evidence and STOP: do not enter the fix loop, and do not change the code. The fix is the developer's broker, credentials, or client permissions. Fix the code only when the output shows a code failure (an exception from the scripts' own logic, or a marker that never appears after a clean connect and bind), up to 3 attempts, then hand the decision to the developer. + +**Close every run with the next steps, stated plainly.** Tell the developer, in order: (1) copy `config.example.json` to `config.json` and fill in their real broker values; (2) run the subscriber, then the publisher, with the commands above, and check the markers. Step 4 already wrote the tailored `solace-verification-checklist.md`; invite the developer to add their own items in its Your additional items section and to record any skill-undelivered concerns under Developer-owned items. + +### Traps to avoid + +- **Improvised verdicts.** A connect that does not raise, or a publish call that returns, proves nothing about the round trip. Judge only by the markers in each script's output. +- **Publishing direct-to-queue instead of pub/sub.** Publish to a topic; let the durable queue carry a topic subscription. See [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md). +- **DIRECT delivery where Guaranteed is required.** Use the persistent publisher for the guaranteed pub/sub journey. See [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md). +- **Seconds instead of milliseconds.** `RetryStrategy.parametrized_retry(retries, retry_interval)` takes its interval in MILLISECONDS per the API reference; a value of `3` means 3 milliseconds. The sample code on the Messaging Service page comments `parametrized_retry(20, 3)` as "over an interval of 3 seconds"; that comment is wrong, and the API reference wins. Whenever a generated script sets a retry interval, write it in milliseconds, as the samples' named constants do. +- **Hardcoded credentials.** Never inline the host, VPN, username, or password. Every script builds its properties through the shared helper. +- **Hardcoded versions.** Resolve the `solace-pubsubplus` version at generation time (Step 3); never carry a version from memory, from an old `requirements.txt`, or from the upstream samples repository. +- **Wildcard imports.** Keep every import explicit, as the samples do; never write `from solace.messaging... import *`. +- **The system interpreter.** Run and check everything with `.venv/bin/python`, so the project uses the version it installed. +- **Running the publisher before the subscriber has provisioned the queue.** A persistent message published before the queue subscribes to its topic is dropped and never received. By default the broker still ACKs it, so the publisher can print `VERIFY: PUBLISH_ACKED` for a message no subscriber gets; the publisher sees a NACK only when the client profile rejects messages that match no subscription. Judge the round trip by `VERIFY: MESSAGE_RECEIVED` in the subscriber output, never by `PUBLISH_ACKED` alone. Run the subscriber first. +- **Treating a sample as a spec that overrides the approved design.** See "Adapting a sample to a variant of its leaf" above. + +## Step 6: Tailor the checklist (written at Step 4) and report + +Step 4 writes the tailored `solace-verification-checklist.md` into the generated project as generation output; this step defines its content and adds the report in chat. The Python API has no master checklist template yet; until it lands, write the file from the list below. Resolve each item for the generated code: an item the generation did not deliver moves to Your responsibility. Write it as a Markdown task list (`- [ ]`), ticking only what this run actually checked. Each item in the first two groups carries one canonical doc link, as the master JCSMP template does; the round-trip and conformance items carry none because they check skill conformance, not Solace guidance. + +**Delivered by this generation** + +- Basic-auth connection through the shared `solace_connection_config.py` helper; credentials live only in a gitignored `config.json`. [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) +- Service reconnection and interruption listeners registered before `connect()`. [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) +- Persistent publish to a topic with a publish receipt listener; a NACK is logged with the message id. [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) +- Capacity-bounded back pressure on the publisher (`on_back_pressure_wait`). [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) +- Message time-to-live set once on the publisher's message template. [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) +- Durable queue plus topic subscription provisioned at start (`CREATE_ON_START`), with the access type from the design summary (one durable queue per consuming service on fan-out). [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md) +- Client acknowledgement only after processing. [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md) +- Graceful SIGINT/SIGTERM shutdown: `terminate(grace_period)` and then `disconnect()` on every exit path. [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md) + +**Your responsibility (not delivered here)** + +- TLS secure session (`tcps://` with certificate validation); Quickstart does not generate it. [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) +- Credentials handled safely: `config.json` is never committed, and credentials are not passed as command-line arguments, where they land in shell history and process listings (the handed-back run commands accept them only for convenience). [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) +- Authentication stronger than basic username/password (for example a client certificate or Kerberos) where the environment requires it; this journey generates basic username/password only. [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) +- Dead message queue eligibility on published messages and a dead message queue on the broker. [Configuring Dead Message Queues](https://docs.solace.com/Messaging/Guaranteed-Msg/Setting-Dead-Msg-Queues.md) +- Reconnect tuning for HA failover; Quickstart leaves connect and reconnect behavior at the API defaults. [Solace C API Best Practices](https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md) +- Separate projects or deployments per role (skill guidance: a real deployment lifts each script into its own service or project; no Solace doc page states it). +- `respect-ttl` enabled on the queue, without which the broker ignores the message time-to-live. [Configuring Queues](https://docs.solace.com/Messaging/Guaranteed-Msg/Configuring-Queues.md) +- An administrator-provisioned queue, or the endpoint-create permission for this client, in production. [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md) +- A Python version in active or security support and OpenSSL installed on Linux for TLS ([Supported Environments](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-supported-Environments.md)), and no `multiprocessing` in the application ([solace-pubsubplus on PyPI](https://pypi.org/project/solace-pubsubplus/)). + +**Verified by the round-trip** (developer-run until the Python verify script lands) + +- Every subscriber prints `VERIFY: CONNECTED` and `VERIFY: QUEUE_BOUND`. +- The publisher prints `VERIFY: CONNECTED` and `VERIFY: PUBLISH_ACKED`, then exits with status 0. +- Every subscriber prints `VERIFY: MESSAGE_RECEIVED` (`PUBLISH_ACKED` alone does not prove delivery). +- Ctrl-C on each subscriber prints `Shutdown signal received` and it exits with status 0. + +**Generation conformance** + +- The disclaimer header is the first two lines of every generated `.py` file (Step 5 header check). +- `requirements.txt` pins the `solace-pubsubplus` version that PyPI reported as `info.version` at generation time, or the developer's version when PyPI was unreachable (Step 5 drift check). +- This tailored checklist exists at the project root (Step 5 checklist check). +- `py_compile` and the import smoke passed (Step 5), with the result recorded here. +- Comments follow their code (Step 4). +- Every page named in the design summary's Grounding docs field was WebFetched in this session (`none fetched` is an honest value; an unfetched citation is not). +- Verification: record `verify script not yet bundled for Python; round trip handed back`, or `manual run` with the observed markers when the developer asked you to run it. + +**Developer-owned items (not delivered by this skill)**: any item the skill does not deliver, recorded as developer-owned rather than skill-satisfied. A consented deviation is recorded here. + +**Your additional items**: open space for the developer to keep growing the list. + +The emitted checklist is an artifact, not source, so it carries NO disclaimer header. Report the same resolution in chat: what landed under each group, the compile, import, and conformance results, and that the round trip is handed back. + +## Grounding references + +Live `docs.solace.com` `.md` pages (WebFetch on demand): + +- [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) +- [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) +- [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md) +- [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md) +- [Supported Environments](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-supported-Environments.md) +- [Solace C API Best Practices](https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md) + +Public URLs (not docs.solace.com doc pages; use the live URL directly): + +- [Python API reference](https://docs.solace.com/API-Developer-Online-Ref-Documentation/python/index.html) +- [Python API Release Notes](https://products.solace.com/download/PYTHON_API_RN) +- [solace-pubsubplus on PyPI](https://pypi.org/project/solace-pubsubplus/) diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/prerequisites.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/prerequisites.md new file mode 100644 index 0000000..b81bb69 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/prerequisites.md @@ -0,0 +1,29 @@ +# Prerequisites + +Read this when a developer does not yet have a broker to build against, or when the developer's Python environment is in question. Both Design mode and Implement mode route here first if the developer lacks a reachable broker. + +## Table of contents + +- Obtain a broker +- Supported environments +- Learn the Python API basics + +## Obtain a broker + +You need one reachable broker. The primary, recommended choice is [Solace Cloud](https://docs.solace.com/Get-Started/Getting-Started-Try-Broker.md): it is the simplest path to a running broker for a greenfield app and for the run-and-observe round-trip. Default to it unless the developer has a reason not to. + +As a brief alternative, the same page also covers setting up a free broker locally, for developers who cannot use Solace Cloud. + +The skill does not provision or configure the broker. It assumes the broker is reachable and that the developer has connection details (host, message VPN, client username, password). A broker discovered running in the environment (for example a local container) is a fact to report, never an answer: still ask which broker the developer wants to target. + +## Supported environments + +WebFetch [Supported Environments for the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-supported-Environments.md) when the developer's environment is in question, and state the answer from the fetched page. Three points matter for every generated project: + +- **Python version.** Solace supports the Python versions that are in active support and also have security update support; the page links the python.org list. The `requires_python` floor on PyPI is an install floor, not a support statement, so never recommend a Python version that is past its end of life because the package still installs on it. +- **OpenSSL.** TLS connections need OpenSSL. The page says which platforms and package versions bundle it and that the bundled libraries are for developer convenience only, so a Linux deployment installs its own. +- **No `multiprocessing`.** The Solace Python API cannot be used in an application that uses the Python `multiprocessing` module. Only the [solace-pubsubplus PyPI page](https://pypi.org/project/solace-pubsubplus/) states this; cite that page, not a `docs.solace.com` page. + +## Learn the Python API basics + +For a tutorial-style, code-first introduction to connecting, publishing, and receiving with the Python API, point the developer at the official Python tutorials: https://tutorials.solace.dev/python/. These walk the same connect, publish, and subscribe flow the skill builds on. From 04b621bdcc99d2f0342f764463ad914ec10a348f Mon Sep 17 00:00:00 2001 From: Adi El-Sammak <71227923+aelsammak@users.noreply.github.com> Date: Thu, 8 Oct 2026 14:21:50 -0400 Subject: [PATCH 04/12] EBP-3229: Add CLAUDE.md and refresh the root README (#11) - Add a root CLAUDE.md for AI coding agents: where the skills, references, evals, and tools live, the exact check and eval commands, repo conventions, a gotchas section, and the PR workflow - Update the README intro and the solace-application-development row to describe the current JCSMP and Python coverage instead of future releases - Add a Contributing section that routes bugs and feature requests to Solace Support, the Solace Ideas portal, or the Solace Community - List CLAUDE.md in the repository layout and describe tools/ by role --- CLAUDE.md | 71 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 11 ++++++--- 2 files changed, 79 insertions(+), 3 deletions(-) create mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..7a753f4 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,71 @@ +# CLAUDE.md + +Orientation for AI coding agents in this repository. For humans, the [README](README.md) covers installation, development, CI, and releases, and the [evals README](plugins/solace-messaging-skills/evals/README.md) covers the eval corpora and runners. + +This repository is a Claude Code plugin marketplace (`solace-agent-plugins`) of Agent Skills for building Solace messaging applications. The product is Markdown skill files, reference samples, and eval corpora. Nothing is compiled or packaged. + +## Where things live + +| Path | What it is | +|------|------------| +| `.claude-plugin/marketplace.json` | The marketplace. One entry per plugin directory. | +| `plugins//.claude-plugin/plugin.json` | Plugin manifest, including the plugin's version. | +| `plugins//skills//SKILL.md` | A skill. The frontmatter `description` alone decides when the skill fires. | +| `plugins/solace-messaging-skills/skills/solace-application-development/references/.md` | Per-API entry file (`jcsmp.md`, `python.md`). Mode files, baked samples, and scripts sit in the matching `references//` folder. | +| `plugins//evals/` | Trigger and output eval corpora. Maintainer tooling, not shipped functionality. | +| `tools/` | CI check scripts, the two eval runners, and `output-evals.schema.json`. | +| `.github/workflows/ci.yml` | The only CI workflow. | + +Nothing in the repository is generated or vendored. The Eclipse files under `references/jcsmp/evals/compile-fixture/` are IDE metadata for the compile fixture; leave them alone. + +## Commands + +Run from the repository root. These match the README. + +```shell +gh skill publish --dry-run # skills against the Agent Skills spec (gh 2.90 or later) +claude plugin validate . --strict # marketplace and plugin manifests +./tools/check-plugin-versions.sh --sync-only # marketplace.json matches plugins/ +./tools/check-links.sh plugins # broken and soft-404 doc links +./plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/compile-fixture/compile.sh # compile the baked JCSMP samples +claude --plugin-dir plugins/solace-messaging-skills # load the plugin from the clone +``` + +The commands above need no broker and no API key. The evals need an exported `ANTHROPIC_API_KEY` or `CLAUDE_CODE_OAUTH_TOKEN`; an existing `claude` login does not work, because the runners use a scratch config directory. + +```shell +./tools/run-trigger-evals.sh --model haiku # then --model sonnet +./tools/run-output-evals.sh --model sonnet # then --model opus; a full leg takes about an hour +./tools/run-output-evals.sh --case # one output eval case +``` + +The trigger runner has no single-case mode; set `TRIGGER_EVAL_RUNS=1` for a quicker check. Only the output evals can touch a broker, and only when all four `OUTPUT_EVAL_BROKER_*` variables are exported (see the evals README). + +## Conventions + +- Never hardcode a library version in skill content. Generated builds resolve the latest release at generation time (Invariant 3 in the application-development `SKILL.md`). +- Link documentation to the live `docs.solace.com` `.md` URL. Do not copy doc content into a skill. Every link under `plugins/` must pass `check-links.sh`. +- A new skill needs cases in both `trigger-evals.json` (positive and near-miss negative prompts) and `output-evals.json` (validate the latter against `tools/output-evals.schema.json`). +- A new, removed, or renamed plugin directory needs the matching `marketplace.json` change in the same pull request. +- A new baked JCSMP sample must be added to the sample roster in `compile.sh`, or the compile check does not cover it. +- Do not put exact counts (of cases, skills, or graders) in READMEs or skill docs. They go stale. + +## Gotchas / do not + +- **Do not bump a `plugin.json` version in a feature pull request.** Versions change only when a release is prepared. Claude Code caches installed plugins by version, so the bump is what ships a release, and a stray bump confuses what users have. Changes under `plugins//evals/`, `tools/`, `.github/`, or the repository root never need a bump. +- **Both repositories are public.** Never write internal links, internal system names, credentials, API keys, or broker details into files, commit messages, branch names, or pull request text. +- **Run `gh repo set-default SolaceDev/agent-plugins` in a fresh clone.** With the `upstream` remote present, a bare `gh pr` command otherwise targets `SolaceProducts/agent-plugins`. +- **A green `trigger-evals` check asserts nothing.** The job is disabled (`if: false`) because the public repositories hold no API key. Run the evals locally and record the results in the pull request description. +- **A trigger eval leg can go red on one `must_pass` case from model variance.** If the prompt and skill descriptions did not change, re-run the full leg before you treat it as a regression. +- **Keep shell scripts on LF line endings.** `.gitattributes` enforces it. Bash rejects CRLF scripts, and an output eval grader compares the generated `verify.sh` byte for byte with the bundled copy in `references/jcsmp/scripts/`. +- **Keep the pins in `ci.yml`.** Every `uses:` step is pinned to a commit SHA with the tag in a trailing comment, and the Claude Code CLI version is pinned in `CLAUDE_CODE_VERSION`. Update a SHA and its comment together. +- **`check-links.sh` sweeps `plugins/` only.** It does not check links in the root README or this file. +- **A tool permission denial during an output eval is an infrastructure failure, not a skill failure.** Managed settings, hooks, or local command shims can deny a command. Fix the environment, then re-run. + +## Agent workflow + +- Open pull requests as drafts. Watch the checks with `gh pr checks --watch`, fix failures, and mark the pull request ready only when all checks are green. +- Print a failed CI job's log with `gh run view --log-failed`, then reproduce it with the matching command above. +- For a change to skill content or skill descriptions, run the trigger evals on Haiku and Sonnet and the output evals on Sonnet and Opus, and record the results in the pull request description. +- Do not pick reviewers. Ask the user who to tag. +- Never merge a release pull request. A human reviewer merges it. diff --git a/README.md b/README.md index c94a5b6..38e9548 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # AI Agent Plugins by Solace -A collection of Solace AI agent plugins that package skills for building event-driven messaging applications. These skills help developers using AI coding assistants (such as Claude Code) design and implement Solace messaging applications that follow Solace best practices out of the gate. The solace-application-development skill designs and generates Solace messaging applications (publishing to topics, consuming from queues, and guaranteed messaging), starting with the Solace Messaging API for JCSMP. A companion solace-topic-best-practices skill looks up the canonical Topic Architecture Best Practices page online and applies it to topic-hierarchy decisions. +A collection of Solace AI agent plugins that package skills for building event-driven messaging applications. These skills help developers using AI coding assistants (such as Claude Code) design and implement Solace messaging applications that follow Solace best practices out of the gate. The solace-application-development skill designs and generates Solace messaging applications (publishing to topics, consuming from queues, and guaranteed messaging), with the Solace JCSMP API and the Solace Python API. A companion solace-topic-best-practices skill looks up the canonical Topic Architecture Best Practices page online and applies it to topic-hierarchy decisions. > [!WARNING] > These skills are powered by language models, which are nondeterministic and may change over time. Generated code can vary between runs and may be incorrect, insecure, incomplete, or unsuitable for your environment. You are responsible for reviewing, testing, and validating all generated code before use, especially before deploying to production. @@ -42,7 +42,7 @@ This repository includes the following skills: | Skill | Description | |-------|-------------| -| **solace-application-development** | Routes Solace messaging design and implementation requests to grounded Solace documentation. Full guidance for messaging pattern selection, runnable code generation, and message round trip verification arrives in upcoming releases. | +| **solace-application-development** | Front door for Solace API development (JCSMP and Python). Routes each request to the right API, helps choose a messaging pattern, and generates a runnable application from reference samples, grounded in canonical Solace documentation. | | **solace-topic-best-practices** | Navigation-only lookup skill that reads the canonical Topic Architecture Best Practices page online and applies it to topic-hierarchy decisions, without generating application code. | | **solace-messaging-feedback** | Formats the current session into copy-paste-ready feedback about the skills, routed by support-contract status: contract holders get an email to Solace Support for bugs and a Solace Ideas portal submission for feature ideas, users without a contract get a Solace Community post for both. Formatter only: it produces the text for you to review and send or post, and never sends anything itself. | @@ -62,8 +62,9 @@ agent-plugins/ │ │ └── SKILL.md # Navigation-only manifest; reads the live docs page │ └── solace-messaging-feedback/ # Feedback formatter skill │ └── SKILL.md # Formats and routes session feedback (Support, Community, or Ideas portal) -├── tools/ # Re-runnable scripts (check-links, run-trigger-evals, run-output-evals) +├── tools/ # CI check scripts and the trigger and output eval runners ├── README.md # This file +├── CLAUDE.md # Orientation for AI coding agents that work in this repository └── .claude-plugin/ # Claude marketplace definition (marketplace.json) ``` @@ -73,6 +74,10 @@ Each plugin ships a trigger eval corpus under `plugins//evals/` that che Each plugin also ships an output eval corpus in the same directory that checks a skill's output honors the skill contract (design before code, the door question, scrubbed feedback drafts, grounded answers). Output evals run locally only. See the [output evals section](plugins/solace-messaging-skills/evals/README.md#output-evals) of the same README for the corpus format, the graders, and how to run them. +## Contributing + +This repository does not accept external pull requests or GitHub issues. If you have a Solace support contract, report bugs to Solace Support and suggest features on the [Solace Ideas portal](https://ideas.solace.com/ideas). Otherwise, post both on the [Solace Community](https://community.solace.com/). The `solace-messaging-feedback` skill formats a session into a ready-to-send report for each channel. + ## License This project is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file for details. From 772d8a73f81f1eeb6ee34d09f6438fe43a5d220c Mon Sep 17 00:00:00 2001 From: Adi El-Sammak <71227923+aelsammak@users.noreply.github.com> Date: Wed, 14 Oct 2026 13:29:25 -0400 Subject: [PATCH 05/12] EBP-3106: Add Python direct leaves and the Solace Suggested and Custom overlays (#12) * EBP-3106: Add Python direct leaves and the Solace Suggested and Custom overlays - Add implement-direct-pubsub.md and implement-request-reply.md leaf wiring files for the Python API, mirroring the JCSMP leaf files and their VERIFY: marker strings; the blocking requestor is the default and the non-blocking requestor is a variant - Add solace-suggested-mode.md (TLS with explicit certificate checks and the developer's trust store directory, the HA question mapped to the RetryStrategy values, DMQ eligibility on the persistent publisher, separate Python projects) and custom-mode.md (the same five a la carte knobs as JCSMP) - Remove the "not yet available" stops for the direct leaves and the hardened doors from implement-mode.md and design-mode.md, and extend the run commands, the manual-run rule, and the interim checklist to every leaf and door - Route a prompt that needs a persisted request to Guaranteed Pub/Sub with an application correlation id, never to Request-Reply (Direct) - Explain the trust store to the developer: the Python API has no default trust store, so a tcps:// host fails at build() without one; Quickstart now asks plaintext or secure and gives the same guidance on a secure host - Note that the Python API re-applies direct subscriptions on reconnect, so the JCSMP reapply-subscriptions property has no Python equivalent * EBP-3106: Hold the design gate when the user asks to skip questions - State in the jcsmp.md and python.md design-contract gate that a request to skip the design or all questions does not waive the gate or the door question - Direct the shortest allowed path instead: derive the summary, present it, and wait for one confirm before writing any file - Sonnet failed appdev-code-before-confirm-negative in 2 of 2 runs before the change and passed 3 of 3 after it - Add an exclude_pattern to the appdev-quickstart-implement-full verify.sh grader so verify.sh text inside a heredoc file write is not counted as a run * EBP-3106: Align Python Custom TLS and checklist rules with JCSMP - Ask the scheme question on Custom when the secure-session knob is unticked, and give a tcps:// host the Quickstart trust store guidance, example config, next steps, and checklist items - Split the DMQ checklist item into the API eligibility flag and the broker-side DMQ, which always stays the developer's responsibility - Move the HA reconnect values to Delivered only when the HA-failover question is answered yes, matching the JCSMP checklist * EBP-3106: Generate only the request leg for a persisted Python request - Record only the application correlation id in the design summary, and tell the developer the answer path is theirs to build - Add a persisted-request variant to the Guaranteed Pub/Sub leaf: the publisher sets CORRELATION_ID, no reply wiring or extra VERIFY markers, and the answer path is listed as developer-owned - Name the reconnect baseline on the unticked HA knob in the Python and JCSMP Custom modes, so the checklist no longer calls it the fail-fast API defaults --- .../evals/output-evals.json | 2 +- .../references/jcsmp.md | 2 +- .../references/jcsmp/custom-mode.md | 2 +- .../references/python.md | 9 +- .../references/python/custom-mode.md | 69 +++++++++++++ .../references/python/design-mode.md | 11 +-- .../python/implement-direct-pubsub.md | 38 ++++++++ .../python/implement-guaranteed-pubsub.md | 2 + .../references/python/implement-mode.md | 96 ++++++++++-------- .../python/implement-request-reply.md | 44 +++++++++ .../python/solace-suggested-mode.md | 97 +++++++++++++++++++ 11 files changed, 322 insertions(+), 50 deletions(-) create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/custom-mode.md create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-direct-pubsub.md create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-request-reply.md create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/solace-suggested-mode.md diff --git a/plugins/solace-messaging-skills/evals/output-evals.json b/plugins/solace-messaging-skills/evals/output-evals.json index 7b2a7ac..388a55b 100644 --- a/plugins/solace-messaging-skills/evals/output-evals.json +++ b/plugins/solace-messaging-skills/evals/output-evals.json @@ -89,7 +89,7 @@ { "type": "tool_use", "tool": "*", "input_pattern": "repo1\\.maven\\.org/maven2/com/solacesystems/sol-jcsmp/maven-metadata\\.xml", "expect": "present" }, { "type": "tool_use", "tool": "Bash", "input_pattern": "solrsearch|search\\.maven\\.org", "expect": "absent" }, { "type": "tool_use", "tool": "WebFetch", "input_pattern": "solrsearch|search\\.maven\\.org", "expect": "absent" }, - { "type": "tool_use", "tool": "Bash", "input_pattern": "verify\\.sh +(consumer|publisher|roundtrip|app)", "expect": "absent" }, + { "type": "tool_use", "tool": "Bash", "input_pattern": "verify\\.sh +(consumer|publisher|roundtrip|app)", "exclude_pattern": "<<.*verify\\.sh +(consumer|publisher|roundtrip|app)", "expect": "absent" }, { "type": "assistant_grep", "pattern": "./verify.sh", "fixed": true, "expect": "present" } ] }, diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp.md index f1d6cfb..4da00ea 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp.md @@ -23,7 +23,7 @@ If unclear, default to **Design**. Understand the messaging problem before gener Three gates hold on every path: -- **The design-contract gate.** Implement mode never starts without a design contract, and never invents design values. Exactly three sources satisfy it: (1) a summary Design mode confirmed in this session — use it as-is, with no re-confirm; (2) an explicit summary the developer supplied in chat (the eight fields or an equivalent statement of them) — use it directly; (3) a saved `solace-design.md` in the project — restate it in one line and proceed on confirm. A prose build request that merely mentions pattern details is NOT a contract: route it through Design mode, whose fully-specified path derives the summary, echoes it, and takes one confirm. The one carve-out is the mechanical-edit row above. An experienced developer therefore never has to walk the design tree, but never skips the contract either. +- **The design-contract gate.** Implement mode never starts without a design contract, and never invents design values. Exactly three sources satisfy it: (1) a summary Design mode confirmed in this session — use it as-is, with no re-confirm; (2) an explicit summary the developer supplied in chat (the eight fields or an equivalent statement of them) — use it directly; (3) a saved `solace-design.md` in the project — restate it in one line and proceed on confirm. A prose build request that merely mentions pattern details is NOT a contract: route it through Design mode, whose fully-specified path derives the summary, echoes it, and takes one confirm. The one carve-out is the mechanical-edit row above. An experienced developer therefore never has to walk the design tree, but never skips the contract either. A request to skip the design, skip all questions, or generate the full project right now does not waive this gate or the door question below: write no file before the confirm. Take the shortest path the gate allows instead: derive the summary from the prompt, present it, say in one line that one confirm starts the build, and wait for that confirm. - **Implement mode opens with a mandatory door question.** Ask Quickstart, Solace Suggested, or Custom — before you request or accept ANY broker details (`jcsmp/implement-mode.md` Step 0). Ask it even when a `config.json` with credentials already exists, even when a local broker is already running, and even when the requested app is bigger than the canonical generated shape; none of those answers the question. A confirmed design summary does not answer it either: the door question always follows the design contract, never merges into it. - **Environment discovery never answers a question.** A running broker container, an existing config file, or found credentials are facts to report, not answers to consume. Report what you found, then still ask which broker the developer wants to target. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/custom-mode.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/custom-mode.md index e80db0c..4af578b 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/custom-mode.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/custom-mode.md @@ -13,7 +13,7 @@ When the developer picks Custom, establish the design summary first (per `implem Custom offers FIVE knobs, and only these five. There is no sixth knob and no authentication knob; Basic username/password is the only auth this journey generates, exactly as on Quickstart and Solace Suggested, so it is never a Custom checklist item. Four of the five knobs reuse a `solace-suggested-mode.md` step; the fifth is defined in this file: 1. **Secure session (TLS).** Apply `solace-suggested-mode.md` Step 1, the Secure branch: a `tcps://` host with explicit certificate validation. Do not restate that logic here; the overlay step is the source of truth. -2. **HA channel values.** Apply `solace-suggested-mode.md` Step 2: the HA-failover question and the four reconnect channel values it gates. +2. **HA channel values.** Apply `solace-suggested-mode.md` Step 2: the HA-failover question and the four reconnect channel values it gates. Unticked keeps the samples' reconnect baseline (`reconnectRetries` 20, `connectRetriesPerHost` 3), not the fail-fast default-channel posture; ticked applies the four HA values when the HA-failover question is answered yes. 3. **DMQ eligibility.** Apply `solace-suggested-mode.md` Step 3: `setDMQEligible(true)` on every PERSISTENT publish, with the broker-side prerequisite stated as the developer's responsibility. 4. **Decoupled projects.** Apply `solace-suggested-mode.md` Step 4: two separate, independent single-module Maven projects, one per role. 5. **Admin-provisioned queue.** The one knob that is a real code change rather than a reused overlay step. It is defined in full in the "Admin-provisioned-queue knob" section below. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python.md index d22ee78..a726fa3 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/python.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python.md @@ -23,7 +23,7 @@ If unclear, default to **Design**. Understand the messaging problem before gener Three gates hold on every path: -- **The design-contract gate.** Implement mode never starts without a design contract, and never invents design values. Exactly three sources satisfy it: (1) a summary Design mode confirmed in this session: use it as-is, with no re-confirm; (2) an explicit summary the developer supplied in chat (the eight fields or an equivalent statement of them): use it directly; (3) a saved `solace-design.md` in the project: restate it in one line and proceed on confirm. A prose build request that merely mentions pattern details is NOT a contract: route it through Design mode, whose fully-specified path derives the summary, echoes it, and takes one confirm. The one carve-out is the mechanical-edit row above. An experienced developer therefore never has to walk the design tree, but never skips the contract either. +- **The design-contract gate.** Implement mode never starts without a design contract, and never invents design values. Exactly three sources satisfy it: (1) a summary Design mode confirmed in this session: use it as-is, with no re-confirm; (2) an explicit summary the developer supplied in chat (the eight fields or an equivalent statement of them): use it directly; (3) a saved `solace-design.md` in the project: restate it in one line and proceed on confirm. A prose build request that merely mentions pattern details is NOT a contract: route it through Design mode, whose fully-specified path derives the summary, echoes it, and takes one confirm. The one carve-out is the mechanical-edit row above. An experienced developer therefore never has to walk the design tree, but never skips the contract either. A request to skip the design, skip all questions, or generate the full project right now does not waive this gate or the door question below: write no file before the confirm. Take the shortest path the gate allows instead: derive the summary from the prompt, present it, say in one line that one confirm starts the build, and wait for that confirm. - **Implement mode opens with a mandatory door question.** Ask Quickstart, Solace Suggested, or Custom, before you request or accept ANY broker details (`python/implement-mode.md` Step 0). Ask it even when a `config.json` with credentials already exists, even when a local broker is already running, and even when the requested app is bigger than the canonical generated shape; none of those answers the question. A confirmed design summary does not answer it either: the door question always follows the design contract, never merges into it. - **Environment discovery never answers a question.** A running broker container, an existing config file, or found credentials are facts to report, not answers to consume. Report what you found, then still ask which broker the developer wants to target. @@ -68,7 +68,12 @@ Live exceptions (these have no `docs.solace.com` `.md` form; use the live URL di - `python/design-mode.md`: choose and confirm the messaging pattern and topology before any code is generated; every new build without a design contract routes here first. - `python/implement-mode.md`: generate a runnable Python project from a confirmed design; its Step 4 dispatches onto the per-leaf wiring files below. - `python/implement-guaranteed-pubsub.md`: the Guaranteed Pub/Sub leaf wiring (read from implement-mode Step 4). +- `python/implement-direct-pubsub.md`: the Direct Pub/Sub leaf wiring (read from implement-mode Step 4). +- `python/implement-request-reply.md`: the Request-Reply (Direct) leaf wiring (read from implement-mode Step 4). +- `python/solace-suggested-mode.md`: the leaf-agnostic Solace Suggested overlay (TLS secure session with the developer's trust store, DMQ eligibility on persistent publishes, HA failover, separate Python projects) reached from implement-mode Step 0 on the Solace Suggested path. +- `python/custom-mode.md`: the thin a la carte Custom overlay reached from implement-mode Step 0 on the Custom path; a leaf-aware checklist of the hardening knobs (secure session, DMQ, HA, decoupled projects, admin-provisioned queue) that generates exactly the ticked subset, reusing the Solace Suggested overlay steps for the mapped knobs. - `python/python-guaranteed-publisher-sample.py`: best-practices publisher sample (basic-auth connect, PERSISTENT publish to a topic, publish receipt handling, graceful shutdown). - `python/python-guaranteed-subscriber-sample.py`: best-practices receiver sample (provisions a durable queue plus a topic subscription, CLIENT-ack receiver). - `python/python-solace-connection-config.py`: the shared connection-config helper every generated app uses. -- `python/python-direct-*-sample.py`: the direct pub/sub and direct request-reply samples. Implement mode does not generate those leaves yet (see `python/implement-mode.md` Step 4). +- `python/python-direct-publisher-sample.py` and `python/python-direct-subscriber-sample.py`: the direct pub/sub samples (bounded-buffer publish with a publish failure listener; direct topic subscription with discard detection). +- `python/python-direct-requestor-blocking-sample.py`, `python/python-direct-requestor-sample.py`, and `python/python-direct-replier-sample.py`: the direct request-reply samples (blocking and non-blocking requestor; replier answering with `replier.reply()`). diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/custom-mode.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/custom-mode.md new file mode 100644 index 0000000..2601432 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/custom-mode.md @@ -0,0 +1,69 @@ +# Custom Mode + +The Custom overlay for Implement mode of the Python API. This file is reached from `implement-mode.md` Step 0 when the developer chooses the Custom door. Custom is a la carte: the developer ticks which hardening knobs to include, and generation applies exactly the ticked subset and nothing more. It is a thin overlay. It does NOT re-describe how each knob works. For the four knobs that map to a `solace-suggested-mode.md` step, it points at that step and reuses it, so `solace-suggested-mode.md` stays the single source of truth for each hardening delta. + +This overlay defers to `implement-mode.md` for all leaf and sample mechanics, exactly the way `solace-suggested-mode.md` does: the Step 2 design-summary input contract, the Step 4 leaf dispatch (which leaf wiring file and samples each leaf reads, and the near-verbatim adaptation discipline), the AI-assisted disclaimer header on every generated script, explicit imports (never `from solace.messaging... import *`), and the fixed `VERIFY:` marker contract. Custom changes NO `VERIFY:` marker string. Any live run on the Custom path prints the same one-line live-broker heads-up (SKILL.md Invariant 6) immediately before it starts: name the broker host and the run's actual broker-side effects. + +## The checklist model: one up-front multi-select + +When the developer picks Custom, establish the design summary first (per `implement-mode.md` Step 2, or reuse one already in hand), THEN present a SINGLE up-front multi-select checklist of the knobs that apply to the resulting `Pattern` leaf, capture the ticked subset once, and generate exactly that subset. Do NOT walk the knobs one at a time as sequential per-knob questions. One checklist, one capture, one generation pass. The property that defines Custom is "exactly the ticked subset, nothing more", so a knob that was not ticked leaves the generated output at its base shape for that concern. + +## The five knobs + +Custom offers FIVE knobs, and only these five. There is no sixth knob and no authentication knob; basic username/password is the only auth this journey generates, exactly as on Quickstart and Solace Suggested, so it is never a Custom checklist item. Four of the five knobs reuse a `solace-suggested-mode.md` step; the fifth is defined in this file: + +1. **Secure session (TLS).** Apply `solace-suggested-mode.md` Step 1: a `tcps://` host, explicit certificate validation, and the developer's trust store directory in `config.json`. Do not restate that logic here; the overlay step is the source of truth. +2. **HA reconnect values.** Apply `solace-suggested-mode.md` Step 2: the HA-failover question and the retry strategies and retries-per-host value it gates. Unticked keeps the samples' reconnect baseline (20 retries 3000 ms apart), not the fail-fast API defaults; ticked applies the HA values when the HA-failover question is answered yes. +3. **DMQ eligibility.** Apply `solace-suggested-mode.md` Step 3: `PERSISTENT_DMQ_ELIGIBLE` set explicitly on the persistent publisher's message template, with the broker-side prerequisite stated as the developer's responsibility. +4. **Decoupled projects.** Apply `solace-suggested-mode.md` Step 4: separate, independent Python projects, one per role. +5. **Admin-provisioned queue.** The one knob that is a real code change rather than a reused overlay step. It is defined in full in the "Admin-provisioned-queue knob" section below. + +For knobs 1 through 4, point at the named overlay step and reuse it. Do NOT duplicate the knob logic in this file. + +## The Quickstart-shaped floor + +Zero knobs ticked yields a Quickstart-shaped base app: one Python project with both scripts, a self-provisioned durable queue on the guaranteed leaves, the Quickstart transport rule (the scheme the developer's host carries, asked in `implement-mode.md` Step 1), and the `config.json`-or-command-line connection convention. Each ticked knob layers its Suggested delta on top of that floor. There is NO minimum-one-knob guard: zero ticks is a valid result, and it is the Quickstart-shaped base app. One exception to the Quickstart shape: the floor KEEPS the reference samples' reconnect baseline (20 retries 3000 ms apart); the fail-fast API-default posture is the Quickstart door's alone, and on Custom the HA knob decides between that baseline (unticked) and the HA values (ticked). Custom therefore spans from about Quickstart (nothing ticked) to about Solace Suggested (everything ticked). + +When the secure-session knob is unticked, the base follows the Quickstart transport rule: `implement-mode.md` Step 1 asks the scheme. On a plaintext `tcp://` host the base carries an honest plaintext caveat: unencrypted transport, suitable for a dev or test broker with no TLS, not for real credentials or real data over untrusted networks. On a `tcps://` host, apply the Quickstart trust store guidance from `implement-mode.md` Step 0 and never downgrade the host to `tcp://`. When the secure-session knob is ticked, the session upgrades to the `solace-suggested-mode.md` Step 1 `tcps://` path. + +The command-line fallback of the shared helper follows the knobs: it stays when both the secure-session and the decoupled-projects knobs are unticked (the Quickstart convention), and it is dropped, as on Solace Suggested, when either one is ticked, so the trust store path and the per-project credentials come only from each project's gitignored `config.json`. The helper requires the trust store key only when the secure-session knob is ticked. With the knob unticked on a `tcps://` host, the helper passes the trust store key through from `config.json` as on Quickstart, and the command-line fallback cannot carry it. + +## Leaf-aware offering + +Only offer knobs that apply to the design summary's chosen `Pattern` leaf. Read the `Pattern` field the same way `implement-mode.md` Step 2 does, then filter the checklist: + +- Secure session, HA reconnect values, and decoupled projects apply on EVERY leaf. +- DMQ eligibility and the admin-provisioned queue apply ONLY on the leaves that carry persistent delivery and a durable queue: the three Guaranteed Pub/Sub leaves. The Python API has no guaranteed request-reply leaf. +- HIDE both DMQ eligibility and the admin-provisioned queue on the direct at-most-once leaves (Direct Pub/Sub and Request-Reply (Direct)). Those leaves have no queue, so both knobs would be a silent no-op there; a developer must not be able to tick a knob that does nothing. + +## Admin-provisioned-queue knob (the one real code change) + +Ticking the admin-provisioned-queue knob changes the generated code so the subscriber binds a pre-existing, administrator-owned queue instead of provisioning its own. The administrator owns the queue lifecycle AND its topic subscription on the broker; the subscriber only binds a persistent receiver to that already-provisioned, already-subscribed endpoint. Ground it in [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md) (`CREATE_ON_START` creates the queue given to `build()`, and the default, `DO_NOT_CREATE`, disables any attempt to create missing resources) and [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md). WebFetch these live pages on demand; do not paste doc content. + +This is a generation-time OMISSION from the generated output. It is NEVER an edit to any `.py` reference sample and never touches any `VERIFY:` marker string. + +On **Guaranteed Pub/Sub**, ticking the knob edits the subscriber's generation. DROP these two from the generated subscriber's receiver builder: + +- `with_missing_resources_creation_strategy(MissingResourcesCreationStrategy.CREATE_ON_START)` and its then-unused import (the in-process queue creation). +- `with_subscriptions([TopicSubscription.of(topic)])` and its then-unused import (the client-side topic-to-queue mapping; the administrator configured this subscription on the broker). + +KEEP the rest of the subscriber unchanged: + +- The local `Queue.durable_exclusive_queue(queue_name)` or `Queue.durable_non_exclusive_queue(queue_name)` handle from the `Access type` field (the receiver still needs a queue object to bind to; the access type itself is now the administrator's broker setting, so state in a comment that it must match). +- `with_message_client_acknowledgement()`, `with_activation_passivation_support(...)`, `receive_async(...)`, `start()`, and `ack()` after processing. +- The sample's `PubSubPlusClientError` branch around `start()`: it now reports a queue the administrator has not provisioned. + +All of the Guaranteed Pub/Sub leaf's `VERIFY:` markers still emit, unchanged: `VERIFY: CONNECTED` after connect, `VERIFY: QUEUE_BOUND` after `start()`, and `VERIFY: MESSAGE_RECEIVED` after `ack()`. The publisher is untouched, so its `VERIFY: PUBLISH_ACKED` is unaffected. On the fan-out leaf, the drop applies to every subscriber script. + +Because the ticked app no longer provisions anything, a round trip needs the administrator to have provisioned the queue and its subscription first. Say so in the handed-back next steps and record it in the checklist under Your responsibility. + +## Return to the implement-mode contracts + +After capturing the ticked subset and applying the deltas above, return to `implement-mode.md` for the unchanged leaf mechanics: the Step 4 dispatch generates the chosen leaf's scripts and writes the tailored `solace-verification-checklist.md` into each project as generation output (one copy per project when the decoupled-projects knob is ticked, otherwise one at the project root), and Step 6 resolves each item to its state for the Custom ticked subset, invites the developer's additions, and reports the same in chat. This overlay selects which hardening deltas to layer; it does not replace those contracts. + +## Grounding references + +Live `docs.solace.com` `.md` pages (WebFetch on demand): + +- [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md) +- [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md) diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/design-mode.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/design-mode.md index 32ac8ff..f3caa2b 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/design-mode.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/design-mode.md @@ -17,6 +17,7 @@ Design mode is ask-or-recommend, not interrogate-first. Before walking the tree, | Stated intent in the prompt | Recommended leaf (or branch to lock if partial) | |---|---| | "request a reply", "needs a correlated response", "ask and wait for an answer" | Request-Reply (Direct) (the delivery question still applies; see the no-guaranteed-request-reply rule below) | +| "the request must not be lost", "the request must survive the responder being down", "persistent request" | a Guaranteed Pub/Sub leaf with an application correlation id (the no-guaranteed-request-reply rule below; never Request-Reply (Direct)) | | "fan out to many services", "broadcast to several independent apps", "every team gets its own copy" | Guaranteed Pub/Sub (fan-out) | | "competing workers", "share the load across consumers", "scale out processing of one stream" | Guaranteed Pub/Sub (single service, non-exclusive) | | "one active consumer", "strict ordering", "only one worker at a time" | Guaranteed Pub/Sub (single service, exclusive) | @@ -46,20 +47,18 @@ Ask only the questions the prompt left open, in this order. Before you answer ea Questions 3 and 4 apply ONLY on the guaranteed publish/subscribe branch. Request-reply and direct publish/subscribe resolve at the delivery-guarantee answer and never reach them. -**No guaranteed request-reply in the Python API.** When question 1 resolves to Request-Reply and question 2 resolves to guaranteed, there is no Python leaf for it. WebFetch [Request-Reply Messaging in the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Request-Reply.md) and quote its note that request-reply messaging can only be used with direct messages in the Python API. Then give the developer the choice in one question: Request-Reply (Direct), which keeps the request-reply shape and accepts direct delivery, or a Guaranteed Pub/Sub leaf, which keeps persistence and leaves the reply correlation to the application. Never invent a guaranteed request-reply leaf and never record `Request-Reply (Guaranteed)` in a Python summary. +**No guaranteed request-reply in the Python API.** When question 1 resolves to Request-Reply and question 2 resolves to guaranteed (the request must be persisted), there is no Python request-reply leaf for it. WebFetch [Request-Reply Messaging in the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Request-Reply.md) and quote its note that request-reply messaging can only be used with direct messages in the Python API. Then route to Guaranteed Pub/Sub, never to Request-Reply (Direct): the request travels as a persistent message to a durable queue, and the application sets a correlation id on it and matches the answer by that id. Continue the tree at question 3 to pick the Guaranteed Pub/Sub leaf, then record the design as a VARIANT of that leaf (Step 1): name the application correlation id in the `Consumption endpoint` field with a short inline reason, for example `durable queue q.acme.orders + topic subscription; application correlation id (persisted request, no Python guaranteed request-reply)`. Tell the developer that Implement generates the request leg only: the answer path (how the consumer returns the answer and how the requestor matches it by the correlation id) is theirs to build, and the checklist records it as developer-owned. Never offer Request-Reply (Direct) for a request that must be persisted, never invent a guaranteed request-reply leaf, and never record `Request-Reply (Guaranteed)` in a Python summary. ## Step 3: Resolve to one of the five leaves The tree resolves to exactly five leaves. Spell the chosen leaf in the summary exactly as written here; these strings are the contract Implement mode reads. -- **Request-Reply (Direct)**. Request-reply over direct messaging. -- **Direct Pub/Sub**. Publish/subscribe over direct messaging, fire-and-forget. +- **Request-Reply (Direct)** GENERATED. Request-reply over direct messaging. +- **Direct Pub/Sub** GENERATED. Publish/subscribe over direct messaging, fire-and-forget. - **Guaranteed Pub/Sub (single service, exclusive)** GENERATED. One durable queue, exclusive access, one active consumer at a time. - **Guaranteed Pub/Sub (single service, non-exclusive)** GENERATED. One durable queue, non-exclusive access, competing workers share the load. - **Guaranteed Pub/Sub (fan-out)** SNIPPET. One durable queue plus a topic subscription per independent consuming service. Implement mode reuses the single-service guaranteed generator, so there is no dedicated fan-out sample; the leaf is realized by repeating the single-service queue-and-subscription setup once per consuming service. -The two direct leaves are valid designs, and Design mode confirms and saves them like any other leaf, but Implement mode does not generate them in this release (see `implement-mode.md` Step 4). When the confirmed leaf is one of them, say so in the Step 4 close-out instead of offering to move into Implement mode. - Inside the guaranteed publish/subscribe single-service leaf, the exclusive versus non-exclusive choice is realized as a queue-factory SNIPPET, not a separate sample: the same generated app builds an exclusive or a non-exclusive durable queue based on the developer's answer to question 4. **A leaf can carry a variant.** The five strings name the pattern family; a design may adapt a leaf's default topology to a stated requirement (the variant path in Step 1). In every case the `Pattern` string stays one of the five verbatim, and the deviation is recorded in the topology fields of the Step 4 summary (for example the `Consumption endpoint`), never by inventing a sixth leaf string. @@ -104,5 +103,3 @@ Field semantics: - **Grounding docs.** ONLY the pages this session actually WebFetched. Never list a page you did not fetch. Write `none fetched` when no page was fetched. After presenting the summary in chat, close Design mode explicitly so the developer knows exactly what happens next; do NOT just display the summary and stop. On the fully-specified path (Step 1) this close-out is the ONLY confirm of the run; it carries the leaf confirmation too. If the design is a variant of its leaf, state the deviation and its rationale in one line here too. Ask them directly, in one step, both whether they are happy with this design AND whether to save it to `solace-design.md` in their project (for example: "Happy with this design? If so, I can save it to `solace-design.md` and move into Implement mode to generate the runnable Python project."). Write the file only on their OK; never write it unprompted. Once they confirm, state the next step plainly: the work moves into Implement mode, which generates the runnable Python project from this summary. Implement mode treats this summary, whether it lives in the chat or in the saved `solace-design.md`, as its input contract. Implement mode OPENS with its own Step 0 door question (Quickstart, Solace Suggested, or Custom), asked before any broker details are requested or accepted; an existing `config.json` or a running broker does not answer it, so ask it on the way in. - -For a direct leaf (Direct Pub/Sub or Request-Reply (Direct)), the close-out differs: offer to save the summary to `solace-design.md`, and say in one line that generating a Python project for this leaf is not yet available in this release, so the saved design is ready for when it is. Do not offer to move into Implement mode, do not generate code from the direct samples by hand, and do not offer to write the code another way (from scratch, "outside the generator", or as unverified code). When the developer asks for the code anyway, repeat that generation for this leaf is not yet available and point them at the [Python tutorials](https://tutorials.solace.dev/python/); write no code. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-direct-pubsub.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-direct-pubsub.md new file mode 100644 index 0000000..53488c5 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-direct-pubsub.md @@ -0,0 +1,38 @@ +# Implement: Direct Pub/Sub leaf + +The per-leaf wiring for the Direct Pub/Sub leaf of the Python API. Read this file from `implement-mode.md` Step 4 when the `Pattern` field is `Direct Pub/Sub`. The shared Step 4 rules in `implement-mode.md` (the disclaimer header, the shared connection helper, the fixed file names, the concurrency rules, the variant discipline, the demo-harness rule, the comments rule, and the Quickstart fail-fast rule) apply here unchanged. + +## Direct pub/sub flow + +Generation reads the direct pub/sub sample pair and adapts each into its own generated script, the same near-verbatim discipline the guaranteed flow uses: `direct_publisher.py` is a near-verbatim adaptation of the publisher sample, and `direct_subscriber.py` is a near-verbatim adaptation of the subscriber sample. Direct messaging is at-most-once: there is no broker receipt and no redelivery, so this leaf has NO queue, NO provisioning, NO client acknowledgement, and NO publish receipt listener. + +- `python-direct-publisher-sample.py`: basic-auth connect, a continuous DIRECT binary publish loop with a rotating payload to a dynamic topic, a publish failure listener for the asynchronous failures the API reports, capacity-bounded back pressure, service event listeners registered before `connect()`, and a graceful SIGINT/SIGTERM shutdown. No receipt listener and no user context (direct has no broker receipt). Generation STRIPS the continuous loop, the rotating payload, the dynamic topic, and the stats thread down to a short bounded burst that self-exits (publisher steps below). +- `python-direct-subscriber-sample.py`: basic-auth connect, a plain topic subscription on a direct receiver (no queue, no provisioning), capacity-bounded back pressure with discard detection, service event listeners registered before `connect()`, and a graceful SIGINT/SIGTERM shutdown. The subscriber is the long-running SIGINT target. + +Each script builds its OWN `MessagingService` (two services, one per script, matching the samples). Keep the samples' structure: the shared state dataclass, `setup_solace(...)` that builds the service and the receiver or publisher before any connection exists, `connect_solace(...)`, the main loop, and `teardown_solace(...)` in `main()`'s `finally`. Keep every listener class; they are part of the best-practices contract and must not be stripped during adaptation. + +**Ground before you write.** Before writing either script, WebFetch [Publishing Direct Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-DM-Publish.md) and [Consuming Direct Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-DM-Receive.md) in this session (SKILL.md Invariant 2). + +## Subscriber script (`direct_subscriber.py`) + +The subscriber subscribes directly to the topic (no queue), receives asynchronously, and is the long-running process the developer runs FIRST. Start the file with the disclaimer header, then wire it in this sequence: + +1. Build the connection properties with `load_service_properties(args, APP_NAME)`, then the service with `MessagingService.builder().from_properties(properties).build()` (on Quickstart without the reconnection strategy call, per the fail-fast rule). +2. Build the direct receiver as the sample does: `create_direct_message_receiver_builder()` with `with_subscriptions([TopicSubscription.of(topic)])` for the design summary's topic and `on_back_pressure_drop_latest(RECEIVE_BUFFER_CAPACITY)`, then `build()`. Register the three service event listeners BEFORE `connect()`. There is NO queue and NO missing-resources creation strategy on the direct path. +3. `messaging_service.connect()`. Immediately after `connect()` returns, emit the connect marker: `trace("VERIFY: CONNECTED")`. +4. `receiver.receive_async(handler)` BEFORE `receiver.start()`, as the sample orders them, then `receiver.start()`. `start()` applies the subscription and blocks until the broker confirms it. Immediately after `start()` returns, emit the subscribed marker: `trace("VERIFY: SUBSCRIBED")`. **No reapply-subscriptions setting.** The JCSMP direct subscriber must set `REAPPLY_SUBSCRIPTIONS` to `true`, or its subscription is lost on an automatic reconnect. The Python API has no such setting and needs none: the [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) page states that the API re-applies the application's direct subscriptions on reconnect. Do not search for or invent a Python equivalent. Keep the sample's `on_reconnected` comment that says so. When reconnect attempts are exhausted, the same page states that the application must recreate the session with all its subscriptions; the sample's `on_service_interrupted` handler covers that case by shutting down. +5. In the handler's `on_message`, keep the sample's payload read and its discard check (`get_message_discard_notification()`), then emit the receipt marker: `trace("VERIFY: MESSAGE_RECEIVED")`. There is NO `ack()` on the direct path. +6. KEEP the long-running wait loop, the SIGINT/SIGTERM handler that only sets the shutdown event, and `teardown_solace(...)` with `terminate(grace_period)` and `disconnect()`. The subscriber is the SIGINT target, and its shutdown line (`Shutdown signal received ...`) stays a `trace(...)` call. + +## Publisher script (`direct_publisher.py`) + +The publisher only connects and publishes a short bounded burst of DIRECT messages, then exits on its own. It does NOT provision anything and does NOT wait for a key press or a signal. Start the file with the disclaimer header, then wire it in this sequence: + +1. Build the connection properties and the service exactly as the subscriber does (step 1 above), then the publisher with `create_direct_message_publisher_builder().on_back_pressure_wait(PUBLISH_BUFFER_CAPACITY).build()`. Keep `set_publish_failure_listener(...)`: direct has no broker receipt, so a buffered message that cannot be sent is reported only through that listener. Register the three service event listeners BEFORE `connect()`. +2. `messaging_service.connect()`. Immediately after `connect()` returns, emit the connect marker: `trace("VERIFY: CONNECTED")`. Then `publisher.start()`. +3. Publish a SHORT BOUNDED BURST to the design summary's topic with `publish(payload, Topic.of(topic))` in a small fixed-count loop, then return from the publish step. REMOVE the sample's continuous loop, its rotating payload and dynamic topic, and its stats thread. Keep the sample's `PubSubPlusClientError` branch, which sets the shared state's `exit_code` to 1. `teardown_solace(...)` in `main()`'s `finally` calls `terminate(grace_period)`, which gives the buffered messages up to the grace period to leave before `disconnect()`. Keep the SIGINT/SIGTERM handler, so Ctrl-C during a back pressure wait still reaches teardown. When the request describes a real application, its own messages replace the burst (the demo-harness rule). +4. The publisher emits NO send-side marker. Direct has no broker receipt, so there is no `PUBLISH_ACKED` analog; a marker after `publish()` would only prove the local call returned. The proof of the round trip is `VERIFY: MESSAGE_RECEIVED` in the subscriber output. + +## The marker contract + +These three `VERIFY:` marker strings (`VERIFY: CONNECTED`, `VERIFY: SUBSCRIBED`, `VERIFY: MESSAGE_RECEIVED`) are the same strings the JCSMP Direct Pub/Sub leaf emits, character for character. They are read PER PROCESS: the subscriber output carries `SUBSCRIBED` and `MESSAGE_RECEIVED`, and both scripts print `CONNECTED`. Until the Python verify script lands, the developer reads them from the two terminals (`implement-mode.md` Step 5); the verify script will read the same strings, so never change one. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-guaranteed-pubsub.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-guaranteed-pubsub.md index 2eed182..aaa19cf 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-guaranteed-pubsub.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-guaranteed-pubsub.md @@ -48,6 +48,8 @@ The publisher only connects and publishes; it does NOT provision the queue or th 4. Publish persistent messages to the design summary's topic as the sample does: one `message_builder()` template with `PERSISTENT_TIME_TO_LIVE` set once, `build(payload, additional_message_properties={APPLICATION_MESSAGE_ID: msg_id})` per message, and `publish(message, topic, user_context=msg_id)`. See [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md). 5. EXIT on its own: replace the sample's continuous publish loop with the application's own messages, or with one message on a try-it run, then return from the publish step. `teardown_solace(...)` in `main()`'s `finally` calls `terminate(grace_period)`, which sends what is buffered and waits for the broker's receipt of every published message, so the `VERIFY: PUBLISH_ACKED` marker prints before the script exits. Drop the stats thread with the loop. Keep the SIGINT/SIGTERM handler, so Ctrl-C during a back pressure wait still reaches teardown. +**Persisted-request variant (application correlation id).** When the `Consumption endpoint` field records an application correlation id (a persisted request that `design-mode.md` routed here), generate the request leg only. The publisher adds `message_properties.CORRELATION_ID: msg_id` to the same `additional_message_properties` dict in step 4, and the subscriber stays unchanged. Do NOT generate an answer path: no reply topic, no reply subscription, no reply publish, and no `VERIFY:` marker beyond the four below. List the answer path under Developer-owned items in the checklist (`implement-mode.md` Step 6), and name it in the close-out as the developer's next step. + ## The marker contract These four `VERIFY:` marker strings (`VERIFY: CONNECTED`, `VERIFY: PUBLISH_ACKED`, `VERIFY: QUEUE_BOUND`, `VERIFY: MESSAGE_RECEIVED`) are the same strings the JCSMP guaranteed pub/sub leaf emits, character for character. They are read PER PROCESS: the subscriber output carries `QUEUE_BOUND` and `MESSAGE_RECEIVED`, the publisher output carries `PUBLISH_ACKED`, and both print `CONNECTED`. Until the Python verify script lands, the developer reads them from the two terminals (`implement-mode.md` Step 5); the verify script will read the same strings, so never change one. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-mode.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-mode.md index 5b9eff4..5f5d465 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-mode.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-mode.md @@ -1,12 +1,12 @@ # Implement Mode -Generate a complete, runnable Python project from a confirmed design summary. The canonical output is TWO scripts (a publisher and a subscriber) plus the shared connection-config helper in ONE project directory with its own virtual environment. Each script is a near-verbatim adaptation of its reference sample. Near-verbatim adaptation of two samples is more reliable than merging them into one program, and a real deployment lifts each script into its own service or project. +Generate a complete, runnable Python project from a confirmed design summary. The canonical output is TWO scripts (a publisher and a subscriber, or a requestor and a replier) plus the shared connection-config helper in ONE project directory with its own virtual environment. Each script is a near-verbatim adaptation of its reference sample. Near-verbatim adaptation of two samples is more reliable than merging them into one program, and a real deployment lifts each script into its own service or project. Implement mode is NOT only for the canonical shape. A web app, a dashboard, a service, or any embedded shape where the Python messaging layer lives inside a larger application follows the SAME steps: the same door question, the same leaf wiring for the messaging layer, the same generation outputs, and the same verification fallback. Building an off-catalog shape never waives a step in this file. Work through the steps in order. Read this file's grounding links only as each step needs them. Link the canonical doc and state the decision in one line, never paraphrase doc content. Resolve every dependency version at generation time; pin nothing in skill content (SKILL.md Invariant 3). -**Coverage in this release.** Implement mode generates the three Guaranteed Pub/Sub leaves on the Quickstart door. The Direct Pub/Sub and Request-Reply (Direct) leaves (Step 4) and the Solace Suggested and Custom doors (Step 0) are not yet available for Python. The Python API does not bundle a verify script yet, so every run ends in the compile-only fallback of Step 5. +**Coverage in this release.** Implement mode generates all five Python leaves on all three doors. The Python API does not bundle a verify script yet, so every run ends in the compile-only fallback of Step 5. ## Generation outputs: the file contract @@ -15,13 +15,13 @@ Every Implement run writes ALL of these as generation output, on every app shape 1. The generated scripts for the chosen leaf, plus the shared `solace_connection_config.py` helper. Every source file starts with the AI-assisted disclaimer header (Step 4). 2. `requirements.txt` with the `solace-pubsubplus` version resolved per Step 3. 3. `config.example.json` (placeholders) and a `.gitignore` that ignores `config.json` and the virtual environment (Step 4). -4. A tailored `solace-verification-checklist.md` at the project root (tailoring rules in Step 6; the WRITE happens here at generation time, like the scripts). +4. A tailored `solace-verification-checklist.md` at each project root (tailoring rules in Step 6; the WRITE happens here at generation time, like the scripts). There is no `verify.sh` and no `verify-hooks.sh` in a Python project yet. Do not copy the JCSMP `verify.sh` into a Python project: it drives Maven and cannot run Python scripts. ## Step 0: Ask Quickstart, Solace Suggested, or Custom, before any broker details -This is the FIRST question, and you ask it before you request or accept ANY broker connection details. Ask the developer plainly which of three doors they want: Quickstart, Solace Suggested, or Custom. Ask it as a direct question and wait for the answer. Frame the three doors honestly: Quickstart is a learning and try-it path to see a round-trip work, NOT a deployment target; Solace Suggested is the hardened baseline Solace suggests as a starting point (TLS, dead message queue eligibility, HA failover, separate projects), and Solace suggests it without guaranteeing it is ready for deployment; Custom lets the developer tick exactly which hardening knobs to include. State in the same question that Solace Suggested and Custom are not yet available for the Python API in this release. +This is the FIRST question, and you ask it before you request or accept ANY broker connection details. Ask the developer plainly which of three doors they want: Quickstart, Solace Suggested, or Custom. Ask it as a direct question and wait for the answer. Frame the three doors honestly: Quickstart is a learning and try-it path to see a round-trip work, NOT a deployment target; Solace Suggested is the hardened baseline Solace suggests as a starting point (TLS, dead message queue eligibility, HA failover, separate projects), and Solace suggests it without guaranteeing it is ready for deployment; Custom lets the developer tick exactly which hardening knobs to include and generates that ticked subset and nothing more. The answer decides WHERE broker connection details belong, and it drives behavior for the rest of Implement mode. Every door generates the same scripts for the chosen leaf plus the shared `solace_connection_config.py` helper; they differ in where the credential VALUES live: Solace Suggested puts them ONLY in a gitignored `config.json`, while Quickstart reads `config.json` when present and otherwise falls back to command-line arguments; Custom follows the Quickstart convention by default, and `custom-mode.md` says when it drops the command-line fallback. Three things do NOT answer this question, and none of them waives it: @@ -31,14 +31,17 @@ Three things do NOT answer this question, and none of them waives it: The doors: -- **Quickstart.** Each script builds its connection through the shared `solace_connection_config.py` helper, which reads the connection properties from a `config.json` in the working directory when present and otherwise falls back to the command-line arguments (` [password]`). Quickstart adds no TLS hardening and has no TLS requirement: it connects with whatever scheme the developer's host carries, and a plaintext `tcp://` dev broker is a normal Quickstart target. Generate the scripts without per-stage preview gates. Quickstart output fails fast (the fail-fast rule in Step 4). -- **Solace Suggested or Custom.** Not yet available for Python. Say so plainly, and do not describe the hardened doors beyond the one-line framing in the Step 0 question. Offer the developer two honest choices: Quickstart now, framed as a learning run, or stop here with the design saved to `solace-design.md` for when the hardened doors are available. Never generate a Quickstart project under the Solace Suggested or Custom name. +- **Quickstart.** Each script builds its connection through the shared `solace_connection_config.py` helper, which reads the connection properties from a `config.json` in the working directory when present and otherwise falls back to the command-line arguments (` [password]`). Quickstart adds no TLS hardening and has no TLS requirement: it connects with whatever scheme the developer's host carries, and a plaintext `tcp://` dev broker is a normal Quickstart target. A `tcps://` host (the Solace Cloud default) needs a trust store even on Quickstart, because the Python API has no default one and fails at `build()` with "Failed to load trust store" without it. Step 1 asks the developer whether the broker URL is plaintext or secure; when the answer is secure (`tcps://`), give the developer the trust store explanation from `solace-suggested-mode.md` Step 1 and have them set `solace.messaging.tls.trust-store-path` in `config.json`; the helper passes that key through unchanged, and the command-line fallback cannot carry it, so a `tcps://` host runs from `config.json`. Quickstart adds no validation code: the API defaults already validate the certificate. Never downgrade a `tcps://` host to `tcp://` to avoid the trust store: the plaintext port may be closed, the port differs, and the password would cross the network in clear text. Generate the scripts without per-stage preview gates. Quickstart output fails fast (the fail-fast rule in Step 4). +- **Solace Suggested.** On the Solace Suggested path, read `solace-suggested-mode.md` and follow the Solace Suggested overlay (TLS with the developer's trust store, DMQ eligibility on persistent publishes, HA failover, separate projects, and the `config.json` convention); the leaf and sample mechanics, the Step 4 dispatch, and the fixed `VERIFY:` contract stay here in implement-mode.md and its leaf files. Solace Suggested is TLS-secure by definition: the session is `tcps://` with server-certificate validation, and it REQUIRES a broker that already has TLS configured (Solace Cloud has it out of the box) plus a trust store directory that holds the broker's CA certificate, which the developer supplies (`solace-suggested-mode.md` Step 1 explains it to the developer). There is no non-secure variant of this door and no sub-question to ask; the door question stays a clean three-way choice. When the developer's target broker has no TLS configured (for example, a local dev container), say plainly that Solace Suggested requires a TLS-capable broker and offer the two honest alternatives: Custom with the equivalent knobs ticked minus the secure session, or Quickstart for a learning run. Never generate a plaintext session under the Solace Suggested name. The overlay is leaf-agnostic: it rides on whatever `Pattern` leaf the design summary chose and only adds the Solace Suggested deltas. It is what Solace suggests as a hardened baseline, not a guarantee that the output is ready for deployment. +- **Custom.** On the Custom path, read `custom-mode.md` and follow it. Custom opens a SINGLE up-front multi-select checklist of the applicable hardening knobs, captured once, not a sequence of per-knob questions; generation then applies exactly the ticked subset and nothing more. Nothing ticked lands a Quickstart-shaped base app, everything ticked lands about the Solace Suggested shape, and each ticked knob layers one Suggested delta on top of that floor. `custom-mode.md` owns the leaf-aware checklist content and the knob application; the leaf and sample mechanics, the Step 4 dispatch, and the fixed `VERIFY:` contract stay here in implement-mode.md and its leaf files, exactly as they do on the Solace Suggested path. -State plainly that a real deployment lifts each script into its own service or project. Basic username/password is the only auth this journey generates; ground it in the authentication section of [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md). +State plainly that a real deployment lifts each script into its own service or project. Every door generates the scripts in one project, unless the developer opts into separate projects on Solace Suggested or Custom. Basic username/password is the only auth this journey generates; ground it in the authentication section of [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md). ## Step 1: Confirm broker access and grounding -With the door chosen, confirm the developer has access to a reachable broker, and confirm WHICH broker they want to target. Confirm that they HAVE one; do NOT ask them to paste connection details into the chat. When Design mode already confirmed broker access in this session (its Step 0), do not re-ask whether a broker exists; ask only WHICH broker to target. A broker discovered in the environment is a fact to report while asking, never an answer. The credential VALUES reach the app through a `config.json` when present, otherwise as command-line arguments at run time; the skill itself never needs the values typed into the chat. +With the door chosen, confirm the developer has access to a reachable broker, and confirm WHICH broker they want to target. Confirm that they HAVE one; do NOT ask them to paste connection details into the chat. When Design mode already confirmed broker access in this session (its Step 0), do not re-ask whether a broker exists; ask only WHICH broker to target. A broker discovered in the environment is a fact to report while asking, never an answer. Where the credential VALUES live depends on the Step 0 door: for Solace Suggested they go ONLY into the gitignored `config.json`; for Quickstart they reach the app through a `config.json` when present, otherwise as command-line arguments at run time. The skill itself never needs the values typed into the chat. + +**On Quickstart, and on Custom with the secure-session knob unticked, also ask the scheme.** Ask in one direct question: "Is your broker URL plaintext (`tcp://`) or secure (`tcps://`)?" The scheme is not a credential, so asking it never conflicts with the rule against connection details in the chat; Solace Cloud URLs are `tcps://` by default. An existing `config.json` or a discovered broker is a fact to report, never the answer: still ask. The answer decides the Quickstart trust store guidance (Step 0) and the `config.example.json` placeholder (Step 4). Do not ask it on Solace Suggested, which is always `tcps://`, or on Custom with the secure-session knob ticked, which is always `tcps://`. If the developer has no reachable broker, route through `prerequisites.md` first, then return here. @@ -85,14 +88,12 @@ Generation is keyed on the `Pattern` leaf read in Step 2. Read ONLY the leaf fil | `Guaranteed Pub/Sub (single service, exclusive)` | `implement-guaranteed-pubsub.md` | `python-guaranteed-publisher-sample.py` + `python-guaranteed-subscriber-sample.py` | | `Guaranteed Pub/Sub (single service, non-exclusive)` | `implement-guaranteed-pubsub.md` | `python-guaranteed-publisher-sample.py` + `python-guaranteed-subscriber-sample.py` | | `Guaranteed Pub/Sub (fan-out)` | `implement-guaranteed-pubsub.md` | `python-guaranteed-publisher-sample.py` + `python-guaranteed-subscriber-sample.py` | -| `Direct Pub/Sub` | not yet available | none | -| `Request-Reply (Direct)` | not yet available | none | - -**The two direct leaves stop here.** Say in one line that generating a Python project for this leaf is not yet available in this release. Offer to save the confirmed summary to `solace-design.md` if it is not saved yet, and stop. Do not read the direct samples and do not adapt them by hand: without a wiring file there is no marker contract and no conformance check for the result. Do not offer to write the code another way (from scratch, "outside the generator", or as unverified code), and do not write it when the developer asks again: the stop is the ENTIRE answer, the same way the Debug redirect is. Point the developer at the saved design and at the [Python tutorials](https://tutorials.solace.dev/python/) instead. +| `Direct Pub/Sub` | `implement-direct-pubsub.md` | `python-direct-publisher-sample.py` + `python-direct-subscriber-sample.py` | +| `Request-Reply (Direct)` | `implement-request-reply.md` | `python-direct-requestor-blocking-sample.py` (or `python-direct-requestor-sample.py` on the non-blocking variant) + `python-direct-replier-sample.py` | **Every leaf also generates the shared connection helper.** Read `python-solace-connection-config.py` and generate it as `solace_connection_config.py` (a module name needs underscores). Each script builds its connection dict by calling `load_service_properties(args, APP_NAME)` from that module and passes it to `MessagingService.builder().from_properties(...)`, rather than setting the host, VPN, username, or password inline. -**Fixed file names.** Generate the scripts as `guaranteed_publisher.py` and `guaranteed_subscriber.py`, the names each sample's docstring gives, next to `solace_connection_config.py` (the fan-out leaf adds one subscriber script per further consuming service, named in `implement-guaranteed-pubsub.md`). The run commands in Step 5 and the checklist address these names; the names of functions, constants, and the `APP_NAME` value inside each script stay your discretion. +**Fixed file names.** Generate each script under the name its sample's docstring gives, next to `solace_connection_config.py`: `guaranteed_publisher.py` and `guaranteed_subscriber.py` (the fan-out leaf adds one subscriber script per further consuming service, named in `implement-guaranteed-pubsub.md`), `direct_publisher.py` and `direct_subscriber.py`, and `direct_replier.py` with `direct_requestor_blocking.py` (or `direct_requestor.py` on the non-blocking variant). The run commands in Step 5 and the checklist address these names; the names of functions, constants, and the `APP_NAME` value inside each script stay your discretion. **Disclaimer header on EVERY generated source file.** Stamp the top of each generated `.py` file with these two comment lines, before the module docstring and before any import: @@ -105,7 +106,7 @@ Stamp the header AT WRITE TIME, as the first content of each new file. `requirem **Embedded and web-app shapes.** When the Python messaging layer sits inside a larger application, the chosen leaf's wiring still governs the messaging layer: the same connection helper, the same service, receiver, and publisher listeners, the same `VERIFY:` markers emitted through a stdout-reaching `trace(...)`, and the disclaimer header on every generated file, messaging or not. The `VERIFY:` markers are NOT demo harness. -**Write the verification artifacts in this same step.** Alongside the scripts, write the tailored `solace-verification-checklist.md` at the project root (Step 6), `config.example.json`, and `.gitignore`. A session that ends early still leaves the checklist in the project. +**Write the verification artifacts in this same step.** Alongside the scripts, write the tailored `solace-verification-checklist.md` at each project root (Step 6), `config.example.json`, and `.gitignore`. A session that ends early still leaves the checklist in the project. ### Concurrency: threads, not asyncio, and no multiprocessing @@ -131,11 +132,11 @@ Hold every invariant while adapting: keep the shared connection helper, the serv ### Quickstart fail-fast reconnect defaults -On the Quickstart door, do not carry the samples' reconnect-tuning block into the generated scripts. The samples set a reconnection budget (`with_reconnection_retry_strategy(RetryStrategy.parametrized_retry(...))` with the `RECONNECT_RETRIES` and `RECONNECT_RETRY_INTERVAL_MS` constants) and carry an HA failover comment block; Quickstart generation OMITS the builder call, the two constants, the HA comment block, and the then-unused `RetryStrategy` import, leaving connect and reconnect behavior at the API defaults, so a wrong host, port, or credential in this learning setup fails immediately instead of retrying. Drop ONE short comment at the `MessagingService.builder()` site in each generated script stating that posture: connect and reconnect settings are at the API defaults so failures surface immediately; see the [Solace C API Best Practices](https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md) for reconnect tuning beyond Quickstart. +On the Quickstart door ONLY, do not carry the samples' reconnect-tuning block into the generated scripts. The samples set a reconnection budget (`with_reconnection_retry_strategy(RetryStrategy.parametrized_retry(...))` with the `RECONNECT_RETRIES` and `RECONNECT_RETRY_INTERVAL_MS` constants) and carry an HA failover comment block; Quickstart generation OMITS the builder call, the two constants, the HA comment block, and the then-unused `RetryStrategy` import, leaving connect and reconnect behavior at the API defaults, so a wrong host, port, or credential in this learning setup fails immediately instead of retrying. Drop ONE short comment at the `MessagingService.builder()` site in each generated script stating that posture: connect and reconnect settings are at the API defaults so failures surface immediately; see the [Solace C API Best Practices](https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md) for reconnect tuning beyond Quickstart. The Solace Suggested door is unchanged: it keeps the samples' baseline or upgrades it per `solace-suggested-mode.md` Step 2. Custom follows its floor exception in `custom-mode.md`. ### Scaffold the config files -Generate two small project files next to the scripts: +Generate two small project files next to the scripts (on Solace Suggested, on Custom when the secure-session knob is ticked, and on Quickstart or on Custom with the secure-session knob unticked when the Step 1 answer is secure, use the `config.example.json` from `solace-suggested-mode.md` Step 0 instead, with its `tcps://` placeholder host and its trust store key): - A committed `config.example.json` that carries placeholder values for the four keys the shared helper requires, spelled as the Solace Python API property names the helper documents. The placeholder values must be obviously fake: @@ -157,7 +158,7 @@ Tell the developer they opt in to the config file by copying `config.example.jso The Python API does not bundle a verify script yet, so Implement mode ends in the compile-only fallback on every run, whatever the contents of `config.json` (SKILL.md Invariant 5). Never fabricate a run result and never call a check below a verified round trip. 1. **Compile check.** Run `.venv/bin/python -m py_compile` on every generated `.py` file. A failure is a code failure: fix it and re-run, up to 3 attempts, then report the evidence and hand the decision to the developer. -2. **Import smoke.** From the project root, run `.venv/bin/python -c "import solace_connection_config, guaranteed_publisher, guaranteed_subscriber"`, adding every further generated module (for example each fan-out subscriber). Each script keeps its `if __name__ == "__main__":` guard, so the import resolves every `solace.messaging` name the scripts use without connecting to a broker. A failure (for example an `ImportError` on a misspelled module path) is a code failure under the same fix loop. +2. **Import smoke.** From the project root, run `.venv/bin/python -c "import solace_connection_config, guaranteed_publisher, guaranteed_subscriber"` with the module names of the chosen leaf (for example `direct_publisher, direct_subscriber` or `direct_replier, direct_requestor_blocking`), adding every further generated module (for example each fan-out subscriber). On separate projects, run the checks in each project root against that project's modules. Each script keeps its `if __name__ == "__main__":` guard, so the import resolves every `solace.messaging` name the scripts use without connecting to a broker. A failure (for example an `ImportError` on a misspelled module path) is a code failure under the same fix loop. 3. **Conformance checks.** From the project root, run these three checks and record each result in the checklist. A missing header or a missing checklist is a code failure under the same fix loop; a version mismatch is reported to the developer, not fixed silently. ```bash @@ -169,7 +170,7 @@ The Python API does not bundle a verify script yet, so Implement mode ends in th grep -qxF "solace-pubsubplus==$(curl -s https://pypi.org/pypi/solace-pubsubplus/json | python3 -c 'import json, sys; print(json.load(sys.stdin)["info"]["version"])')" requirements.txt || echo "VERSION DRIFT" ``` -4. **Report and hand back.** Report the compile, import, and conformance results, then hand the developer the exact commands to run against their own broker from the project root, subscriber first so the durable queue and its subscription exist before the first publish: +4. **Report and hand back.** Report the compile, import, and conformance results, then hand the developer the exact commands to run against their own broker from the project root. The long-running role always starts first: the subscriber, so the durable queue (guaranteed) or the topic subscription (direct) exists before the first publish, and the replier, so the request topic subscription exists before the first request. For the Guaranteed Pub/Sub leaves: ``` # terminal 1: provisions the queue, binds, and waits (Ctrl-C stops it) @@ -178,15 +179,23 @@ The Python API does not bundle a verify script yet, so Implement mode ends in th .venv/bin/python guaranteed_publisher.py [ [password]] ``` - On the fan-out leaf, hand back one subscriber command per subscriber script (`guaranteed_subscriber.py` and each `guaranteed_subscriber_.py`), each in its own terminal and all started before the publisher; every subscriber prints its own markers. + For the Direct Pub/Sub leaf, the same two terminals run `direct_subscriber.py` (subscribes and waits; Ctrl-C stops it), then `direct_publisher.py` (publishes a short burst and exits). For the Request-Reply (Direct) leaf, they run `direct_replier.py` (subscribes to the request topic and waits; Ctrl-C stops it), then `direct_requestor_blocking.py` or `direct_requestor.py` (sends, waits for the reply, and exits). + + On the fan-out leaf, hand back one subscriber command per subscriber script (`guaranteed_subscriber.py` and each `guaranteed_subscriber_.py`), each in its own terminal and all started before the publisher; every subscriber prints its own markers. On separate projects (Solace Suggested, or the Custom decoupled-projects knob), hand back each command from its own project directory with that project's `.venv/bin/python`. + + The arguments are needed only when there is no `config.json`, and they are never offered on a path that drops the command-line fallback. Tell the developer what a passing run prints, per leaf: + + - Guaranteed Pub/Sub: the subscriber prints `VERIFY: CONNECTED` and then `VERIFY: QUEUE_BOUND`; the publisher prints `VERIFY: CONNECTED` and `VERIFY: PUBLISH_ACKED` and exits with status 0; the subscriber then prints `VERIFY: MESSAGE_RECEIVED`. + - Direct Pub/Sub: the subscriber prints `VERIFY: CONNECTED` and then `VERIFY: SUBSCRIBED`; the publisher prints `VERIFY: CONNECTED` and exits with status 0; the subscriber then prints `VERIFY: MESSAGE_RECEIVED`. + - Request-Reply (Direct): the replier prints `VERIFY: CONNECTED` and then `VERIFY: SUBSCRIBED`; the requestor prints `VERIFY: CONNECTED`; the replier prints `VERIFY: REQUEST_RECEIVED`; the requestor prints `VERIFY: REPLY_RECEIVED` and exits with status 0. - The arguments are needed only when there is no `config.json`. Tell the developer what a passing run prints: the subscriber prints `VERIFY: CONNECTED` and then `VERIFY: QUEUE_BOUND`; the publisher prints `VERIFY: CONNECTED` and `VERIFY: PUBLISH_ACKED` and exits with status 0; the subscriber then prints `VERIFY: MESSAGE_RECEIVED`; Ctrl-C on the subscriber prints `Shutdown signal received` and it exits with status 0. + On every leaf, Ctrl-C on the long-running role prints `Shutdown signal received` and it exits with status 0. -**When the developer asks you to run it.** If the developer explicitly asks you to run the two scripts against their broker, print the live-broker heads-up first (SKILL.md Invariant 6: name the host, every durable queue the subscribers provision, one connection per script, and the messages the publisher sends, scaled to the generated scripts). Then start each subscriber in the background with its output captured, wait for `VERIFY: QUEUE_BOUND` from each, run the publisher in the foreground, wait for `VERIFY: MESSAGE_RECEIVED` in each subscriber's output, and stop the subscribers with SIGINT. Report the markers you observed as evidence of a manual run. It is not a verify verdict: record it in the checklist as a manual run, and never describe it as a passed verify stage. +**When the developer asks you to run it.** If the developer explicitly asks you to run the scripts against their broker, print the live-broker heads-up first (SKILL.md Invariant 6: name the host, every durable queue the subscribers provision, one connection per script, and the messages or requests the foreground role sends, scaled to the generated scripts). Then start each long-running role in the background with its output captured, wait for its readiness marker (`VERIFY: QUEUE_BOUND` on the guaranteed leaves, `VERIFY: SUBSCRIBED` on the direct leaves) from each, run the foreground role, wait for the pass marker (`VERIFY: MESSAGE_RECEIVED` in each subscriber's output, or `VERIFY: REPLY_RECEIVED` in the requestor's output), and stop the long-running roles with SIGINT. Report the markers you observed as evidence of a manual run. It is not a verify verdict: record it in the checklist as a manual run, and never describe it as a passed verify stage. -**Environment failure on a manual run: do NOT modify the code.** When the captured output shows that the broker is unreachable, the credentials or VPN are wrong, or the client may not create the queue (the subscriber's `Could not bind to queue` error), the code is not the problem. Report the matched output line as evidence and STOP: do not enter the fix loop, and do not change the code. The fix is the developer's broker, credentials, or client permissions. Fix the code only when the output shows a code failure (an exception from the scripts' own logic, or a marker that never appears after a clean connect and bind), up to 3 attempts, then hand the decision to the developer. +**Environment failure on a manual run: do NOT modify the code.** When the captured output shows that the broker is unreachable, the credentials or VPN are wrong, the TLS handshake or the trust store fails, or the client may not create the queue (the subscriber's `Could not bind to queue` error), the code is not the problem. Report the matched output line as evidence and STOP: do not enter the fix loop, and do not change the code. The fix is the developer's broker, credentials, or client permissions. Fix the code only when the output shows a code failure (an exception from the scripts' own logic, or a marker that never appears after a clean connect and bind), up to 3 attempts, then hand the decision to the developer. -**Close every run with the next steps, stated plainly.** Tell the developer, in order: (1) copy `config.example.json` to `config.json` and fill in their real broker values; (2) run the subscriber, then the publisher, with the commands above, and check the markers. Step 4 already wrote the tailored `solace-verification-checklist.md`; invite the developer to add their own items in its Your additional items section and to record any skill-undelivered concerns under Developer-owned items. +**Close every run with the next steps, stated plainly.** Tell the developer, in order: (1) copy `config.example.json` to `config.json` and fill in their real broker values; on any `tcps://` host (always on Solace Suggested and with the Custom secure-session knob, and on Quickstart or on Custom with the secure-session knob unticked when the Step 1 answer was secure), also put the broker's CA certificate into a trust store directory and set `solace.messaging.tls.trust-store-path` to it, as `solace-suggested-mode.md` Step 1 explains. On a Quickstart run, or a Custom run with the secure-session knob unticked, whose Step 1 answer was plaintext, still add one line: if the host later changes to `tcps://`, the project needs a trust store directory set in `solace.messaging.tls.trust-store-path` in `config.json`, or the connection fails with "Failed to load trust store"; (2) run the long-running role, then the foreground role, with the commands above, and check the markers. Step 4 already wrote the tailored `solace-verification-checklist.md`; invite the developer to add their own items in its Your additional items section and to record any skill-undelivered concerns under Developer-owned items. ### Traps to avoid @@ -203,37 +212,45 @@ The Python API does not bundle a verify script yet, so Implement mode ends in th ## Step 6: Tailor the checklist (written at Step 4) and report -Step 4 writes the tailored `solace-verification-checklist.md` into the generated project as generation output; this step defines its content and adds the report in chat. The Python API has no master checklist template yet; until it lands, write the file from the list below. Resolve each item for the generated code: an item the generation did not deliver moves to Your responsibility. Write it as a Markdown task list (`- [ ]`), ticking only what this run actually checked. Each item in the first two groups carries one canonical doc link, as the master JCSMP template does; the round-trip and conformance items carry none because they check skill conformance, not Solace guidance. +Step 4 writes the tailored `solace-verification-checklist.md` into the generated project as generation output; this step defines its content and adds the report in chat. The Python API has no master checklist template yet; until it lands, write the file from the list below. Resolve each item for the generated code: keep only the items for the chosen leaf (a leaf in parentheses marks a leaf-specific item), and move an item the generation did not deliver to Your responsibility. On Solace Suggested, and for each ticked Custom knob, an item under Your responsibility that the door or the knob delivered (the TLS secure session, DMQ eligibility on published messages, separate projects, and the HA reconnect values only when the HA-failover question was answered yes) moves to Delivered by this generation. The broker-side dead message queue always stays under Your responsibility, because the skill never provisions it. Write it as a Markdown task list (`- [ ]`), ticking only what this run actually checked. Each item in the first two groups carries one canonical doc link, as the master JCSMP template does; the round-trip and conformance items carry none because they check skill conformance, not Solace guidance. **Delivered by this generation** - Basic-auth connection through the shared `solace_connection_config.py` helper; credentials live only in a gitignored `config.json`. [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) - Service reconnection and interruption listeners registered before `connect()`. [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) -- Persistent publish to a topic with a publish receipt listener; a NACK is logged with the message id. [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) -- Capacity-bounded back pressure on the publisher (`on_back_pressure_wait`). [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) -- Message time-to-live set once on the publisher's message template. [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) -- Durable queue plus topic subscription provisioned at start (`CREATE_ON_START`), with the access type from the design summary (one durable queue per consuming service on fan-out). [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md) -- Client acknowledgement only after processing. [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md) +- (Guaranteed Pub/Sub) Persistent publish to a topic with a publish receipt listener; a NACK is logged with the message id. [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) +- (Guaranteed Pub/Sub) Capacity-bounded back pressure on the publisher (`on_back_pressure_wait`). [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) +- (Guaranteed Pub/Sub) Message time-to-live set once on the publisher's message template. [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) +- (Guaranteed Pub/Sub) Durable queue plus topic subscription provisioned at start (`CREATE_ON_START`), with the access type from the design summary (one durable queue per consuming service on fan-out); with the Custom admin-provisioned-queue knob, the subscriber binds the administrator's queue instead. [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md) +- (Guaranteed Pub/Sub) Client acknowledgement only after processing. [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md) +- (Direct Pub/Sub) Direct publish to a topic with a publish failure listener and capacity-bounded back pressure (`on_back_pressure_wait`). [Publishing Direct Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-DM-Publish.md) +- (Direct Pub/Sub) Direct topic subscription with capacity-bounded back pressure (`on_back_pressure_drop_latest`) and discard detection on received messages. [Consuming Direct Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-DM-Receive.md) +- (Request-Reply (Direct)) Requests sent through a `RequestReplyMessagePublisher` with a positive reply timeout, and a replier that answers with `replier.reply()` and ignores a message with no reply-to. [Request-Reply Messaging in the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Request-Reply.md) - Graceful SIGINT/SIGTERM shutdown: `terminate(grace_period)` and then `disconnect()` on every exit path. [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md) **Your responsibility (not delivered here)** -- TLS secure session (`tcps://` with certificate validation); Quickstart does not generate it. [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) +- TLS secure session (`tcps://` with certificate validation set explicitly in code). Quickstart, and Custom with the secure-session knob unticked, open a TLS session only when the host is `tcps://`, and then validate only through the API defaults, which a `config.json` key can turn off; neither sets the certificate checks in code. [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) +- (Any `tcps://` host: Solace Suggested, the Custom secure-session knob, or Quickstart or Custom without that knob with a `tcps://` host) A trust store directory that holds the broker's CA certificate as a PEM file, named by `solace.messaging.tls.trust-store-path` in `config.json`; the Python API has no default trust store, so the connection fails without it. [Using Client Certificates with Solace Cloud](https://docs.solace.com/Cloud/ght_client_certs.md) +- (Custom admin-provisioned-queue knob) The queue and its topic subscription provisioned on the broker by an administrator, with the access type the design summary records. [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md) - Credentials handled safely: `config.json` is never committed, and credentials are not passed as command-line arguments, where they land in shell history and process listings (the handed-back run commands accept them only for convenience). [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) - Authentication stronger than basic username/password (for example a client certificate or Kerberos) where the environment requires it; this journey generates basic username/password only. [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) -- Dead message queue eligibility on published messages and a dead message queue on the broker. [Configuring Dead Message Queues](https://docs.solace.com/Messaging/Guaranteed-Msg/Setting-Dead-Msg-Queues.md) -- Reconnect tuning for HA failover; Quickstart leaves connect and reconnect behavior at the API defaults. [Solace C API Best Practices](https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md) +- (Guaranteed Pub/Sub) Dead message queue eligibility (`PERSISTENT_DMQ_ELIGIBLE`) set explicitly on published messages. [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) +- (Guaranteed Pub/Sub) A dead message queue provisioned on the broker with a max-redelivery count or a time-to-live; the eligibility flag alone does nothing without it. [Configuring Dead Message Queues](https://docs.solace.com/Messaging/Guaranteed-Msg/Setting-Dead-Msg-Queues.md) +- Reconnect tuning for HA failover; Quickstart leaves connect and reconnect behavior at the API defaults, and Custom without the HA knob keeps the samples' baseline of 20 retries 3000 ms apart. [Solace C API Best Practices](https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md) - Separate projects or deployments per role (skill guidance: a real deployment lifts each script into its own service or project; no Solace doc page states it). -- `respect-ttl` enabled on the queue, without which the broker ignores the message time-to-live. [Configuring Queues](https://docs.solace.com/Messaging/Guaranteed-Msg/Configuring-Queues.md) -- An administrator-provisioned queue, or the endpoint-create permission for this client, in production. [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md) +- (Guaranteed Pub/Sub) `respect-ttl` enabled on the queue, without which the broker ignores the message time-to-live. [Configuring Queues](https://docs.solace.com/Messaging/Guaranteed-Msg/Configuring-Queues.md) +- (Guaranteed Pub/Sub) An administrator-provisioned queue, or the endpoint-create permission for this client, in production. [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md) - A Python version in active or security support and OpenSSL installed on Linux for TLS ([Supported Environments](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-supported-Environments.md)), and no `multiprocessing` in the application ([solace-pubsubplus on PyPI](https://pypi.org/project/solace-pubsubplus/)). **Verified by the round-trip** (developer-run until the Python verify script lands) -- Every subscriber prints `VERIFY: CONNECTED` and `VERIFY: QUEUE_BOUND`. -- The publisher prints `VERIFY: CONNECTED` and `VERIFY: PUBLISH_ACKED`, then exits with status 0. -- Every subscriber prints `VERIFY: MESSAGE_RECEIVED` (`PUBLISH_ACKED` alone does not prove delivery). -- Ctrl-C on each subscriber prints `Shutdown signal received` and it exits with status 0. +- (Guaranteed Pub/Sub) Every subscriber prints `VERIFY: CONNECTED` and `VERIFY: QUEUE_BOUND`. +- (Guaranteed Pub/Sub) The publisher prints `VERIFY: CONNECTED` and `VERIFY: PUBLISH_ACKED`, then exits with status 0. +- (Guaranteed Pub/Sub) Every subscriber prints `VERIFY: MESSAGE_RECEIVED` (`PUBLISH_ACKED` alone does not prove delivery). +- (Direct Pub/Sub) The subscriber prints `VERIFY: CONNECTED` and `VERIFY: SUBSCRIBED`; the publisher prints `VERIFY: CONNECTED` and exits with status 0; the subscriber then prints `VERIFY: MESSAGE_RECEIVED`. +- (Request-Reply (Direct)) The replier prints `VERIFY: CONNECTED`, `VERIFY: SUBSCRIBED`, and `VERIFY: REQUEST_RECEIVED`; the requestor prints `VERIFY: CONNECTED` and `VERIFY: REPLY_RECEIVED`, then exits with status 0. +- Ctrl-C on each long-running role (subscriber or replier) prints `Shutdown signal received` and it exits with status 0. **Generation conformance** @@ -258,6 +275,9 @@ Live `docs.solace.com` `.md` pages (WebFetch on demand): - [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) - [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) - [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md) +- [Publishing Direct Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-DM-Publish.md) +- [Consuming Direct Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-DM-Receive.md) +- [Request-Reply Messaging in the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Request-Reply.md) - [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md) - [Supported Environments](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-supported-Environments.md) - [Solace C API Best Practices](https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md) diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-request-reply.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-request-reply.md new file mode 100644 index 0000000..5552d16 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-request-reply.md @@ -0,0 +1,44 @@ +# Implement: Request-Reply (Direct) leaf + +The per-leaf wiring for the Request-Reply (Direct) leaf of the Python API. Read this file from `implement-mode.md` Step 4 when the `Pattern` field is `Request-Reply (Direct)`. The shared Step 4 rules in `implement-mode.md` (the disclaimer header, the shared connection helper, the fixed file names, the concurrency rules, the variant discipline, the demo-harness rule, the comments rule, and the Quickstart fail-fast rule) apply here unchanged. + +Request-reply in the Python API uses direct messages only, so this is the ONLY Python request-reply leaf. There is no Request-Reply (Guaranteed) leaf and no guaranteed requestor or replier sample; a design that needs a persisted request is a Guaranteed Pub/Sub leaf (`design-mode.md` Step 2), never this one. + +## Request-reply flow + +Generation reads one requestor sample and the replier sample and adapts each into its own generated script, the same near-verbatim discipline the other leaves use. Direct request-reply is at-most-once on both legs: there is no broker receipt and no redelivery, so this leaf has NO queue, NO provisioning, and NO client acknowledgement. The API manages the reply: the requestor's `RequestReplyMessagePublisher` sets a unique reply-to on each request and correlates the reply, and the replier's `Replier.reply()` routes to that reply-to and copies the correlation. The requestor therefore needs no reply subscription and no correlation id of its own. + +- `python-direct-requestor-blocking-sample.py` (the BASE shape): basic-auth connect, a `RequestReplyMessagePublisher`, ONE blocking `publish_await_response()` with a POSITIVE `REQUEST_TIMEOUT_MS`, a `PubSubTimeoutError` catch with one retry to absorb a cold-start race, service event listeners registered before `connect()`, and a graceful SIGINT/SIGTERM shutdown. It exits 0 on a reply and 1 without one. +- `python-direct-requestor-sample.py` (the VARIANT shape): the same, but several non-blocking `publish()` calls that each return a `concurrent.futures.Future`, so the requests are in flight together, then each Future collected for its reply or its `PubSubTimeoutError`. It exits 0 only when every request sent got a reply. +- `python-direct-replier-sample.py`: basic-auth connect, a `RequestReplyMessageReceiver` built with the request topic subscription (no queue, no provisioning), an async `RequestMessageHandler` that guards on a `None` replier and answers with `replier.reply()`, service event listeners registered before `connect()`, and a graceful SIGINT/SIGTERM shutdown. It answers both requestor shapes. + +**Base shape and variant.** Generate the blocking requestor when the design summary's `Consumption endpoint` reads `API-managed reply (RequestReplyMessagePublisher)` (the default): one request at a time, the blocking call as the synchronization point, the same shape as the JCSMP direct requestor. When the summary records the VARIANT (for example `API-managed reply (RequestReplyMessagePublisher, non-blocking publish)`, chosen because many requests must be in flight at once), read the non-blocking sample instead. Read only the requestor sample the design chose, never both. + +Each script builds its OWN `MessagingService`. Keep the samples' structure: the shared state dataclass, `setup_solace(...)`, `connect_solace(...)`, the main step, and `teardown_solace(...)` in `main()`'s `finally`. Keep every listener and handler class. + +**Ground before you write.** Before writing either script, WebFetch [Request-Reply Messaging in the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Request-Reply.md) in this session (SKILL.md Invariant 2). + +## Replier script (`direct_replier.py`) + +The replier subscribes directly to the request topic (no queue), answers each request, and is the long-running process the developer runs FIRST. Start the file with the disclaimer header, then wire it in this sequence: + +1. Build the connection properties with `load_service_properties(args, APP_NAME)`, then the service with `MessagingService.builder().from_properties(properties).build()` (on Quickstart without the reconnection strategy call, per the fail-fast rule). +2. Build the receiver as the sample does: `request_reply().create_request_reply_message_receiver_builder().build(TopicSubscription.of(request_topic))` for the design summary's topic. Register the three service event listeners BEFORE `connect()`. +3. `messaging_service.connect()`. Immediately after `connect()` returns, emit the connect marker: `trace("VERIFY: CONNECTED")`. +4. `receiver.receive_async(handler)` BEFORE `receiver.start()`, as the sample orders them, then `receiver.start()`, which applies the request topic subscription and blocks until the broker confirms it. Immediately after `start()` returns, emit the subscribed marker: `trace("VERIFY: SUBSCRIBED")`. **No reapply-subscriptions setting.** The JCSMP direct replier must set `REAPPLY_SUBSCRIPTIONS` to `true`, or its request topic subscription is lost on an automatic reconnect. The Python API has no such setting and needs none: request-reply in the Python API uses direct messages only, and the [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) page states that the API re-applies the application's direct subscriptions on reconnect. Do not search for or invent a Python equivalent. Keep the sample's `on_reconnected` comment that says so. +5. In the handler's `on_message(message, replier)`, keep the sample's guard: a `None` replier means the message has no reply-to, so log it and return. After the guard, emit the request marker: `trace("VERIFY: REQUEST_RECEIVED")`. Then build the reply with the handler's one `message_builder()` and answer with `replier.reply(reply)`, inside the sample's `PubSubPlusClientError` catch. Use `replier.reply()`, never a manual publish to the reply-to: the API routes and correlates the reply. +6. KEEP the long-running wait loop, the SIGINT/SIGTERM handler that only sets the shutdown event, and `teardown_solace(...)` with `terminate(grace_period)` and `disconnect()`. The replier is the SIGINT target, and its shutdown line (`Shutdown signal received ...`) stays a `trace(...)` call. + +## Requestor script (`direct_requestor_blocking.py`, or `direct_requestor.py` on the variant) + +The requestor connects, sends its request (or requests), collects the reply, and exits on its own. It does NOT provision anything and does NOT wait for a key press or a signal. Start the file with the disclaimer header, then wire it in this sequence: + +1. Build the connection properties and the service exactly as the replier does (step 1 above), then the requestor with `request_reply().create_request_reply_message_publisher_builder().build()`. Register the three service event listeners BEFORE `connect()`. +2. `messaging_service.connect()`. Immediately after `connect()` returns, emit the connect marker: `trace("VERIFY: CONNECTED")`. Then `requestor.start()`. +3. Base shape: build the request with `message_builder().build(payload)` and send it to the design summary's topic with `publish_await_response(..., reply_timeout=REQUEST_TIMEOUT_MS, ...)`. Keep the POSITIVE `REQUEST_TIMEOUT_MS` (the API has no default), the `PubSubTimeoutError` catch, and the single retry (`REQUEST_ATTEMPTS = 2`). Variant: keep the sample's send phase (one `publish(...)` per request, each returning a Future) and its collect phase (`future.result()` per request, a `PubSubTimeoutError` counted as a missing reply). +4. When a reply arrives, emit the reply marker: `trace("VERIFY: REPLY_RECEIVED")`. On the variant, emit it once for each reply collected. Keep the sample's exit code rule: 1 when a request sent got no reply, 0 otherwise (a shutdown signal before the reply is a clean stop). +5. Self-exit: return from the main step, and `teardown_solace(...)` in `main()`'s `finally` calls `terminate(grace_period)` and `disconnect()`. Keep the SIGINT/SIGTERM handler the sample carries, so Ctrl-C during the reply wait still reaches teardown. When the request describes a real application, its own request payload replaces the sample's text (the demo-harness rule). + +## The marker contract + +These four `VERIFY:` marker strings (`VERIFY: CONNECTED`, `VERIFY: SUBSCRIBED`, `VERIFY: REQUEST_RECEIVED`, `VERIFY: REPLY_RECEIVED`) are the same strings the JCSMP Request-Reply (Direct) leaf emits, character for character. They are read PER PROCESS: the replier output carries `SUBSCRIBED` and `REQUEST_RECEIVED`, the requestor output carries `REPLY_RECEIVED`, and both print `CONNECTED`. Until the Python verify script lands, the developer reads them from the two terminals (`implement-mode.md` Step 5); the verify script will read the same strings, so never change one. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/solace-suggested-mode.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/solace-suggested-mode.md new file mode 100644 index 0000000..3d6e7d3 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/solace-suggested-mode.md @@ -0,0 +1,97 @@ +# Solace Suggested Mode + +The Solace Suggested overlay for Implement mode of the Python API. This file is reached from `implement-mode.md` Step 0 when the developer chooses the Solace Suggested path. It hardens whatever the design summary chose without re-describing how the chosen leaf is generated. Work through the steps in order. Read this file's grounding links only as each step needs them; link the canonical doc and state the decision in one line, never paraphrase doc content. + +This overlay is TLS-secure by definition. It assumes a broker whose TLS server certificate is already configured; Solace Cloud, the primary broker default, has TLS out of the box. There is NO non-secure variant of Solace Suggested. When the target broker has no TLS configured, do not proceed on this path: state that Solace Suggested requires a TLS-capable broker and route back to the `implement-mode.md` Step 0 door, which offers Custom with the equivalent knobs minus the secure session, or Quickstart. The session is built from the connection details the developer puts into a gitignored `config.json`. The overlay does NOT provision or configure any broker (no DMQ, no queues, no TLS certificates). It states the broker-side prerequisites; it never sets them up. + +## This is a leaf-agnostic overlay + +The Solace Suggested hardening is described ONCE here and applies on top of whatever `Pattern` leaf the design summary selected. It is not duplicated per leaf. The session-level deltas (TLS and HA failover) are identical code in every leaf, the message-level delta (DMQ eligibility) lands only on persistent publishes, and the separate-projects layout applies to every pattern, since each has two roles (publisher and subscriber, or requestor and replier). + +This overlay defers to `implement-mode.md` for all leaf and sample mechanics: the Step 2 design-summary input contract, the Step 4 leaf dispatch (which leaf wiring file and samples each leaf reads, the near-verbatim adaptation discipline), and the fixed `VERIFY:` marker contract. It ADDS only the Solace Suggested deltas below. It changes NO `VERIFY:` marker string. The marker contract, the leaf dispatch table, and the checklist write (Step 4) and report (Step 6) all stay in `implement-mode.md` and its leaf wiring files, and run unchanged on the Solace Suggested path. + +Every generated script this overlay describes carries the AI-assisted disclaimer header (the exact two lines `implement-mode.md` Step 4 defines) and keeps every import explicit; never write `from solace.messaging... import *`. This overlay names identifiers, constants, and method calls; it does not paste whole generated scripts. + +## Step 0: Carry over the config.json convention + +The Solace Suggested path reads broker connection details ONLY from a `config.json` file. Its keys are the Solace Python API property names the shared helper documents (`solace.messaging.transport.host`, `solace.messaging.service.vpn-name`, `solace.messaging.authentication.basic.username`, `solace.messaging.authentication.basic.password`), plus the trust store key that Step 1 adds. State plainly to the developer that `config.json` holds broker credentials, so it MUST be added to `.gitignore` and never committed, and that the credential VALUES MUST be placed directly into `config.json`, never pasted into the chat. Never inline a host, message VPN, username, or password into a generated script. + +On this path there are no command-line connection arguments. Generate the shared `solace_connection_config.py` helper WITHOUT its command-line fallback: when `config.json` is absent, it prints one line that names the missing file and exits with status 1. Add the trust store key to its required keys, so a missing trust store fails at start with a clear message. Keep everything else in the helper (the pass-through of every key, the blank-value checks, the empty-password rule). The generated `config.example.json` carries the four keys plus the trust store key, with a `tcps://` placeholder host: + +```json +{ + "solace.messaging.transport.host": "tcps://HOST:55443", + "solace.messaging.service.vpn-name": "YOUR_VPN", + "solace.messaging.authentication.basic.username": "YOUR_USERNAME", + "solace.messaging.authentication.basic.password": "YOUR_PASSWORD", + "solace.messaging.tls.trust-store-path": "PATH_TO_TRUST_STORE_DIRECTORY" +} +``` + +Generate the scripts directly, with no per-stage confirmation gate, the same way Quickstart does, then let the developer review the written files. The credential VALUES live only in the gitignored `config.json`, never in a generated script or the chat, so there is nothing credential-bearing to gate on. + +## Step 1: Build the TLS secure session + +The `host` in `config.json` carries a `tcps://` scheme: the Messaging Service page states that TLS needs the secure TCP protocol (`tcps`) in the host property. If the developer gives a `tcp://` (plaintext) host, tell them to use the broker's `tcps://` URL and state the TLS-already-configured assumption; never open a plaintext session on the Solace Suggested path. If the broker genuinely has no TLS configured, this overlay does not fit: route back to the `implement-mode.md` Step 0 door (Custom minus the secure-session knob, or Quickstart) rather than degrading the session here. + +Set server-certificate validation EXPLICITLY in code. After `load_service_properties(...)` returns and BEFORE the service is built, set these three `transport_layer_security_properties` keys to `True` on the loaded properties dict: `CERT_VALIDATED`, `CERT_REJECT_EXPIRED`, and `CERT_VALIDATE_SERVERNAME`. The values are the API defaults, but setting them explicitly makes the secure posture auditable in code review, the same reason the JCSMP overlay sets its validation properties explicitly. Because the code writes them after the file is loaded, a `config.json` value cannot turn validation off. NEVER set any of the three to `False`, and NEVER call `without_certificate_validation()`; the Messaging Service page restricts that call to development environments. + +**The trust store is the developer's.** The Python API documents no default trust store, so this overlay always sets one: the `solace.messaging.tls.trust-store-path` key in `config.json` (the `TRUST_STORE_PATH` property) names a directory that holds the certificate of the CA that signed the broker's server certificate. The developer supplies the directory and its contents; for Solace Cloud that is the public CA certificate the service uses. The generated code never downloads a certificate, never creates the directory, and never hardcodes the path. + +**Tell the developer about the trust store, twice.** This step is new to a developer who comes from JCSMP (Java ships a default trust store, so JCSMP needs no such step) and easy to miss for a developer new to TLS, so do not leave it to the checklist alone: + +1. BEFORE generating, in plain words: with TLS, the app checks that the broker's certificate was signed by a certificate authority (CA) it trusts. The Python API has no built-in list of trusted CAs, so the developer must give the app one: a directory that holds the CA certificate as a PEM file. For Solace Cloud, the service's **Connect** tab has a **Download PEM** link for that certificate; link [Using Client Certificates with Solace Cloud](https://docs.solace.com/Cloud/ght_client_certs.md), which states that a client needs the certificate for the public CA used by the event broker service and that not all clients have a default trust store. For another broker, the broker administrator supplies the CA certificate. Without the directory, the connection fails at start. +2. In the handed-back next steps (`implement-mode.md` Step 5): create a directory (for example `trust-store/` in the project), save the downloaded PEM file into it, and set `solace.messaging.tls.trust-store-path` in `config.json` to that directory. Recommend that the directory sits outside version control or holds only the public CA certificate; never put a private key in it. + +Record the same item in the checklist under Your responsibility (`implement-mode.md` Step 6). + +Ground TLS in the TLS section of [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md). On Linux, TLS also needs OpenSSL installed ([Supported Environments](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-supported-Environments.md)). + +## Step 2: Ask the HA-failover question; upgrade the reconnect values on yes, keep the sample baseline on no + +After broker confirmation and before generating the service, ask the developer plainly: is HA failover set up? Ask it as a direct question and wait for the answer. + +The reference samples already ship a reasonable reconnect baseline: `with_reconnection_retry_strategy(RetryStrategy.parametrized_retry(RECONNECT_RETRIES, RECONNECT_RETRY_INTERVAL_MS))` with 20 retries 3000 ms apart, the recommended default when the design says nothing about high availability. When the answer is no, KEEP this baseline as-is; do not strip it back to the API defaults (the fail-fast posture belongs to the Quickstart door alone), and do not add the HA-only values. + +ONLY when the answer is yes, apply the HA failover values the samples' HA comment block names: `with_connection_retry_strategy(RetryStrategy.parametrized_retry(1, 3000))`, `with_reconnection_retry_strategy(RetryStrategy.parametrized_retry(20, 3000))`, and `transport_layer_properties.CONNECTION_RETRIES_PER_HOST` set to 5 on the properties dict. These map to the JCSMP overlay's `connectRetries` 1, `reconnectRetries` 20, `reconnectRetryWaitInMillis` 3000, and `connectRetriesPerHost` 5. Keep both strategy calls BEFORE `from_properties(...)`, as the samples order them. Replace the samples' HA comment block with a one-line comment that names the applied values. Every interval is in MILLISECONDS (the seconds-versus-milliseconds trap in `implement-mode.md` Step 5). Ground the values in [Solace C API Best Practices](https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md), which the Python API inherits because it wraps the C API. + +## Step 3: Set DMQ eligibility on every persistent publish (Solace Suggested path only) + +On the Solace Suggested path, add `.with_property(message_properties.PERSISTENT_DMQ_ELIGIBLE, True)` to the persistent publisher's `message_builder()` template, next to the `PERSISTENT_TIME_TO_LIVE` property it already sets, so every message built from the template carries it. In the Python API this applies to the Guaranteed Pub/Sub publisher only: there is no guaranteed request-reply leaf, so no other script builds a persistent message. The API reference gives this property a default of `True`; set it explicitly anyway, for the same auditable reason as the TLS properties in Step 1. + +The direct leaves are EXCLUDED. The API reference states that the property is valid only for persistent messages, so it is a misleading no-op on a direct message; do NOT set it on Direct Pub/Sub or Request-Reply (Direct). Quickstart and the reference samples are unchanged; the explicit property is a Solace Suggested addition only. + +At the call site, state the broker-side prerequisite as the developer's responsibility in one comment: a message moves to a dead message queue only when a DMQ is configured on the broker AND a max-redelivery count or a time-to-live applies. The flag alone does not make DMQ work. Ground the persistent publish in [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) and the broker side in [Configuring Dead Message Queues](https://docs.solace.com/Messaging/Guaranteed-Msg/Setting-Dead-Msg-Queues.md). + +## Step 4: Generate separate, independent Python projects + +The Solace Suggested path decouples the roles. The DEFAULT shape is one standalone Python project per role (publisher and subscriber, or requestor and replier), each in its own directory. On the fan-out leaf, every subscriber script gets its own project beside the publisher project. Each project is self-contained: its own `requirements.txt` with the `solace-pubsubplus` version resolved per `implement-mode.md` Step 3, its own `.venv`, its own copy of `solace_connection_config.py`, its own `config.example.json`, `.gitignore`, and gitignored `config.json`, and its own tailored `solace-verification-checklist.md`. There is no shared package and no shared virtual environment between the projects. + +**When the developer's ask requires a single deliverable** (one command, one app, one dashboard), the separate-projects default conflicts with the ask. Surface that conflict plainly and let the developer resolve it; NEVER merge silently. Offer two resolutions: (1) keep the independent projects plus a thin launcher script that starts each one with its own `.venv/bin/python`; (2) drop the separate-projects shape with the developer's explicit consent. Whichever they choose is recorded in the emitted `solace-verification-checklist.md`; a consented drop lands under Developer-owned items, so the deviation is written down, not lost. + +The Solace Suggested path adds no new dependency. TLS, DMQ, and the retry strategies all live in the same `solace-pubsubplus` package Quickstart already uses. + +**Verification.** The Python API does not bundle a verify script yet, so the Solace Suggested path ends in the same check-and-hand-back fallback as Quickstart (`implement-mode.md` Step 5), run once per project: `py_compile`, the import smoke, and the conformance checks from each project root. Hand back the run commands per project, long-running role first, each from its own directory with its own interpreter, and with NO connection arguments (each script reads its own `config.json`). When the developer asks you to run them, follow the manual-run rule in `implement-mode.md` Step 5, including the live-broker heads-up (SKILL.md Invariant 6). A live Solace Suggested run needs a TLS-capable broker and the trust store from Step 1; a TLS or trust store error is an environment failure, never a code failure, so do not modify the code for it. + +## Step 5: Topic architecture + +For how to structure, order, name, and place wildcards in the topic hierarchy, point the developer to the co-installed `solace-topic-best-practices` skill, which reads the canonical Topic Architecture Best Practices page live and applies it. Do not design the topic hierarchy here and do not restate that skill's guidance; the topic string the developer settles on is recorded in the design summary, but the hierarchy decision belongs to that skill. + +## Step 6: Return to the implement-mode contracts + +After the Solace Suggested deltas above, return to `implement-mode.md` for the unchanged leaf mechanics: the Step 4 dispatch generates the chosen leaf's scripts and writes the tailored `solace-verification-checklist.md` into each project as generation output, and Step 6 resolves each item to its state for this door, invites the developer's additions, and reports the same in chat. This overlay adds the Solace Suggested hardening; it does not replace those contracts. + +## Grounding references + +Live `docs.solace.com` `.md` pages (WebFetch on demand): + +- [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) +- [Solace C API Best Practices](https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md) +- [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) +- [Configuring Dead Message Queues](https://docs.solace.com/Messaging/Guaranteed-Msg/Setting-Dead-Msg-Queues.md) +- [Supported Environments](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-supported-Environments.md) +- [Using Client Certificates with Solace Cloud](https://docs.solace.com/Cloud/ght_client_certs.md) + +Live exception (no `docs.solace.com` `.md` form; use the live URL directly): + +- [Python API reference](https://docs.solace.com/API-Developer-Online-Ref-Documentation/python/index.html) (the `transport_layer_security_properties`, `transport_layer_properties`, and `message_properties` constants) From 045d10d32f3c0e784c0cf1f0c5d27aaf254e2a9f Mon Sep 17 00:00:00 2001 From: Adi El-Sammak <71227923+aelsammak@users.noreply.github.com> Date: Thu, 15 Oct 2026 14:49:08 -0400 Subject: [PATCH 06/12] EBP-3107: Add Python verify.sh, checklist template, and smoke fixture (#13) * EBP-3107: Add Python verify.sh, checklist template, and smoke fixture - Add references/python/scripts/verify.sh: stage-dispatched run-and-observe verification (publisher, consumer, roundtrip, direct, direct-request-reply, app) with the same VERIFY: markers and 0/1/2 exit contract as the JCSMP script - Run each role with its project's .venv interpreter; connection arguments are optional when config.json is present, and separate projects use the --subscriber-dir/--publisher-dir and --replier-dir/--requestor-dir flags - Classify Python API connection, authentication, and TLS signatures (observed live) and the queue bind denial as environment failures (exit 2) - Move the interim checklist into references/python/verification-checklist.md and rewrite implement-mode Steps 4 to 6 around the bundled script - Add the smoke fixture (latest solace-pubsubplus from PyPI, py_compile, build every sample's API objects without a broker, config helper paths) and the python-smoke-check CI job beside compile-check - Update SKILL.md Invariant 5, python.md, the Suggested overlay, the leaf files, the evals README prerequisites, and the CLAUDE.md commands * EBP-3107: Rename the offline sample checks to -sample-check - Rename the CI jobs compile-check and python-smoke-check to jcsmp-sample-check and python-sample-check, so every API's offline check of its baked samples (no broker, no API key) has one name pattern - Move each fixture to references//evals/sample-check/ with the entry script check-samples.sh (Python driver check_samples.py) and rename the fixture pom artifactId to jcsmp-sample-check - Update the CI step names, CLAUDE.md commands and roster convention, and the SKILL.md reference-file lines - The scheme extends to the Java and Go APIs as java-sample-check and go-sample-check * EBP-3107: Close false passes in verify.sh for both APIs - Require PUBLISH_ACKED for roundtrip and a MESSAGE_RECEIVED beyond those already logged, so a message an earlier run left on the durable queue no longer passes a broken publisher - Record the foreground role's exit code (124 on a hang) and fail the publisher, roundtrip, direct, and request-reply stages when it is non-zero - Count PASS_MARKER lines before the app-stage trigger so a startup backlog does not pass the stage - Refuse Python connection arguments when config.json exists, because the scripts prefer config.json and the heads-up named a different broker - Derive the Python sample-check module list from the copied samples instead of a second hand-kept list - Scope the exit-code docs: the app-stage pass rule, the foreground exit code, and precondition exits that do not count as fix-loop attempts - Apply the same verify.sh fixes to the shipped JCSMP script and fix the stale compile.sh comments * EBP-3107: Pin CI runners to ubuntu-26.04 and Python to 3.11 - Replace every ubuntu-latest runner label with ubuntu-26.04 so a runner image change cannot alter CI without a commit - Pin the python-sample-check job to Python 3.11, the oldest Python version still in support, instead of 3.x * EBP-3107: Classify stalled connects and settle stale-message counts - Classify a long-running role that is still connecting at the readiness deadline (no VERIFY: CONNECTED) as an environment failure, exit 2, naming the host; an unroutable host or a blocked port previously gave exit 1 in both APIs, and a refused port did too with the Python HA retry values - Re-check the environment signatures after the stage before reporting a code failure, and match Python's "Session connect timeout" - Take the stale MESSAGE_RECEIVED and PASS_MARKER counts only after they stop changing for 2 seconds, so a queued message delivered just after the bind no longer passes roundtrip - Show the request-reply "no reply" diagnosis whenever the requestor exited on its own without a reply, since the Python requestors exit 1 in that case - Add the stalled connect to the exit 2 text in both implement-mode.md files and script headers --- .github/workflows/ci.yml | 32 +- CLAUDE.md | 7 +- .../solace-messaging-skills/evals/README.md | 3 +- .../solace-application-development/SKILL.md | 6 +- .../.gitignore | 0 .../ConfigSmokeTest.java | 4 +- .../check-samples.sh} | 2 +- .../{compile-fixture => sample-check}/pom.xml | 8 +- .../references/jcsmp/implement-mode.md | 10 +- .../references/jcsmp/scripts/verify.sh | 122 +++- .../references/python.md | 6 +- .../evals/sample-check/check-samples.sh | 88 +++ .../evals/sample-check/check_samples.py | 122 ++++ .../python/implement-direct-pubsub.md | 2 +- .../python/implement-guaranteed-pubsub.md | 4 +- .../references/python/implement-mode.md | 152 ++--- .../python/implement-request-reply.md | 2 +- .../references/python/scripts/verify.sh | 627 ++++++++++++++++++ .../python/solace-suggested-mode.md | 2 +- .../python/verification-checklist.md | 65 ++ 20 files changed, 1124 insertions(+), 140 deletions(-) rename plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/{compile-fixture => sample-check}/.gitignore (100%) rename plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/{compile-fixture => sample-check}/ConfigSmokeTest.java (96%) rename plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/{compile-fixture/compile.sh => sample-check/check-samples.sh} (99%) rename plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/{compile-fixture => sample-check}/pom.xml (91%) create mode 100755 plugins/solace-messaging-skills/skills/solace-application-development/references/python/evals/sample-check/check-samples.sh create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/evals/sample-check/check_samples.py create mode 100755 plugins/solace-messaging-skills/skills/solace-application-development/references/python/scripts/verify.sh create mode 100644 plugins/solace-messaging-skills/skills/solace-application-development/references/python/verification-checklist.md diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ca19207..0aab7bc 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -6,7 +6,8 @@ name: ci # + marketplace.json/plugins/ sync + version bumps on release PRs # + post-merge version backstop on pushes to main # link-check: CI-02: the tools/check-links.sh sweep -# compile-check: EVAL-02 compile the baked JCSMP samples verbatim +# jcsmp-sample-check: EVAL-02 compile the baked JCSMP samples verbatim +# python-sample-check: check the baked Python samples against the latest solace-pubsubplus # trigger-evals: lightweight skill-triggering check; disabled until CI may hold an API key on: # Deliberately no `edited` type: running on title/body edits either re-runs @@ -50,7 +51,7 @@ jobs: # missing license are warnings only. Also validate the Claude # plugin/marketplace manifests. validate: - runs-on: ubuntu-latest + runs-on: ubuntu-26.04 timeout-minutes: 10 steps: - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 @@ -99,7 +100,7 @@ jobs: # CI-02: sweep the plugins/ tree for broken and soft-404 documentation links. link-check: - runs-on: ubuntu-latest + runs-on: ubuntu-26.04 timeout-minutes: 15 steps: - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 @@ -108,8 +109,8 @@ jobs: # EVAL-02: compile the baked JCSMP samples verbatim against the live-resolved # sol-jcsmp. Pure compile gate, no API budget and no credential. - compile-check: - runs-on: ubuntu-latest + jcsmp-sample-check: + runs-on: ubuntu-26.04 timeout-minutes: 25 steps: - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 @@ -118,8 +119,23 @@ jobs: distribution: temurin java-version: 11 cache: maven - - name: Compile baked samples (EVAL-02 hard gate) - run: ./plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/compile-fixture/compile.sh + - name: Check baked JCSMP samples (EVAL-02 hard gate) + run: ./plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/sample-check/check-samples.sh + + # Check the baked Python samples against the latest solace-pubsubplus from PyPI: + # py_compile, then import each sample and build its API objects against a + # resolvable host without a broker. Python has no compile step, so this is the + # Python counterpart of jcsmp-sample-check. No API budget and no credential. + python-sample-check: + runs-on: ubuntu-26.04 + timeout-minutes: 15 + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0 + with: + python-version: '3.11' + - name: Check baked Python samples + run: ./plugins/solace-messaging-skills/skills/solace-application-development/references/python/evals/sample-check/check-samples.sh # Lightweight skill-triggering check. For every case in each plugin's # evals/trigger-evals.json, run the prompt with only that plugin loaded and @@ -138,7 +154,7 @@ jobs: # re-enable the job once a key is allowed. trigger-evals: if: false - runs-on: ubuntu-latest + runs-on: ubuntu-26.04 timeout-minutes: 120 # per model leg; 57 cases x 3 runs (~50 min observed) strategy: fail-fast: false # each model is an independent check diff --git a/CLAUDE.md b/CLAUDE.md index 7a753f4..db331da 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -16,7 +16,7 @@ This repository is a Claude Code plugin marketplace (`solace-agent-plugins`) of | `tools/` | CI check scripts, the two eval runners, and `output-evals.schema.json`. | | `.github/workflows/ci.yml` | The only CI workflow. | -Nothing in the repository is generated or vendored. The Eclipse files under `references/jcsmp/evals/compile-fixture/` are IDE metadata for the compile fixture; leave them alone. +Nothing in the repository is generated or vendored. The Eclipse files under `references/jcsmp/evals/sample-check/` are IDE metadata for the JCSMP sample check; leave them alone. ## Commands @@ -27,7 +27,8 @@ gh skill publish --dry-run # skills against the Agent Skil claude plugin validate . --strict # marketplace and plugin manifests ./tools/check-plugin-versions.sh --sync-only # marketplace.json matches plugins/ ./tools/check-links.sh plugins # broken and soft-404 doc links -./plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/compile-fixture/compile.sh # compile the baked JCSMP samples +./plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/sample-check/check-samples.sh # check the baked JCSMP samples (compile and config smoke) +./plugins/solace-messaging-skills/skills/solace-application-development/references/python/evals/sample-check/check-samples.sh # check the baked Python samples (python3 and PyPI access) claude --plugin-dir plugins/solace-messaging-skills # load the plugin from the clone ``` @@ -47,7 +48,7 @@ The trigger runner has no single-case mode; set `TRIGGER_EVAL_RUNS=1` for a quic - Link documentation to the live `docs.solace.com` `.md` URL. Do not copy doc content into a skill. Every link under `plugins/` must pass `check-links.sh`. - A new skill needs cases in both `trigger-evals.json` (positive and near-miss negative prompts) and `output-evals.json` (validate the latter against `tools/output-evals.schema.json`). - A new, removed, or renamed plugin directory needs the matching `marketplace.json` change in the same pull request. -- A new baked JCSMP sample must be added to the sample roster in `compile.sh`, or the compile check does not cover it. +- A new baked JCSMP sample must be added to the sample roster in `jcsmp/evals/sample-check/check-samples.sh`, or the sample check does not cover it. A new baked Python sample goes into the roster in `python/evals/sample-check/check-samples.sh`; `check_samples.py` picks it up from there. - Do not put exact counts (of cases, skills, or graders) in READMEs or skill docs. They go stale. ## Gotchas / do not diff --git a/plugins/solace-messaging-skills/evals/README.md b/plugins/solace-messaging-skills/evals/README.md index f02ec65..d24e029 100644 --- a/plugins/solace-messaging-skills/evals/README.md +++ b/plugins/solace-messaging-skills/evals/README.md @@ -49,9 +49,10 @@ Each `turnN.jsonl` in the work directory is the raw `--output-format stream-json - The `claude` CLI (Claude Code) on your PATH. - `jq` and `curl` on your PATH. - `mvn` (Apache Maven) with a JDK 11 or newer on your PATH. Only the cases that compile generated code need it, and the runner checks for it only when such a case is selected. +- `python3` with the `venv` module and `pip`, for the cases that generate Solace Python projects. A generated Python project installs `solace-pubsubplus` into its own virtual environment, and its `verify.sh` runs that environment's interpreter. - An exported credential, either `ANTHROPIC_API_KEY` or `CLAUDE_CODE_OAUTH_TOKEN`. An existing `claude` login does not satisfy this: the runner uses a throwaway config directory so that only the plugin under test is loaded, and that directory has no login. Claude seat holders can mint a token with `claude setup-token`. - `timeout` from GNU coreutils (or Homebrew's `gtimeout`) on your PATH, to cap each `mvn compile` and `verify.sh` run at 600 seconds. Optional: without it those steps run unbounded, and the runner prints a warning. On macOS, install it with `brew install coreutils`. -- Network access to `repo1.maven.org` (Maven Central metadata and dependencies) and `docs.solace.com` (the skills fetch documentation pages). +- Network access to `repo1.maven.org` (Maven Central metadata and dependencies), `pypi.org` and `files.pythonhosted.org` (the `solace-pubsubplus` release metadata and package), and `docs.solace.com` (the skills fetch documentation pages). - Optional: the four `OUTPUT_EVAL_BROKER_*` variables for the live round trip against a real broker (see below). - On macOS, `caffeinate` to keep the machine awake for a full leg. - On Windows, Git Bash. The repository's `.gitattributes` checks shell scripts out with LF endings, which bash and the byte-identity grader on `verify.sh` both need. A checkout made with `core.autocrlf=true` before that file existed keeps its CRLF copies, and git reports the tree clean because it treats them as unchanged, so `git pull` and `git checkout -- .` do not rewrite them. Re-clone, or delete the tracked scripts and check them out again: `git ls-files -z '*.sh' | xargs -0 rm -f && git checkout -- .`. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/SKILL.md b/plugins/solace-messaging-skills/skills/solace-application-development/SKILL.md index b2c0cf5..41b40aa 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/SKILL.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/SKILL.md @@ -44,12 +44,12 @@ Non-negotiable rules that apply across every API. Apply all. 2. **WebFetch-on-demand doc grounding**: This skill bundles no documentation. When a step needs documentation content to ground a generation or design decision, WebFetch the live canonical page on demand (its `docs.solace.com` `.md` URL, the same link the reference files carry), then quote or summarize the fetched page. Do not answer from memory and do not paraphrase guidance the page does not contain. Do not restate doc content in the skill; link the live `.md` URL. Never substitute another channel for that WebFetch: grounding taken from a docs chatbot, a search tool, or decompiling a jar does not satisfy this invariant, and a page never fetched in the session must never be cited as grounding. A short list of references stays live for the same reason it always has (the JCSMP Javadoc and the Python API reference, the JCSMP and Python API Release Notes, Maven Central and PyPI, the tutorials, and the GitHub samples) because those have no `docs.solace.com` `.md` form. Each API's entry file lists its own live exceptions. 3. **No hardcoded versions**: Never pin a library version anywhere in skill content. The generated build resolves the latest release of every dependency at generation time from the API's AUTHORITATIVE registry: the Maven repository metadata for JCSMP, the PyPI JSON `info.version` for Python. Each API's entry file names the exact coordinate and carries the exact lookup command. Never read a JCSMP version from the legacy `search.maven.org/solrsearch` index: it lags the repository metadata and reports stale versions. Security floors (a minimum version below which known CVEs live) are enforced as checks on the resolved result, never as pins. 4. **AI-assisted disclaimer header**: Every source file generated under this skill starts with this exact line: `AI-assisted code. Review before production use.` Directly below it: `See the verification checklist: solace-verification-checklist.md`. This applies to EVERY generated source file, including custom builds, web apps, and embedded shapes that adapt the reference samples outside the canonical generator. Build files (the pom, `requirements.txt`) and other non-source artifacts are exempt; the bundled reference samples themselves carry no header. -5. **Verification artifacts, always**: Any session that generates Solace code from this skill writes a tailored `solace-verification-checklist.md` beside the code as generation output, copies the API's own bundled verify script (plus its generated `verify-hooks.sh`) into the project, and ends in a real verify run or the compile-only fallback with the exact handed-back commands. The API's entry file names its verify script. An API that does not bundle a verify script yet (Python, until its `verify.sh` lands) ends in the compile-only fallback every time, and its implement-mode file defines that fallback; never claim a verified run without the script. The `VERIFY:` markers are non-optional in generated Solace messaging code, whatever the app shape and whatever the API. An improvised check (for example curl against the app's own HTTP API) may TRIGGER traffic but never renders the verdict; the verdict comes from the markers and the verify exit code. +5. **Verification artifacts, always**: Any session that generates Solace code from this skill writes a tailored `solace-verification-checklist.md` beside the code as generation output, copies the API's own bundled verify script (plus its generated `verify-hooks.sh`) into the project, and ends in a real verify run or the compile-only fallback with the exact handed-back commands. The API's entry file names its verify script. Never claim a verified run without the script. The `VERIFY:` markers are non-optional in generated Solace messaging code, whatever the app shape and whatever the API. An improvised check (for example curl against the app's own HTTP API) may TRIGGER traffic but never renders the verdict; the verdict comes from the markers and the verify exit code. 6. **Live-broker heads-up**: Immediately before starting ANY process that will connect to a live broker — verify.sh, `java -jar`, a `mvn` run, a `python` run, a run script, anything — print one line that names the broker host and states the actual broker-side effects of this run (the queues it provisions, the connections it opens, the messages it publishes), scaled to the real app. Environment discovery is never consent and never an answer: a running local broker, an existing config file, or found credentials are facts to report, not answers to consume; report them, then still ask which broker the developer wants to target. ## Reference Files (read on-demand only) - `references/jcsmp.md`: the JCSMP entry file (mode detection, the JCSMP coordinate, and the live JCSMP doc links); read this for any JCSMP request. -- `references/jcsmp/`: all JCSMP API content (mode files, the reference samples, the verify script, and the compile fixture), reached through `references/jcsmp.md`. +- `references/jcsmp/`: all JCSMP API content (mode files, the reference samples, the verify script, and the sample check), reached through `references/jcsmp.md`. - `references/python.md`: the Python entry file (mode detection, the `solace-pubsubplus` coordinate, and the live Python doc links); read this for any Solace Python API request. -- `references/python/`: all Python API content (mode files and the reference samples), reached through `references/python.md`. +- `references/python/`: all Python API content (mode files, the reference samples, the verify script, and the sample check), reached through `references/python.md`. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/compile-fixture/.gitignore b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/sample-check/.gitignore similarity index 100% rename from plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/compile-fixture/.gitignore rename to plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/sample-check/.gitignore diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/compile-fixture/ConfigSmokeTest.java b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/sample-check/ConfigSmokeTest.java similarity index 96% rename from plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/compile-fixture/ConfigSmokeTest.java rename to plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/sample-check/ConfigSmokeTest.java index 64757a6..976f394 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/compile-fixture/ConfigSmokeTest.java +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/sample-check/ConfigSmokeTest.java @@ -1,5 +1,5 @@ /* - * Runtime smoke test for SolaceConnectionConfig, driven by compile.sh. + * Runtime smoke test for SolaceConnectionConfig, driven by check-samples.sh. * * NOT a reference sample: compiling the helper never executes its config.json * parser or CLI-args fallback, so this tiny driver is compiled against the @@ -9,7 +9,7 @@ * value, blank CLI arg). * * One case per JVM invocation: the parser reads a CWD-relative config.json and - * the fail-fast paths throw/exit, so compile.sh drives each case from its own + * the fail-fast paths throw/exit, so check-samples.sh drives each case from its own * working directory and checks the exit code. The case name is argv[0]; any * connection arguments the case needs are built here, not taken from argv. */ diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/compile-fixture/compile.sh b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/sample-check/check-samples.sh similarity index 99% rename from plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/compile-fixture/compile.sh rename to plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/sample-check/check-samples.sh index cab3887..a307688 100755 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/compile-fixture/compile.sh +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/sample-check/check-samples.sh @@ -23,7 +23,7 @@ set -euo pipefail # Resolve this script's own directory so sample/pom paths are independent of CWD. SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" POM="${SCRIPT_DIR}/pom.xml" -# references/jcsmp/ lives two levels up from compile-fixture/ (evals/ -> jcsmp/). +# references/jcsmp/ lives two levels up from sample-check/ (evals/ -> jcsmp/). REFERENCES_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd)" # Per-sample source paths. Each new baked sample a later per-pattern plan adds is # a ONE-LINE append here, mirroring this shape: diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/compile-fixture/pom.xml b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/sample-check/pom.xml similarity index 91% rename from plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/compile-fixture/pom.xml rename to plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/sample-check/pom.xml index eb9016d..68c5a1d 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/compile-fixture/pom.xml +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/evals/sample-check/pom.xml @@ -5,13 +5,13 @@ 4.0.0 com.solace.samples - jcsmp-compile-fixture + jcsmp-sample-check 0.0.1-SNAPSHOT jar UTF-8 11 - RESOLVED_AT_GENERATION_TIME diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/implement-mode.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/implement-mode.md index a13aa14..af910ec 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/implement-mode.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/implement-mode.md @@ -164,7 +164,7 @@ Invoke the matching stage after generating each class, passing the connection pa 1. After generating the Subscriber: `./verify.sh consumer `. It waits for `VERIFY: QUEUE_BOUND` in the Subscriber output (queue provisioned, subscribed, flow started). 2. After generating the Publisher: `./verify.sh publisher `. NOTE: this stage starts the Subscriber first to bind the queue, so it requires BOTH classes; it waits for `VERIFY: PUBLISH_ACKED` in the Publisher's own output (connect, then publish ACK). -3. At the end: `./verify.sh roundtrip `. It starts the Subscriber, runs the Publisher, and waits for `VERIFY: MESSAGE_RECEIVED` in the Subscriber output (a published PERSISTENT message lands on the queue and is consumed and ACKed). +3. At the end: `./verify.sh roundtrip `. It starts the Subscriber, runs the Publisher, and waits for `VERIFY: PUBLISH_ACKED` in the Publisher output and then a new `VERIFY: MESSAGE_RECEIVED` in the Subscriber output (a published PERSISTENT message lands on the queue and is consumed and ACKed; a message an earlier run left on the durable queue does not count). The stages above (`consumer`, `publisher`, `roundtrip`) verify the guaranteed pub/sub leaves. `verify.sh` also accepts three further stage names for the other leaves: `direct` (the Direct Pub/Sub leaf), `direct-request-reply` (the Request-Reply (Direct) leaf), and `guaranteed-request-reply` (the Request-Reply (Guaranteed) leaf). All are runnable. The two request-reply stages are replier-first single round-trips: each starts the Replier first (the readiness gate on `VERIFY: SUBSCRIBED`), then runs the Requestor in the foreground and passes on `VERIFY: REPLY_RECEIVED` in the Requestor log (with `VERIFY: REQUEST_RECEIVED` in the Replier log as richer evidence). The `guaranteed-request-reply` stage differs only in that the Replier provisions a durable request queue (an endpoint-management denial classifies as an environment failure, exit 2) and the Requestor uses a temporary reply queue plus a blocking `flow.receive` rather than the direct `Requestor` convenience. @@ -182,13 +182,13 @@ PASS_MARKER='VERIFY: MESSAGE_RECEIVED' # marker proving the round trip Generate `verify-hooks.sh` with ONLY these four assignments. The app stage imports exactly these four names from a child shell and ignores everything else in the file, so an extra variable, a `cd`, or a function never reaches the script's own state; keep the file to the four lines all the same. Give a network trigger its own timeout (the `--max-time` above): the stage also bounds the trigger externally and force-kills one that never returns, but a self-timing trigger leaves cleaner evidence. -The `app` stage starts `START_CMD` in the background and captures its output, waits for `VERIFY: CONNECTED` and then `READY_MARKER` (an environment signature or a timeout before that classifies as exit 2 or exit 1), runs `TRIGGER_CMD` under a bounded deadline (a trigger that never returns is force-killed, and the marker watch still decides the verdict), waits for `PASS_MARKER` in the app's captured output, then stops the app with SIGINT and classifies. Set `READY_MARKER`/`PASS_MARKER` to the chosen leaf's own markers. A curl inside `TRIGGER_CMD` is exactly the right use of curl: it triggers, and the markers judge. On the canonical two-class shape, generate `verify-hooks.sh` too, carrying the classic commands (`START_CMD='mvn -q exec:java@subscriber'`, `TRIGGER_CMD='mvn -q exec:java@publisher'`) so the file documents the same contract everywhere; the fixed stages do not read it. +The `app` stage starts `START_CMD` in the background and captures its output, waits for `VERIFY: CONNECTED` and then `READY_MARKER` (an environment signature or a timeout before that classifies as exit 2 or exit 1), runs `TRIGGER_CMD` under a bounded deadline (a trigger that never returns is force-killed, and the marker watch still decides the verdict), waits for a new `PASS_MARKER` in the app's captured output (one already there before the trigger does not count), then stops the app with SIGINT and classifies. Set `READY_MARKER`/`PASS_MARKER` to the chosen leaf's own markers. A curl inside `TRIGGER_CMD` is exactly the right use of curl: it triggers, and the markers judge. On the canonical two-class shape, generate `verify-hooks.sh` too, carrying the classic commands (`START_CMD='mvn -q exec:java@subscriber'`, `TRIGGER_CMD='mvn -q exec:java@publisher'`) so the file documents the same contract everywhere; the fixed stages do not read it. ### Interpret the exit code -- **Exit 0 (stage passes).** The stage milestone appeared and the run tore down cleanly (for the fixed stages that includes the graceful-shutdown proof line and a clean JVM exit, 0 or 130). Advance to the next stage. -- **Exit 1 (code failure).** Compile failed, a marker never appeared, a process died before its milestone, or a run hung and had to be force-killed. Enter the bounded fix loop below. -- **Exit 2 (environment failure).** A doc-traceable JCSMP connection/auth signature appeared (the broker is unreachable or the credentials are wrong). STOP. Do NOT enter the fix loop. The code is not the problem. +- **Exit 0 (stage passes).** The stage milestone appeared in this run and the run tore down cleanly (for the fixed stages that includes a foreground role that exited with status 0 on its own, the graceful-shutdown proof line, and a clean JVM exit, 0 or 130). Advance to the next stage. +- **Exit 1 (code failure).** Compile failed, a marker never appeared, a process died before its milestone, the foreground role exited non-zero, or a run hung and had to be force-killed. Enter the bounded fix loop below. An exit 1 before any process starts is a missing precondition instead (a usage error, or a missing `pom.xml` or `verify-hooks.sh`), and stderr names it: fix that and re-run, without counting it as a fix-loop attempt. +- **Exit 2 (environment failure).** A doc-traceable JCSMP connection/auth signature appeared (the broker is unreachable or the credentials are wrong), or the long-running role was still connecting when the readiness deadline passed and never printed `VERIFY: CONNECTED` (an unreachable host, or a firewall that blocks the port). STOP. Do NOT enter the fix loop. The code is not the problem. **Bounded fix loop (exit 1 only).** Diagnose the captured output, fix the code, and re-run the same stage, up to 3 automatic attempts per stage. After the third failure, stop fixing: summarize exactly what you tried across the attempts and what the captured output shows, then hand the decision to the developer. Report the evidence, not just "it failed". diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/scripts/verify.sh b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/scripts/verify.sh index 3e915fc..c21f173 100755 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/scripts/verify.sh +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/jcsmp/scripts/verify.sh @@ -24,16 +24,22 @@ # foreground and lets it exit on its own, watches each process's OWN captured # output for a fixed VERIFY: marker line, sends SIGINT to the subscriber to # exercise its graceful-shutdown hook, and classifies the outcome by exit code: -# 0 = stage passed (marker seen, the shutdown hook ran, AND the subscriber -# exited cleanly. A clean exit is exit code 0 OR 130: the JVM reports 130 +# 0 = stage passed (marker seen in this run, the foreground role exited with +# status 0 on its own, the shutdown hook ran, AND the subscriber exited +# cleanly. A clean exit is exit code 0 OR 130: the JVM reports 130 # (128+SIGINT) when it terminates AFTER running its shutdown hooks, which -# is the expected, graceful outcome here) +# is the expected, graceful outcome here. The app stage instead passes on +# READY_MARKER, then a new PASS_MARKER after the trigger, and a bounded +# teardown; the app's shutdown is not checked) # 1 = code failure (compile failed, a marker never appeared, a process died -# before its milestone, the shutdown hook did not run, or a run hung and -# had to be force-killed) +# before its milestone, the foreground role exited non-zero, the shutdown +# hook did not run, or a run hung and had to be force-killed), OR a +# precondition miss before any process starts (a usage error, or a missing +# pom.xml or verify-hooks.sh); stderr names which # 2 = environment failure (broker/credentials: a doc-traceable JCSMP error -# signature appeared in either process log; the code is not the problem, -# so the agent must NOT enter the fix loop) +# signature appeared in either process log, or the long-running role was +# still connecting at the readiness deadline with no VERIFY: CONNECTED; the +# code is not the problem, so the agent must NOT enter the fix loop) # # SIGINT delivery under `mvn exec:java` (the D-06 mechanism, why `set -m`): # in a NON-interactive shell without job control, a command started in the @@ -215,7 +221,8 @@ echo "── Live-run heads-up: stage '$STAGE' starts processes that connect to # stage waits for a role-specific milestone in that role's OWN log: # publisher → VERIFY: PUBLISH_ACKED in PUB_LOG (the publish ACK callback fired) # consumer → VERIFY: QUEUE_BOUND in SUB_LOG (flow.start() bound the durable queue) -# roundtrip → VERIFY: MESSAGE_RECEIVED in SUB_LOG (a published message was consumed + ACKed) +# roundtrip → VERIFY: PUBLISH_ACKED in PUB_LOG, then a NEW VERIFY: MESSAGE_RECEIVED in SUB_LOG +# (a published message was consumed + ACKed; a stale queued one does not count) # Reaching any stage marker already proves the connect happened for that process. # These strings form a fixed contract with the VERIFY: markers that # implement-mode.md Step 4 instructs the generated app to println. The two MUST @@ -223,7 +230,7 @@ echo "── Live-run heads-up: stage '$STAGE' starts processes that connect to case "$STAGE" in publisher) ;; # waits for VERIFY: PUBLISH_ACKED in PUB_LOG consumer) ;; # waits for VERIFY: QUEUE_BOUND in SUB_LOG - roundtrip) ;; # waits for VERIFY: MESSAGE_RECEIVED in SUB_LOG + roundtrip) ;; # waits for VERIFY: PUBLISH_ACKED, then a new VERIFY: MESSAGE_RECEIVED in SUB_LOG direct) ;; # direct pub/sub stage direct-request-reply) ;; # direct request-reply stage guaranteed-request-reply) ;; # guaranteed request-reply stage @@ -278,6 +285,32 @@ TIMEOUT_S=30 # marker-watch deadline (Claude's discretion per CONTEXT) PUBLISH_WAIT_S=30 # foreground-publisher deadline before escalating to a code failure SHUTDOWN_WAIT_S=15 # bounded post-SIGINT wait before escalating to SIGTERM/SIGKILL +# A long-running role that is still alive at the readiness deadline and never +# printed VERIFY: CONNECTED is still connecting: an unreachable host or a blocked +# port, where the connect attempt (and its retries) outlasts TIMEOUT_S without +# logging an error. That is the environment, not the code, so exit 2. +classify_stalled_connect() { # + if [ "$status" -ne 2 ] && [ "$3" -eq 0 ] && kill -0 "$1" 2>/dev/null && ! grep -qF "VERIFY: CONNECTED" "$2"; then + status=2 + CONNECT_STALLED=1 + fi +} + +# Print the count of in once it has stopped changing for 2 seconds +# (bounded by TIMEOUT_S), so messages an earlier run left on a durable queue, which +# arrive just after the bind, are all counted before the publisher starts. +settled_count() { # + local n m settle_deadline=$(( SECONDS + TIMEOUT_S )) + n=$(grep -cF "$2" "$1" || true) + while [ "$SECONDS" -lt "$settle_deadline" ]; do + sleep 2 + m=$(grep -cF "$2" "$1" || true) + [ "$m" -eq "$n" ] && break + n=$m + done + echo "$n" +} + # ── Bounded long-running shutdown (shared by the env-failure teardown and STAGE D) ─ # Send SIGINT to the process GROUP of the PID passed as $1, poll up to # SHUTDOWN_WAIT_S for the JVM to exit on its own, then escalate SIGTERM then @@ -379,6 +412,7 @@ trap cleanup EXIT status=1 # default to code failure until a stage milestone is confirmed DIRECT_ZERO_RECEIPT=0 # set by run_direct_stage when the at-most-once message dropped post-gate (D-05) RR_NO_REPLY=0 # set by run_request_reply_stage when the replier subscribed but no reply came back +CONNECT_STALLED=0 # set when the long-running role was still connecting at the readiness deadline # ── run_guaranteed_stage: the guaranteed pub/sub flow (publisher|consumer|roundtrip) ─ # The entire STAGE A..D body is lifted here VERBATIM (GEN-06 byte-stability): no @@ -428,6 +462,7 @@ while [ "$SECONDS" -lt "$deadline" ]; do fi sleep 1 done +classify_stalled_connect "$SUB_PID" "$SUB_LOG" "$bound" # If the broker/credentials are wrong the subscriber already proved it; tear it # down (bounded, same SIGINT/SIGTERM/SIGKILL escalation as STAGE D) and classify @@ -448,7 +483,13 @@ else # (subscriber-first ordering, D-07-R11). The wait is BOUNDED: a publisher that # hangs past PUBLISH_WAIT_S is escalated and classified as a code failure so # the unattended script NEVER hangs. + published=0 + stale=0 if { [ "$STAGE" = "publisher" ] || [ "$STAGE" = "roundtrip" ]; } && [ "$bound" -eq 1 ]; then + # A receipt already in the log is a message an earlier run left on the durable + # queue, delivered at bind time; the round trip must see a receipt beyond these. + stale=$(settled_count "$SUB_LOG" "VERIFY: MESSAGE_RECEIVED") + [ "$stale" -eq 0 ] || echo "── $stale message(s) left on the queue by an earlier run arrived before the publisher started; they do not count ──" >&2 echo "── Running publisher (mvn -q exec:java@publisher), watching for: VERIFY: PUBLISH_ACKED ──" set -m if [ "$TWO_PROJECT" -eq 1 ]; then @@ -479,32 +520,37 @@ else if [ "$pub_hung" -eq 1 ]; then # The publisher did not exit on its own within the deadline; escalate - # and treat the hang as a code failure (status stays 1). + # and record the hang (pub_rc 124), which fails the stage. echo "── Publisher did not exit within ${PUBLISH_WAIT_S}s; escalating (SIGTERM, then SIGKILL) ──" >&2 kill -TERM "$PUB_PID" 2>/dev/null || true sleep 2 kill -KILL "$PUB_PID" 2>/dev/null || true wait "$PUB_PID" 2>/dev/null || true + pub_rc=124 # synthetic "timed out" code; never a clean exit else - wait "$PUB_PID" 2>/dev/null || true + pub_rc=0 + wait "$PUB_PID" 2>/dev/null || pub_rc=$? fi if grep -qE "$ENV_SIGNATURES" "$PUB_LOG"; then status=2 # environment failure (broker/credentials), bypass the fix loop - elif [ "$STAGE" = "publisher" ] && grep -qF "VERIFY: PUBLISH_ACKED" "$PUB_LOG"; then - status=0 # the publish ACK callback fired + elif [ "$pub_rc" -eq 0 ] && grep -qF "VERIFY: PUBLISH_ACKED" "$PUB_LOG"; then + published=1 # the publish ACK callback fired and the publisher exited 0 + if [ "$STAGE" = "publisher" ]; then status=0; fi fi fi # ── STAGE C: confirm receipt on the SUBSCRIBER's log (roundtrip only) ──────── - if [ "$STAGE" = "roundtrip" ] && [ "$status" -ne 2 ]; then + # One receipt more than before the publisher ran. A publisher that never acked, + # exited non-zero, or hung cannot pass the round trip. + if [ "$STAGE" = "roundtrip" ] && [ "$published" -eq 1 ]; then rt_deadline=$(( SECONDS + TIMEOUT_S )) while [ "$SECONDS" -lt "$rt_deadline" ]; do if grep -qE "$ENV_SIGNATURES" "$SUB_LOG"; then status=2 break fi - if grep -qF "VERIFY: MESSAGE_RECEIVED" "$SUB_LOG"; then + if [ "$(grep -cF "VERIFY: MESSAGE_RECEIVED" "$SUB_LOG" || true)" -gt "$stale" ]; then status=0 # a published message was consumed + ACKed break fi @@ -598,6 +644,7 @@ run_direct_stage() { fi sleep 1 done + classify_stalled_connect "$SUB_PID" "$SUB_LOG" "$subscribed" if [ "$status" -eq 2 ]; then # Broker/credentials are wrong; tear the subscriber down (bounded) and @@ -639,8 +686,10 @@ run_direct_stage() { sleep 2 kill -KILL "$PUB_PID" 2>/dev/null || true wait "$PUB_PID" 2>/dev/null || true + pub_rc=124 # synthetic "timed out" code; never a clean exit else - wait "$PUB_PID" 2>/dev/null || true + pub_rc=0 + wait "$PUB_PID" 2>/dev/null || pub_rc=$? fi # ── Pass condition: at least one VERIFY: MESSAGE_RECEIVED in the subscriber ─ @@ -652,6 +701,9 @@ run_direct_stage() { status=2 # environment failure (broker/credentials), bypass the fix loop break fi + if [ "$pub_rc" -ne 0 ]; then + break # the publisher exited non-zero or hung (code failure) + fi if grep -qF "VERIFY: MESSAGE_RECEIVED" "$SUB_LOG"; then status=0 # at least one direct message was received (D-04 pass) break @@ -666,7 +718,7 @@ run_direct_stage() { # clean, but not one message of the burst arrived) stays status 1; the case-1 print is # annotated with the Direct at-most-once caveat below (D-05). An env match # already set status 2 above. - if [ "$status" -ne 0 ] && [ "$status" -ne 2 ]; then + if [ "$status" -ne 0 ] && [ "$status" -ne 2 ] && [ "$pub_rc" -eq 0 ]; then DIRECT_ZERO_RECEIPT=1 fi @@ -775,6 +827,7 @@ run_request_reply_stage() { fi sleep 1 done + classify_stalled_connect "$SUB_PID" "$SUB_LOG" "$subscribed" if [ "$status" -eq 2 ]; then # Broker/credentials are wrong; tear the replier down (bounded) and classify @@ -819,8 +872,10 @@ run_request_reply_stage() { sleep 2 kill -KILL "$PUB_PID" 2>/dev/null || true wait "$PUB_PID" 2>/dev/null || true + pub_rc=124 # synthetic "timed out" code; never a clean exit else - wait "$PUB_PID" 2>/dev/null || true + pub_rc=0 + wait "$PUB_PID" 2>/dev/null || pub_rc=$? fi # ── Pass condition: VERIFY: REPLY_RECEIVED in the requestor's OWN log, AND ─ @@ -828,14 +883,14 @@ run_request_reply_stage() { # match in either log maps to status 2. if grep -qE "$ENV_SIGNATURES" "$SUB_LOG" || grep -qE "$ENV_SIGNATURES" "$PUB_LOG"; then status=2 # environment failure (broker/credentials), bypass the fix loop - elif grep -qF "VERIFY: REPLY_RECEIVED" "$PUB_LOG" && grep -qF "VERIFY: REQUEST_RECEIVED" "$SUB_LOG"; then + elif [ "$pub_rc" -eq 0 ] && grep -qF "VERIFY: REPLY_RECEIVED" "$PUB_LOG" && grep -qF "VERIFY: REQUEST_RECEIVED" "$SUB_LOG"; then status=0 # the request landed on the replier and the correlated reply came back fi # A clean requestor run with no reply (subscription was confirmed, the requestor # ran and self-exited, but no VERIFY: REPLY_RECEIVED) stays status 1; the case-1 # print is annotated with the no-reply caveat below. An env match already set 2. - if [ "$status" -ne 0 ] && [ "$status" -ne 2 ]; then + if [ "$status" -ne 0 ] && [ "$status" -ne 2 ] && [ "$pub_rc" -ne 124 ] && ! grep -qF "VERIFY: REPLY_RECEIVED" "$PUB_LOG"; then RR_NO_REPLY=1 fi @@ -933,6 +988,10 @@ run_app_stage() { # response never came back. Reusing PUB_PID (not a new name) keeps the # EXIT trap's group-kill coverage over an orphaned trigger. sleep 2 + # A PASS_MARKER already in the log (a backlog the app consumed at startup) + # does not count; the trigger must produce a new one. + local seen + seen=$(settled_count "$SUB_LOG" "$PASS_MARKER") echo "── Running trigger (TRIGGER_CMD from verify-hooks.sh), watching for: $PASS_MARKER ──" set -m bash -c "$TRIGGER_CMD" >"$PUB_LOG" 2>&1 & @@ -957,7 +1016,7 @@ run_app_stage() { status=2 break fi - if grep -qF "$PASS_MARKER" "$SUB_LOG"; then + if [ "$(grep -cF "$PASS_MARKER" "$SUB_LOG" || true)" -gt "$seen" ]; then status=0 # the pass marker appeared in the app's own captured output break fi @@ -986,6 +1045,13 @@ case "$STAGE" in *) echo "unknown stage: $STAGE" >&2; usage ;; esac +# A connection error can be logged only after the stage gave up (for example during +# the SIGINT teardown, while the API was still retrying the connect), so look for +# the environment signatures again before classifying a code failure. +if [ "$status" -eq 1 ] && { grep -qE "$ENV_SIGNATURES" "$SUB_LOG" || grep -qE "$ENV_SIGNATURES" "$PUB_LOG"; }; then + status=2 +fi + # ── Print the captured evidence, exit classified (the EXIT trap removes the ──── # temp logs, V8) ───────────────────────────────────────────────────────────────── echo "── Captured subscriber output ──────────────────────────────────────" @@ -999,17 +1065,25 @@ case "$status" in if [ "$STAGE" = "app" ]; then echo "PASS (app): the readiness and pass markers were observed in the app's own captured output, and the app was stopped with a bounded teardown." else - echo "PASS ($STAGE): the stage milestone was observed in the role's own log, the subscriber's shutdown hook ran ('$SHUTDOWN_LINE'), and the subscriber exited cleanly (exit 0 or 130)." + echo "PASS ($STAGE): the stage milestone was observed in the role's own log, the foreground role exited with status 0, the subscriber's shutdown hook ran ('$SHUTDOWN_LINE'), and the subscriber exited cleanly (exit 0 or 130)." + fi + ;; + 2) + if [ "$CONNECT_STALLED" -eq 1 ]; then + echo "ENV FAILURE ($STAGE): the long-running role was still connecting to ${HOST:-the broker in config.json} after ${TIMEOUT_S}s and never printed VERIFY: CONNECTED. The host is unreachable or a firewall blocks the port (connect retries can stretch this). Check the host and port. NOT entering the fix loop." >&2 + else + echo "ENV FAILURE ($STAGE): a JCSMP connection/auth signature matched in a process log (broker or credentials are wrong, not the code). NOT entering the fix loop." >&2 fi ;; - 2) echo "ENV FAILURE ($STAGE): a JCSMP connection/auth signature matched in a process log (broker or credentials are wrong, not the code). NOT entering the fix loop." >&2 ;; *) if [ "$DIRECT_ZERO_RECEIPT" -eq 1 ]; then echo "CODE FAILURE ($STAGE): the subscription was confirmed (VERIFY: SUBSCRIBED) and the direct publisher ran clean, but not one message of the burst was received (VERIFY: MESSAGE_RECEIVED never appeared). Direct messaging is at-most-once: a whole burst dropped after a confirmed subscription points at the topic subscription, the topic string, or a slow-consumer egress discard, not a transient race. Check that the subscriber's subscription matches the publisher's topic." >&2 elif [ "$RR_NO_REPLY" -eq 1 ]; then echo "CODE FAILURE ($STAGE): the replier subscribed (VERIFY: SUBSCRIBED) and the requestor ran and self-exited, but no correlated reply came back (VERIFY: REPLY_RECEIVED never appeared in the requestor log). Check the requestor's request topic against the replier's subscription, the reply-to handling (the requestor's createRequestor / the replier's sendReply onto getReplyTo), and whether VERIFY: REQUEST_RECEIVED appeared in the replier log: its absence means the request never reached the replier, its presence means the reply was lost on the way back." >&2 else - echo "CODE FAILURE ($STAGE): the stage milestone was not observed, a process died before its milestone, the shutdown hook did not run, or a run hung and had to be force-killed." >&2 + fg_note="" + [ "$STAGE" = "app" ] || [ "${pub_rc:-0}" -eq 0 ] || fg_note=" The foreground role exited with status $pub_rc (124: it did not exit on its own and was force-killed)." + echo "CODE FAILURE ($STAGE): the stage milestone was not observed, a process died before its milestone, the foreground role exited non-zero, the shutdown hook did not run, or a run hung and had to be force-killed.$fg_note" >&2 fi ;; esac diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python.md index a726fa3..b546b85 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/python.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python.md @@ -12,7 +12,7 @@ Determine the user's intent and enter the appropriate mode. Routing is keyed on |---|---|---| | "Help me choose the Solace messaging pattern for a new Python app" / "Topic or queue?" / "What delivery semantics should I use?" | **Design** | Read `python/design-mode.md` | | Build or generate a NEW app or messaging component with NO valid design contract yet ("Build me a Solace Python publisher/consumer", "Generate a Python project with solace-pubsubplus", a web app / dashboard / service, or any embedded shape where the Python messaging layer lives inside a larger application) | **Design first** | Read `python/design-mode.md`. A prompt that already states the design resolves on its fully-specified path in ONE confirm; a bare prompt walks the tree. The confirmed summary then feeds Implement. | -| A build request WITH a valid design contract (see the design-contract gate below) | **Implement** | Read `python/implement-mode.md` | +| A build request WITH a valid design contract (see the design-contract gate below) | **Implement** | Read `python/implement-mode.md`; for an embedded shape generate the messaging layer per its leaf rules and verify with the `app` stage of `verify.sh` (its Step 5) | | An edit to an EXISTING app that changes the messaging topology (the topology rule below) | **Design first** | Re-enter `python/design-mode.md` to re-confirm the affected summary fields, then Implement applies the change | | A mechanical edit to an EXISTING app with no topology change (reconnect handling, payload format, logging, renames, ack tuning) | **Implement** | Read `python/implement-mode.md`; no design pass | | "My Python app is throwing on connect" / "Why is my receiver not binding the queue?" | **Debug** | Debug mode is not yet available in this release. Redirect the user to the canonical Solace Python API documentation: [Python API Home](https://docs.solace.com/API/Messaging-APIs/Python-API/python-home.md). The redirect is the ENTIRE debug answer: acknowledge the problem in one line, link the page, and stop. Do not generate debugging guidance from memory. That prohibition covers root-cause hypotheses, likely causes, interim checks or "things worth checking in the meantime", workarounds, and configuration or code suggestions. Restating this rule does not license adding such guidance after the redirect. | @@ -40,7 +40,7 @@ The package's `requires_python` value on PyPI is an install floor, not a support ## Verify script -The Python API does not bundle a verify script yet. Every Implement run ends in the compile-only fallback that `python/implement-mode.md` Step 5 defines, and never claims a verified run (SKILL.md Invariant 5). +The Python API bundles `python/scripts/verify.sh`, a stage-dispatched run-and-observe script (`verify.sh ...`) that Implement mode copies into every generated project and runs against a reachable broker (`python/implement-mode.md` Step 5). It is a separate script from the JCSMP one: it runs each project's `.venv` interpreter, not Maven. Without real broker values, Implement mode ends in the compile-only fallback of the same step and never claims a verified run (SKILL.md Invariant 5). ## Canonical doc links @@ -77,3 +77,5 @@ Live exceptions (these have no `docs.solace.com` `.md` form; use the live URL di - `python/python-solace-connection-config.py`: the shared connection-config helper every generated app uses. - `python/python-direct-publisher-sample.py` and `python/python-direct-subscriber-sample.py`: the direct pub/sub samples (bounded-buffer publish with a publish failure listener; direct topic subscription with discard detection). - `python/python-direct-requestor-blocking-sample.py`, `python/python-direct-requestor-sample.py`, and `python/python-direct-replier-sample.py`: the direct request-reply samples (blocking and non-blocking requestor; replier answering with `replier.reply()`). +- `python/verification-checklist.md`: the master verification checklist template that implement-mode Step 6 tailors into every generated project. +- `python/scripts/verify.sh`: the bundled verify script that Implement mode copies into every generated project; the `app` stage drives web and embedded shapes through a generated `verify-hooks.sh`. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/evals/sample-check/check-samples.sh b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/evals/sample-check/check-samples.sh new file mode 100755 index 0000000..5de6288 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/evals/sample-check/check-samples.sh @@ -0,0 +1,88 @@ +#!/usr/bin/env bash +# +# Sample check for the baked Solace Python samples. Python has no compile step, so the +# gate installs the latest solace-pubsubplus, runs py_compile on every baked sample, +# and runs check_samples.py, which imports each sample, builds its API objects without a +# broker, and exercises the shared connection helper. The script's exit code IS the +# gate: exit 0 only if every check passes. +# +# Runnable locally with one command from any working directory. Resolves the +# solace-pubsubplus version from PyPI at run time, so no version number is committed +# anywhere (SKILL.md Invariant 3). All work happens in a throwaway temp directory, so +# the repository tree is left unmodified. + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +# references/python/ lives two levels up from sample-check/ (evals/ -> python/). +REFERENCES_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd)" +PYTHON="${PYTHON:-python3}" + +# Sample roster: each baked sample and the name a generated project gives it +# (implement-mode.md Step 4). A new baked sample is a one-line append here. +SAMPLES=( + "python-solace-connection-config.py:solace_connection_config.py" + "python-guaranteed-publisher-sample.py:guaranteed_publisher.py" + "python-guaranteed-subscriber-sample.py:guaranteed_subscriber.py" + "python-direct-publisher-sample.py:direct_publisher.py" + "python-direct-subscriber-sample.py:direct_subscriber.py" + "python-direct-requestor-blocking-sample.py:direct_requestor_blocking.py" + "python-direct-requestor-sample.py:direct_requestor.py" + "python-direct-replier-sample.py:direct_replier.py" +) + +# Existence guard: fail early if any input is missing, including a new roster entry +# whose sample is not yet committed. +for entry in "${SAMPLES[@]}"; do + src="${REFERENCES_DIR}/${entry%%:*}" + [[ -f "$src" ]] || { echo "ERROR: required input not found: $src" >&2; exit 1; } +done +for f in "${SCRIPT_DIR}/check_samples.py" "${REFERENCES_DIR}/scripts/verify.sh"; do + [[ -f "$f" ]] || { echo "ERROR: required input not found: $f" >&2; exit 1; } +done + +# (1) Resolve the latest solace-pubsubplus release from the PyPI JSON API, the same +# source implement-mode.md Step 3 uses. On any failure, exit non-zero rather than +# guessing a version. +echo "Resolving the latest solace-pubsubplus version from PyPI..." +VERSION="$(curl -s --max-time 20 https://pypi.org/pypi/solace-pubsubplus/json \ + | "$PYTHON" -c 'import json, sys; print(json.load(sys.stdin)["info"]["version"])' 2>/dev/null || true)" +if [[ -z "${VERSION// /}" ]]; then + echo "ERROR: could not resolve the latest solace-pubsubplus version from PyPI" >&2 + echo " (network failure or unreachable API). Not guessing a version." >&2 + exit 1 +fi +echo "Resolved solace-pubsubplus version: ${VERSION}" + +# (2) Build a throwaway working tree and virtual environment. +WORK_DIR="$(mktemp -d)" +cleanup() { rm -rf "$WORK_DIR"; } +trap cleanup EXIT + +"$PYTHON" -m venv "${WORK_DIR}/.venv" +VENV_PY="${WORK_DIR}/.venv/bin/python" +echo "Installing solace-pubsubplus==${VERSION} with $("$VENV_PY" --version)..." +"$VENV_PY" -m pip install -q --disable-pip-version-check "solace-pubsubplus==${VERSION}" + +# (3) Copy the samples under their generated names. Content is verbatim; only the +# file name changes, so the samples import each other as a generated project does. +for entry in "${SAMPLES[@]}"; do + cp "${REFERENCES_DIR}/${entry%%:*}" "${WORK_DIR}/${entry##*:}" +done +cp "${SCRIPT_DIR}/check_samples.py" "${WORK_DIR}/check_samples.py" + +# (4) py_compile every sample. +echo "Running py_compile on every baked sample..." +( cd "$WORK_DIR" && "$VENV_PY" -m py_compile ./*.py ) +echo "PASS: py_compile" + +# (5) Import each sample and build its API objects without a broker, then exercise +# the connection helper. +echo "Building the API objects against tcp://localhost:55555 without a broker..." +( cd "$WORK_DIR" && "$VENV_PY" check_samples.py ) + +# (6) The bundled verify script must at least parse. +bash -n "${REFERENCES_DIR}/scripts/verify.sh" +echo "PASS: scripts/verify.sh parses" + +echo "PASS: the baked Python samples against solace-pubsubplus ${VERSION}" diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/evals/sample-check/check_samples.py b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/evals/sample-check/check_samples.py new file mode 100644 index 0000000..5bc89c1 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/evals/sample-check/check_samples.py @@ -0,0 +1,122 @@ +"""Sample check for the baked Solace Python samples, run by check-samples.sh. + +check-samples.sh copies every sample into the working directory under its generated name and +runs this file with the virtual environment's interpreter. Without a broker, it: + +1. imports each sample module (each keeps its __main__ guard, so nothing connects), +2. calls each sample's setup_solace(), which builds the MessagingService and the + role's publisher, receiver, or request-reply objects, then teardown_solace() on the + never-connected service, and +3. exercises the shared solace_connection_config helper's config.json, command-line + fallback, and fail-fast paths. + +build() resolves the host, so every build uses tcp://localhost:55555, which resolves +whether or not a broker listens there. Any failure raises, and the non-zero exit fails +the gate. +""" + +import glob +import importlib +import json +import os +import tempfile +import threading + +# Every sample check-samples.sh copied in, so a new roster entry is covered without a +# second list to keep in step. The shared helper has no setup_solace() of its own. +NOT_SAMPLES = {"check_samples", "solace_connection_config"} +MODULES = sorted( + name + for name in (os.path.splitext(path)[0] for path in glob.glob("*.py")) + if name not in NOT_SAMPLES +) + +HOST_KEY = "solace.messaging.transport.host" +VPN_KEY = "solace.messaging.service.vpn-name" +USER_KEY = "solace.messaging.authentication.basic.username" +PASSWORD_KEY = "solace.messaging.authentication.basic.password" + +PROPERTIES = { + HOST_KEY: "tcp://localhost:55555", + VPN_KEY: "smoke-vpn", + USER_KEY: "smoke-user", + PASSWORD_KEY: "smoke-pass", +} + + +def build_samples() -> None: + if not MODULES: + raise RuntimeError("no sample modules found in the working directory") + for name in MODULES: + module = importlib.import_module(name) + state = module.setup_solace(dict(PROPERTIES), threading.Event()) + module.teardown_solace(state) + print(f"PASS: {name} builds its service and role objects without a broker", flush=True) + + +def expect_exit(call, case: str) -> None: + try: + call() + except SystemExit as exit_error: + if exit_error.code == 1: + return + raise AssertionError(f"{case}: exit code {exit_error.code}, expected 1") from exit_error + raise AssertionError(f"{case}: expected the helper to exit") + + +def expect_value_error(call, case: str) -> None: + try: + call() + except ValueError: + return + raise AssertionError(f"{case}: expected a ValueError") + + +def check_config_helper() -> None: + from solace_connection_config import load_service_properties + + start_dir = os.getcwd() + with tempfile.TemporaryDirectory() as work: + os.chdir(work) + try: + def write_config(config: dict) -> None: + with open("config.json", "w", encoding="utf-8") as config_file: + json.dump(config, config_file) + + # a valid config.json round-trips every key, including a pass-through key + write_config({**PROPERTIES, "solace.messaging.tls.trust-store-path": "/smoke/trust"}) + loaded = load_service_properties([], "smoke") + assert loaded[HOST_KEY] == PROPERTIES[HOST_KEY], loaded + assert loaded["solace.messaging.tls.trust-store-path"] == "/smoke/trust", loaded + + # config.json wins over command-line arguments + loaded = load_service_properties(["tcp://other:55555", "v", "u"], "smoke") + assert loaded[HOST_KEY] == PROPERTIES[HOST_KEY], loaded + + # a missing required key fails fast + write_config({VPN_KEY: "smoke-vpn", USER_KEY: "smoke-user"}) + expect_value_error(lambda: load_service_properties([], "smoke"), "missing-key") + + # a present but blank required value fails fast + write_config({**PROPERTIES, HOST_KEY: " "}) + expect_value_error(lambda: load_service_properties([], "smoke"), "blank-value") + + # with no config.json, the command-line arguments are the source, and an + # omitted password is sent as an empty string + os.remove("config.json") + loaded = load_service_properties(["tcp://localhost:55555", "smoke-vpn", "smoke-user"], "smoke") + assert loaded[USER_KEY] == "smoke-user" and loaded[PASSWORD_KEY] == "", loaded + + # a blank command-line argument fails fast + expect_value_error(lambda: load_service_properties(["", "smoke-vpn", "smoke-user"], "smoke"), "args-blank") + + # neither source: print the usage line and exit 1 + expect_exit(lambda: load_service_properties([], "smoke"), "no-source") + finally: + os.chdir(start_dir) + print("PASS: solace_connection_config file, fallback, and fail-fast paths", flush=True) + + +if __name__ == "__main__": + build_samples() + check_config_helper() diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-direct-pubsub.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-direct-pubsub.md index 53488c5..b2dc851 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-direct-pubsub.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-direct-pubsub.md @@ -35,4 +35,4 @@ The publisher only connects and publishes a short bounded burst of DIRECT messag ## The marker contract -These three `VERIFY:` marker strings (`VERIFY: CONNECTED`, `VERIFY: SUBSCRIBED`, `VERIFY: MESSAGE_RECEIVED`) are the same strings the JCSMP Direct Pub/Sub leaf emits, character for character. They are read PER PROCESS: the subscriber output carries `SUBSCRIBED` and `MESSAGE_RECEIVED`, and both scripts print `CONNECTED`. Until the Python verify script lands, the developer reads them from the two terminals (`implement-mode.md` Step 5); the verify script will read the same strings, so never change one. +These three `VERIFY:` marker strings (`VERIFY: CONNECTED`, `VERIFY: SUBSCRIBED`, `VERIFY: MESSAGE_RECEIVED`) are the same strings the JCSMP Direct Pub/Sub leaf emits, character for character. They are read PER PROCESS: the subscriber output carries `SUBSCRIBED` and `MESSAGE_RECEIVED`, and both scripts print `CONNECTED`. `verify.sh` reads these same strings in each process's own output (`implement-mode.md` Step 5), so never change one. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-guaranteed-pubsub.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-guaranteed-pubsub.md index aaa19cf..826e4b9 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-guaranteed-pubsub.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-guaranteed-pubsub.md @@ -36,7 +36,7 @@ queue = Queue.durable_exclusive_queue(queue_name) # one active consumer / When the `Access type` field reads exclusive, keep the `durable_exclusive_queue` line: only one bound receiver is active at a time, and the receiver state listener reports the passive standby. When it reads non-exclusive, use the `durable_non_exclusive_queue` line instead, so several subscriber processes bound to the same queue share the messages. Emit exactly one of the two lines. Ground the choice in [Topic Endpoints and Queues](https://docs.solace.com/Get-Started/topic-endpoints-queues.md). -**Fan-out.** On the fan-out leaf, every consuming service gets its own durable queue with the same topic subscription, so each service receives its own full copy of every event. Generate one subscriber script per consuming service: `guaranteed_subscriber.py` for the first service, and `guaranteed_subscriber_.py` for each further service, where `` is a lowercase Python identifier (letters, digits, and underscores, for example `billing` or `audit_log`) so the import smoke can import the module, each with its own queue name from the design summary and an exclusive durable queue. Hand back one run command per subscriber in Step 5, and add each script to the import smoke. +**Fan-out.** On the fan-out leaf, every consuming service gets its own durable queue with the same topic subscription, so each service receives its own full copy of every event. Generate one subscriber script per consuming service: `guaranteed_subscriber.py` for the first service, and `guaranteed_subscriber_.py` for each further service, where `` is a lowercase Python identifier (letters, digits, and underscores, for example `billing` or `audit_log`) so the import smoke can import the module, each with its own queue name from the design summary and an exclusive durable queue. The `verify.sh` stages drive `guaranteed_subscriber.py` only, so hand back one run command per further subscriber in Step 5, and add each script to the import smoke. ## Publisher script (`guaranteed_publisher.py`) @@ -52,4 +52,4 @@ The publisher only connects and publishes; it does NOT provision the queue or th ## The marker contract -These four `VERIFY:` marker strings (`VERIFY: CONNECTED`, `VERIFY: PUBLISH_ACKED`, `VERIFY: QUEUE_BOUND`, `VERIFY: MESSAGE_RECEIVED`) are the same strings the JCSMP guaranteed pub/sub leaf emits, character for character. They are read PER PROCESS: the subscriber output carries `QUEUE_BOUND` and `MESSAGE_RECEIVED`, the publisher output carries `PUBLISH_ACKED`, and both print `CONNECTED`. Until the Python verify script lands, the developer reads them from the two terminals (`implement-mode.md` Step 5); the verify script will read the same strings, so never change one. +These four `VERIFY:` marker strings (`VERIFY: CONNECTED`, `VERIFY: PUBLISH_ACKED`, `VERIFY: QUEUE_BOUND`, `VERIFY: MESSAGE_RECEIVED`) are the same strings the JCSMP guaranteed pub/sub leaf emits, character for character. They are read PER PROCESS: the subscriber output carries `QUEUE_BOUND` and `MESSAGE_RECEIVED`, the publisher output carries `PUBLISH_ACKED`, and both print `CONNECTED`. `verify.sh` reads these same strings in each process's own output (`implement-mode.md` Step 5), so never change one. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-mode.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-mode.md index 5f5d465..fb95d3a 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-mode.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-mode.md @@ -6,7 +6,7 @@ Implement mode is NOT only for the canonical shape. A web app, a dashboard, a se Work through the steps in order. Read this file's grounding links only as each step needs them. Link the canonical doc and state the decision in one line, never paraphrase doc content. Resolve every dependency version at generation time; pin nothing in skill content (SKILL.md Invariant 3). -**Coverage in this release.** Implement mode generates all five Python leaves on all three doors. The Python API does not bundle a verify script yet, so every run ends in the compile-only fallback of Step 5. +**Coverage in this release.** Implement mode generates all five Python leaves on all three doors and verifies them with the bundled `scripts/verify.sh` (Step 5). ## Generation outputs: the file contract @@ -16,8 +16,9 @@ Every Implement run writes ALL of these as generation output, on every app shape 2. `requirements.txt` with the `solace-pubsubplus` version resolved per Step 3. 3. `config.example.json` (placeholders) and a `.gitignore` that ignores `config.json` and the virtual environment (Step 4). 4. A tailored `solace-verification-checklist.md` at each project root (tailoring rules in Step 6; the WRITE happens here at generation time, like the scripts). +5. A copy of the bundled Python `scripts/verify.sh` at each project root, plus a generated `verify-hooks.sh` beside it (Step 5), so the verification commands resolve from inside the project. -There is no `verify.sh` and no `verify-hooks.sh` in a Python project yet. Do not copy the JCSMP `verify.sh` into a Python project: it drives Maven and cannot run Python scripts. +Copy the Python `verify.sh` from `references/python/scripts/`, never the JCSMP one: the JCSMP script drives Maven and cannot run Python scripts. ## Step 0: Ask Quickstart, Solace Suggested, or Custom, before any broker details @@ -93,7 +94,7 @@ Generation is keyed on the `Pattern` leaf read in Step 2. Read ONLY the leaf fil **Every leaf also generates the shared connection helper.** Read `python-solace-connection-config.py` and generate it as `solace_connection_config.py` (a module name needs underscores). Each script builds its connection dict by calling `load_service_properties(args, APP_NAME)` from that module and passes it to `MessagingService.builder().from_properties(...)`, rather than setting the host, VPN, username, or password inline. -**Fixed file names.** Generate each script under the name its sample's docstring gives, next to `solace_connection_config.py`: `guaranteed_publisher.py` and `guaranteed_subscriber.py` (the fan-out leaf adds one subscriber script per further consuming service, named in `implement-guaranteed-pubsub.md`), `direct_publisher.py` and `direct_subscriber.py`, and `direct_replier.py` with `direct_requestor_blocking.py` (or `direct_requestor.py` on the non-blocking variant). The run commands in Step 5 and the checklist address these names; the names of functions, constants, and the `APP_NAME` value inside each script stay your discretion. +**Fixed file names.** Generate each script under the name its sample's docstring gives, next to `solace_connection_config.py`: `guaranteed_publisher.py` and `guaranteed_subscriber.py` (the fan-out leaf adds one subscriber script per further consuming service, named in `implement-guaranteed-pubsub.md`), `direct_publisher.py` and `direct_subscriber.py`, and `direct_replier.py` with `direct_requestor_blocking.py` (or `direct_requestor.py` on the non-blocking variant). `verify.sh` (Step 5) finds the scripts by these names; the names of functions, constants, and the `APP_NAME` value inside each script stay your discretion. **Disclaimer header on EVERY generated source file.** Stamp the top of each generated `.py` file with these two comment lines, before the module docstring and before any import: @@ -104,9 +105,9 @@ Generation is keyed on the `Pattern` leaf read in Step 2. Read ONLY the leaf fil Stamp the header AT WRITE TIME, as the first content of each new file. `requirements.txt`, `config.example.json`, and `.gitignore` are not source and carry no header. The reference samples themselves carry NO disclaimer. This rule is SKILL.md Invariant 4. -**Embedded and web-app shapes.** When the Python messaging layer sits inside a larger application, the chosen leaf's wiring still governs the messaging layer: the same connection helper, the same service, receiver, and publisher listeners, the same `VERIFY:` markers emitted through a stdout-reaching `trace(...)`, and the disclaimer header on every generated file, messaging or not. The `VERIFY:` markers are NOT demo harness. +**Embedded and web-app shapes.** When the Python messaging layer sits inside a larger application, the chosen leaf's wiring still governs the messaging layer: the same connection helper, the same service, receiver, and publisher listeners, the same `VERIFY:` markers emitted through a stdout-reaching `trace(...)`, and the disclaimer header on every generated file, messaging or not. The `VERIFY:` markers are NOT demo harness; they stay in the messaging layer whatever the shape, because the `app` verify stage (Step 5) greps for them in the app's captured output. -**Write the verification artifacts in this same step.** Alongside the scripts, write the tailored `solace-verification-checklist.md` at each project root (Step 6), `config.example.json`, and `.gitignore`. A session that ends early still leaves the checklist in the project. +**Write the verification artifacts in this same step.** Alongside the scripts, write the tailored `solace-verification-checklist.md` at each project root (Step 6), `config.example.json`, `.gitignore`, a copy of the bundled `scripts/verify.sh` at each project root, and the generated `verify-hooks.sh` (Step 5 defines its contract). A session that ends early still leaves the checklist and the verify entry points in the project. Make the `verify.sh` copy with a read-and-redirect, `cat "/references/python/scripts/verify.sh" > verify.sh && chmod +x verify.sh`, where the skill directory is the one announced when this skill loaded. Do not use `cp` on the bundled file: Claude Code protects the plugin directory and blocks a `cp` that names a file inside it, and a blocked copy pushes the script through a retype that must stay byte-identical. Never edit the bundled script itself. ### Concurrency: threads, not asyncio, and no multiprocessing @@ -153,13 +154,19 @@ Generate two small project files next to the scripts (on Solace Suggested, on Cu Tell the developer they opt in to the config file by copying `config.example.json` to `config.json` and filling in their real broker values. Both scripts then read it from the working directory, so they run from the project root. -## Step 5: Check the project, then hand back the run commands +## Step 5: Check the project, then verify with verify.sh -The Python API does not bundle a verify script yet, so Implement mode ends in the compile-only fallback on every run, whatever the contents of `config.json` (SKILL.md Invariant 5). Never fabricate a run result and never call a check below a verified round trip. +Verification is a `verify.sh` run, on every door and every app shape. Step 4 copied `scripts/verify.sh` into each project root and generated `verify-hooks.sh` beside it, so every command below resolves from inside the project. An improvised check (curl against the app's own HTTP API, a manual browser click) may TRIGGER traffic, but it NEVER renders the verdict: the verdict is the `VERIFY:` markers plus the script's exit code, which is what separates "your code is broken" (exit 1) from "your broker, credentials, or trust store are wrong" (exit 2). Never fabricate a run result. + +**Live-broker heads-up (every run, every launcher).** Immediately before starting ANY process that will connect to a live broker (a `verify.sh` stage, a `.venv/bin/python` run, a launcher script), print ONE line that names the broker host and states the actual broker-side effects of this run: the durable queues the subscribers provision, the connections it opens, and the messages or requests the foreground role sends, scaled to the generated scripts (SKILL.md Invariant 6). + +### Check first: compile, import, and conformance + +Run these checks on every run, before any `verify.sh` stage. On the compile-only fallback below they are the whole check. 1. **Compile check.** Run `.venv/bin/python -m py_compile` on every generated `.py` file. A failure is a code failure: fix it and re-run, up to 3 attempts, then report the evidence and hand the decision to the developer. 2. **Import smoke.** From the project root, run `.venv/bin/python -c "import solace_connection_config, guaranteed_publisher, guaranteed_subscriber"` with the module names of the chosen leaf (for example `direct_publisher, direct_subscriber` or `direct_replier, direct_requestor_blocking`), adding every further generated module (for example each fan-out subscriber). On separate projects, run the checks in each project root against that project's modules. Each script keeps its `if __name__ == "__main__":` guard, so the import resolves every `solace.messaging` name the scripts use without connecting to a broker. A failure (for example an `ImportError` on a misspelled module path) is a code failure under the same fix loop. -3. **Conformance checks.** From the project root, run these three checks and record each result in the checklist. A missing header or a missing checklist is a code failure under the same fix loop; a version mismatch is reported to the developer, not fixed silently. +3. **Conformance checks.** From the project root, run these three checks and record each result in the checklist. A missing header or a missing checklist is a code failure under the same fix loop; a version mismatch is reported to the developer, not fixed silently. The `verify.sh` preflight repeats them as warnings on every run. ```bash # every generated .py file starts with the disclaimer header @@ -170,36 +177,69 @@ The Python API does not bundle a verify script yet, so Implement mode ends in th grep -qxF "solace-pubsubplus==$(curl -s https://pypi.org/pypi/solace-pubsubplus/json | python3 -c 'import json, sys; print(json.load(sys.stdin)["info"]["version"])')" requirements.txt || echo "VERSION DRIFT" ``` -4. **Report and hand back.** Report the compile, import, and conformance results, then hand the developer the exact commands to run against their own broker from the project root. The long-running role always starts first: the subscriber, so the durable queue (guaranteed) or the topic subscription (direct) exists before the first publish, and the replier, so the request topic subscription exists before the first request. For the Guaranteed Pub/Sub leaves: +### The canonical two-script shape: the fixed stages - ``` - # terminal 1: provisions the queue, binds, and waits (Ctrl-C stops it) - .venv/bin/python guaranteed_subscriber.py [ [password]] - # terminal 2: publishes, waits for the broker's acknowledgement, and exits - .venv/bin/python guaranteed_publisher.py [ [password]] - ``` +`verify.sh` checks every script with `py_compile`, starts the long-running role FIRST with the project's own `.venv` interpreter (the subscriber, so the durable queue or the topic subscription exists before the first publish; the replier, so the request topic subscription exists before the first request), waits for its readiness marker, runs the foreground role and lets it exit on its own, watches each process's own output for the stage's `VERIFY:` marker, sends SIGINT to the long-running role to exercise its graceful shutdown, and returns an exit code that classifies the outcome. It finds the scripts by the fixed file names of Step 4, so never rename a generated script. + +| `Pattern` leaf | Stages, in order | Each stage waits for | +|----------------|------------------|----------------------| +| The three Guaranteed Pub/Sub leaves | `consumer`, then `publisher`, then `roundtrip` | `VERIFY: QUEUE_BOUND` in the subscriber output; `VERIFY: PUBLISH_ACKED` in the publisher output (this stage starts the subscriber first, so it needs both scripts); `VERIFY: PUBLISH_ACKED` in the publisher output and then a new `VERIFY: MESSAGE_RECEIVED` in the subscriber output (a message an earlier run left on the durable queue does not count) | +| `Direct Pub/Sub` | `direct` | `VERIFY: SUBSCRIBED` in the subscriber output as the readiness gate, then at least one `VERIFY: MESSAGE_RECEIVED` from the publisher's short burst | +| `Request-Reply (Direct)` | `direct-request-reply` | `VERIFY: SUBSCRIBED` in the replier output as the readiness gate, then `VERIFY: REPLY_RECEIVED` in the requestor output and `VERIFY: REQUEST_RECEIVED` in the replier output | - For the Direct Pub/Sub leaf, the same two terminals run `direct_subscriber.py` (subscribes and waits; Ctrl-C stops it), then `direct_publisher.py` (publishes a short burst and exits). For the Request-Reply (Direct) leaf, they run `direct_replier.py` (subscribes to the request topic and waits; Ctrl-C stops it), then `direct_requestor_blocking.py` or `direct_requestor.py` (sends, waits for the reply, and exits). +The `direct-request-reply` stage runs `direct_requestor_blocking.py`, or `direct_requestor.py` when the project has the non-blocking variant instead. The Python API has no guaranteed request-reply, so there is no `guaranteed-request-reply` stage. On the fan-out leaf, the stages drive `guaranteed_subscriber.py` only; hand back one command per further subscriber script (`guaranteed_subscriber_.py`), each started in its own terminal before the publisher, and tell the developer that each prints its own `VERIFY: MESSAGE_RECEIVED`. - On the fan-out leaf, hand back one subscriber command per subscriber script (`guaranteed_subscriber.py` and each `guaranteed_subscriber_.py`), each in its own terminal and all started before the publisher; every subscriber prints its own markers. On separate projects (Solace Suggested, or the Custom decoupled-projects knob), hand back each command from its own project directory with that project's `.venv/bin/python`. +Invoke each stage from the project root in one of these forms: - The arguments are needed only when there is no `config.json`, and they are never offered on a path that drops the command-line fallback. Tell the developer what a passing run prints, per leaf: +- **With `config.json`** (always the form for a `tcps://` host, because the trust store path has no argument form): `./verify.sh `. Each script reads `config.json` from the working directory. +- **With connection arguments** (a plaintext host and no `config.json`): `./verify.sh [password]`, the same contract the scripts' command-line fallback uses. +- **On separate projects** (Solace Suggested, or the Custom decoupled-projects knob): `./verify.sh --subscriber-dir --publisher-dir ` on the pub/sub leaves, or `./verify.sh direct-request-reply --replier-dir --requestor-dir `. Run it from either project; the directories are relative to the working directory. It passes NO connection details on the command line: each script runs from inside its own project and reads that project's `config.json`, with that project's `.venv`. + +### Every other shape: the `app` stage and `verify-hooks.sh` + +A web app, an embedded service, or any single-process shape that the fixed stages cannot drive is verified with the `app` stage: `./verify.sh app`. The stage sources the generated `verify-hooks.sh`, which carries the ONLY app-specific facts (how to start the app and how to cause one publish), while the script keeps the universal logic: marker watching, timeouts, environment-signature classification, and the 0/1/2 exit contract. Generation writes `verify-hooks.sh` because the generator just wrote the app and knows both commands: + +```bash +# verify-hooks.sh: generated with the project; verify.sh's app stage sources it. +START_CMD='.venv/bin/python app.py' # starts the app (long-running) +TRIGGER_CMD='curl -s --max-time 10 -X POST localhost:8081/api/orders -d "{\"demo\":1}"' # causes exactly one publish +READY_MARKER='VERIFY: QUEUE_BOUND' # marker proving the consuming side is live +PASS_MARKER='VERIFY: MESSAGE_RECEIVED' # marker proving the round trip +``` - - Guaranteed Pub/Sub: the subscriber prints `VERIFY: CONNECTED` and then `VERIFY: QUEUE_BOUND`; the publisher prints `VERIFY: CONNECTED` and `VERIFY: PUBLISH_ACKED` and exits with status 0; the subscriber then prints `VERIFY: MESSAGE_RECEIVED`. - - Direct Pub/Sub: the subscriber prints `VERIFY: CONNECTED` and then `VERIFY: SUBSCRIBED`; the publisher prints `VERIFY: CONNECTED` and exits with status 0; the subscriber then prints `VERIFY: MESSAGE_RECEIVED`. - - Request-Reply (Direct): the replier prints `VERIFY: CONNECTED` and then `VERIFY: SUBSCRIBED`; the requestor prints `VERIFY: CONNECTED`; the replier prints `VERIFY: REQUEST_RECEIVED`; the requestor prints `VERIFY: REPLY_RECEIVED` and exits with status 0. +Generate `verify-hooks.sh` with ONLY these four assignments. The app stage imports exactly these four names from a child shell and ignores everything else in the file; keep the file to the four lines all the same. Give a network trigger its own timeout (the `--max-time` above): the stage also bounds the trigger and force-kills one that never returns, but a self-timing trigger leaves cleaner evidence. Set `READY_MARKER` and `PASS_MARKER` to the chosen leaf's own markers. On the canonical two-script shape, generate `verify-hooks.sh` too, carrying the classic commands (for example `START_CMD='.venv/bin/python guaranteed_subscriber.py'` and `TRIGGER_CMD='.venv/bin/python guaranteed_publisher.py'`), so the file documents the same contract everywhere; the fixed stages do not read it. - On every leaf, Ctrl-C on the long-running role prints `Shutdown signal received` and it exits with status 0. +The `app` stage starts `START_CMD` in the background, waits for `READY_MARKER` in the app's captured output, runs `TRIGGER_CMD` under a bounded deadline, waits for a new `PASS_MARKER` (one already in the output before the trigger does not count), then stops the app with SIGINT and classifies. A curl inside `TRIGGER_CMD` is exactly the right use of curl: it triggers, and the markers judge. -**When the developer asks you to run it.** If the developer explicitly asks you to run the scripts against their broker, print the live-broker heads-up first (SKILL.md Invariant 6: name the host, every durable queue the subscribers provision, one connection per script, and the messages or requests the foreground role sends, scaled to the generated scripts). Then start each long-running role in the background with its output captured, wait for its readiness marker (`VERIFY: QUEUE_BOUND` on the guaranteed leaves, `VERIFY: SUBSCRIBED` on the direct leaves) from each, run the foreground role, wait for the pass marker (`VERIFY: MESSAGE_RECEIVED` in each subscriber's output, or `VERIFY: REPLY_RECEIVED` in the requestor's output), and stop the long-running roles with SIGINT. Report the markers you observed as evidence of a manual run. It is not a verify verdict: record it in the checklist as a manual run, and never describe it as a passed verify stage. +### Interpret the exit code -**Environment failure on a manual run: do NOT modify the code.** When the captured output shows that the broker is unreachable, the credentials or VPN are wrong, the TLS handshake or the trust store fails, or the client may not create the queue (the subscriber's `Could not bind to queue` error), the code is not the problem. Report the matched output line as evidence and STOP: do not enter the fix loop, and do not change the code. The fix is the developer's broker, credentials, or client permissions. Fix the code only when the output shows a code failure (an exception from the scripts' own logic, or a marker that never appears after a clean connect and bind), up to 3 attempts, then hand the decision to the developer. +- **Exit 0 (stage passes).** For the fixed stages: the stage milestone appeared in this run, the foreground role exited with status 0 on its own, the long-running role printed `Shutdown signal received`, and it exited with status 0. For the `app` stage: `READY_MARKER` appeared, a new `PASS_MARKER` followed the trigger, and the app was stopped with a bounded teardown (its own shutdown is not checked). Advance to the next stage. +- **Exit 1 (code failure).** `py_compile` failed, a marker never appeared, a process died before its milestone, the foreground role exited non-zero, the shutdown line did not print, or a run hung and had to be force-killed. Enter the bounded fix loop below. An exit 1 before any process starts is a missing precondition instead (a usage error, or a missing script, `.venv`, `config.json`, or `verify-hooks.sh`), and stderr names it: fix that and re-run, without counting it as a fix-loop attempt. +- **Exit 2 (environment failure).** A Solace Python API connection, authentication, or TLS signature appeared (the broker is unreachable, the credentials or VPN are wrong, or the trust store is missing or holds the wrong CA), or the subscriber could not bind its queue because the client may not create it (`Could not bind to queue`), or the long-running role was still connecting when the readiness deadline passed and never printed `VERIFY: CONNECTED` (an unreachable host, or a firewall that blocks the port). STOP. Do NOT enter the fix loop. The code is not the problem. -**Close every run with the next steps, stated plainly.** Tell the developer, in order: (1) copy `config.example.json` to `config.json` and fill in their real broker values; on any `tcps://` host (always on Solace Suggested and with the Custom secure-session knob, and on Quickstart or on Custom with the secure-session knob unticked when the Step 1 answer was secure), also put the broker's CA certificate into a trust store directory and set `solace.messaging.tls.trust-store-path` to it, as `solace-suggested-mode.md` Step 1 explains. On a Quickstart run, or a Custom run with the secure-session knob unticked, whose Step 1 answer was plaintext, still add one line: if the host later changes to `tcps://`, the project needs a trust store directory set in `solace.messaging.tls.trust-store-path` in `config.json`, or the connection fails with "Failed to load trust store"; (2) run the long-running role, then the foreground role, with the commands above, and check the markers. Step 4 already wrote the tailored `solace-verification-checklist.md`; invite the developer to add their own items in its Your additional items section and to record any skill-undelivered concerns under Developer-owned items. +**Bounded fix loop (exit 1 only).** Diagnose the captured output, fix the code, and re-run the same stage, up to 3 automatic attempts per stage. After the third failure, stop fixing: summarize exactly what you tried and what the captured output shows, then hand the decision to the developer. Fixes apply without a confirmation gate, on every door. + +**Environment-failure path (exit 2).** Report the matched line from the captured output as evidence, and do NOT modify the code. The fix is the developer's broker, credentials, trust store, or client permissions. + +### When to run verify.sh, and when to hand it back + +**Run it from `config.json` with real values.** Run the stages for real, with NO confirmation prompt, ONLY when a `config.json` is present at every project root involved AND its values are real, meaning they are NOT the `config.example.json` placeholders (`tcp://HOST:55555` or `tcps://HOST:55443`, `YOUR_VPN`, `YOUR_USERNAME`, `YOUR_PASSWORD`, `PATH_TO_TRUST_STORE_DIRECTORY`). Filling in real values is the opt-in; a `config.json` that still carries a placeholder is not. Use the `config.json` form of the stage command (single project) or the separate-projects form, and run the leaf's stages in the table's order. Print the live-broker heads-up immediately before the run, naming the host from `config.json`; the real-values `config.json` is the consent, the heads-up is not a prompt. The developer can also ask you to run `verify.sh` with connection arguments they give for a plaintext host, but only when no `config.json` exists: the scripts prefer `config.json`, so `verify.sh` refuses connection arguments when one is present. + +**Honesty rule (no fabricated runs).** When `config.json` is absent or still holds placeholders, do NOT run the scripts and do NOT fabricate a result. Report the compile, import, and conformance results, and hand back the exact `verify.sh` commands for the developer to run from the project root, for example on the Guaranteed Pub/Sub leaves: + +``` +./verify.sh consumer +./verify.sh publisher +./verify.sh roundtrip +``` + +Add the connection arguments only for a plaintext host with no `config.json`, and never on a path that drops the command-line fallback. On separate projects, hand back the separate-projects form. For an `app`-stage shape, the handed-back command is `./verify.sh app` after the developer fills in the app's config. + +**Close every run with the next steps, stated plainly.** Tell the developer, in order: (1) copy `config.example.json` to `config.json` (in each project on separate projects) and fill in their real broker values; on any `tcps://` host (always on Solace Suggested and with the Custom secure-session knob, and on Quickstart or on Custom with the secure-session knob unticked when the Step 1 answer was secure), also put the broker's CA certificate into a trust store directory and set `solace.messaging.tls.trust-store-path` to it, as `solace-suggested-mode.md` Step 1 explains. On a Quickstart run, or a Custom run with the secure-session knob unticked, whose Step 1 answer was plaintext, still add one line: if the host later changes to `tcps://`, the project needs a trust store directory set in `solace.messaging.tls.trust-store-path` in `config.json`, or the connection fails with "Failed to load trust store"; (2) then just ask you to verify, and you will run `verify.sh` for them against their broker (naming the host and side effects first). Make the fill-in-`config.json`-then-ask-me path the recommended one; if they would rather run it themselves, point them at the handed-back commands. Step 4 already wrote the tailored `solace-verification-checklist.md`; invite the developer to add their own items in its Your additional items section and to record any skill-undelivered concerns under Developer-owned items. ### Traps to avoid -- **Improvised verdicts.** A connect that does not raise, or a publish call that returns, proves nothing about the round trip. Judge only by the markers in each script's output. +- **Improvised verdicts.** A connect that does not raise, a publish call that returns, or a curl against the app's own API proves nothing about the round trip. Trigger with whatever fits; judge ONLY by the markers and the `verify.sh` exit code. - **Publishing direct-to-queue instead of pub/sub.** Publish to a topic; let the durable queue carry a topic subscription. See [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md). - **DIRECT delivery where Guaranteed is required.** Use the persistent publisher for the guaranteed pub/sub journey. See [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md). - **Seconds instead of milliseconds.** `RetryStrategy.parametrized_retry(retries, retry_interval)` takes its interval in MILLISECONDS per the API reference; a value of `3` means 3 milliseconds. The sample code on the Messaging Service page comments `parametrized_retry(20, 3)` as "over an interval of 3 seconds"; that comment is wrong, and the API reference wins. Whenever a generated script sets a retry interval, write it in milliseconds, as the samples' named constants do. @@ -207,66 +247,14 @@ The Python API does not bundle a verify script yet, so Implement mode ends in th - **Hardcoded versions.** Resolve the `solace-pubsubplus` version at generation time (Step 3); never carry a version from memory, from an old `requirements.txt`, or from the upstream samples repository. - **Wildcard imports.** Keep every import explicit, as the samples do; never write `from solace.messaging... import *`. - **The system interpreter.** Run and check everything with `.venv/bin/python`, so the project uses the version it installed. -- **Running the publisher before the subscriber has provisioned the queue.** A persistent message published before the queue subscribes to its topic is dropped and never received. By default the broker still ACKs it, so the publisher can print `VERIFY: PUBLISH_ACKED` for a message no subscriber gets; the publisher sees a NACK only when the client profile rejects messages that match no subscription. Judge the round trip by `VERIFY: MESSAGE_RECEIVED` in the subscriber output, never by `PUBLISH_ACKED` alone. Run the subscriber first. +- **Running the publisher before the subscriber has provisioned the queue.** A persistent message published before the queue subscribes to its topic is dropped and never received. By default the broker still ACKs it, so the publisher can print `VERIFY: PUBLISH_ACKED` for a message no subscriber gets; the publisher sees a NACK only when the client profile rejects messages that match no subscription. Judge the round trip by `VERIFY: MESSAGE_RECEIVED` in the subscriber output, never by `PUBLISH_ACKED` alone. Run the subscriber first; `verify.sh` enforces that order. - **Treating a sample as a spec that overrides the approved design.** See "Adapting a sample to a variant of its leaf" above. ## Step 6: Tailor the checklist (written at Step 4) and report -Step 4 writes the tailored `solace-verification-checklist.md` into the generated project as generation output; this step defines its content and adds the report in chat. The Python API has no master checklist template yet; until it lands, write the file from the list below. Resolve each item for the generated code: keep only the items for the chosen leaf (a leaf in parentheses marks a leaf-specific item), and move an item the generation did not deliver to Your responsibility. On Solace Suggested, and for each ticked Custom knob, an item under Your responsibility that the door or the knob delivered (the TLS secure session, DMQ eligibility on published messages, separate projects, and the HA reconnect values only when the HA-failover question was answered yes) moves to Delivered by this generation. The broker-side dead message queue always stays under Your responsibility, because the skill never provisions it. Write it as a Markdown task list (`- [ ]`), ticking only what this run actually checked. Each item in the first two groups carries one canonical doc link, as the master JCSMP template does; the round-trip and conformance items carry none because they check skill conformance, not Solace guidance. - -**Delivered by this generation** - -- Basic-auth connection through the shared `solace_connection_config.py` helper; credentials live only in a gitignored `config.json`. [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) -- Service reconnection and interruption listeners registered before `connect()`. [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) -- (Guaranteed Pub/Sub) Persistent publish to a topic with a publish receipt listener; a NACK is logged with the message id. [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) -- (Guaranteed Pub/Sub) Capacity-bounded back pressure on the publisher (`on_back_pressure_wait`). [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) -- (Guaranteed Pub/Sub) Message time-to-live set once on the publisher's message template. [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) -- (Guaranteed Pub/Sub) Durable queue plus topic subscription provisioned at start (`CREATE_ON_START`), with the access type from the design summary (one durable queue per consuming service on fan-out); with the Custom admin-provisioned-queue knob, the subscriber binds the administrator's queue instead. [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md) -- (Guaranteed Pub/Sub) Client acknowledgement only after processing. [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md) -- (Direct Pub/Sub) Direct publish to a topic with a publish failure listener and capacity-bounded back pressure (`on_back_pressure_wait`). [Publishing Direct Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-DM-Publish.md) -- (Direct Pub/Sub) Direct topic subscription with capacity-bounded back pressure (`on_back_pressure_drop_latest`) and discard detection on received messages. [Consuming Direct Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-DM-Receive.md) -- (Request-Reply (Direct)) Requests sent through a `RequestReplyMessagePublisher` with a positive reply timeout, and a replier that answers with `replier.reply()` and ignores a message with no reply-to. [Request-Reply Messaging in the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Request-Reply.md) -- Graceful SIGINT/SIGTERM shutdown: `terminate(grace_period)` and then `disconnect()` on every exit path. [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md) - -**Your responsibility (not delivered here)** - -- TLS secure session (`tcps://` with certificate validation set explicitly in code). Quickstart, and Custom with the secure-session knob unticked, open a TLS session only when the host is `tcps://`, and then validate only through the API defaults, which a `config.json` key can turn off; neither sets the certificate checks in code. [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) -- (Any `tcps://` host: Solace Suggested, the Custom secure-session knob, or Quickstart or Custom without that knob with a `tcps://` host) A trust store directory that holds the broker's CA certificate as a PEM file, named by `solace.messaging.tls.trust-store-path` in `config.json`; the Python API has no default trust store, so the connection fails without it. [Using Client Certificates with Solace Cloud](https://docs.solace.com/Cloud/ght_client_certs.md) -- (Custom admin-provisioned-queue knob) The queue and its topic subscription provisioned on the broker by an administrator, with the access type the design summary records. [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md) -- Credentials handled safely: `config.json` is never committed, and credentials are not passed as command-line arguments, where they land in shell history and process listings (the handed-back run commands accept them only for convenience). [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) -- Authentication stronger than basic username/password (for example a client certificate or Kerberos) where the environment requires it; this journey generates basic username/password only. [Messaging Service](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Messaging-Service.md) -- (Guaranteed Pub/Sub) Dead message queue eligibility (`PERSISTENT_DMQ_ELIGIBLE`) set explicitly on published messages. [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md) -- (Guaranteed Pub/Sub) A dead message queue provisioned on the broker with a max-redelivery count or a time-to-live; the eligibility flag alone does nothing without it. [Configuring Dead Message Queues](https://docs.solace.com/Messaging/Guaranteed-Msg/Setting-Dead-Msg-Queues.md) -- Reconnect tuning for HA failover; Quickstart leaves connect and reconnect behavior at the API defaults, and Custom without the HA knob keeps the samples' baseline of 20 retries 3000 ms apart. [Solace C API Best Practices](https://docs.solace.com/API/API-Developer-Guide/C-API-Best-Practices.md) -- Separate projects or deployments per role (skill guidance: a real deployment lifts each script into its own service or project; no Solace doc page states it). -- (Guaranteed Pub/Sub) `respect-ttl` enabled on the queue, without which the broker ignores the message time-to-live. [Configuring Queues](https://docs.solace.com/Messaging/Guaranteed-Msg/Configuring-Queues.md) -- (Guaranteed Pub/Sub) An administrator-provisioned queue, or the endpoint-create permission for this client, in production. [Creating Queues with the Solace Python API](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-Create-Queues.md) -- A Python version in active or security support and OpenSSL installed on Linux for TLS ([Supported Environments](https://docs.solace.com/API/API-Developer-Guide-Python/Python-API-supported-Environments.md)), and no `multiprocessing` in the application ([solace-pubsubplus on PyPI](https://pypi.org/project/solace-pubsubplus/)). - -**Verified by the round-trip** (developer-run until the Python verify script lands) - -- (Guaranteed Pub/Sub) Every subscriber prints `VERIFY: CONNECTED` and `VERIFY: QUEUE_BOUND`. -- (Guaranteed Pub/Sub) The publisher prints `VERIFY: CONNECTED` and `VERIFY: PUBLISH_ACKED`, then exits with status 0. -- (Guaranteed Pub/Sub) Every subscriber prints `VERIFY: MESSAGE_RECEIVED` (`PUBLISH_ACKED` alone does not prove delivery). -- (Direct Pub/Sub) The subscriber prints `VERIFY: CONNECTED` and `VERIFY: SUBSCRIBED`; the publisher prints `VERIFY: CONNECTED` and exits with status 0; the subscriber then prints `VERIFY: MESSAGE_RECEIVED`. -- (Request-Reply (Direct)) The replier prints `VERIFY: CONNECTED`, `VERIFY: SUBSCRIBED`, and `VERIFY: REQUEST_RECEIVED`; the requestor prints `VERIFY: CONNECTED` and `VERIFY: REPLY_RECEIVED`, then exits with status 0. -- Ctrl-C on each long-running role (subscriber or replier) prints `Shutdown signal received` and it exits with status 0. - -**Generation conformance** - -- The disclaimer header is the first two lines of every generated `.py` file (Step 5 header check). -- `requirements.txt` pins the `solace-pubsubplus` version that PyPI reported as `info.version` at generation time, or the developer's version when PyPI was unreachable (Step 5 drift check). -- This tailored checklist exists at the project root (Step 5 checklist check). -- `py_compile` and the import smoke passed (Step 5), with the result recorded here. -- Comments follow their code (Step 4). -- Every page named in the design summary's Grounding docs field was WebFetched in this session (`none fetched` is an honest value; an unfetched citation is not). -- Verification: record `verify script not yet bundled for Python; round trip handed back`, or `manual run` with the observed markers when the developer asked you to run it. - -**Developer-owned items (not delivered by this skill)**: any item the skill does not deliver, recorded as developer-owned rather than skill-satisfied. A consented deviation is recorded here. - -**Your additional items**: open space for the developer to keep growing the list. - -The emitted checklist is an artifact, not source, so it carries NO disclaimer header. Report the same resolution in chat: what landed under each group, the compile, import, and conformance results, and that the round trip is handed back. +Step 4 writes the tailored `solace-verification-checklist.md` into the generated project as generation output; this step defines its tailoring and adds the report in chat. Read `verification-checklist.md` (the master template) and resolve each item for the generated code: keep only the items for the chosen leaf (a leaf in parentheses marks a leaf-specific item), and move an item the generation did not deliver to Your responsibility. On Solace Suggested, and for each ticked Custom knob, an item under Your responsibility that the door or the knob delivered (the TLS secure session, DMQ eligibility on published messages, separate projects, and the HA reconnect values only when the HA-failover question was answered yes) moves to Delivered by this generation. The broker-side dead message queue always stays under Your responsibility, because the skill never provisions it. Write it as a Markdown task list (`- [ ]`), ticking only what this run actually checked. Keep the master template's groups and doc links, drop its template introduction and the leaf names in parentheses from the emitted copy, and keep its two closing sections (Developer-owned items, where a consented deviation is recorded, and Your additional items). On separate projects, write ONE copy into EACH project root. + +The emitted checklist is an artifact, not source, so it carries NO disclaimer header. Report the same resolution in chat: what landed under each group and the compile, import, and conformance results. Record each `verify.sh` stage that ran and its exit code in the checklist's Generation conformance group; on the compile-only fallback, record that the run was handed back instead. ## Grounding references diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-request-reply.md b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-request-reply.md index 5552d16..8bb33e5 100644 --- a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-request-reply.md +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/implement-request-reply.md @@ -41,4 +41,4 @@ The requestor connects, sends its request (or requests), collects the reply, and ## The marker contract -These four `VERIFY:` marker strings (`VERIFY: CONNECTED`, `VERIFY: SUBSCRIBED`, `VERIFY: REQUEST_RECEIVED`, `VERIFY: REPLY_RECEIVED`) are the same strings the JCSMP Request-Reply (Direct) leaf emits, character for character. They are read PER PROCESS: the replier output carries `SUBSCRIBED` and `REQUEST_RECEIVED`, the requestor output carries `REPLY_RECEIVED`, and both print `CONNECTED`. Until the Python verify script lands, the developer reads them from the two terminals (`implement-mode.md` Step 5); the verify script will read the same strings, so never change one. +These four `VERIFY:` marker strings (`VERIFY: CONNECTED`, `VERIFY: SUBSCRIBED`, `VERIFY: REQUEST_RECEIVED`, `VERIFY: REPLY_RECEIVED`) are the same strings the JCSMP Request-Reply (Direct) leaf emits, character for character. They are read PER PROCESS: the replier output carries `SUBSCRIBED` and `REQUEST_RECEIVED`, the requestor output carries `REPLY_RECEIVED`, and both print `CONNECTED`. `verify.sh` reads these same strings in each process's own output (`implement-mode.md` Step 5), so never change one. diff --git a/plugins/solace-messaging-skills/skills/solace-application-development/references/python/scripts/verify.sh b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/scripts/verify.sh new file mode 100755 index 0000000..327ddb2 --- /dev/null +++ b/plugins/solace-messaging-skills/skills/solace-application-development/references/python/scripts/verify.sh @@ -0,0 +1,627 @@ +#!/usr/bin/env bash +# verify.sh: run-and-observe verification for the generated Solace Python projects +# (guaranteed pub/sub, direct pub/sub, and direct request-reply). +# +# Usage (three modes): +# Single project (both scripts in the working directory, one .venv): +# verify.sh [ [pass]] +# With no connection arguments, each script reads ./config.json (the only way +# to reach a tcps:// host, because the trust store path has no argument form). +# Connection arguments are refused when ./config.json exists, because the +# scripts prefer config.json and would connect to a different broker. +# Separate projects (Solace Suggested, or the Custom decoupled-projects knob; +# each project has its own .venv and reads its own gitignored config.json): +# verify.sh --subscriber-dir --publisher-dir +# verify.sh direct-request-reply --replier-dir --requestor-dir +# Any other app shape (web app, embedded service; the app reads its own config): +# verify.sh app +# Sources ./verify-hooks.sh (generated with the project) for the ONLY +# app-specific facts: START_CMD, TRIGGER_CMD, READY_MARKER, PASS_MARKER. +# +# What it does: checks every generated script with py_compile, starts the +# long-running role (subscriber or replier) FIRST with the project's own +# .venv interpreter, waits for its readiness marker, runs the foreground role +# (publisher or requestor) and lets it exit on its own, watches each process's +# OWN captured output for the fixed VERIFY: marker lines, sends SIGINT to the +# long-running role to exercise its graceful shutdown, and classifies the +# outcome by exit code: +# 0 = stage passed. For the fixed stages: the marker was seen in this run, the +# foreground role exited with status 0 on its own, the shutdown line +# printed, AND the long-running role exited with status 0. For the app +# stage: READY_MARKER, then a new PASS_MARKER after the trigger, and a +# bounded teardown (the app's shutdown is not checked). +# 1 = code failure (py_compile failed, a marker never appeared, a process died +# before its milestone, the foreground role exited non-zero, the shutdown +# line never printed, or a run hung and had to be force-killed), OR a +# precondition miss before any process starts (a usage error, or a missing +# script, .venv, config.json, or verify-hooks.sh). stderr names which. +# 2 = environment failure (a Solace Python API connection, authentication, or +# TLS signature appeared in a process log, or the long-running role was +# still connecting at the readiness deadline with no VERIFY: CONNECTED; the +# code is not the problem, so the agent must NOT enter the fix loop) +# +# The Python API has no guaranteed request-reply, so there is no +# guaranteed-request-reply stage. +# +# Script resolution: the script does NOT read a build file. It relies on the fixed +# script names that implement-mode.md Step 4 gives every generated project +# (guaranteed_subscriber.py, direct_replier.py, ...), so the same script works +# across generations whatever the function and constant names inside each file. +# On the fan-out leaf it drives guaranteed_subscriber.py only. +# +# This is run-and-observe, NOT a test framework. It does not stand up a broker. +# +# Windows: run it under WSL or Git Bash; it also finds .venv/Scripts/python.exe. +# +# Credential handling (accepted Quickstart tradeoff): connection arguments are a +# pass-through, mirroring the generated scripts' own command-line fallback, so on a +# shared host the password may appear in process listings. With config.json (and +# always in separate-projects mode) no credential appears on the command line. The +# script never echoes the password, never enables shell command tracing, and never +# evals its arguments. + +set -euo pipefail + +# ── Arguments ────────────────────────────────────────────────────────────────── +# The --subscriber-dir/--replier-dir flag names the LONG-RUNNING role's project (the +# SIGINT target); --publisher-dir/--requestor-dir names the FOREGROUND role's +# project. Both directories default to "." so the single-project path needs no flag. +usage() { + cat >&2 <<'EOF' +usage: + # Single project (scripts read ./config.json; pass connection arguments only when there is no config.json): + verify.sh [ [pass]] + + # Separate projects (each project reads its own gitignored config.json): + verify.sh --subscriber-dir --publisher-dir + verify.sh direct-request-reply --replier-dir --requestor-dir + + # Any other app shape (web app, embedded service; reads ./verify-hooks.sh): + verify.sh app +EOF + exit 1 +} + +STAGE="${1:-}" +[ -z "$STAGE" ] && usage +shift || true + +SUB_DIR="." # long-running role's project (subscriber, or replier for request-reply) +PUB_DIR="." # foreground role's project (publisher, or requestor for request-reply) +TWO_PROJECT=0 +SUB_SET=0; PUB_SET=0 +CONN_ARGS=() +while [ "$#" -gt 0 ]; do + case "$1" in + --subscriber-dir|--replier-dir) + [ -n "${2:-}" ] || { echo "$1 needs a directory value" >&2; usage; } + SUB_DIR="$2"; SUB_SET=1; TWO_PROJECT=1; shift 2 ;; + --publisher-dir|--requestor-dir) + [ -n "${2:-}" ] || { echo "$1 needs a directory value" >&2; usage; } + PUB_DIR="$2"; PUB_SET=1; TWO_PROJECT=1; shift 2 ;; + -*) + echo "unknown flag: $1" >&2; usage ;; + *) + CONN_ARGS+=("$1"); shift ;; + esac +done + +case "$STAGE" in + publisher|consumer|roundtrip) SUB_SCRIPT=guaranteed_subscriber.py; PUB_SCRIPT=guaranteed_publisher.py ;; + direct) SUB_SCRIPT=direct_subscriber.py; PUB_SCRIPT=direct_publisher.py ;; + direct-request-reply) SUB_SCRIPT=direct_replier.py; PUB_SCRIPT=direct_requestor_blocking.py ;; + app) ;; + guaranteed-request-reply) + echo "the Solace Python API has no guaranteed request-reply, so there is no $STAGE stage" >&2; usage ;; + *) echo "unknown stage: $STAGE" >&2; usage ;; +esac + +if [ "$STAGE" = "app" ]; then + { [ "$TWO_PROJECT" -eq 0 ] && [ "${#CONN_ARGS[@]}" -eq 0 ]; } || { + echo "the app stage takes no arguments; the app reads its own config" >&2; usage; } +elif [ "$TWO_PROJECT" -eq 1 ]; then + { [ "$SUB_SET" -eq 1 ] && [ "$PUB_SET" -eq 1 ]; } || { + echo "separate-projects mode needs BOTH role directories" >&2; usage; } + [ "${#CONN_ARGS[@]}" -eq 0 ] || { + echo "separate-projects mode takes no connection arguments; each project reads its own config.json" >&2; usage; } + # Resolve to absolute paths so each role runs from INSIDE its own project + # (its config.json is read relative to that directory) whatever the cwd. + SUB_DIR="$(cd "$SUB_DIR" && pwd)" + PUB_DIR="$(cd "$PUB_DIR" && pwd)" +else + case "${#CONN_ARGS[@]}" in + 0) [ -f config.json ] || { + echo "no connection details: pass [pass], or create config.json from config.example.json" >&2; usage; } ;; + 3|4) [ ! -f config.json ] || { + echo "config.json exists and the scripts prefer it over connection arguments; omit the arguments, or remove config.json" >&2; usage; } ;; + *) echo "connection arguments are [pass]" >&2; usage ;; + esac +fi + +# The request-reply leaf generates the blocking requestor by default and +# direct_requestor.py on the non-blocking variant; use whichever the project has. +if [ "$STAGE" = "direct-request-reply" ] && [ ! -f "$PUB_DIR/$PUB_SCRIPT" ] && [ -f "$PUB_DIR/direct_requestor.py" ]; then + PUB_SCRIPT=direct_requestor.py +fi + +# Each project runs with its own virtual environment's interpreter, never the +# system python3 (implement-mode.md Step 3). +venv_python() { + if [ -x "$1/.venv/bin/python" ]; then + echo "$1/.venv/bin/python" + elif [ -x "$1/.venv/Scripts/python.exe" ]; then + echo "$1/.venv/Scripts/python.exe" + fi +} + +if [ "$STAGE" != "app" ]; then + for pair in "$SUB_DIR:$SUB_SCRIPT" "$PUB_DIR:$PUB_SCRIPT"; do + d="${pair%:*}"; s="${pair##*:}" + [ -f "$d/$s" ] || { echo "no $s found in: $d (implement-mode.md Step 4 fixes the script names)" >&2; exit 1; } + [ -n "$(venv_python "$d")" ] || { + echo "no virtual environment in: $d (create it: python3 -m venv .venv && .venv/bin/pip install -r requirements.txt)" >&2; exit 1; } + done + SUB_PY="$(venv_python "$SUB_DIR")" + PUB_PY="$(venv_python "$PUB_DIR")" +fi + +# ── Preflight conformance warnings (warn-only; never changes the exit contract) ── +# Reports the generation-conformance misses that the verification checklist's +# "Generation conformance" group records: a missing tailored checklist, scripts +# without the AI-assisted disclaimer header, and a solace-pubsubplus pin that +# drifted from the release PyPI reports now. +preflight() { + local d="$1" f missing="" pin rel + if [ ! -f "$d/solace-verification-checklist.md" ]; then + echo "PREFLIGHT WARN: $d/solace-verification-checklist.md is missing (every generation must emit it; implement-mode.md Step 4)" >&2 + fi + for f in "$d"/*.py; do + [ -f "$f" ] || continue + head -n 1 "$f" | grep -qxF '# AI-assisted code. Review before production use.' || missing="$missing$f"$'\n' + done + if [ -n "$missing" ]; then + echo "PREFLIGHT WARN: generated scripts missing the AI-assisted disclaimer header:" >&2 + printf '%s' "$missing" >&2 + fi + if [ -f "$d/requirements.txt" ]; then + pin=$(grep -oE '^solace-pubsubplus==[^[:space:]]+' "$d/requirements.txt" 2>/dev/null | sed 's/^solace-pubsubplus==//' | head -1) || true + rel=$(curl -s --max-time 5 https://pypi.org/pypi/solace-pubsubplus/json 2>/dev/null \ + | python3 -c 'import json, sys; print(json.load(sys.stdin)["info"]["version"])' 2>/dev/null) || true + if [ -z "${pin:-}" ]; then + echo "PREFLIGHT WARN: $d/requirements.txt does not pin solace-pubsubplus== (implement-mode.md Step 3)" >&2 + elif [ -n "${rel:-}" ] && [ "$pin" != "$rel" ]; then + echo "PREFLIGHT WARN: requirements.txt pins solace-pubsubplus $pin in $d, but PyPI reports $rel as the latest release" >&2 + fi + fi +} +if [ "$TWO_PROJECT" -eq 1 ]; then + preflight "$SUB_DIR" + preflight "$PUB_DIR" +else + preflight "." +fi + +# ── Live-run heads-up (SKILL.md Invariant 6): name the target and the side effects ─ +# The host comes from the connection arguments, else from the long-running role's +# config.json (the app stage's own config is not read). +config_host() { + [ -f "$1/config.json" ] || return 0 + grep -oE '"solace\.messaging\.transport\.host"[[:space:]]*:[[:space:]]*"[^"]*"' "$1/config.json" 2>/dev/null \ + | head -1 | sed -E 's/.*:[[:space:]]*"([^"]*)"$/\1/' || true +} +TARGET_HOST="${CONN_ARGS[0]:-}" +if [ -z "$TARGET_HOST" ] && [ "$STAGE" != "app" ]; then + TARGET_HOST="$(config_host "$SUB_DIR")" +fi +echo "── Live-run heads-up: stage '$STAGE' starts processes that connect to broker ${TARGET_HOST:-}; broker-side effects can include provisioned durable queues, opened connections, and published messages ──" + +# ── Environment-failure signatures (doc-traceable API symbols) ────────────────── +# The API's named error classes and the SOLCLIENT_SUBCODE_* names it logs when the +# broker or the transport rejects the session. Observed live with solace-pubsubplus: +# wrong password AuthenticationError, SOLCLIENT_SUBCODE_LOGIN_FAILURE +# wrong VPN SOLCLIENT_SUBCODE_MSG_VPN_NOT_ALLOWED +# unresolvable host ServiceUnreachableError +# unroutable host Session connect timeout (SOLCLIENT_SUBCODE_TIMEOUT, also used by non-connect timeouts) +# refused port SOLCLIENT_SUBCODE_COMMUNICATION_ERROR +# no trust store SOLCLIENT_SUBCODE_FAILED_LOADING_TRUSTSTORE +# wrong CA SOLCLIENT_SUBCODE_UNTRUSTED_CERTIFICATE +# AuthorizationError, CLIENT_USERNAME_IS_SHUTDOWN, and INVALID_VIRTUAL_ADDRESS are +# the remaining broker rejections of a session. A match means the broker, +# credentials, or trust store are wrong, NOT the code, so exit 2 bypasses the fix loop. +ENV_SIGNATURES='AuthenticationError|AuthorizationError|ServiceUnreachableError|SOLCLIENT_SUBCODE_LOGIN_FAILURE|SOLCLIENT_SUBCODE_MSG_VPN_NOT_ALLOWED|SOLCLIENT_SUBCODE_CLIENT_USERNAME_IS_SHUTDOWN|SOLCLIENT_SUBCODE_INVALID_VIRTUAL_ADDRESS|SOLCLIENT_SUBCODE_COMMUNICATION_ERROR|SOLCLIENT_SUBCODE_FAILED_LOADING_TRUSTSTORE|SOLCLIENT_SUBCODE_UNTRUSTED_CERTIFICATE|Session connect timeout' + +# Every baked sample's signal handler prints this line on SIGINT. Seeing it in the +# long-running role's log is the observable proof that the graceful shutdown ran. +SHUTDOWN_LINE='Shutdown signal received' + +# The baked guaranteed subscriber logs this line and exits when receiver.start() +# fails to provision or bind the queue, which is what a client without the +# endpoint-create permission sees. The code is not the problem: exit 2. +ENDPOINT_DENIED='Could not bind to queue' + +TIMEOUT_S=30 # marker-watch deadline +PUBLISH_WAIT_S=30 # foreground-role deadline before escalating to a code failure +SHUTDOWN_WAIT_S=15 # bounded post-SIGINT wait before escalating to SIGTERM/SIGKILL + +# A long-running role that is still alive at the readiness deadline and never +# printed VERIFY: CONNECTED is still connecting: an unreachable host or a blocked +# port, where the connect attempt (and its retries) outlasts TIMEOUT_S without +# logging an error. That is the environment, not the code, so exit 2. +classify_stalled_connect() { # + if [ "$status" -ne 2 ] && [ "$3" -eq 0 ] && kill -0 "$1" 2>/dev/null && ! grep -qF "VERIFY: CONNECTED" "$2"; then + status=2 + CONNECT_STALLED=1 + fi +} + +# Print the count of in once it has stopped changing for 2 seconds +# (bounded by TIMEOUT_S), so messages an earlier run left on a durable queue, which +# arrive just after the bind, are all counted before the publisher starts. +settled_count() { # + local n m settle_deadline=$(( SECONDS + TIMEOUT_S )) + n=$(grep -cF "$2" "$1" || true) + while [ "$SECONDS" -lt "$settle_deadline" ]; do + sleep 2 + m=$(grep -cF "$2" "$1" || true) + [ "$m" -eq "$n" ] && break + n=$m + done + echo "$n" +} + +# ── 1. py_compile first (a syntax error is a code failure, exit 1) ────────────── +compile_project() { + local d="$1" py="$2" files=() + local f + for f in "$d"/*.py; do [ -f "$f" ] && files+=("$f"); done + [ "${#files[@]}" -gt 0 ] || return 0 + echo "── Checking $d with py_compile ──" + if ! "$py" -m py_compile "${files[@]}"; then + echo "py_compile failed in $d (code failure, exit 1)" >&2 + exit 1 + fi +} +if [ "$STAGE" = "app" ]; then + if [ -n "$(venv_python .)" ]; then + compile_project . "$(venv_python .)" + else + echo "── No .venv in the working directory; skipping py_compile (the app stage runs through its own START_CMD) ──" + fi +else + compile_project "$SUB_DIR" "$SUB_PY" + [ "$PUB_DIR" = "$SUB_DIR" ] || compile_project "$PUB_DIR" "$PUB_PY" +fi + +# ── Per-process logs: each role writes its own log, because both print ───────── +# VERIFY: CONNECTED and a merged log could not say which process reached a marker. +SUB_LOG="$(mktemp)" +PUB_LOG="$(mktemp)" + +# If the script itself dies (agent-harness timeout, Ctrl-C, CI cancellation, or a +# set -e exit), the backgrounded long-running role would keep running detached and +# hold its queue. The EXIT trap stops both roles and removes the logs. SIGKILL of +# this script is the one residual: `set -m` puts each child in its own process +# group, so a hard kill of this script's group cannot reach them. +cleanup() { + if [ -n "${SUB_PID:-}" ]; then kill -TERM -- "-$SUB_PID" 2>/dev/null || true; fi + if [ -n "${PUB_PID:-}" ]; then kill -TERM -- "-$PUB_PID" 2>/dev/null || true; fi + rm -f "${SUB_LOG:-}" "${PUB_LOG:-}" +} +trap cleanup EXIT + +status=1 # default to code failure until a stage milestone is confirmed +DIRECT_ZERO_RECEIPT=0 # set when the at-most-once burst was lost after a confirmed subscription +RR_NO_REPLY=0 # set when the replier subscribed but no reply came back +CONNECT_STALLED=0 # set when the long-running role was still connecting at the readiness deadline + +# ── Launchers ────────────────────────────────────────────────────────────────── +# `set -m` starts each role in its own process group, so the group-directed kills +# below reach the whole role (and a `bash -c` wrapper's child on the app stage). +# In separate-projects mode each role runs from inside its own project, so it reads +# that project's config.json; in single-project mode the connection arguments, when +# given, pass through as the scripts' command-line fallback. +start_role() { #