Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions plugins/solace-messaging-skills/evals/output-evals.json
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@
"max_agent_turns": 25,
"graders": [
{ "type": "assistant_grep", "pattern": "[Bb]roker", "expect": "present", "turn": 1 },
{ "type": "assistant_grep", "pattern": "[Mm]ulti-threaded|[Mm]inimal threading", "expect": "present", "turn": 1 },
{ "type": "assistant_grep", "pattern": "[Mm]ulti-?threaded|[Mm]inimal( threading| \\(blocking)|[Ss]ingle[- ]threaded|[Tt]hread of control", "expect": "present", "turn": 1 },
{ "type": "assistant_grep", "pattern": "Pattern\\**:", "expect": "present" },
{ "type": "assistant_grep", "pattern": "Delivery\\**:", "expect": "present" },
{ "type": "assistant_grep", "pattern": "Access type\\**:", "expect": "present" },
Expand Down Expand Up @@ -199,7 +199,7 @@
"graders": [
{ "type": "tool_use", "tool": "*", "input_pattern": "src/solace_connection_config\\.py", "expect": "present" },
{ "type": "tool_use", "tool": "Write", "input_pattern": "\"file_path\": *\"[^\"]*\\.py\"", "exclude_pattern": "\"file_path\": *\"[^\"]*/src/[^\"/]+\\.py\"", "expect": "absent" },
{ "type": "assistant_grep", "pattern": "[Mm]ulti-threaded|[Mm]inimal threading", "expect": "present", "turn": 1 },
{ "type": "assistant_grep", "pattern": "[Mm]ulti-?threaded|[Mm]inimal( threading| \\(blocking)|[Ss]ingle[- ]threaded|[Tt]hread of control", "expect": "present", "turn": 1 },
{ "type": "assistant_grep", "pattern": "Threading[*:`[:space:]]+Async", "expect": "present" },
{ "type": "assistant_grep", "pattern": "Quickstart", "fixed": true, "expect": "present", "turn": 3 },
{ "type": "assistant_grep", "pattern": "Solace Suggested", "fixed": true, "expect": "present", "turn": 3 },
Expand Down Expand Up @@ -274,7 +274,7 @@
"graders": [
{ "type": "tool_use", "tool": "*", "input_pattern": "src/solace_connection_config\\.py", "expect": "present" },
{ "type": "tool_use", "tool": "Write", "input_pattern": "\"file_path\": *\"[^\"]*\\.py\"", "exclude_pattern": "\"file_path\": *\"[^\"]*/src/[^\"/]+\\.py\"", "expect": "absent" },
{ "type": "assistant_grep", "pattern": "[Mm]ulti-threaded|[Mm]inimal threading", "expect": "present", "turn": 1 },
{ "type": "assistant_grep", "pattern": "[Mm]ulti-?threaded|[Mm]inimal( threading| \\(blocking)|[Ss]ingle[- ]threaded|[Tt]hread of control", "expect": "present", "turn": 1 },
{ "type": "assistant_grep", "pattern": "Threading[*:`[:space:]]+Async", "expect": "present" },
{ "type": "assistant_grep", "pattern": "[Ff]ailover", "expect": "present", "turn": 4 },
{ "type": "assistant_grep", "pattern": "ght_client_certs.md", "fixed": true, "expect": "present" },
Expand Down Expand Up @@ -311,7 +311,7 @@
"graders": [
{ "type": "tool_use", "tool": "WebFetch", "input_pattern": "Python-API-Request-Reply\\.md", "expect": "present" },
{ "type": "assistant_grep", "pattern": "Pattern[*:`[:space:]]+Guaranteed Pub/Sub \\(", "expect": "present" },
{ "type": "assistant_grep", "pattern": "[Mm]ulti-threaded|[Mm]inimal threading", "expect": "present", "turn": 1 },
{ "type": "assistant_grep", "pattern": "[Mm]ulti-?threaded|[Mm]inimal( threading| \\(blocking)|[Ss]ingle[- ]threaded|[Tt]hread of control", "expect": "present", "turn": 1 },
{ "type": "assistant_grep", "pattern": "Threading[*:`[:space:]]+Async", "expect": "present" },
{ "type": "assistant_grep", "pattern": "Pattern[*:`[:space:]]+Request-Reply", "expect": "absent" },
{ "type": "assistant_grep", "pattern": "[Cc]orrelation", "expect": "present" },
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Ask only the questions the prompt left open, in this order. Before you answer ea

1. **Interaction shape: request-reply or publish/subscribe?** Grounded in [Message Exchange Patterns](https://docs.solace.com/Get-Started/message-exchange-patterns.md). Decision: a request that expects a correlated reply is Request-Reply; an event fanned out to one or many independent consumers is publish/subscribe.
2. **Delivery guarantee: direct or guaranteed?** This ONE question applies to both interaction shapes, so ask it next regardless of the answer to question 1. Grounded in [Message Delivery Modes](https://docs.solace.com/Get-Started/message-delivery-modes.md). Decision: direct is fire-and-forget with no broker-side persistence, for high-rate flows that tolerate occasional loss; guaranteed (persistent) is persisted and survives consumer downtime.
3. **Threading on the messaging data path: multi-threaded, or minimal threading? (Direct Pub/Sub and guaranteed publish/subscribe branches only.)** Grounded in [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md) on the guaranteed branch and in [Consuming Direct Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-DM-Receive.md) on the direct branch; both pages state that `receive_message(timeout)` blocks the routine until a message arrives, the timeout passes, or a service interruption occurs, and the first states that `receive_async()` invokes the callback on a new Python thread for every message. Frame the choice as WHERE the application's thread of control sits on the messaging data path, never as speed: make no throughput claim for either answer. Name the two answers with the labels written here, multi-threaded and minimal threading, whatever words the framing adds: the developer's reply maps onto those labels. The reason the question exists, stated in one line when you ask it: in Python the GIL lets only one thread run Python code at a time, so the extra API threads of the callback shape buy less than they do in other languages, and a developer who wants one thread of control can have it. Three answers:
3. **Threading on the messaging data path: multi-threaded, or minimal threading? (Direct Pub/Sub and guaranteed publish/subscribe branches only.)** Grounded in [Consuming Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Receive.md) on the guaranteed branch and in [Consuming Direct Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-DM-Receive.md) on the direct branch; both pages state that `receive_message(timeout)` blocks the routine until a message arrives, the timeout passes, or a service interruption occurs, and the first states that `receive_async()` invokes the callback on a new Python thread for every message. Frame the choice as WHERE the application's thread of control sits on the messaging data path, never as speed: make no throughput claim for either answer. Name the two answers so the developer can tell them apart. The first answer can be called multi-threaded, async, callback, or push; the second can be called minimal threading, blocking, pull, or single thread of control. Any of these names is fine, and the developer's reply maps onto the matching answer. The reason the question exists, stated in one line when you ask it: in Python the GIL lets only one thread run Python code at a time, so the extra API threads of the callback shape buy less than they do in other languages, and a developer who wants one thread of control can have it. Three answers:
- **Multi-threaded app** (the push model): the API pushes each received message, and on the guaranteed branch each publish receipt, to an application callback on an API thread, so the application manages the concurrency between its own threads and the API's threads. This is the async shape and the current default. Record `Threading: Async`.
- **Minimal threading preferred** (the pull model): the application's own thread pulls each message with a blocking `receive_message(timeout)` call, and on the guaranteed branch publishes each message with a blocking `publish_await_acknowledgement()` call, so no application callback runs on the data path and no application state is shared with an API thread; only the service event listeners still run on API threads. The cost is one message in flight at a time on each side. This is the blocking shape. Record `Threading: Minimal (blocking publish and receive)`.
- **Unsure**: the async shape. Record `Threading: Async`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ The publisher only connects and publishes a short bounded burst of DIRECT messag

1. Build the connection properties and the service exactly as the subscriber does (step 1 above), then the publisher with `create_direct_message_publisher_builder().on_back_pressure_wait(PUBLISH_BUFFER_CAPACITY).build()`. Keep `set_publish_failure_listener(...)`: direct has no broker receipt, so a buffered message that cannot be sent is reported only through that listener. Register the three service event listeners BEFORE `connect()`.
2. `messaging_service.connect()`. Immediately after `connect()` returns, emit the connect marker: `trace("VERIFY: CONNECTED")`. Then `publisher.start()`.
3. Publish a SHORT BOUNDED BURST to the design summary's topic with `publish(payload, Topic.of(topic))` in a small fixed-count loop, then return from the publish step. REMOVE the sample's continuous loop, its rotating payload and dynamic topic, and its stats thread. Keep the sample's `PubSubPlusClientError` branch, which sets the shared state's `exit_code` to 1. `teardown_solace(...)` in `main()`'s `finally` calls `terminate(grace_period)`, which gives the buffered messages up to the grace period to leave before `disconnect()`. Keep the SIGINT/SIGTERM handler, so Ctrl-C during a back pressure wait still reaches teardown. When the request describes a real application, its own messages replace the burst (the demo-harness rule). That replacement changes the payload and the topic, never the shape: a request that describes a continuous or high-rate feed (readings several times a second, a telemetry stream) still gets a publisher that sends a bounded batch of the application's real messages and exits on its own, because the `direct` stage waits for that exit. Do not keep the sample's open-ended loop behind an environment-variable cap, and do not move the project to the `app` stage to drive it; the continuous trigger is the developer's integration step after verification.
3. Publish a SHORT BOUNDED BURST to the design summary's topic with `publish(payload, Topic.of(topic))` in a small fixed-count loop, then return from the publish step. REMOVE the sample's continuous loop, its rotating payload and dynamic topic, and its stats thread. Keep the sample's `PubSubPlusClientError` branch, which sets the shared state's `exit_code` to 1. `teardown_solace(...)` in `main()`'s `finally` calls `terminate(grace_period)`, which gives the buffered messages up to the grace period to leave before `disconnect()`. Keep the SIGINT/SIGTERM handler, so Ctrl-C during a back pressure wait still reaches teardown. When the request describes a real application, its own messages replace the burst (the demo-harness rule), and each payload stays a `bytearray` or a `str`, never `bytes`. That replacement changes the payload and the topic, never the shape: a request that describes a continuous or high-rate feed (readings several times a second, a telemetry stream) still gets a publisher that sends a bounded batch of the application's real messages and exits on its own, because the `direct` stage waits for that exit. Do not keep the sample's open-ended loop behind an environment-variable cap, and do not move the project to the `app` stage to drive it; the continuous trigger is the developer's integration step after verification.
4. The publisher emits NO send-side marker. Direct has no broker receipt, so there is no `PUBLISH_ACKED` analog; a marker after `publish()` would only prove the local call returned. The proof of the round trip is `VERIFY: MESSAGE_RECEIVED` in the subscriber output.

## The marker contract
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ The publisher only connects and publishes; it does NOT provision the queue or th
1. Build the connection properties and the service exactly as the subscriber does (step 1 above), then the publisher. Callback shape: `create_persistent_message_publisher_builder().on_back_pressure_wait(PUBLISH_BUFFER_CAPACITY).build()`. Blocking shape: `create_persistent_message_publisher_builder().build()` with NO back-pressure setting, as the blocking sample explains: one message is in flight at a time, so the publish buffer cannot grow. Register the three service event listeners BEFORE `connect()`.
2. `messaging_service.connect()`. Immediately after `connect()` returns, emit the connect marker: `trace("VERIFY: CONNECTED")`.
3. `publisher.start()`. Callback shape: then `publisher.set_message_publish_receipt_listener(...)` AFTER `start()`, as the sample orders them. In the receipt handler's ACK branch, emit the publish-ACK marker: `trace("VERIFY: PUBLISH_ACKED")`. CRITICAL: the sample logs the ACK with `logger.debug`, which the INFO-level logging never shows, so the marker would never appear. Emit it through `trace(...)`, not through the logger. In the NACK branch, keep the sample's warning and also set the shared state's `exit_code` to 1, so a rejected message makes the publisher exit non-zero. Pass the shared state to the handler to do this. Blocking shape: NO receipt listener; the marker moves into step 4.
4. Publish persistent messages to the design summary's topic as the sample does: one `message_builder()` template with `PERSISTENT_TIME_TO_LIVE` set once, and `build(payload, additional_message_properties={APPLICATION_MESSAGE_ID: msg_id})` per message. Callback shape: `publish(message, topic, user_context=msg_id)`. Blocking shape: `publish_await_acknowledgement(message, topic, PUBLISH_ACK_TIMEOUT_MS)` with the sample's POSITIVE timeout in milliseconds and no `user_context`; a normal return IS the broker's ACK, so emit `trace("VERIFY: PUBLISH_ACKED")` immediately after the call returns. Keep the blocking sample's three discard branches (`PubSubTimeoutError`, the NACK error classes, and `PublisherOverflowError`), which log the message id at INFO and continue (the Discard the Message strategy from the Publishing Persistent Messages page), but set the shared state's `exit_code` to 1 in EACH of them, the same way the callback shape's NACK branch does: the loop still sends the remaining messages, and the publisher exits non-zero at the end, so a run that lost one message out of many fails the `verify.sh` stage instead of passing on the markers of the others. Keep its `PubSubPlusClientError` branch too, which sets `exit_code` to 1 and stops. See [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md).
4. Publish persistent messages to the design summary's topic as the sample does: one `message_builder()` template with `PERSISTENT_TIME_TO_LIVE` set once, and `build(payload, additional_message_properties={APPLICATION_MESSAGE_ID: msg_id})` per message, where `payload` is a `bytearray` or a `str`, never `bytes` (the demo-harness rule in `implement-mode.md`). Callback shape: `publish(message, topic, user_context=msg_id)`. Blocking shape: `publish_await_acknowledgement(message, topic, PUBLISH_ACK_TIMEOUT_MS)` with the sample's POSITIVE timeout in milliseconds and no `user_context`; a normal return IS the broker's ACK, so emit `trace("VERIFY: PUBLISH_ACKED")` immediately after the call returns. Keep the blocking sample's three discard branches (`PubSubTimeoutError`, the NACK error classes, and `PublisherOverflowError`), which log the message id at INFO and continue (the Discard the Message strategy from the Publishing Persistent Messages page), but set the shared state's `exit_code` to 1 in EACH of them, the same way the callback shape's NACK branch does: the loop still sends the remaining messages, and the publisher exits non-zero at the end, so a run that lost one message out of many fails the `verify.sh` stage instead of passing on the markers of the others. Keep its `PubSubPlusClientError` branch too, which sets `exit_code` to 1 and stops. See [Publishing Persistent Messages](https://docs.solace.com/API/API-Developer-Guide-Python/Python-PM-Publish.md).
5. EXIT on its own: replace the sample's continuous publish loop with the application's own messages, or with one message on a try-it run, then return from the publish step. `teardown_solace(...)` in `main()`'s `finally` calls `terminate(grace_period)`. Callback shape: it sends what is buffered and waits for the broker's receipt of every published message, so the `VERIFY: PUBLISH_ACKED` marker prints before the script exits. Blocking shape: every call has already returned with its ACK and printed the marker, so `terminate()` returns at once; keep the sample's `IncompleteMessageDeliveryError` branch for a message the API never handed to the transport. Drop the stats thread (or the blocking sample's inline rate report) with the loop. Keep the SIGINT/SIGTERM handler, so Ctrl-C during a back pressure wait or an acknowledgement wait still reaches teardown.

**Persisted-request variant (application correlation id).** When the `Consumption endpoint` field records an application correlation id (a persisted request that `design-mode.md` routed here), generate the request leg only. The publisher adds `message_properties.CORRELATION_ID: msg_id` to the same `additional_message_properties` dict in step 4 (on either threading shape), and the subscriber stays unchanged. Do NOT generate an answer path: no reply topic, no reply subscription, no reply publish, and no `VERIFY:` marker beyond the four below. List the answer path under Developer-owned items in the checklist (`implement-mode.md` Step 6), and name it in the close-out as the developer's next step.
Expand Down
Loading
Loading