Skip to content

Add a Swift runtime for Prompty - #439

Open
sethjuarez wants to merge 47 commits into
mainfrom
sejuare-microsoft/swift-runtime
Open

Add a Swift runtime for Prompty#439
sethjuarez wants to merge 47 commits into
mainfrom
sejuare-microsoft/swift-runtime

Conversation

@sethjuarez

@sethjuarez sethjuarez commented Aug 4, 2026

Copy link
Copy Markdown
Member

Adds a Swift implementation of the Prompty runtime. There was no Swift runtime in this repo before; nothing existing changes behaviour.

The Rust runtime is the behavioural reference throughout, and both are held to the same cross-runtime vectors in spec/vectors.

Important

Status: complete and green on the pinned @typra/emitter@0.4.2. Merge is gated on an upstream emitter release.

The runtime is finished and passes on Linux, macOS and Windows: 77 tests, 0 failures, including live provider E2E. Generated model files are never hand-edited; a scripted post-generation repair shim (schema/scripts/patch-swift-emitter-defects.mjs) stands in for unfixed emitter defects and must be deleted before this is considered done.

Six published emitter releases were probed and rejected, two of them prepared as commits and then deliberately reverted. 0.4.10 is the closest yet — 23 of the 24 shim patch sites are fixed upstream — but one residual and one new defect remain, and its effect on C#/TypeScript is unmeasured. 0.4.9 fixes the last Swift defect (Connection.unknown) and the C# break, but a new named-collection feature emits struct-style accessors against polymorphic enums — 48 fresh compile errors. 0.4.8 leaves just one Swift defect (Connection.unknown) but still breaks C#. 0.4.6 fixes six Swift defects and regresses C# and TypeScript. 0.4.5 was announced upstream as carrying the generated-test fixes, but measured worse than 0.4.9 on Swift. 0.4.3 fixes four and was committed, until CI showed the same C# break — a regression the local run had masked. A shared emitter pin cannot be bumped for one language, so that belongs in its own repo-wide PR. See Validated against each upstream emitter release.

Remaining merge acceptance: consume an emitter fixing the Swift defects without the regressions carried by 0.4.3 through 0.4.9 → delete the shim and its call site → restore Swift test-dir in schema/tspconfig.yaml (still unusable: 1184 compile errors at 0.4.3, and 180 at both 0.4.8 and 0.4.9, across four test-generator defect classes) → regenerate cleanly → run the generated model tests, the 77 runtime tests, and the live E2E → push the final reviewed commit. See Upstream emitter defects.

What's here

Two SwiftPM packages, and the split is the point:

Package Module Contents Hand-written?
prompty-model PromptyModel Domain types and pipeline protocols No — Typra-generated
prompty Prompty Loader, renderers, parser, registry, harness Yes
prompty PromptyOpenAI OpenAI executor and processor Yes

135 generated model files, 26 hand-written sources, 14 test files.

The runtime defines no domain types of its own. Prompty, Model, Message, ContentPart, Tool and the four pipeline protocols (Renderer, Parser, Executor, Processor) all come from the generated model, and the hand-written code conforms to them. Partway through this work I found that the generated model already emits the pipeline protocols, so I deleted the hand-written declarations I had started with and adopted the generated ones — a competing type layer is exactly what this port is meant to avoid.

Covered: loading (including ${env:} / ${file:}), rendering, parsing, preparation, protocols and registries, provider execution and processing, tools and turns, structured output, streaming, and harness durability/replay.

Commits

99ab256b Generate the Swift Prompty model via Typra
86eb1265 Add the Swift Prompty core runtime
68f04e2e Add the OpenAI provider
e5e4c6b7 Add the turn harness and durability replay
a64ed071 Fix Windows line-ending handling; add live OpenAI E2E tests
56f638d8 Document the Swift runtime
3d60d232 Guard inherited fields on composite Property subtypes
05f70409 Conform the hand-written runtime to swift-format defaults
97c3a7a7 Add the PROMPTY_SKIP_SWIFT_SHIM emitter-validation escape hatch
d13109af Add the Swift CI workflow
2a724c03 Stop silently discarding CRLF frontmatter
66b60ec3 Adopt @typra/emitter 0.4.3, minimize and harden the Swift shim
0a5a5e99 Revert the 0.4.3 pin after CI caught a C# regression; keep the hardening
15f7e620 Correct the name classification in the shim header (retirement criterion)
fa15d77e Record the @typra/emitter 0.4.8 probe: Swift near-clean, C# still blocking
66de5c52 Record the @typra/emitter 0.4.9 probe: both blockers fixed, new enum defect
406fc882 Record the @typra/emitter 0.4.5 probe: announced as fixed, measured worse
75a38073 Map the per-patch residual shim at 0.4.5: 13 of 24 sites fixed upstream, 11 residual
dde5c617 Record the @typra/emitter 0.4.10 probe: 23 of 24 sites fixed upstream, one residual
dfb3aee6 Reconcile the test-dir error counts: raw lines vs primary unique
383058c2 Pin the Connection base-field gap with measured load/save evidence

Each of the first four was verified to build and test standalone, by moving the not-yet-committed files aside — so no commit depends on a later one.

Continuous integration

Every other runtime has a prompty-<lang>-check.yml; Swift had none, so nothing in CI compiled or ran this code. .github/workflows/prompty-swift-check.yml builds, tests and format-checks on ubuntu-latest and macos-latest.

Three decisions worth flagging:

  • spec/** is in the trigger paths. The Swift tests are validated against the shared cross-runtime vectors, so a spec change has to re-run them. The other runtimes' check workflows watch only their own directory and have the same blind spot — fixing those is out of scope here.
  • The SwiftPM cache keys on a stamp of swift --version, not just the manifests, so a runner-image or toolchain change can never restore artifacts built by a different compiler. Package.resolved is untracked — a library must not pin its consumers — so it can't contribute to the key.
  • windows-latest is deliberately absent. swift-actions/setup-swift maps Windows releases only through 5.6.3, and the swift format subcommand needs 6.x. Windows is covered by local development, and the line-ending tests build their inputs from explicit \r\n literals so they're platform-independent anyway.

swift test is hermetic on CI because the live tests skip themselves without credentials.

The formatting commit is not cosmetic drive-by work: the 135 generated PromptyModel files already pass swift-format lint --strict with zero findings, which establishes swift-format defaults as the house Swift style here. The hand-written runtime didn't conform, so CI would have failed on its first run. Only the hand-written code is linted — the generated sources are the emitter's output, and an upstream formatting change must not break CI. Formatting was authored under Swift 6.3.3 and the lint gate runs under 6.0.3, so the first green run also confirms the two formatter versions agree.

Evidence

CI, this branch — run 30904239140, both jobs green:

Runner Toolchain Build Tests Format
ubuntu-latest Swift 6.0.3 0 errors, 0 warnings 77 executed, 7 skipped, 0 failures 0 findings
macos-latest Apple Swift 6.3.3 0 errors, 0 warnings 77 executed, 7 skipped, 0 failures — (Linux only)

The 7 skips are the live OpenAI tests declining to run without OPENAI_API_KEY — the intended hermetic CI shape.

Locally, Swift 6.3.3 on x86_64-unknown-windows-msvc, where the credentials are present:

swift package clean && swift build --build-tests      → Build complete!, zero warnings
swift test  (credentials present)                     → Executed 77 tests, with 0 failures
swift test  (.env removed)                            → Executed 77 tests, 7 skipped, 0 failures
swift format lint --strict --recursive Sources Tests  → 0 findings

Between them that's the runtime building and passing on Linux, macOS and Windows, and across two different Swift versions.

Live tests are real. Seven tests hit the actual OpenAI API — chat, token caps, streaming, streamed accumulation, structured output, tool calling, and a .prompty file with ${env:} references — and they pass. They were run repeatedly with fresh random values to confirm they aren't flaky.

I deliberately hardened them after review, because the first versions could pass for the wrong reasons:

  • the token cap is now a relative two-call comparison rather than a substring guess;
  • structured output is proven by a prompt that explicitly demands prose and forbids JSON, so a successful decode proves the schema won;
  • tool use is proven by an opaque run-time station code the model cannot guess. An earlier version used a random temperature, and the model rejected 641582 °C as a tool error — a genuine finding from the test, not a flake. The sentinel is alphabetic so it can't accidentally satisfy the temperature assertion;
  • the streaming test asserts ordered reassembly of all twenty items instead of comparing two independent requests, which assumed a determinism the API doesn't promise.

A Swift-specific bug worth calling out

Swift treats a CRLF pair as a single Character. For "a\r\nb", both components(separatedBy: "\n") and split(separator: "\n") return the whole string as one element, and text.contains("\r") is false.

So a prompt assembled in memory with Windows endings missed every role marker and collapsed into one lumped message; a CRLF journal replayed as zero records; the buffered SSE path yielded one unparseable line. Rust splits at the byte level and degrades gracefully, so this hazard has no counterpart in the reference and no LF-only spec vector can catch it.

Fixed with Lines, a scalar-based splitter. Scanning unicodeScalars rather than Characters is what makes it correct: normalising first and splitting on characters afterwards still fails, because rewriting "a\r\r\nb" yields "a\r\nb" whose CR and LF are now adjacent and cluster into one grapheme again.

Character.isNewline is deliberately not used — it also matches U+0085, U+2028 and U+2029, which are legal unescaped inside JSON strings, so splitting on them tore journal records in half and discarded them silently.

A later pass found one more instance of the same hazard, and it was the worst of them. Frontmatter.split located its delimiters with firstIndex(of: "\n"), which matches nothing in a CRLF document — so the opening --- looked unterminated and the function returned empty frontmatter and an empty body, with no error at all. Loader happened to normalise before calling in, so files read from disk were unaffected and every existing test passed; but Frontmatter.split is public API, and any direct caller silently lost the entire document. It is now line-based on the same scalar scanner. The regression tests cover it directly, and also add the two cases the suite had never exercised: a .prompty file whose bytes on disk contain CRLF, and CRLF-delimited SSE. SSE.payload was trimming with CharacterSet.whitespaces, which is space and tab only, so a surviving CR defeated the [DONE] sentinel comparison.

Fixing it surfaced two subtler faults that review caught before the commit landed. Once split normalises, Loader doing so as well is not merely redundant but lossy: the first pass turns a\r\r\nb into a\r\nb, and the second reads that residual CR + LF as a terminator and deletes a lone CR that both Lines and the Rust reference preserve. Loader no longer normalises, leaving exactly one pass. And skipping leading blank lines with CharacterSet.whitespaces is wrong for the same reason Character.isNewline is wrong elsewhere — the set is horizontal-only, so a document opening with a vertical tab, form feed, NEL, or a Unicode separator would have been demoted to body-only, a genuine LF-input regression against the replaced algorithm. It now uses allSatisfy(isWhitespace).

Because that rewrite sits under every vector test's load path, equivalence with the replaced algorithm was not argued but measured: a temporary differential harness reimplemented the original character-index algorithm and asserted the two agree across a 54-sample corpus, including four documents that must throw. A negative control — reverting the whitespace predicate — made it fail on exactly the vertical-tab and NEL samples, confirming the harness discriminates rather than passing vacuously.

Every fix in this area was mutation-verified: reverted deliberately, and confirmed to produce the exact predicted failure.

Upstream emitter defects

Twelve defects in the Typra Swift emitter were characterised and reported to the schema owner across five change requests. The highest-value one drops extends base fields from three Property subtypes, all five Tool subtypes and the Connection subtypes — silent data corruption. Four of the twelve are fixed in 0.4.3, that one in 0.4.6, and the last Swift source defect in 0.4.9 — but every release from 0.4.3 on is rejected, the earlier ones for regressions in other runtimes and 0.4.9 for a new Swift defect of its own; see Validated against each upstream emitter release.

schema/scripts/patch-swift-emitter-defects.mjs runs as part of generation and repairs the output. It is a scripted post-generation step, so generated files are still never hand-edited. It is pinned to the emitter version it was written against and fails loudly — rather than silently mis-patching — on an unrecognised version, a missing anchor, or a half-patched file. GeneratedModelRoundTripTests covers every field it injects, so a silently regressed patch fails the suite rather than the runtime. It's designed to shrink to nothing and be deleted as upstream fixes land.

Validated against each upstream emitter release

Setting PROMPTY_SKIP_SWIFT_SHIM=1 skips the patch so native emitter output can be evaluated. Every candidate release is regenerated and built this way, and the results reported upstream with line-level citations.

The pin remains 0.4.2. Every later release was probed by generating unpatched Swift under it and diffing — so every verdict below is a per-file determination, not an inference from release notes — and all were rejected. Through 0.4.8 the reason was that the emitter is shared, so a bump regenerates every runtime and Swift gains arrived with C#/TypeScript regressions. At 0.4.9 those regressions are gone and the reason is Swift's own: a new defect that makes the shim larger, not smaller.

Contract item 0.4.2 (pinned) 0.4.3 0.4.5 0.4.6 0.4.8 0.4.9 0.4.10
Tool.customTool(CustomTool) wildcard fallback, with load/save ❌ shimmed ✅ fixed ✅ ✅
indirect only on genuinely recursive polymorphic enums ❌ shimmed ✅ fixed ✅ ✅
Factories emit typed cases (role: .assistant, .textPart(TextPart(…))) ❌ shimmed ✅ fixed ✅ ✅
unknown?Any?, Record<unknown>?[String: Any]?, Message[][Message] ❌ shimmed ✅ fixed ✅ ✅
Inherited extends fields on Property / Tool / Connection subtypes ❌ shimmed ❌ shimmed measured missing ✅ fixed ✅ ✅ incl. memberwise init
Connection fallback .unknown([String: Any]) ❌ shimmed ❌ shimmed ❌ 30 errors ❌ not emitted ❌ only defect left fixed ⚠️ case + save only; load still throws
Named-collection helpers work on polymorphic enums ✅ n/a — feature absent ✅ n/a ✅ n/a ✅ n/a ✅ n/a new: 48 errors10 errors
Map-form bindings loading (array-only today) ❌ shimmed ❌ shimmed — not probed ❌ not emitted ❌ not emitted — not re-probed — not re-probed
C# *Yaml* generated tests ✅ green 16 break — not measured 16 break 16 break 48/48 green — not measured
TypeScript = [] on fields with an explicit = #[] ✅ kept ✅ kept — not measured 13 lost — not re-probed ❌ still lost — not measured

0.4.3 is a real Swift gain — it would cut the shim from 419 to 345 lines and its generated Swift is byte-identical to what the current shim produces. It was adopted in 66b60ec3 and reverted in 0a5a5e99 once CI showed the C# break. Swift gains cannot be bought with a C# regression, and a shared pin cannot be regenerated for one language.

At 0.4.8 the Swift source generator is close to clean. Compiling native 0.4.8 output yields exactly 30 errors, every one type 'Connection' has no member 'unknown', every one in tool.swift. Probe-patching only that single enum case into the native output makes the model package compile — so a release adding Connection.unknown would likely collapse this shim to nothing. Likely, not certainly: compiling is not the same as being correct. Dropped fields produce zero diagnostics, so each patch still has to be retired against swift build --build-tests and swift test, never a clean build alone.

0.4.8 also fixes the inherited-extends defect including name, which matters: name is not extends-inherited (model Property and model Tool never declare it; it arrives via the Named<…> spread in core/core.tsp), so a fix scoped to extends alone would have restored every other base field and silently dropped that one.

0.4.9 clears both blockers that were reported upstream, and is still rejected. Connection.unknown is emitted in exactly the requested shape, and the C# literal asymmetry is gone — Prompty.Core.Tests reports 48 passed, 0 failed, which beats even the pinned 0.4.2 baseline (it fails 16 *Json* tests locally to a CRLF artifact). Go improves to 17 FAIL lines from 44; Rust stays green.

What blocks it is new. 0.4.9 adds named-dict collections — the feature Prompty needs so inputs: and tools: can be read as name-keyed maps. The generated helper assumes its element type is a struct, so for the polymorphic enums Property and Tool it emits item.name = name and Property.shorthandProperty, neither of which those enums declare (only the concrete ArrayProperty / FunctionTool / … structs do). That is 48 fresh compile errorsagent/prompty.swift 30, core/property.swift 10, tools/tool.swift 8 — where 0.4.8 had 30. The shim would have to grow, not shrink. Routing the name through the dictionary before load, mirroring the save path that already calls removeValue(forKey: "name"), would make the helper agnostic to struct-vs-enum elements; that has been reported upstream.

The generated test-dir is no better either. After probe-patching those 48 source errors so the library would compile and the test target would actually build, it produced the same 180 errors as 0.4.8 — which is the whole reason the library must be made to compile first: if it fails, the test target is never built and the error count looks deceptively small.

0.4.5 is the one release probed out of order, and it is a cautionary case. It was announced upstream as carrying fixes for the four test-generator defect classes, so it was probed even though it is numerically behind the already-rejected 0.4.9. It is not a candidate: published before both 0.4.6 and 0.4.9, it predates the inherited-extends fix and still omits Connection.unknown, so native output is 30 errors in tool.swift. Probe-patching only that case makes the library compile and then yields 228 test-build error lines, 57 unique, across six files — against 0.4.9's 180 and 45 across five. The extra file is tools/ToolTests.swift, whose eight errors are all FunctionTool missing name and description; ReferenceConnection drops authenticationMode and usageDescription too, but silently, which is precisely why restoring test-dir matters. The four columns marked ✅ above are inherited from 0.4.3 and corroborated by the library compiling once Connection alone was patched. Rejected on Swift, so the other runtimes were not measured.

Because 0.4.5 is the first probed release to clear a majority of this shim, it was also classified patch by patch — mechanically, against native output, using the shim's own anchors. 13 of the 24 patch sites are fixed upstream and 11 are residual: the three Connection.unknown patches and all eight base-field injections. Applying only those 11 compiles the library, passes all 76 runtime tests including live E2E, and clears ToolTests entirely — the first direct evidence that the base-field injections are load-bearing rather than defensive, since defect 10 otherwise fails silently. 49 generated-test errors remain across five files, so test-dir stays out and the shim cannot be retired. Two measurement traps found here are now documented next to the code they affect: the parser.swift patch false-negatives against a fixed emitter because its replacement text is pre-wrapped to the formatter's width, and baseFieldInjections deliberately excludes Connection subtypes even though the generated tests assert them.

0.4.10 is the closest any release has come to making this shim deletable, and it is still rejected. Published after 0.4.9 and now latest, it fixes 23 of the 24 patch sites — every base-field injection, and the inherited fields land in the memberwise initializer too, which would settle the construction limitation recorded at the top of the shim. Connection gains its unknown case and save arm. The single residual is Connection.load's default:, which still throws unknownDiscriminator instead of returning .unknown, leaving that case declared but unreachable by the loader.

The 0.4.9 named-collection defect persists, though it is much reduced: item.name = name and Property.shorthandProperty are still emitted against the polymorphic enums, now costing 10 errors confined to agent/prompty.swift where 0.4.9 cost 48 across three files. Hand-synthesising the enum-level forwarders plus that one Connection line takes swift build to exit 0. That measures compilation only — the forwarders are themselves a workaround, and the generated test-dir is still unusable at 45 errors across five files — so deleting the shim needs both fixes upstream and a green generated-test run. C# and TypeScript were not re-measured, and every Swift-clean release so far has failed on exactly that gate.

One contract wrinkle worth recording: Tool now emits both customTool(CustomTool) and unknown([String: Any]), but Tool.load's default: still routes the wildcard to .customTool, so Prompty's * semantics survive and unknown tool kinds still arrive typed. .unknown is therefore unreachable through load while still forcing any exhaustive consumer switch without a catch-all to grow an arm — it broke this runtime's own boundParameterNames. Both fixes and that observation are specified upstream.

Compiling is not the bar for a type layer whose failure mode is silent, so the 0.4.10 probe was extended to load/save round-trips: a dropped field emits no diagnostic at all. Over three Property kinds, five Tool kinds including the wildcard, ReferenceConnection, and the Connection unknown fallback, 0.4.10-generated output scores 42/42 where this pinned configuration scores 40/42. The two-check delta is not a regression — it is the Connection scope gap the shim has always declined to patch, and the probe covered only ReferenceConnection of the six connection subtypes, so the 0.4.10 result is scoped to that type.

That gap was documented but never measured, which left nothing separating "deliberately unpatched" from "quietly broken". It is now pinned by a test walking all six Connection subtypes: every one silently loses authenticationMode and usageDescription between load and save. The scoping decision stands — injecting them restores data the runtime never reads and no vector covers, at the price of two more patches to delete on adoption — but it is recorded as an accepted limitation of the published model package rather than a free win. The test fails if either field starts surviving, which is the signal to re-audit and assert preservation instead. One further difference must be settled before adopting 0.4.10, not after: this shim writes name only when non-empty while 0.4.10 writes it unconditionally, and Prompty.save maps inputs/outputs/tools straight through without stripping name, so an unnamed composite can reach vector output as "name": "".
The 0.4.10 probe also invalidated the mechanical classifier used at 0.4.5: keyed on the shim's own anchors, it reported all eight base-field sites as still residual when the generated source proves them fixed, because the structural anchors had moved. Every verdict above was re-confirmed by reading generated source. That is now the documented rule, and the second known false-negative mechanism after the pre-wrapped replace text.

Adoption of 0.4.8 is blocked by C#, and the cause is now pinned down: schema/model/agent/agent.tsp:166 genuinely ends with a trailing space; the emitter writes 56 escaped expected-value literals that preserve it and 24 verbatim input-YAML literals that lose it. At 0.4.2 both forms dropped it consistently and the suite passed. The escaped form is arguably the faithful one — the defect is the asymmetry, not the escaping.

Three defects therefore still need shimming at the pinned 0.4.2:

  • Connection.unknown is never declared, yet the emitter writes .unknown([:]) as a stored-property default at six sites in its own tool.swift. The generated model package does not compile standalone. Property already emits exactly the right shape, so the fix is one enum case, a save arm and a load fallback. Rust tolerates unknown kinds via _ => ConnectionKind::default(), so accepting them is parity-correct.
  • Inherited extends fields are dropped from derived structs, so three Property subtypes and all five Tool subtypes silently lose their base fields on both load and save.
  • bindings only loads array form. The shared spec (spec/fixtures/tools_function.prompty) and the Rust reference (load_bindings) both accept map form, keyed by parameter name.

Worth recording: those defect classes fail very differently. Missing Connection.unknown is a hard compile failure. Missing inherited Property fields produce zero compile errors, because the runtime reads those through the raw dictionary — the data simply disappears. That asymmetry is why InheritedPropertyFieldTests exists and why the shim's retirement criterion is swift build --build-tests and swift test, never a clean build alone.

Cross-runtime effect of a bump

Every suite was measured against HEAD rather than assumed:

Runtime Effect of 0.4.3 Verdict
Swift four defects fixed, shim 419 → 345 lines ✅ gain
Rust byte-identical output, ~896 tests green ✅ neutral
Go unreachable default: clauses removed, so unknown discriminators fall through to the map load instead of returning an empty value 44 → 17 failures, fixes 27
C# generated-test string literals reshaped 16 *Yaml* tests break

The C# result is why compare-the-counts is not good enough, and this PR got it wrong once before CI caught it. On this Windows checkout, PromptyConversionTests already fails 16 tests at HEAD — but as LoadJsonInput*/RoundtripJson*, a CRLF artifact of the local checkout. Under 0.4.3 it also fails 16 — but as LoadYamlInput*/RoundtripYaml*. Identical totals, completely disjoint identities. A count comparison reports "unchanged"; the truth is that a pre-existing local failure masked a real regression. The 0.4.3 failures are content-based ("some \npersonal" vs "some\npersonal"), so they reproduce on Linux CI. Both releases and this trap are now documented next to PINNED_EMITTER_VERSION.

The shim, the omitted test-dir, and this section all come out once an emitter carries the Swift fixes without the C#/TypeScript regressions.

Scope

This port is not parity-complete, and the README says so explicitly rather than implying coverage that isn't there. Six of the ten shared vector files are exercised — load (25), render (23), parse (15), wire (22 of 27), process (17 of 21) and harness replay (5). The nine skipped wire and process cases are Anthropic ones: this package ships the OpenAI provider only, so both suites filter on input.provider and an Anthropic package would pick them up unchanged. Four files are not run at all:

Vector file Cases Missing capability
engine/turn_vectors.json 5 not wired; delegated provider state and cancellation absent
agent/agent_vectors.json 28 agent layer
discovery/discovery_vectors.json 7 model discovery
discovery/enrichment_vectors.json 9 discovery enrichment

These are a deliberate scoping decision for an initial port. The engine row is the nuanced one, and worth stating precisely rather than overclaiming the gap: ReferenceTurnRunner already implements the iteration loop, permission mediation, host tool execution and checkpointing, so three of the five engine vectors (final_output, ordered_tool_round, permission_denial_is_model_visible) describe behaviour that exists but is not yet asserted against the shared file. Only delegated_provider_state and cancel_before_context need capabilities this port lacks. Tool calling is covered at the wire and processing layers — request serialization and response extraction. Wiring the engine vectors and closing those two capabilities is follow-up work tracked separately from this PR.

Notes

.env is gitignored and was never committed — verified against the whole branch history. Outside runtime/swift, this branch touches .gitignore, the new workflow, schema/tspconfig.yaml, the generation scripts, and the Swift emitter shim. The emitter pin stays at 0.4.2 and no other runtime's generated output is modified — see Cross-runtime effect of a bump.

sethjuarez and others added 6 commits August 4, 2026 00:03
Add a Swift emit target so the canonical Prompty types are emitted from
schema/model/**/*.tsp like every other runtime, rather than hand-written.
The generated package lands at runtime/swift/prompty-model as a standalone
SwiftPM package because the emitter owns Package.swift.

test-dir is deliberately omitted from the emit target: the emitter's
generated Swift tests do not compile (297 errors across 4 further defects),
and driver.js treats testRoot as optional, so a library-only package is a
supported configuration.

@typra/emitter@0.4.2 has defects that make its Swift output either
uncompilable or lossy, so generation is followed by a scripted, idempotent
normalization pass. It is never a hand edit of generated files: the shim runs
as part of `npm run generate` and is designed to retire itself once the
emitter is fixed. Change requests for all of these have been sent upstream.

  1. Wildcard enum cases referenced but never declared, with no save branch
  2. Recursive enums not marked `indirect` (Property <- ArrayProperty.items)
  3. Convenience factories built from raw literals instead of enum values
  4. Unmapped placeholder type names in protocol signatures
  5. Array/optional suffixes dropped from protocol signatures
 10. `extends` base fields dropped from derived structs -- silent data loss
     across 3 Property subtypes and all 5 Tool subtypes

Defect 10 is the dangerous one, so it is patched structurally rather than by
literal find/replace: base declarations, load branches and save branches are
injected at known anchors in each derived struct. The shim pins the emitter
version, throws when the Swift output root is missing, and refuses to run
against partially-patched or ambiguous input, so a future emitter release
cannot silently produce half-patched code.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Introduces the `Prompty` module: loading, rendering, parsing, preparation,
and the invoker registry, built on the Typra-generated `PromptyModel`
package. The generated model is the only domain type layer -- this package
adds behavior, not a parallel set of types, and conforms to the pipeline
protocols the emitter already produces rather than declaring its own.

The Rust runtime is the behavioral reference. Loading resolves `${env:}`
and `${file:}` references, migrates legacy frontmatter, and injects the
prompt discriminator. Preparation stamps role markers with a nonce before
rendering and requires that nonce when parsing, so markers that appear only
after interpolation are rejected instead of silently becoming new turns.
Nonce substitution happens in exactly one place for the same reason.

Templates are evaluated by a real tokenizer and precedence-climbing parser
rather than shape matching, so an expression the runtime does not support
is reported instead of quietly resolving to nothing.

Conformance runs against the shared spec vectors -- 25 load, 23 render,
and 15 parse -- plus round-trip guards over every base field the Swift
generation shim injects, and regression tests covering defects that the
vectors do not reach.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adds `PromptyOpenAI`: the executor that turns a prepared conversation into
an OpenAI request, and the processor that turns the response back into a
result. Connection details, structured output, and tool declarations come
from the generated model, so the provider translates between that model and
the wire rather than defining shapes of its own.

Providers register themselves. Importing the module and calling
`registerOpenAI()` installs the executor and processor, which keeps the core
runtime free of provider dependencies.

Both directions are checked against the shared spec vectors:
`spec/vectors/wire` for the request shape and `spec/vectors/process` for
response handling, including structured output and streamed responses.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adds the reference harness adapters and a turn runner that drives the loop
the durability contract describes: ask the model, checkpoint, resolve
permission, execute the tool, report the result, and go again until the
model answers or the iteration budget runs out.

Every event is handed to the sink before it is journaled, so no observer
can see an event that was never recorded. The clock and id factory are
injected, which makes a run byte-reproducible and lets replay compare
against a recorded journal rather than a re-derived one.

The journal writer appends and flushes per record, so a run interrupted
mid-flight still leaves a readable prefix -- covered by a test that reads
the file while the run is in progress. Tool failures are contained and
reported as results rather than propagated, since a failing tool is an
outcome the model has to see, not a crash.

Verified against `spec/vectors/harness` across all five replay scenarios,
with adapter-level tests for crash durability, checkpoint ordering and
session scoping, permission decisions, tool failure containment, and
verifier drift detection.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Swift treats a CRLF pair as a single Character, so the obvious line
splits silently do nothing: for "a\r\nb" both components(separatedBy:)
and split(separator:) return the whole string as one element, and
text.contains("\r") is false. A prompt assembled in memory with Windows
endings therefore missed every role marker and collapsed into one lumped
message, a CRLF journal replayed as zero records, and the buffered SSE
path yielded a single unparseable line.

Rust splits at the byte level and degrades gracefully, so this hazard has
no counterpart in the reference runtime and no LF-only spec vector can
catch it.

Add Lines, a scalar-based splitter, and route the parser, the JSONL
journal reader and the SSE fallback through it. Scanning unicodeScalars
rather than Characters is what makes this correct: normalizing first and
splitting on characters afterwards still fails, because rewriting
"a\r\r\nb" yields "a\r\nb" whose CR and LF are now adjacent and cluster
into one grapheme again.

Character.isNewline is deliberately not used. It also matches U+0085,
U+2028 and U+2029, which are legal unescaped inside JSON strings, so
splitting on them tore journal records in half and discarded them
silently. Only CR, LF and CRLF terminate a line. Normalization rewrites
CRLF and nothing else, matching the Rust loader, so a lone CR stays as
content.

Also add seven end-to-end tests against the real OpenAI API covering
chat, token caps, streaming, streamed accumulation, structured output,
tool calling and loading a .prompty file with ${env:} references. They
skip when OPENAI_API_KEY is absent so a keyless CI stays green. The
assertions avoid vacuous passes: the token cap is a relative two-call
comparison, structured output is proven by a prompt that demands prose
and forbids JSON, and tool use is proven by an opaque run-time station
code the model cannot guess.

Every fix here was mutation-verified by reverting it and confirming the
predicted failure.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Cover the two-package layout and why it is split, the rule that nothing
under prompty-model/Sources is ever hand-edited, the Windows git escape
hatch SwiftPM needs, how to run the live tests, and the emitter shim —
including that it is pinned, fails loudly, and is meant to shrink to
nothing as upstream fixes land.

The snippets are type-checked by ReadmeSnippetTests rather than trusted:
an earlier draft documented a Prompty.load(path:) entry point that does
not exist. Documented calls are API surface and should break the build
when they drift.

State plainly that spec/vectors/engine/turn_vectors.json describes a turn
engine this port does not implement, so those vectors are not run.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings August 4, 2026 08:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

ArrayProperty, ObjectProperty and UnionProperty each extend Property, so
they inherit description, required, nullable, default, example and
enumValues, plus name via the Named<Property> alias. The Typra emitter
drops these fields from derived Swift structs and the loss is silent: the
code compiles and the values simply disappear. The build-time shim
restores them, but the shared spec vectors only exercise inherited fields
on scalar properties, so a regeneration that reintroduced the defect on
the composite subtypes would go unnoticed along several axes.

Add an acceptance gate covering the axes the existing round-trip suite
does not reach: JSON and YAML text round trips, values constructed in
Swift rather than parsed, repeated round trips reaching a fixed point,
non-scalar values in default/example/enumValues, falsy values that the
save() guards could quietly discard, and inherited fields on nested
object children, union branches and three-level composites.

The assertions target individual inherited fields rather than comparing
whole dictionaries, because a corrected emitter may legitimately begin
materializing schema defaults such as required: false, which a strict
key-count comparison would reject for reasons unrelated to this defect.
These tests assert the required behaviour rather than the shim, so they
are expected to stay green once the emitter is fixed.

Verified by mutation: with the shim's Property injection disabled the
library still builds with zero errors, confirming the defect is silent,
while the construction test fails to compile and the nine runtime tests
report 268 assertion failures spread across every inherited field.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 4, 2026 08:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

sethjuarez and others added 3 commits August 4, 2026 02:20
The 135 generated PromptyModel files already pass `swift-format lint
--strict` with zero findings, so swift-format defaults are the house
Swift style in this repo. The hand-written runtime did not conform.

Applied `swift-format format --in-place` across Sources and Tests, then
hand-fixed one multi-line string literal in LiveOpenAITests that the
formatter reindented only at its closing delimiter -- Swift strips the
closing delimiter's indentation from every line, so the body had to be
indented uniformly to match. The literal's string content is unchanged,
which testLivePromptFile confirms by still round-tripping the same
.prompty document through a live call.

No generated model file is touched.

Verified: swift-format lint --strict => 0 findings; swift build
--build-tests => 0 errors, 0 warnings; swift test => 64 tests,
0 failures, including all 7 live OpenAI E2E tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
runtime/swift carries a temporary post-processing shim that repairs
known Swift emitter defects, and that shim asserts every one of its
find/replace patches still applies. That is the right default -- it
fails loudly instead of rotting silently -- but it also means a
candidate emitter build cannot be measured at all, because generation
aborts as soon as the candidate stops matching the patches.

This flag emits raw, unpatched Swift so a candidate can be evaluated
against the defects the shim compensates for. It was used to validate
two emitter candidates and to produce the defect reports filed upstream.

The comment records the retirement criterion. A clean `swift build` is
deliberately not sufficient: dropped inherited fields on Property
subtypes are read back through save(), so they vanish at runtime with
no compile error at all. `swift test` and InheritedPropertyFieldTests
are the gate for that.

Verified: with the flag unset, `npm run generate` reproduces
runtime/swift byte-identically to HEAD.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Every other runtime has a prompty-<lang>-check.yml; Swift had none, so
nothing in CI compiled or ran the new runtime.

Matrix is ubuntu-latest and macos-latest. Only Linux installs a
toolchain -- macOS runners ship Swift with Xcode. windows-latest is
deliberately omitted: swift-actions/setup-swift maps Windows releases
only through 5.6.3, and this needs Swift 6.x for the `swift format`
subcommand. Windows is covered by local development for now.

`spec/**` is in the trigger paths because the Swift tests are validated
against the shared cross-runtime vectors under spec/vectors, so a spec
change has to re-run them. The other runtimes' check workflows watch
only their own runtime directory and have the same blind spot; fixing
those is out of scope here.

The SwiftPM cache keys on a stamp of `swift --version` as well as the
manifests, so a runner-image or toolchain change can never restore
artifacts produced by a different compiler. Package.resolved is
untracked (a library must not pin its consumers), so it cannot
contribute to the key.

Only the hand-written runtime is format-checked. The generated
PromptyModel sources pass swift-format today, but they are the emitter's
output and an upstream formatting change must not break CI.

swift test is hermetic here: LiveOpenAITests calls XCTSkipIf when
OPENAI_API_KEY is unset.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 4, 2026 09:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

`Frontmatter.split` located its delimiters with `firstIndex(of: "\n")`. Swift
clusters a CRLF pair into a single `Character` that is not equal to `"\n"`, so
in a Windows-authored document the search found nothing, the opening `---`
looked unterminated, and the function returned empty frontmatter and an empty
body. No error was raised.

`Loader` normalized before calling in, so every existing test and every shared
vector passed and the defect was invisible; only direct callers of the public
API lost their document.

Rewrite `split` onto `Lines.splitLineFeeds`. Normalizing and then splitting on
characters is not sufficient -- as `Lines` documents, one pass over `a\r\r\nb`
yields `a\r\nb`, whose CR and LF re-cluster into a single grapheme. Only a
unicode-scalar scanner is safe.

That makes `split` the single normalization pass in the load path, so `Loader`
no longer normalizes first: two passes would read the residual CR + LF of
`\r\r\n` as a terminator and delete a lone CR that `Lines` and the Rust
reference both preserve.

Blank-line skipping uses `allSatisfy(isWhitespace)` rather than
`CharacterSet.whitespaces`, which is horizontal-only and would have demoted a
document opening with a vertical tab, form feed, NEL, or a Unicode separator to
body-only. Equivalence with the replaced algorithm was verified by a temporary
differential harness fuzzing both implementations over a 54-sample corpus,
including a negative control confirming it detects that divergence.

Also widen the SSE payload trim from `.whitespaces` to
`.whitespacesAndNewlines`; the narrow set is space and tab only, so a surviving
CR left `"[DONE]\r"` and defeated the sentinel comparison.

Add 12 line-ending tests covering the regression, CRLF/LF equivalence, .prompty
files with CRLF bytes on disk, lone-CR survival end to end, vertical whitespace
before the delimiter, and CR-tolerant SSE framing.

Correct the README conformance section: report the partial OpenAI-only coverage
of the wire and process vectors, and stop claiming the port has no turn engine --
`ReferenceTurnRunner` implements the loop, permission mediation, and
checkpointing. The genuine gaps are delegated provider state and cancellation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 4, 2026 11:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

0.4.3 ships the Swift source-generator fixes this repo requested. Determine
per-defect what actually landed by generating unpatched Swift with both 0.4.2
and 0.4.3 (PROMPTY_SKIP_SWIFT_SHIM=1) and diffing the two, rather than
inferring from release notes: 10 files, 18 insertions, 16 deletions.

Fixed upstream, so the corresponding 13 patch entries are deleted:
  - Tool wildcard `case customTool(CustomTool)` plus load/save arms
  - `indirect` on recursive polymorphic enums
  - typed factory cases (`role: .assistant`, `.textPart(TextPart(...))`)
  - placeholder types (`Unknown` -> `Any?`, `RecordUnknown` -> `[String: Any]`)
  - array/optional suffix loss in protocol signatures

Still broken at 0.4.3, so those patches are retained and loudly asserted:
  - `Connection` has no `unknown` case although tool.swift defaults six fields
    to `.unknown([:])` (hard compile error)
  - fields inherited via `extends` are dropped from derived structs (silent
    data loss, zero compile errors -- so a clean build cannot be the retirement
    criterion for this shim)
  - map-form `bindings` is never emitted

Net effect: the shim drops from 419 to 345 lines while producing generated
Swift that is byte-identical to the 0.4.2 output except two cosmetic lines.

Also harden the shim against emitter drift, since its whole value is being
deterministic. Patch application now requires exactly one anchor: a duplicated
patched form, or a patched form sitting alongside a stray raw anchor, both
fail instead of silently producing a half-patched file. Single-site replacement
via indexOf/slice replaces split/join. Missing-manifest version assertion now
throws rather than returning silently.

The generated Swift test directory stays disabled. Measured, not assumed:
enabling test-dir emits 125 files and 1184 compile errors in four classes --
unwrapped optionals, struct-style member access on polymorphic enums, compound
values compared to their scalar shorthand, and references to helpers that are
never emitted (`FixtureRoot`). These are test-generator defects, disjoint from
the source-generator defects 0.4.3 fixed; the tspconfig comment now records
that instead of the stale 0.4.2 note.

Cross-runtime effect of the bump, all measured against HEAD rather than
assumed: Go improves from 44 failures to 17, C# is unchanged at 17/1089,
Rust stays fully green. Swift is 76 passed / 0 failed both with live
credentials and without (7 E2E skipped), build and strict-lint clean, and
regeneration is deterministic.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 4, 2026 11:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

Reverts the generated-output and pin changes from 66b60ec. CI is the
authority here: the C# suite was green at 2a724c0 and fails on all three
platforms at 66b60ec, with 16 `LoadYamlInput*`/`RoundtripYaml*` tests in
Prompty.Core.Tests newly broken by the reshaped generated-test string
literals. That is the same regression that already disqualified 0.4.6.

The local pre-merge measurement missed it, and the reason is worth writing
down. Locally the same 16-test class fails at HEAD too -- but as
`LoadJsonInput*`/`RoundtripJson*`, a Windows CRLF artifact of this checkout.
Counts are 16 before and 16 after, so a count comparison reports "unchanged";
the failing identities are in fact completely disjoint. This is precisely the
trap recorded for 0.4.6, and it was walked into anyway by comparing totals
instead of names. The pin note now says so explicitly.

0.4.3 remains a genuine Swift improvement -- it fixes the Tool wildcard,
`indirect`, typed factories, placeholder types and suffix loss, and would cut
this shim from 419 to 345 lines. But the emitter is shared, so a bump
regenerates every runtime, and Swift gains cannot be bought with a C# break.
Both evaluated releases and their exact costs are now recorded next to
PINNED_EMITTER_VERSION so the next reader does not repeat the probe.

Kept from the reverted commit, because none of it depends on the emitter
version:

- Patch application is now fail-closed on ambiguity. It previously used
  `includes()` plus `split/join`, which would silently accept a file that was
  half patched, or rewrite several sites when the anchor was not unique. It
  now requires exactly one occurrence of either the raw anchor or the patched
  form, and rejects a patched form sitting next to a stray raw anchor. Both
  failure modes are exercised directly; all 16 patches satisfy the invariant.
- The missing-manifest branch of the version assertion throws instead of
  returning silently, so a shim run without `npm install` can no longer patch
  against an unverified emitter.
- The omitted Swift `test-dir` is documented from measurement rather than
  memory: enabling it emits 125 files and 1184 compile errors in four classes
  -- unwrapped optionals, struct-style member access on polymorphic enums,
  compound values compared to their scalar shorthand, and references to
  helpers that are never emitted. All four are test-generator defects, still
  present at 0.4.3 and disjoint from the source-generator defects it fixes, so
  those fixes could not have made these compile.

Generated output is byte-identical to 2a724c0. Swift: build clean, strict
lint clean, 76 passed / 0 failed with live credentials.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 4, 2026 12:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

Defect 10 in the shim header listed `name` among the fields that
`ArrayProperty` / `ObjectProperty` / `UnionProperty` and the `Tool`
subtypes lose to the emitter's dropped `extends` inheritance. That is
inaccurate: neither `model Property` nor `model Tool` declares `name`.
It arrives through the `Named<...>` spread in
`schema/model/core/core.tsp`, which the emitter drops by a separate
mechanism.

The distinction is not cosmetic. It decides part of the shim retirement
criterion this PR is blocked on: an upstream fix to `extends` inheritance
restores every other field in that list but leaves `name` missing, so
removing the `name` injection on the strength of such a fix would
reintroduce silent data loss with no compile error to catch it. The header
now says so explicitly and directs the reader to confirm against
regenerated output.

Comment-only; generated output is unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 4, 2026 12:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

Probed the current registry frontier with the shim disabled, so every
claim below is a measurement of native emitter output rather than a
reading of release notes.

Swift source generation is close to clean at 0.4.8. Native output
compiles to exactly 30 errors, all `Connection has no member 'unknown'`,
all in tool.swift; probe-patching only that enum case makes the model
package compile. Inherited `extends` fields are fixed too, including
`name` -- which matters, because `name` arrives via the `Named<...>`
spread rather than `extends`, so a narrower upstream fix would have
restored every other base field and silently dropped that one.

That is still not grounds to retire a patch on a clean build alone.
Dropped fields produce no diagnostics, so the retirement criterion in
defect 10 stands: `swift build --build-tests` and `swift test`.

0.4.8 is rejected all the same. The C# `*Yaml*` break reproduces, and
the cause is now identified: agent.tsp:166 genuinely ends in a trailing
space, which the 56 escaped expected-value literals preserve and the 24
verbatim input-YAML literals drop. At 0.4.2 both forms dropped it
consistently, so the suite passed.

Also re-measures the generated test-dir, which the library must compile
before it is observable at all: 180 errors, down from 1184 at 0.4.3, in
four test-generator classes that are now described from the diagnostics
they actually produce.

Comments only -- generation output is byte-identical.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 4, 2026 12:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

The shim header doubles as the upstream defect report, and it bundled two
unrelated problems into a single 'Defect 1'. That conflation had a real cost:
successive emitter candidates fixed the Tool wildcard, were reported as
fixing Defect 1, and the Connection build errors persisted unchanged.

Split into two entries with distinct owners:

  1a (emitter) Tool is missing a customTool case even though tool.tsp
      declares CustomTool { kind: '*' }.
  1b (emitter) The emitter synthesises Connection = .unknown([:]) defaults
      for six required fields in tools/tool.swift while never declaring the
      case, so raw output does not compile on its own terms. Verified by
      deleting only the injected case declaration and rebuilding: 16 errors,
      all
"
type 'Connection' has no member 'unknown'
"
, at
      tool.swift:145/152/232/242/344/351 plus the two injected arms.
  Schema gap  connection.tsp closes the union over six kind literals, so
      throwing on an unrecognised discriminator is correct emitter output.
      Overriding it is a deliberate choice, not a defect report.

Three claims were corrected after review rather than shipped:

  - The header cited spec.md 2.5 as requiring lossless unknown-kind
    round-trips. It does not; 2.5 only tabulates the six known kinds on this
    branch. The override is now described as extending the 2.3 unknown-
    property rule pending a 2.5 amendment, matching what
    ConnectionRoundTripTests already documents.
  - It claimed fixing 1b retires none of the three patches. This file's own
    0.4.10 log records that release emitting the declaration and save arm
    natively, which retires two. Retirement now depends on the shipped fix.
  - It cited Rust as precedent for lossless round-trips. Rust maps unknown
    kinds to ConnectionKind::default() (connection.rs:258) and kind_str has
    only six arms (274-283), so it rewrites the discriminator and drops the
    payload. It is precedent for not throwing only; Swift is stronger, and
    that divergence is unresolved rather than settled parity.

The exit condition no longer says 'regenerate and delete'. Defect 1a proves a
declared wildcard does not guarantee correct projection, so it now requires
regenerating with the patches removed and measuring that raw output builds,
preserves an unknown kind's discriminator and payload, and passes the suite.

Regenerated at the pinned 0.4.2 so the emitted comment matches the script.
Swift regeneration is otherwise byte-identical. 148 tests, 3 skipped,
0 failures.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 5, 2026 00:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

An open schema question is how to open the closed
"
Connection
"
 union: add a
typed wildcard subtype (as
"
CustomTool { kind:
"
*
"
 }
"
 does for
"
Tool
"
), or open
the discriminator so the raw payload survives. The two are routinely treated
as interchangeable. They are not, and nothing measured that until now.

Adds WildcardPreservationTests (4 tests) covering:

  - the typed subtype drops unknown top-level fields, because CustomTool
    declares only named fields and no raw catch-all;
  - raw passthrough preserves every field, unrecognised discriminator
    included;
  - the two disagree on identical unknown fields, pinned as one assertion so
    the divergence cannot close in either direction unnoticed;
  - a missing *required* connection is accepted without diagnostic and saved
    as an empty, kind-less connection.

That last one sharpens emitter defect 1b. It was reported as a compile break
the shim absorbs; it also produces invalid output. CustomTool.connection is
required (tool.tsp:98), but the loader assigns it only when present
(tool.swift:167-169), so the synthesised .unknown([:]) placeholder survives
and save writes it unconditionally (tool.swift:198). Rust guards the same
write with !connection.is_null() (tool.rs:327-329) and omits the key, so this
is a Swift-specific divergence rather than agreed behaviour.

Scoped deliberately narrowly. What is shown is that a wildcard subtype
declaring only named fields loses unknown ones -- corroborated cross-runtime,
since Rust's ToolKind::Custom captures only connection/options/kind_name
(tool.rs:175-185). It does not follow that any conceivable Connection
wildcard must, as one could declare catch-all storage. And dropping unknown
properties is not a spec violation: spec.md 2.3 permits them to be preserved
or ignored. The transferable point is that the choice is not neutral and the
declared shape decides it.

Header comments in the shim are updated to match, replacing an earlier
categorical claim that mirroring CustomTool onto Connection would be a
regression. Comment-only; generated output verified byte-identical.

All 6 new assertions mutation-proved. Suite 152 tests, 3 skipped, 0 failures.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 5, 2026 01:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

parseAttributes coerced every attribute value to Int or Double before
falling back to String. The strict-mode nonce is 8 random bytes rendered
as 16 hex characters, and hex is a superset of decimal, so the nonce was
sometimes numeric-looking and did not survive the round trip:

  9677e80871924237  -> inf                  (exponent overflows Double)
  0419856025378190  -> 419856025378190      (leading zero lost to Int)
  9789350921772800  -> 9.7893509217728e+15  (normalized on stringify)

buildMessage then compared the corrupted value against the nonce that
produced it, so the parser rejected its own untampered output with
"possible prompt injection detected". Measured 165 failures in 200,000
trials, about 1 in 1,200.

This surfaced as an intermittent failure of testLiveToolCalling, which
has no template inputs at all -- the nonce was the only thing varying
between runs, and the throw happened in about a millisecond, before any
network call. CI never saw it because live tests need credentials CI
does not have.

Exempt nonce from coercion. It is already reserved: it is generated by
the parser, consumed by the parser, and stripped from metadata rather
than surfaced to callers, so coercing it was never meaningful. Coercion
is preserved for every other attribute, since attributes such as
[index=1] are documented to arrive typed.

Rust hit the same problem and worked around it at the comparison site
(parsers/prompty.rs:174-182, "parse_attrs may coerce all-digit hex nonces
to Number"), mapping Number back through to_string. That workaround is
incomplete: parse_attrs strips leading zeros, so a nonce like
0123456789012345 still fails to compare equal. Fixing the root cause is
strictly stronger and behaves identically wherever Rust behaves
correctly, so this diverges from the reference only where the reference
is broken. Reported upstream separately.

The new tests use crafted numeric-looking nonces, one per corruption
mode, so a regression fails deterministically rather than once in 1,200
runs, plus a 20,000-trial sweep of real generated nonces to catch modes
nobody anticipated. Both directions are mutation-proved: removing the
exemption fails 6 of 7 tests, and "fixing" it by dropping coercion
wholesale fails the coercion-preservation test.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 5, 2026 01:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

The named-collection vector suite compares with deliberate subset
semantics, which is correct: fields the runtime adds beyond the fixture
are tolerated while the canonical file is still being revised upstream.

The gap was one step earlier. `canonicalizeExpected` rewrites a fixture's
nested name-keyed `properties` map into the ordered list the wire uses so
the semantic assertions can run at all. That same adaptation erases the
shape difference, so a nested collection saved as the array fallback
compared equal to a fixture asking for the canonical object form. The
top-level fallback was already recorded, and NamedCollectionSaveFormTests
covers duplicate and empty names, but nothing saw a nested one.

Walk the fixture and the saved wire in parallel and record nested
object-form gaps into the existing `blocked` baseline, which is tied to
the emitter pin, so bumping the pin without closing the gap fails rather
than resting on stale prose. Only the fixture's own structure drives it:
a nested map is read as a request for the object form, a nested list as a
request for the array form, and nothing is inferred about collections the
fixture does not mention.

Verified end-to-end by authoring the absent fixture locally: the gap
reached `blocked` and the pin check named the exact path. The fixture was
then removed, so the vector test still skips on absence.

Review follow-ups folded in:

- Pairing children by name alone dropped a saved child that omits `name`
  and reported nothing beneath it. That unnamed composite is precisely
  the canonical case, so pair positionally when both sides are ordered
  lists, and fall back to position when a name lookup fails.
- Every test drove the helper directly, so deleting the call in
  runLoadSaveReload left them all green. Added a test that drives the
  real vector path and asserts the gap arrives in `blocked`.
- Renamed the justification test: `compare` on its own would see the
  shape mismatch. It is the canonicalised pipeline that goes blind, and
  the name now says so.
- Pinned the alphabetical ordering of nested named collections, which
  both the canonicalisation and the positional fallback rely on and
  neither stated.

169 executed, 3 skipped, 0 failures, 7 live OpenAI E2E. Mutating away the
wiring and the unnamed-child fallback each kill exactly one test.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 5, 2026 01:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

The spec coordinator ruled that object-form named collections are keyed
rather than order-bearing, and that only the array fallback carries order.
Post-save positional lookup is therefore invalid for object-form
collections.

`nestedObjectFormGaps` descended by name but fell back to indexing the
saved array whenever a key missed, which is exactly that invalid lookup.
It had two concrete failure modes:

  - misattribution: a differently-named child bound to the wrong key, so
    that child's nested gap was reported against a key it does not belong
    to;
  - silent loss: when a named child resolved to a later index, the earlier
    nameless child was never reached and its gap disappeared.

Pair by elimination instead. Names address every child they can; the one
child a key cannot address is an entry that omits `name` in the array
fallback, and that entry is matched only when a single unmatched key faces
a single nameless child. Anything more ambiguous is left unpaired rather
than guessed, since attributing a gap to an unproven key is the same
misattribution keyed pairing exists to prevent. Nameless now means the
field is absent, so a malformed `name` is treated as a mismatch.

Both guards are load-bearing, and both are caught by the real canonical
vector test rather than only by the synthetic ones:

  - disabling elimination fails testEliminationPairingIgnoresPosition,
    testNestedObjectFormGapIsFoundWhenTheSavedChildOmitsItsName, and
    testCanonicalNamedCollectionVector;
  - removing the ambiguity guard fails
    testAmbiguousNamelessChildrenAreNotGuessed and
    testCanonicalNamedCollectionVector.

Also corrects comments that justified the positional fallback by asserting
both sides order collections alphabetically. The alphabetical-order test
keeps earning its place, but for the array save form, which does carry
order, and because canonicalizeExpected sorts a keyed expectation into a
list that compare then pairs positionally.

172 executed / 3 skipped / 0 failures, including 7/7 live OpenAI E2E.
Emitter pin 0.4.2 unchanged; no generated files touched.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 5, 2026 02:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

sethjuarez and others added 2 commits August 4, 2026 19:59
Canonical rule: an absent optional collection must stay omitted through
save, an explicitly present empty collection may save as empty, and save
must never synthesize an empty collection from absent input.

Adds OptionalCollectionPresenceTests covering both halves of the rule and
the distinction between them, over all three composite Property kinds
(array/object/union) plus the document-level `tools` collection. Swift
conforms at emitter pin 0.4.2: all five gates pass unmodified.

Each absence assertion is paired with a positive control, because an
empty save output or a silent fall-through to the `.unknown` passthrough
- which echoes its source dictionary verbatim - would otherwise satisfy
every "must be nil" assertion while exercising no composite code at all.
The controls assert the loaded enum case, not just the saved shape, since
only the case distinguishes a real composite from the passthrough.

Also removes the `isMaterializedEmptyDefault` exemption from
InheritedPropertyFieldTests. It tolerated an empty array standing in for
an absent value on the rationale that "a corrected emitter materializes
the schema default for enumValues" - which this rule inverts: doing so is
itself the defect. The new gates prove the pinned emitter does not
synthesize, so the exemption was dead code, but it would have silently
accepted a future regression.

Full suite green.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Canonical rule 2 is shared, not a Java special-case: a duplicate-name save
must pre-scan exact uniqueness and fall back to the whole array *before*
any map construction. Applying it to the Swift comparators found two
places that built the map first and let a later entry overwrite an
earlier one - the same silent-loss class as the positional pairing bug,
arriving through map collapse instead of indexing.

NamedCollectionVectorTests.namedChildren now pre-scans and returns an
optional, where nil means "not soundly name-addressable". Refusal is nil
rather than an empty index because the two mean opposite things to the
caller: an empty index says the collection has no named children, which
leaves every expected key unmatched and therefore eligible for the
nameless-child elimination pairing. A duplicate would then bind an
unrelated child to the very key it was hiding - "no proven owner" decays
into "sole remaining candidate". Refusal now suppresses lookup and
elimination together.

A present-but-non-string `name` refuses the index for the same reason: it
falls out of the uniqueness scan while also not counting as nameless, so
it would vanish from both pairing paths without trace.

The expected-list against actual-object path now requires unique expected
names too. Uniqueness on the actual side alone still lets several expected
entries be attributed to one actual child, reporting a nested gap once per
duplicate against entries never separately observed.

LoadVectorTests.bindingPairs throws instead of collapsing duplicates. Its
sole consumer pins exactly one binding and its [String: String] shape
cannot represent duplicates, so collapsing would shrink a spec-vector
expectation and keep the vector passing while checking fewer bindings than
it declares. This is correct for that pin only - under rule 2 duplicate
entries in the array fallback are legitimate ordered entries, so it is not
a general binding-list interpretation.

Three tests cover the refusal modes. All five guards are mutation-proved:
each mutation compiles cleanly and kills exactly one test, including
`return nil` -> `return [:]`, which is what makes the sentinel choice
load-bearing rather than cosmetic.

Full suite 181 executed / 3 skipped / 0 failures; live OpenAI E2E 7/7.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 5, 2026 03:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

The Swift CI job runs `swift format lint --strict`, where warnings are
errors. Two spots in the new tests tripped it: a dictionary literal over
the line limit and an XCTAssertEqual whose first argument was not on its
own line.

Reproducing this locally needs care - a Windows checkout carries CRLF, and
the trailing CR is itself counted as trailing whitespace, so a local run
reports failures in untouched files that CI never sees. Both remaining
local hits are that artifact; only these two were real.

No behavioural change. Full suite 181 executed / 3 skipped / 0 failures.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 5, 2026 03:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

sethjuarez and others added 2 commits August 4, 2026 20:26
…ections

The coordinator ratified the save-form gate design in 393d274 and fixed the
canonical semantics: where the ordered-array fallback is mandatory the
serialized form must be asserted directly, name-payload association must be
checked with composite entries, and neither form nor order may be asserted for
collections that qualify for the object form.

Reviewing the file against that ruling surfaced three gaps.

The property tests read payload only from a *second* save, because a scalar
Property resolves to the .unknown passthrough so the saved dictionary is the
only place the payload is legible. That makes them blind to any involutive
defect: a serializer that reversed entries would reverse them once, reload in
that order, then reverse them back on the re-save, cancelling out and leaving
the assertion green while the first serialized output violated canonical
order. Both now assert the first save as well, keeping the re-save assertion
for reload-side corruption that the first cannot see.

The empty-name disqualifier was proven only on tools. inputs serializes
through a different path and can fail independently, and this is the
disqualifier least likely to be caught by accident: "" is a legal JSON object
key, so an illegal object encoding round-trips cleanly and satisfies every
payload assertion. Only the form check rejects it.

The duplicate-name inputs comparison used a sorted multiset. Both entries are
named dup, so exchanging their defaults leaves the sorted multiset unchanged
and passed; the association is observable only positionally. Asserting order
is legal here because the array form is required for this fixture and the
array fallback is the one representation canonical promises an order for.

Mutation-proved: reordering either expected array kills exactly its own test.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The optional-collection presence rule was gated at document level only for
tools. inputs and outputs serialize through different paths and can fail
independently, so an implementation can synthesize one while correctly
omitting another.

Verified non-vacuous rather than assumed: populating the fixture and inverting
both assertions passes, which proves both key names are real and do appear in
the saved output when present. Without that check a misspelled key would
satisfy XCTAssertNil forever and gate nothing.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 5, 2026 03:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

… alone

validateBindings resolved every expected binding with actual.first(where:),
which is unsound once names repeat. Two opposite defects followed: entries
sharing a name and an input satisfied both expectations from the first entry,
leaving the rest unverified; and a correctly ordered duplicate pair was
falsely rejected, because the second expectation also matched the first entry.

Object form cannot carry one key twice, and an empty key disqualifies it as
well, so either one proves the source used the array fallback - the only
ordered representation. Those entries are now compared positionally, by name
and input at each index. Collections that qualify for object form stay
name-addressed and order-agnostic, since both forms are legal for them and
asserting order there would reject a conforming loader.

No current vector declares a duplicate or empty binding name, so none of this
is reachable through testLoadVectors; the new tests drive validateBindings
directly to pin the rule before PR #447 lands vectors that rely on it.

Mutation-proved: dropping the duplicate pre-scan, the empty-name disqualifier,
the positional name comparison, or the map empty-key guard each kills exactly
the test covering it, and forcing every collection positional kills the
order-agnostic guard.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 5, 2026 03:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants