fix(pii): redact third-party and document PII in remarks

.gitignore

@@ -32,3 +32,6 @@ relay.json
 
 # Raw (unsanitized) supplier payloads — never commit
 /sabre/
+/amadeus_response.xml
+/sabre_response.xml
+/*_response.xml

openspec/changes/close-remark-free-text-pii-gap/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-08-12

openspec/changes/close-remark-free-text-pii-gap/proposal.md

@@ -0,0 +1,71 @@
+## Why
+
+A customer reported PII surviving anonymization, "mainly in remarks". Two real captured responses
+(Amadeus `PNR_Reply`, Sabre `GetReservationRS`) were replayed through the shipped baked ruleset with
+the real engine. The passenger's own identity redacts correctly — structured name, phone, email,
+DOCS/APIS entries, frequent-flyer number and card all scrubbed. Two specific classes still leak:
+
+- **PII that exists only in free text.** The phase-2 reference pass can only scrub values that a
+  phase-1 field rule already collected, so any value with no structured counterpart is invisible to
+  it. This is exactly the third-party population: emergency contact (`EMER-<name>`, `EMER-<phone>`),
+  authority-to-charge (`TPREF-AUTH-AUTHORITY TO CHARGE-<name>`), travel arranger (`TRAVEL ARRANGER/…`,
+  `*84-…`, the `CTC` phone node, `ReceivedFrom/Name`, `FutureTicketing/Comment`) and the Amadeus
+  orderer (`ACEORB-…`, `ACECRM-ORDERER-…`). The observed proof is a single node reading
+  `EMER-YAARA <ENC_…>`: the passenger surname beside it *was* scrubbed by the reference rule, while
+  the emergency contact's given name survived because nothing had collected it.
+- **Sabre history mirrors for one-way types.** Every coded passenger remark is re-emitted three times
+  — `RemarkLine/Text`, `HistoryAssociationElement`, and `AssociationChild`/`AssociationParent/Content`
+  (with the remark code turned into a `P!`/`V!`/`E!` prefix). `ReferenceRule` accepts only
+  `EncryptAction`, so date-of-birth, passport number, nationality and card data — whose policy is
+  one-way replacement — cannot be propagated to those mirrors by the reference pass at all. Confirmed
+  leaks: `BIRTHDATE-07AUG70`, `PSPT-IL <passport> …`, `PSPT/CO-IL/NR-<passport>/IS-…/EX-…/CI-IL`,
+  `CTZN-IL`, and a masked PAN `*CA5XXXXXXXXXXX8630/1127` appearing 20 times.
+
+Two further gaps are structural rather than free-text: no rule covers the Amadeus `FP` (form of
+payment) or `OS` (other service information) `dataElementsIndiv` at all, so a card number sits in
+cleartext in `PAX CCDCXXXXXXXXXX7298/0629`; and person-linked pseudonymous identifiers (employee id,
+traveller profile id, traveller-attached loyalty/contract numbers) are unclassified today and survive
+in both channels.
+
+## What Changes
+
+- **Scope decision: traveller-adjacent third parties are data subjects.** Emergency contacts,
+  authority-to-charge holders, travel arrangers and the Amadeus orderer SHALL be redacted like the
+  passenger. **Agency-agent personal names and sign-in codes remain operational** and SHALL be
+  preserved — this keeps the existing "agent sign-in codes are not PII" stance and keeps remark
+  audit trails legible for the customer.
+- **Scope decision: person-linked pseudonymous identifiers are PII.** Employee ids, traveller profile
+  ids, and loyalty/contract numbers attached to a traveller SHALL be encrypted (reversible — clients
+  legitimately correlate them). Organisation-wide commercial codes (corporate account ids such as
+  `OIN CHEVRON`/`CMP …`, tour codes, discount codes) are NOT personal data and SHALL be preserved,
+  since redacting them breaks fare reissue and policy display.
+- **Identity-document data found in free text gets format-preserving sentinels** (`01JAN00`,
+  `00000000`, `31DEC99`, `ZZ`, `M`) rather than `REDACTED`, extending the existing typed-field
+  sentinel policy from structured nodes to free-text spans so downstream text parsers keep working.
+- **Rules only reach values via literal markers.** Every new `extract_patterns` entry anchors on an
+  operational marker observed in a payload (`EMER-`, `BIRTHDATE-`, `PSPT-`, `PSPT/CO-`, `CTZN-`,
+  `AUTHORITY TO CHARGE-`, `TRAVEL ARRANGER/`, `ACEORB-`, `ACECRM-EMPLOYEE ID-`, `CYTRIC PROFILE REF:`,
+  `PREF HTL ID/`, `DHSNBR-PAX-`, `ISS TKT FR`). No generic shape matching over free text: an
+  unanchored date or digit-run pattern would redact fares, flight dates and city pairs.
+- **One-way types name their mirrors explicitly.** Rules for date-of-birth, gender, passport,
+  nationality and card fragments in Sabre remarks SHALL also target `HistoryAssociationElement` and
+  `AssociationChild`/`AssociationParent/Content`, because the reference pass cannot carry a
+  non-encrypt action.
+- **New Amadeus paths.** The `FP` and `OS` `dataElementsIndiv` free-text nodes gain coverage.
+
+No engine change, no schema change, no new `PiiType`. The whole change is baked-ruleset authoring plus
+contract tests over sanitized fixtures.
+
+## Capabilities
+
+### New Capabilities
+<!-- none -->
+
+### Modified Capabilities
+- `pii-rules`: defines which non-passenger persons and which pseudonymous identifiers are in scope,
+  and requires free-text identity-document spans to use format-preserving sentinels.
+- `amadeus-pii-baseline`: gains orderer-name, remark date-of-birth/gender, employee-id, profile-ref
+  and loyalty-pref rules, plus first-time coverage of the `FP` and `OS` free-text elements.
+- `sabre-pii-baseline`: gains third-party name/phone rules, remark identity-document rules that also
+  cover the two history mirrors, card-fragment rules for history nodes, and employee/traveller-profile
+  id rules.

openspec/changes/close-remark-free-text-pii-gap/specs/amadeus-pii-baseline/spec.md

@@ -0,0 +1,42 @@
+## ADDED Requirements
+
+### Requirement: Amadeus form-of-payment and other-service free text are covered
+
+The Amadeus baseline SHALL cover the `FP` (form of payment) and `OS` (other service information)
+`dataElementsIndiv` free-text nodes, not only the `AP` and `RM` elements it targeted previously. All
+four use the same `otherDataFreetext/longFreetext` carrier; `FP` holds a card number and expiry, and
+`OS` holds traveller-attached contract numbers.
+
+#### Scenario: Card number in the FP element is redacted
+- **WHEN** a `PNR_Reply` carries a form-of-payment line such as `PAX CC<type><masked pan>/<mmyy>`
+- **THEN** the card number and the expiry are redacted and the operational prefix is preserved —
+  supplier-side masking is not trusted
+
+#### Scenario: Traveller contract number in the OS element is encrypted
+- **WHEN** an `OS` line carries a traveller-attached contract or loyalty number (for example
+  `CP/<carrier><digits>`)
+- **THEN** the number is encrypted, while organisation-level corporate account codes in sibling `OS`
+  lines (for example `CMP …`, `OIN …`, `NCA …`) are preserved
+
+### Requirement: Amadeus RM remark rules cover both remark mirrors
+
+Amadeus remark extraction patterns SHALL NOT require a leading `*`, so that both renderings of every
+remark match. Each Amadeus RM/RIR element is emitted twice for the same remark: once under
+`miscellaneousRemarks/remarks/freetext` with the leading `*` inside the text, and once under
+`extendedRemark/structuredRemark/freetext` with that `*` hoisted into the sibling `category` element.
+
+#### Scenario: Orderer name is redacted in both mirrors
+- **WHEN** a `PNR_Reply` carries an orderer name as `*ACEORB-<name>` and `*ACECRM-ORDERER-<name>`, each
+  duplicated into the `structuredRemark` mirror without the leading `*`
+- **THEN** every occurrence is encrypted and the `ACEORB-` / `ACECRM-ORDERER-` markers are preserved
+
+#### Scenario: Date of birth and gender in a remark are redacted
+- **WHEN** a remark carries `DOB-<ddmmmyy>/GENDER-<mf>` in both mirrors
+- **THEN** both spans are replaced with format-preserving sentinels in both mirrors
+
+#### Scenario: Person-linked identifiers in remarks are encrypted
+- **WHEN** remarks carry an employee id (`ACECRM-EMPLOYEE ID-<digits>`), a booking-tool traveller
+  profile reference (`CYTRIC PROFILE REF:<digits>`), or a traveller hotel-loyalty preference id
+  (`PREF HTL ID/SUP-<cc>/ID-<digits>`)
+- **THEN** each identifier is encrypted while its marker text and any template remark that carries the
+  marker with no value are preserved

openspec/changes/close-remark-free-text-pii-gap/specs/pii-rules/spec.md

@@ -0,0 +1,71 @@
+## ADDED Requirements
+
+### Requirement: Traveller-adjacent third parties are data subjects; agency staff are operational
+
+The baked ruleset SHALL treat a natural person recorded in a booking *because of their relationship to
+the traveller* as a data subject, redacted on the same terms as the passenger. This covers at least
+emergency contacts, authority-to-charge / payment-authorising persons, travel arrangers and bookers,
+and the Amadeus orderer.
+
+Personal names and identifiers belonging to **agency staff acting in their professional capacity** —
+full agent names in audit remarks, agent sign-in codes, duty codes, pseudo-city codes — SHALL NOT be
+redacted. Redacting them would destroy the audit legibility the remark trail exists to provide, and
+extends the existing "agent sign-in codes are not PII" classification consistently to the name form.
+
+#### Scenario: Emergency contact is redacted
+- **WHEN** a response carries an emergency-contact name or phone number in free text with no
+  structured counterpart anywhere in the document
+- **THEN** the value is redacted and the operational marker around it (for example `EMER-`) is
+  preserved
+
+#### Scenario: Travel arranger is redacted across every rendering
+- **WHEN** the same travel-arranger name appears in a contact phone node, a remark line, an invoice
+  remark field, and a ticketing comment
+- **THEN** every occurrence is redacted, and occurrences reached by the reference pass carry the same
+  token as the occurrence the field rule collected
+
+#### Scenario: Agent name and sine are preserved
+- **WHEN** a remark records the acting agent, either by full name (for example
+  `ADVISED BY - <agent name>`) or by short sign-in code (for example `12AUG2026-<sine>-…`)
+- **THEN** the value is preserved verbatim in the anonymized response
+
+### Requirement: Person-linked pseudonymous identifiers are PII; organisation codes are not
+
+An identifier that resolves to one natural person SHALL be treated as PII and encrypted (reversible,
+because clients legitimately correlate travellers across bookings by these keys). This covers at least
+employer employee ids, traveller profile ids in every namespace that mirrors them, and
+loyalty/contract numbers attached to an individual traveller.
+
+An identifier that resolves only to an organisation — corporate account ids, corporate contract and
+deal codes, tour codes, negotiated-fare discount codes — SHALL NOT be redacted; it is commercial data,
+and removing it breaks fare reissue and corporate policy display.
+
+#### Scenario: Employee id is encrypted wherever it appears
+- **WHEN** an employee id appears both as a structured attribute and inside remark free text
+- **THEN** both occurrences are replaced by the same reversible token
+
+#### Scenario: Traveller profile id is encrypted in every mirror namespace
+- **WHEN** a traveller profile id is emitted in more than one namespace for the same reservation
+- **THEN** every occurrence is encrypted, while agency-level and corporate-level profile ids for the
+  same reservation are preserved
+
+#### Scenario: Corporate account code is preserved
+- **WHEN** a response carries an organisation-level corporate account, contract, tour or discount code
+- **THEN** the value is preserved verbatim
+
+### Requirement: Free-text identity-document spans use format-preserving sentinels
+
+A rule redacting identity-document data embedded in free text SHALL replace the span with a sentinel
+that preserves the original shape in the format used at that location, rather than a `REDACTED`
+literal. This applies to date of birth, gender, document number, document dates and nationality, and
+extends the existing typed-field sentinel policy to free text so that consumer-side text parsing of
+the surrounding operational line keeps working.
+
+#### Scenario: Date of birth in a remark keeps its date shape
+- **WHEN** a remark line carries a date of birth in a `DDMMMYY` free-text span
+- **THEN** the span is replaced with a `DDMMMYY`-shaped sentinel, and the surrounding marker text is
+  preserved
+
+#### Scenario: Passport number in a remark keeps its length class
+- **WHEN** a remark line carries a passport number in a free-text span
+- **THEN** the span is replaced with a document-number-shaped sentinel, not a `REDACTED` literal

openspec/changes/close-remark-free-text-pii-gap/specs/sabre-pii-baseline/spec.md

@@ -0,0 +1,94 @@
+## ADDED Requirements
+
+### Requirement: Reservation remark PII is redacted in every history mirror
+
+A field rule redacting remark PII with a non-encrypt action SHALL target Sabre's history mirror paths
+explicitly, in addition to the remark path. Sabre re-emits each coded passenger remark up to three
+times for one reservation: as `RemarkLine/Text`, as `HistoryAssociationElement`, and as
+`AssociationChild`/`AssociationParent` `Content`, where the remark's `@code` becomes a `<letter>!` text
+prefix. Reference rules accept only an encrypt action, so a value whose policy is one-way replacement
+can never be propagated to those mirrors by the reference pass.
+
+#### Scenario: Remark date of birth is redacted in all three renderings
+- **WHEN** a `GetReservationRS` carries `BIRTHDATE-<ddmmmyy>` in a passenger remark, and the same line
+  is mirrored as `P!BIRTHDATE-<ddmmmyy>` in both history renderings
+- **THEN** all three occurrences carry the sentinel, and no plaintext date of birth remains in the
+  response
+
+#### Scenario: Remark passport, document dates and nationality are redacted in all three renderings
+- **WHEN** a passenger remark carries `PSPT-<cc> <document number> <expiry>`, or
+  `PSPT/CO-<cc>/NR-<document number>/IS-<date>/EX-<date>/CI-<cc>`, or `CTZN-<cc>`, mirrored into
+  history
+- **THEN** every document number, document date and country span is replaced by its sentinel in all
+  renderings, while the operational markers (`PSPT-`, `/NR-`, `CTZN-`) are preserved
+
+#### Scenario: Masked card fragment in history is redacted
+- **WHEN** a form-of-payment value appears in history nodes as a supplier-masked card fragment with an
+  expiry (for example `*CA<masked pan>/<mmyy>` or the `¥`-delimited remark rendering)
+- **THEN** the card fragment and the expiry are both redacted in every rendering — supplier masking is
+  not trusted, since it still reveals the leading digit and last four
+
+### Requirement: Non-passenger persons in reservation free text are redacted
+
+Each non-passenger person recorded in a `GetReservationRS` SHALL be reached by a field rule anchored on
+its operational marker, so that the value also enters the collector and the reference pass scrubs its
+history mirrors. These persons — emergency contact, authority-to-charge holder, travel arranger — have
+no structured name node anywhere in the document, so nothing else can collect them.
+
+#### Scenario: Emergency contact name and phone
+- **WHEN** a passenger remark carries `EMER-<name>` and `EMER-<phone>`
+- **THEN** the name is encrypted, the phone is redacted, and `EMER-OTHER` and other non-PII `EMER-`
+  values are left untouched
+
+#### Scenario: Authority-to-charge holder
+- **WHEN** a passenger remark carries `TPREF-AUTH-AUTHORITY TO CHARGE-<name>`
+- **THEN** the name is encrypted and the preference marker is preserved
+
+#### Scenario: Travel arranger reached through the contact phone node
+- **WHEN** the arranger's name is embedded in a `CTC` contact phone value (for example
+  `M-<name>-<digits>`), which a separate rule masks as a whole node
+- **THEN** the arranger name rule runs before the whole-node phone rule, so the name is collected and
+  encrypted rather than destroyed unread, and the collected value scrubs the arranger's other
+  occurrences in remarks and the ticketing comment
+
+#### Scenario: Received-from field mixes arranger and agent
+- **WHEN** `ReceivedFrom/Name` carries an arranger name, a phone number, and an agency agent name in
+  one value
+- **THEN** the arranger name and the phone are redacted and the agent name is preserved
+
+### Requirement: A rewritten span must never leave a token abutting adjacent text
+
+A rule rewriting a span inside free text SHALL NOT leave an `ENC_` token immediately adjacent to
+surrounding alphanumeric or `-`/`_` text. `ENC_` payloads are base64url, so `-` and `_` read as token
+characters: the greedy token scan on the request path re-consumes the adjacent text into the token, the
+value never de-anonymizes, and the relay would forward a Wenrix token to the channel — breaking
+transparency. Where an operational suffix would abut, the rule SHALL either fold that suffix into the
+encrypted span (keeping the round-trip exact) or use a one-way sentinel containing no token characters.
+
+#### Scenario: Phone followed by an agent name
+- **WHEN** `ReceivedFrom/Name` holds a phone number followed directly by `-<agent name>`
+- **THEN** the number is replaced with a one-way sentinel rather than a token, so nothing abuts the
+  suffix and the response still de-anonymizes cleanly on the way back upstream
+
+#### Scenario: Name followed by an operational qualifier
+- **WHEN** a remark name is followed directly by a `-<letter>` qualifier (for example
+  `TRAVEL ARRANGER/<name>-B`)
+- **THEN** the qualifier is folded into the encrypted span, so the marker stays visible and
+  de-anonymizing the response restores the original line byte for byte
+
+### Requirement: Person-linked identifiers in a reservation are encrypted
+
+Employee ids and traveller profile ids carried in a `GetReservationRS` SHALL be encrypted in every
+location and namespace that mirrors them, while profile ids belonging to the agency, the corporate
+account, or any other non-traveller entity SHALL be preserved.
+
+#### Scenario: Employee id in attribute and remark
+- **WHEN** an employee id appears as `Passenger/@referenceNumber` and inside a `PROFILE <org>-<id>`
+  remark
+- **THEN** both are encrypted with the same token
+
+#### Scenario: Traveller profile id in both namespaces
+- **WHEN** the traveller's profile id is emitted under the passenger's structured profile node and
+  again in the open-reservation-element mirror namespace
+- **THEN** both are encrypted, and the agency, corporate and other non-traveller profile ids for the
+  same reservation are preserved