1.3.4#70
Merged
Merged
Conversation
* fix: prevent unknown character corruption in translations ## Changed - Set contentType explicitly per parser so Lara no longer auto-detects TextBlock[] as HTML and replaces non-ASCII chars with literal `?` - Engine splits each batch by detected content type so values containing inline HTML are sent as text/html and plain values as text/plain - Retry U+FFFD-corrupted translations up to 3 times as solo calls before failing with a neutral "please retry" message - Bump version to 1.3.4 ## New - contentType utility (hasHtmlMarkup / resolveContentType) and per-parser getContentType() exposed through ParserFactory - Parameterized integration tests covering content-type routing for all 10 supported file formats - Deterministic test reproducing the upstream UTF-8 streaming bug in @translated/lara, plus integration coverage for the retry guard * chore: use #utils path alias in contentType test imports ## Changed - Replace relative `../../utils/contentType.js` import with `#utils/contentType.js` to match the convention used by sibling utility tests (e.g. entities.test.ts).
## Changed - Bump `glob` from `^11.1.0` to `^13.0.6` to remove deprecation warning at install - Override transitive `brace-expansion` in vulnerable range `>=5.0.0 <5.0.6` to `^5.0.6` (CVE-2026-45149, GHSA-jxxr-4gwj-5jf2)
There was a problem hiding this comment.
Pull request overview
This PR prepares the 1.3.4 release by making Lara API contentType explicit per parser/format (and per value when inline HTML is detected) and adding a mitigation for an upstream UTF-8 streaming corruption issue by retrying results containing U+FFFD (�). It also bumps the CLI version and updates dependencies (notably glob) with a pnpm override for brace-expansion.
Changes:
- Add parser-level
getContentType()and aresolveContentType()utility to route plain vs HTML-bearing strings to the correct Lara pipeline. - Update
TranslationEngineto (a) setcontentTypeon all translation calls, (b) split batch calls by detected content type, and (c) retry U+FFFD-corrupted results as solo calls. - Add/extend unit + integration tests for content-type routing and the UTF-8 retry guard; bump package version and update
glob+ lockfile.
Reviewed changes
Copilot reviewed 34 out of 35 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/utils/contentType.ts | Adds heuristic HTML-markup detection and per-value contentType resolution utility. |
| src/parsers/xcode-xcstrings.parser.ts | Implements getContentType() for xcstrings parser. |
| src/parsers/xcode-stringsdict.parser.ts | Implements getContentType() for stringsdict parser. |
| src/parsers/xcode-strings.parser.ts | Implements getContentType() for strings parser. |
| src/parsers/vue.parser.ts | Implements getContentType() for Vue parser. |
| src/parsers/txt.parser.ts | Implements getContentType() for txt parser. |
| src/parsers/ts.parser.ts | Implements getContentType() for TS parser. |
| src/parsers/po.parser.ts | Implements getContentType() for PO parser. |
| src/parsers/parser.factory.ts | Exposes parser getContentType() via the factory. |
| src/parsers/markdown.parser.ts | Implements getContentType() for Markdown/MDX parser. |
| src/parsers/json.parser.ts | Implements getContentType() for JSON parser. |
| src/parsers/android-xml.parser.ts | Implements getContentType() as text/html for Android XML. |
| src/modules/translation/translation.engine.ts | Adds default/per-value contentType routing, batch grouping by contentType, and U+FFFD retry mitigation. |
| src/messages/messages.ts | Adds a neutral error message for retry exhaustion. |
| src/interface/parser.ts | Extends parser interface with getContentType(). |
| src/tests/utils/contentType.test.ts | Unit tests for HTML detection and contentType resolution. |
| src/tests/sdk-utf8-streaming.repro.test.ts | Deterministic reproduction test documenting upstream SDK UTF-8 streaming bug. |
| src/tests/parsers/xcode-xcstrings.parser.test.ts | Tests getContentType() for xcstrings parser. |
| src/tests/parsers/xcode-stringsdict.parser.test.ts | Tests getContentType() for stringsdict parser. |
| src/tests/parsers/xcode-strings.parser.test.ts | Tests getContentType() for strings parser. |
| src/tests/parsers/vue.parser.test.ts | Tests getContentType() for Vue parser. |
| src/tests/parsers/txt.parser.test.ts | Tests getContentType() for txt parser. |
| src/tests/parsers/ts.parser.test.ts | Tests getContentType() for TS parser. |
| src/tests/parsers/po.parser.test.ts | Tests getContentType() for PO parser. |
| src/tests/parsers/parser.factory.test.ts | Tests factory-level contentType selection across formats. |
| src/tests/parsers/markdown.parser.test.ts | Tests getContentType() for Markdown parser. |
| src/tests/parsers/json.parser.test.ts | Tests getContentType() for JSON parser. |
| src/tests/parsers/android-xml.parser.test.ts | Tests getContentType() for Android XML parser. |
| src/tests/integration/utf8-retry.integration.test.ts | Integration coverage for U+FFFD detection + solo retry behavior. |
| src/tests/integration/content-type-routing.integration.test.ts | Integration coverage ensuring contentType routing across supported formats. |
| src/tests/integration/batch-translation.integration.test.ts | Adds regressions around explicit contentType and mixed HTML/plain batching. |
| src/tests/integration/android-xml.integration.test.ts | Asserts Android XML translations use contentType=text/html. |
| README.md | Bumps displayed version badge to 1.3.4. |
| pnpm-lock.yaml | Updates lockfile for glob upgrade and brace-expansion override. |
| package.json | Bumps version to 1.3.4, upgrades glob, and adds pnpm.overrides. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.