Skip to content

[INTER-2546] Honor origin cache - #356

Open
TheUnderScorer wants to merge 3 commits into
feature/INTER-2067-carefrom
feature/INTER-2546-cache-headers
Open

TheUnderScorer wants to merge 3 commits into
feature/INTER-2067-carefrom
feature/INTER-2546-cache-headers

Conversation

@TheUnderScorer

@TheUnderScorer TheUnderScorer commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Honor origin agent cache headers

The JS Agent team now lets the origin own the agent's cache lifetimes. This proxy was overwriting them. It clamped max-age to at most 3600 and injected s-maxage=60 whenever the origin sent none, so CloudFront discarded the agent every minute, and any origin lifetime above an hour reached browsers truncated. It also forwarded the upstream Cache-Tag, which carries a hash derived from the customer's API key.

Verified against Fingerprint staging (procdn.fpjs.sh) through four endpoints in a single run, each with its own API key so every endpoint's first request reaches the origin:

This PR main
Bare path /fpjs/web/v4/<apiKey> /fpjs/web/v4/<apiKey>
.js path /fpjs/web/v4/<apiKey>/iife.min.js /fpjs/web/v4/<apiKey>/iife.min.js

Neither build treats the two path shapes differently: same statuses, same cache behavior, same header handling. Only the origin's own max-age differs, because each endpoint uses its own API key. The table reports one row per step, bare path first, .js path second.

Test results

Step Expected main This PR
1. First request 200, no s-maxage, no age, cache-tag ❌ 200 MISS, s-maxage=60, cache-tag forwarded ⚠️ 200 MISS, max-age=3669 / 3761, no s-maxage, no age, no cache-tag*
2. Second request (hit) 200, no s-maxage, age: 0, no cache-tag ❌ 200 HIT, s-maxage=60, age: 3 / 2, cache-tag still forwarded ⚠️ 200 HIT, no s-maxage, no cache-tag, age: 10 / 7**
3. After the edge entry expired 200, no s-maxage, age: 0, no cache-tag ❌ 200 MISS, s-maxage=60 ✅ 200 MISS, re-fetched, max-age=3669 / 3644, no cache-tag
4a. If-None-Match, answered from cache 304, no s-maxage, age: 0, no cache-tag ❌ 304 HIT, s-maxage=60, age: 1 ⚠️ 304 HIT, no s-maxage, age: 3 / 2**
4b. If-None-Match, revalidated at the origin*** 304 ✅ 304 MISS, but still s-maxage=60 ✅ 304 MISS, age: 0
  • * main forwarded the upstream purge tag on steps 1 and 2, keyed to the customer: cache-tag: procdn,procdn-apiKey-62dd1936… on one endpoint, …7dba44ea… on the other. It stops appearing from step 3 because procdn's own Cloudflare cache was warm by then and no longer emitted it. This PR returns no cache-tag on any step.
  • ** Age is the one expectation this change does not meet, on either build. This function cannot fix it, see the follow-up section. The values differ across endpoints because of the run order. Each step hits the four endpoints in sequence, so the endpoint requested first has the oldest entry by the time the next step comes round. At step 4b this PR reports age: 0 where main sends no Age at all; RFC 9111 treats an absent Age as zero, so the two are equivalent.
  • *** Not in the ticket's plan. The ticket's step 4 is answered by the CloudFront cache, so the function never sees an upstream 304. 4b lets the edge entry expire first, which pushes the conditional request through to the origin. Both builds handle it.

The max-age cap was truncating real values

A second run isolates the cap. All four endpoints shared one API key, so the origin value was identical everywhere. main returned exactly max-age=3600 on all ten of its responses while the origin sent between 3613 and 3715 in the same minute. That cut up to 115 seconds off every agent response. This PR returns what the origin sent.

The fresh-key run above only hints at it. main's bare path got max-age=3478, under the cap and so untouched, while its .js path landed on exactly 3600 against sibling keys serving 3644 to 3761. Suggestive, not conclusive, since 3600 is also a plausible origin value.

s-maxage and the edge TTL

main injected s-maxage=60 into all ten of its responses, which is what CloudFront uses for its edge TTL. On this PR the TTL comes from the cache policy's 180s instead. Step 3 cannot separate the two, since a 190s pause expires both and both re-fetched once. The difference is in how often that happens under real traffic: every 60s on main, every 180s here.

What changed

Area Before After
Cache-Control to clients max-age clamped to at most 3600, truncating the 3613 to 3715 the origin sent during the shared-key run. s-maxage set to 60 when the origin sent none Origin value as it is
Edge TTL 60s, from the injected s-maxage 180s, from the CloudFront cache policy
cache-tag Forwarded, with an API-key-derived hash Stripped on every response
age CloudFront's own value CloudFront's own value, unchanged
If-None-Match 304 from cache and 304 revalidated at the origin, both correct Unchanged

Code:

  • Deleted proxy/utils/cache-control.ts. The origin Cache-Control passes through byte for byte.
  • Dropped the overrideCacheControl parameter from updateResponseHeaders and the isJavascript flag that fed it. The agent response needs no special case.
  • Added cache-tag to BLACKLISTED_HEADERS.

Why

  • The origin decides cache lifetimes. The proxy no longer replaces them with static values.
  • cache-tag is the upstream CDN's purge handle. A browser has no use for it.
  • An upstream 304 has no body. Step 4b checks the function returns it rather than failing, since nothing exercised that path before.

Why cache-tag is stripped from the first response too

The ticket expects cache-tag on a miss and nothing on a hit. A CloudFront proxy cannot do that, and main shows why. Step 2 is a CloudFront HIT and still carries the tag. The Lambda@Edge function runs on origin-request, so it only runs on a cache miss. It fetches the agent itself and returns the response, CloudFront stores that, and every later hit is replayed from the store without the function running. The header is either always present or never present. Stripping it on every response keeps the API-key hash out of client responses.

Follow-up: the edge TTL stays at 180s

The cache policy in fingerprintjs/fingerprint-cloudfront-proxy-integration (v2.0.0) sets default_ttl = max_ttl = 180, and CloudFront clamps origin TTLs to it. Removing the s-maxage=60 injection therefore raised the real edge TTL from 60s to 180s, not to the origin's ~1h. Keeping it there, for now.

A browser loses whatever the CloudFront entry's age was when it fetched, once, off its own window. At max_ttl = 180 that is at most 180s of ~3500s, averaging ~90s, so under 3%. At max_ttl = 3600 the average loss becomes roughly half the window and the unlucky half of users revalidate on nearly every page load.

Pinning Age: 0 does not buy the headroom to raise it. An aws_cloudfront_response_headers_policy can set it, but it changes nothing a browser computes. CloudFront replays the stored Date, frozen at the moment it stored the entry, so RFC 9111's max(apparent_age, corrected_age_value) resolves to the Date-derived value regardless of Age. Measured with the policy: a hit at 10:48:42 serving date: 10:46:27 and age: 0, so a browser reads 135s, not 0.

Fixing Date needs code, since the correct value is "now" and a policy only writes constants. That means a viewer-response function, which the edge function restrictions permit alongside our origin-request one. Worth doing only if someone wants max_ttl raised.

Also ruled out: passing the origin's Age: 0 through. Removing age from BLACKLISTED_HEADERS was deployed and tested, and CloudFront overwrites it with its own count. age stays blacklisted.

@TheUnderScorer TheUnderScorer self-assigned this Oct 5, 2026
@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Coverage report

St.❔
Category Percentage Covered / Total
🟢 Statements
90.63% (-0.26% 🔻)
580/640
🟢 Branches
84.74% (-0.39% 🔻)
261/308
🟢 Functions
91.28% (-0.17% 🔻)
136/149
🟢 Lines
90.54% (-0.27% 🔻)
565/624
Show files with reduced coverage 🔻
St.❔
File Statements Branches Functions Lines
🟢
... / headers.ts
95.24% (-0.22% 🔻)
89.19% (-1.51% 🔻)
100%
94.92% (-0.25% 🔻)
🟢
... / transport.ts
91.67% (-0.33% 🔻)
66.67% 100%
91.67% (-0.33% 🔻)

Test suite run success

205 tests passing in 61 suites.

Report generated by 🧪jest coverage report action from fd7c115

Show full coverage report
St File % Stmts % Branch % Funcs % Lines Uncovered Line #s
🟢 All files 90.62 84.74 91.27 90.54
🟢  mgmt-lambda 98.73 95.74 100 98.73
🟢   ...ltSettings.ts 100 100 100 100
🟢   app.ts 97.36 96.42 100 97.36 31
🟢   auth.ts 100 100 100 100
🟢   exceptions.ts 100 0 100 100 20
🟢   routing.ts 100 100 100 100
🟢  ...ambda/handlers 85.97 72.58 93.33 85.88
🟢   errorHandlers.ts 100 70 100 100 22-38,47
🟡   statusHandler.ts 76.92 50 100 76.92 78-82,86-91
🟢   updateHandler.ts 86.5 76.08 87.5 86.4 ...24,286-287,321
🟡  mgmt-lambda/utils 70 83.33 66.66 62.5
🟢   ...frontUtils.ts 100 83.33 100 100 7
🔴   delay.ts 25 100 0 25 2-4
🟢  proxy/handlers 98.21 86.66 100 98.21
🟢   handleIngress.ts 96.96 85 100 96.96 65
🟢   handleStatus.ts 100 88 100 100 58,72
🟡  proxy/test 66.66 100 50 66.66
🟡   aws.ts 66.66 100 50 66.66 4-5
🟢  ...omer-variables 100 100 100 100
🟢   ...-variables.ts 100 100 100 100
🟢  proxy/utils 86.97 78.88 89.47 86.95
🟢   buffer.ts 100 50 100 100 2
🟢   cache.ts 100 87.5 100 100 17
🟢   cookie.ts 100 100 100 100
🔴   ...orResponse.ts 16.66 100 25 18.18 15-30
🟢   headers.ts 95.23 89.18 100 94.91 232-234
🔴   is-blob.ts 0 0 0 0 7
🟢   is-truthy.ts 100 100 100 100
🟡   log.ts 80 50 100 75 11
🟢   paths.ts 100 87.5 100 100 20
🟢   request.ts 92 75 87.5 91.3 8-9
🟢   routing.ts 100 100 100 100
🔴   string.ts 0 100 0 0 2-8
🟢   traffic.ts 100 100 100 100
🟢   transport.ts 91.66 66.66 100 91.66 35,60
🟢   validation.ts 100 100 100 100
🟢  ...omer-variables 98.5 100 95.45 98.41
🟢   ...-variables.ts 100 100 100 100
🟢   defaults.ts 100 100 100 100
🟢   ...-variables.ts 100 100 100 100
🟢   ...e-variable.ts 100 100 100 100
🟢   selectors.ts 95.45 100 90 95 31
🟢   types.ts 100 100 100 100
🟢  ...ecrets-manager 93.44 94.73 100 93.33
🟢   ...ize-secret.ts 83.33 75 100 80 5
🟢   ...eve-secret.ts 100 100 100 100
🟢   ...-variables.ts 86.36 93.33 100 86.36 36,64-69
🟢   ...ate-secret.ts 100 100 100 100

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Documentation incorrectly states that edge TTL follows the origin despite the configured 180-second maximum.

Review effort: Balanced
Findings: 2 Low severity

Open (2)
What changed in this PR

Removes proxy cache-header overrides so agent responses retain origin directives while preventing Cache-Tag exposure.

Changes:

  • Passes origin Cache-Control through unchanged.
  • Removes cache-control rewriting and JavaScript-specific handling.
  • Strips Cache-Tag and updates tests.
File Description
proxy/​utils/​transport.ts Removes JavaScript-specific response handling.
proxy/​utils/​headers.ts Passes cache directives through and blocks Cache-Tag.
proxy/​utils/​cache-control.ts Deletes cache-control rewriting logic.
proxy/​test/​utils/​headers.test.ts Tests passthrough and tag removal.
proxy/​test/​utils/​cache-control.test.ts Removes obsolete rewriting tests.
proxy/​test/​handlers/​v4/​handleAgentDownloading.test.ts Updates V4 cache-header expectations.
proxy/​test/​handlers/​handleAgentDownloading.test.ts Updates V3 cache-header expectations.
.changeset/​pass-through-agent-cache-headers.md Documents externally visible behavior.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .changeset/pass-through-agent-cache-headers.md Outdated
Comment thread proxy/utils/headers.ts Outdated
TheUnderScorer and others added 2 commits October 6, 2026 15:20
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@TheUnderScorer
TheUnderScorer requested a balanced review from Copilot October 6, 2026 13:21
@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

🚀 Following releases will be created using changesets from this PR:

@fingerprint/aws-cloudfront-proxy@2.2.1-rc.0

Patch Changes

  • Upgrade AWS SDK Clients to latest version (^3.1144.0). (51192a4)
  • Pass the origin's Cache-Control for the agent through as it is. The proxy no longer caps max-age at 3600 or injects s-maxage=60, so the browser cache lifetime follows the origin. CloudFront derives its edge TTL from the origin directives subject to the cache policy's limits, and uses the policy default when the origin sends no cache lifetime. Strip the upstream Cache-Tag header, which carries the upstream CDN's purge tag and an API-key-derived hash. (6dc6eef)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The updated V4 cache tests accidentally exercise the V3 agent route and should use the declared V4 request URI.

Review effort: Balanced
Findings: None

Resolved since last review (2)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Use V4 requestUri to avoid routing test through V3 handler

proxy/​test/​handlers/​v4/​handleAgentDownloading.test.ts:113

This URI matches the configured V3 agent route (fpjs_agent_download_path = greiodsfkljlds), which is checked before the V4 catch-all in app.ts:26-51. As a result, this test no longer exercises the V4 agent path and duplicates the V3 test instead; use the suite's requestUri so the cache pass-through remains covered through the V4 handler.

This issue also appears on line 140 of the same file.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants