Skip to content

feat: support dynamic downstream TLS certificate minting - #425

Open
clement0010 wants to merge 12 commits into
refactor/ct/tls-static-configfrom
feat/ct/tls-static-dynamic
Open

feat: support dynamic downstream TLS certificate minting#425
clement0010 wants to merge 12 commits into
refactor/ct/tls-static-configfrom
feat/ct/tls-static-dynamic

Conversation

@clement0010

@clement0010 clement0010 commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Related Tickets & Documents

Branched out from #424

Changes

  • Add a tls.dynamic mode, mutually exclusive with tls.static (exactly one must be set, enforced by config validation): the Gateway mints short-lived leaf certificates on demand, signed by an operator-provided CA (ca.selfSign with certificate and private key files).
  • Certificates are cached per name set in a bounded cache and re-minted inside the renewal window
  • Leaf settings are configurable via cert.duration, cert.renewBefore, cert.keyType, cert.keyBits (defaults 24h / 8h / ecdsa / 256).

Notes

  • Accessing a web app resource by IP address over TLS does not work for now; use a DNS-name address (aliases are covered).
  • Certificate caching uses hashicorp/golang-lru/v2, which has no transitive dependencies.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Jul 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.33032% with 28 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.50%. Comparing base (4b2cb69) to head (6e9d18b).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
internal/connect/cert.go 76.99% 18 Missing and 8 partials ⚠️
internal/connect/listener.go 81.81% 1 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@                        Coverage Diff                        @@
##           refactor/ct/tls-static-config     #425      +/-   ##
=================================================================
- Coverage                          90.68%   90.50%   -0.19%     
=================================================================
  Files                                 41       43       +2     
  Lines                               2909     3117     +208     
=================================================================
+ Hits                                2638     2821     +183     
- Misses                               177      194      +17     
- Partials                              94      102       +8     
Flag Coverage Δ
integration 54.22% <13.12%> (-3.39%) ⬇️
unit 85.91% <83.71%> (+0.25%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
internal/config/config.go 92.24% <100.00%> (+1.57%) ⬆️
internal/connect/cert_provider.go 100.00% <100.00%> (ø)
internal/connect/cert_reloader.go 100.00% <100.00%> (ø)
internal/connect/conn.go 85.36% <100.00%> (+3.78%) ⬆️
internal/connect/listener.go 84.69% <81.81%> (-1.48%) ⬇️
internal/connect/cert.go 76.99% <76.99%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@clement0010
clement0010 force-pushed the feat/ct/tls-static-dynamic branch from 8ff1d2c to d22f04b Compare July 29, 2026 05:32
@clement0010
clement0010 force-pushed the refactor/ct/tls-static-config branch from ab8c5b8 to bd63558 Compare July 29, 2026 10:18
clement0010 and others added 2 commits July 29, 2026 18:51
Add a `tls.dynamic` mode that mints short-lived downstream leaf certificates
from a configured CA, so operators no longer maintain static SANs for every
resource hostname.

Both modes now sit behind a `CertProvider` interface (Run, GetCertificate,
GetCertificateForHost) selected by `newCertProviderFromConfig`, mirroring the
existing sshhandler caProvider. Static mode keeps the file-watching
CertReloader; the inner post-CONNECT handshake pins the certificate for the
CONNECT-requested host in both modes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GetCertificate serves the pre-CONNECT handshake, so the certificate cache was
keyed by client-controlled SNI with no eviction: distinct SNI values per
connection grew it without limit. Cap it at 1024 entries (~2 MB) with an LRU
backed by hashicorp/golang-lru, sized in line with comparable certificate
caches (mitmproxy 100, Caddy 10000).

The mutex also wrapped the whole lookup, so every caller queued behind an
in-flight mint even on a cache hit -- 349 ms for an rsa-4096 leaf. Take the
lock only on the miss path and re-check the cache under it, which keeps hits
lock-free while still collapsing concurrent cold misses for the same host to
a single mint.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@clement0010
clement0010 force-pushed the feat/ct/tls-static-dynamic branch from cec1ebf to a91f4ad Compare July 29, 2026 14:52
Add a checked-in CA at test/data/ca, embedded as data.CACert/data.CAKey and
produced the same way as the existing proxy and api_server fixtures, so the
tests stop generating an RSA CA per call.

Drop the tests that provoked mint failures through configuration
TLSDynamicCertConfig.Validate rejects — unsupported key types and sizes, and a
renewBefore longer than duration. The remaining renewal test reaches the
renewal window by expiring the cached certificate instead. Cover the provider
failure that getTLSConfig wraps with a stub CertProvider, which needs no
invalid configuration.

Inline the certificate construction each test needs in place of a shared
helper, name each test after the method it exercises, and group them by that
method. CertReloader.GetCertificate now delegates to GetCertificateForHost so
the read lock sits with the implementation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@clement0010
clement0010 force-pushed the feat/ct/tls-static-dynamic branch from a91f4ad to 4178249 Compare July 29, 2026 16:12
CertProvider only supplied a certificate per host; deriving that host from the
ClientHello was a second method every provider had to implement, and the static
reloader had to ignore its argument. Move the derivation into
getCertificateForHello, wired by the listener, and drop GetCertificate from the
interface.

The inner upgrade now resolves through a GetCertificate closure rather than
pinning Certificates. The closure ignores the ClientHello, so the served
certificate still follows the validated CONNECT address and not client SNI.
getTLSConfig can no longer fail, so it returns a config alone.

Drop the unused data.CAKey embed; tests reference the CA key by path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@clement0010
clement0010 requested a lite review from Copilot July 30, 2026 09:44
@clement0010
clement0010 marked this pull request as ready for review July 30, 2026 09:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a dynamic downstream TLS mode to the Gateway’s CONNECT/TLS flow by introducing a certificate provider abstraction that can either hot-reload a static certificate or mint short-lived leaf certificates on demand, and by extending GAT claims to indicate when downstream TLS should be upgraded for WebApp resources.

Changes:

  • Introduces CertProvider and dynamic certificate minting with caching/renewal behavior, wired into the CONNECT listener and inner TLS upgrade path.
  • Extends GAT claims (Downstream.TLS) and updates fake client + local tooling to exercise WebApp downstream TLS.
  • Extends config schema and validation to require exactly one of tls.static or tls.dynamic, plus dynamic CA/leaf settings.

Reviewed changes

Copilot reviewed 23 out of 24 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tools/local/main.go Adds a second fake WebApp client instance for downstream TLS and updates local dev output/config template.
test/fake/client.go Adds WithDownstreamTLS() option and populates Downstream.TLS in minted GAT requests.
internal/webapphandler/handler_test.go Updates connect.NewProxyConn call sites for the new signature.
internal/token/gat_claims.go Adds Downstream.TLS field and updates ShouldUpgradeTLS() for WebApp.
internal/token/gat_claims_test.go Adds coverage for Downstream.TLS upgrade behavior and JSON unmarshal.
internal/sshhandler/proxy_test.go Updates connect.NewProxyConn call sites for the new signature.
internal/kuberneteshandler/handler_test.go Updates connect.NewProxyConn call sites for the new signature.
internal/httpproxy/proxy_test.go Updates connect.NewProxyConn call sites for the new signature.
internal/connect/listener.go Switches from CertReloader to CertProvider and wires cert selection into TLS GetCertificate.
internal/connect/listener_test.go Updates test fixtures to use certProvider field.
internal/connect/connect.go Adds DownstreamAddress to parsed CONNECT info to pin inner TLS to the CONNECT host.
internal/connect/conn.go Pins inner TLS certificate to CONNECT host via CertProvider and stores DownstreamAddress.
internal/connect/conn_test.go Adds tests for pinned TLS config behavior and error cases during inner TLS upgrade.
internal/connect/cert.go Implements dynamic leaf certificate minting + caching.
internal/connect/cert_test.go Adds unit tests for dynamic minting, SANs, caching, renewal, and concurrency behavior.
internal/connect/cert_reloader.go Adapts static cert reloader to implement CertProvider.
internal/connect/cert_reloader_test.go Updates tests to use GetCertificateForHost.
internal/connect/cert_provider.go Adds CertProvider interface, SNI/local-IP host selection, and config-to-provider factory.
internal/connect/cert_provider_test.go Adds tests for SNI/local-IP selection and provider factory behavior.
internal/config/config.go Adds tls.dynamic schema + validation and enforces exactly one of static/dynamic.
internal/config/config_test.go Adds load/validate coverage for dynamic TLS config and edge cases.
go.mod Adds github.com/hashicorp/golang-lru/v2 dependency.
go.sum Adds checksums for the new LRU dependency.
.golangci.yml Updates linter configuration to account for CertProvider.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread internal/connect/cert.go
Comment thread internal/connect/cert.go
Comment thread internal/connect/cert.go Outdated
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown

This pull request is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days.

@github-actions github-actions Bot added the wontfix This will not be worked on label Aug 9, 2026
@clement0010 clement0010 removed the wontfix This will not be worked on label Aug 10, 2026
The Twingate client can dial a web app by an alias while the tunnel is
established against the resource address, so the minted certificate now
covers the token aliases alongside the CONNECT host, cached per name
set.

Fold the test/data/ca fixture into the proxy certificate, which is
already a CA, and assert leaf contents in the DynamicCert unit tests
instead of the UpgradeToTLS handshake tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tokens can carry the same aliases in different orders, which changed
the cache key and minted a duplicate certificate per ordering.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@clement0010
clement0010 force-pushed the feat/ct/tls-static-dynamic branch from a2e192f to 98469da Compare August 11, 2026 12:05
Isolate the signing backend behind a certIssuer interface so alternative
CA backends only add an issuer, and thread the TLS handshake context
through CertProvider so issuers doing network I/O are canceled when the
handshake is abandoned.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@clement0010
clement0010 force-pushed the feat/ct/tls-static-dynamic branch from 98469da to b7273ef Compare August 11, 2026 12:25
@github-actions

Copy link
Copy Markdown

This pull request is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days.

@github-actions github-actions Bot added the wontfix This will not be worked on label Aug 21, 2026
@clement0010 clement0010 removed the wontfix This will not be worked on label Aug 21, 2026
Copilot AI review requested due to automatic review settings August 25, 2026 05:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 19 changed files in this pull request and generated no new comments.

Suppressed comments (1)

internal/connect/cert.go:35

  • Issue #423 / PR description specify dynamic leaf certs should be backdated 5m for clock skew, but DynamicCert uses a 30s clockSkewBuffer, reducing tolerance for client/server time drift and not matching the stated requirement.
const (
	clockSkewBuffer = 30 * time.Second
	maxCachedCerts  = 1024
)

Copilot AI review requested due to automatic review settings August 25, 2026 12:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 19 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

internal/connect/cert.go:35

  • Issue #423 calls for backdating minted leaf certificates by 5 minutes for clock skew, but clockSkewBuffer is currently 30s, which may be too small for clients with larger clock drift and doesn't match the stated requirement.
const (
	clockSkewBuffer = 30 * time.Second
	maxCachedCerts  = 1024
)

Comment thread internal/connect/cert_reloader_test.go Outdated
@clement0010
clement0010 requested a review from minhtule August 25, 2026 12:59
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Clement Tee <56408987+clement0010@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 25, 2026 13:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 19 changed files in this pull request and generated no new comments.

Suppressed comments (5)

Previously missed (3) — in code that hasn't changed since the last review.

internal/connect/cert.go:149

  • certNames currently treats DNS names with different casing as distinct (e.g. "Example.com" vs "example.com"), which reduces cache hit rate and can inflate the bounded cache with duplicate certificates for the same DNS name set. Normalizing DNS names to lowercase before dedup/sort avoids this.
func certNames(host string, aliases []string) []string {
	names := make([]string, 0, len(aliases)+1)
	names = append(names, host)

	for _, alias := range aliases {
		if alias != "" && !slices.Contains(names, alias) {
			names = append(names, alias)
		}
	}

internal/config/config_test.go:772

  • If ErrNegativeDuration’s message is updated to be field-agnostic (e.g. "value must be non-negative"), this test will fail because it currently asserts the old message substring.

This issue also appears on line 781 of the same file.

			errContains: "cert: duration must be non-negative: duration",

internal/config/config.go:426

  • ErrNegativeDuration is also used for a negative renewBefore value, which yields a misleading message (“duration must be non-negative: renewBefore”). Consider making the base message field-agnostic so it stays accurate for both duration and renewBefore.
	ErrNegativeDuration     = errors.New("duration must be non-negative")

internal/connect/cert.go:35

  • Issue #423 specifies leaf certificates should be backdated by 5m for clock skew, but the implementation uses a 30s buffer. This can cause clients with larger clock skew to reject freshly issued certs as “not yet valid”.
const (
	clockSkewBuffer = 30 * time.Second
	maxCachedCerts  = 1024
)

internal/config/config_test.go:781

  • If ErrNegativeDuration’s message is updated to be field-agnostic (e.g. "value must be non-negative"), this test will fail because it currently asserts the old message substring (it also currently says “duration … renewBefore”, which is misleading).
			errContains: "cert: duration must be non-negative: renewBefore",

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants