Preflight checklist
Describe the background of your feature request
heimdall uses the Endpoint abstraction for communicating with external services and retrieving remote resources. It is used in a number of different contexts, including, among others:
- generic authenticators and contextualizers
- remote authorizers
- OAuth2 token introspection
- OAuth2 client credentials token retrieval
- OAuth2/OIDC metadata retrieval
- JWKS retrieval
- remotely hosted rule sets
At the moment, redirect handling is not explicitly modeled by heimdall. The HTTP clients created for an Endpoint effectively inherit the default redirect behavior of Go's net/http client.
This is not necessarily appropriate for all usages of Endpoint, as redirects have different semantics depending on the purpose of the request.
For example, when retrieving a remote resource such as a rule set, metadata document, or JWKS, following a redirect can be perfectly legitimate and may even be required depending on how the resource is hosted, e.g. behind a reverse proxy, CDN, or object storage.
For RPC- or protocol-like calls, such as authorization decisions, OAuth2 token introspection, or token retrieval, a redirect is much less obviously part of the expected interaction. Following it implicitly can change the actual dependency serving the request and, depending on the redirect status code, even affect the HTTP method and request body.
There are also further aspects which make a single implicit policy problematic:
- redirects can cross origin boundaries and therefore extend the configured dependency boundary
- HTTPS endpoints can redirect to HTTP endpoints
- headers, including custom or forwarded headers, require careful handling across origin boundaries
- redirects of requests using HTTP Message Signatures are problematic because the redirected request can differ from the originally signed request
- redirects and configured retries interact, since redirected requests pass through the transport and can therefore have their own retry attempts
- Go's redirect behavior for
301, 302, and 303 differs from 307 and 308, particularly for non-GET requests
- changing the behavior globally, e.g. by simply disabling redirects, could break existing and legitimate usages such as remote rule set or metadata retrieval
Describe your idea
Introduce explicit redirect handling for outbound HTTP communication instead of implicitly relying on the default behavior of Go's net/http client and let the implementation take take of the different purposes for which Endpoint is used.
Things to consider:
- whether redirects should be configurable as part of the generic
Endpoint configuration
- whether individual consumers of
Endpoint should constrain or define the allowed redirect behavior
- distinguishing between no redirects, same-origin redirects, and cross-origin redirects
- defining an appropriate maximum number of redirect hops
- preventing unintended HTTPS-to-HTTP downgrades
- defining redirect semantics for non-safe HTTP methods, especially the different behavior of
301/302/303 versus 307/308
- defining which headers may be propagated when the origin changes
- handling authentication strategies which cannot safely or correctly be reused for a redirected request, such as HTTP Message Signatures
- taking the interaction between redirects, retries, timeouts, and the overall request budget into account
- making redirects observable so operators can understand when requests are served by a redirected endpoint
- providing a backwards-compatible migration path for existing configurations
Are there any workarounds or alternatives?
One alternative would be to disable redirects globally for all Endpoint usages. Although this would be simple, it would apply RPC semantics to resource-fetching use cases as well and could break existing configurations where redirects are a legitimate part of retrieving rule sets, metadata, JWKS, or resources from generic APIs.
Another alternative would be to add isolated hardening measures, for example preventing only HTTPS-to-HTTP redirects or disabling redirects for selected calls. While such measures could improve individual cases, they would leave the underlying redirect semantics implicit and result in different special cases being added over time without a coherent model.
Keeping Go's current default behavior is also possible.
Version
0.17.20
Additional Context
The motivation for this feature came from investigating whether redirects should generally be disabled for resilience reasons.
Reviewing the different usages of Endpoint showed that there is no universally correct answer: heimdall uses the same abstraction both for service/API calls and for retrieving remote resources. Redirects can therefore be undesirable for some consumers while being legitimate, or potentially required for compatibility, for others.
The goal of this feature is consequently to make redirect semantics deliberate, constrained, and appropriate for the respective use case.
Preflight checklist
Describe the background of your feature request
heimdall uses the
Endpointabstraction for communicating with external services and retrieving remote resources. It is used in a number of different contexts, including, among others:At the moment, redirect handling is not explicitly modeled by heimdall. The HTTP clients created for an
Endpointeffectively inherit the default redirect behavior of Go'snet/httpclient.This is not necessarily appropriate for all usages of
Endpoint, as redirects have different semantics depending on the purpose of the request.For example, when retrieving a remote resource such as a rule set, metadata document, or JWKS, following a redirect can be perfectly legitimate and may even be required depending on how the resource is hosted, e.g. behind a reverse proxy, CDN, or object storage.
For RPC- or protocol-like calls, such as authorization decisions, OAuth2 token introspection, or token retrieval, a redirect is much less obviously part of the expected interaction. Following it implicitly can change the actual dependency serving the request and, depending on the redirect status code, even affect the HTTP method and request body.
There are also further aspects which make a single implicit policy problematic:
301,302, and303differs from307and308, particularly for non-GET requestsDescribe your idea
Introduce explicit redirect handling for outbound HTTP communication instead of implicitly relying on the default behavior of Go's
net/httpclient and let the implementation take take of the different purposes for whichEndpointis used.Things to consider:
EndpointconfigurationEndpointshould constrain or define the allowed redirect behavior301/302/303versus307/308Are there any workarounds or alternatives?
One alternative would be to disable redirects globally for all
Endpointusages. Although this would be simple, it would apply RPC semantics to resource-fetching use cases as well and could break existing configurations where redirects are a legitimate part of retrieving rule sets, metadata, JWKS, or resources from generic APIs.Another alternative would be to add isolated hardening measures, for example preventing only HTTPS-to-HTTP redirects or disabling redirects for selected calls. While such measures could improve individual cases, they would leave the underlying redirect semantics implicit and result in different special cases being added over time without a coherent model.
Keeping Go's current default behavior is also possible.
Version
0.17.20
Additional Context
The motivation for this feature came from investigating whether redirects should generally be disabled for resilience reasons.
Reviewing the different usages of
Endpointshowed that there is no universally correct answer: heimdall uses the same abstraction both for service/API calls and for retrieving remote resources. Redirects can therefore be undesirable for some consumers while being legitimate, or potentially required for compatibility, for others.The goal of this feature is consequently to make redirect semantics deliberate, constrained, and appropriate for the respective use case.