Skip to content

[DOCS]: Snowflake profiler network prerequisites — result-stage egress is undocumented, PAT network policy is buried #2609

Description

@vladsagot

Summary

The Snowflake profiler page (docs/lakebridge/docs/assessment/profiler/snowflake.mdx) understates two network prerequisites. Both cost a real customer engagement a working session each, and neither is discoverable from the error you get when you hit it.

1. Egress to Snowflake's result stage is required, and isn't mentioned

The Prerequisites section currently says only:

Network access from the machine running Lakebridge to your Snowflake account.

That is not sufficient, and the omission is easy to miss because it only bites on large accounts. When a result set exceeds the inline threshold, Snowflake does not return rows over the SQL connection — it writes chunk files to its own internal cloud storage stage and hands the client pre-signed URLs, which the connector then downloads in parallel. The client therefore needs egress to two distinct hostnames: the Snowflake endpoint and the result-stage storage host (*.blob.core.windows.net for Azure-hosted deployments, S3 for AWS).

In a corporate network that allowlists *.snowflakecomputing.com but not object storage, the failure mode is confusing:

  • Small steps (account_info, database_objects) succeed, because their results come back inline and never touch storage. Connectivity looks fine.
  • Large steps fail during the TLS handshake to the storage host, retry with backoff, and eventually error out.

The resulting error names the storage host, not Snowflake, so it reads as an unrelated infrastructure problem:

snowflake.connector.vendored.requests.exceptions.SSLError: HTTPSConnectionPool(
  host='<snowflake-result-stage>.blob.core.windows.net', port=443):
  Max retries exceeded with url: /results/... (Caused by SSLError(SSLError(
  "bad handshake: SysCallError(10054, 'WSAECONNRESET')")))

A connection reset arriving mid-handshake is the signature of a TLS-inspecting proxy with no rule for that destination, rather than a transient network fault — it reproduces on every chunk and every retry.

Suggested addition to Prerequisites, something like:

Network access from the machine running Lakebridge to your Snowflake account and to Snowflake's result-stage object storage. Large result sets are not returned over the SQL connection; Snowflake stages them in cloud storage and the client downloads them directly. Run SELECT SYSTEM$ALLOWLIST(); in Snowflake to get the exact host list for your deployment (including the STAGE entry) and give it to your network team. These hosts must be allowlisted and exempted from TLS inspection.

The SYSTEM$ALLOWLIST() pointer is the valuable part — it turns a vague "please open some object storage" request into a specific, reviewable one.

2. The PAT network policy is a prerequisite, not a footnote

Snowflake requires a network policy to be attached before programmatic access tokens can be used, and the profiler authenticates exclusively with a PAT. Today this appears only at the bottom of the Authentication section, framed as an optional aside for short-term testing from a dynamic IP:

If your account or user has a network policy, also see Snowflake's docs on bypassing the network policy on a PAT for short-term testing from a dynamic IP.

Read top-to-bottom, that does not signal "you will be blocked at connection time until this exists." Suggest promoting it into Prerequisites as a required setup step, with the bypass note kept where it is as the testing shortcut.

Why both matter

Each of these surfaces at a different point (credential configuration, then the first large extract step), so a team can hit them sequentially across separate sessions and burn days on what is really one round-trip to a network team. Documenting them together up front lets someone raise a single, complete firewall request before starting.

Happy to open a PR for the docs change if that's useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    feat/profilerIssues related to profilers

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions