You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Lineage observability module points its web UI and lineage-sink contract at a hostname no compose service answers #1029
Found while implementing #1010 (PR #1028): the experimental lineage
observability module is a multi-service module (marquez-api, marquez-web), so no compose service named after the instance id is
ever rendered. Yet three places substitute ${service.host} — the
instance id — as if it were a resolvable DNS name:
modules-experimental/observability/marquez/module.yamllineage-sink
provide: host: ${service.host} and connectionUri: http://${service.host}:${config.apiPort} — a
consumer of the lineage-sink contract would try to reach http://marquez:5000, which nothing answers on the Compose network.
The marquez-web service's MARQUEZ_HOST: "${service.host}"
environment variable — the web UI's link to the API backend — points
at the same non-existent hostname.
marquez-api's hostname: "${service.host}" (cosmetic; the container
hostname only, not a DNS name).
The multi-service half of the ${service.dns} / serviceName mechanism
from #1010 does not help here: ${service.dns} also keeps the instance
id for multi-service modules, and serviceName is currently only
honored for http-service contracts (E136 route generation and the new
E147 host check). The lineage-sink kind is not E147-checked at all, so
this fails silently.
Proposed solution
Point the web UI's MARQUEZ_HOST (and the lineage-sink contract's host/connectionUri) at the API service's actual DNS name. The
reference must go through the renderer's service-naming rule
(_compose_service_name(id, service)) so it survives non-conventional
instance ids, mirroring how http-service providers advertise serviceName.
Consider extending the serviceName mechanism (and E147 coverage) to
the lineage-sink contract kind, so this class of mismatch fails
loudly instead of silently.
Reproduction
Render profiles/local-dagster-postgres-marquez/profile.yaml (it
instantiates the module as id: marquez) and inspect the plan: the lineage-sink contract advertises host: marquez; the rendered compose
file contains only marquez-api and marquez-web services, so http://marquez does not resolve from any sibling container.
Impact
Module-level, experimental: the shipped lineage profile's web UI cannot
reach its API backend, and any consumer binding the lineage-sink
contract gets an unresolvable endpoint. No silent production impact
today (the module is explicitly not production-suitable), but it is the
same contract-hostname class of bug as #1010, found one layer deeper.
Activity
fix(changelog): restore blank lines between Unreleased Fixed entries
fix(marquez): point lineage-sink endpoint at the API service's render…
Merge origin/main into fix/1036-e150-host-port-check