Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions AGENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,8 +120,15 @@ repository, in addition to `CONTRIBUTING.md`.
external roots in CLI and test scenarios.
- Preserve the interpolation vocabulary across planning and rendering:
`${config.*}` reads normalized module config, `${bindings.*}` reads
consumed contract fields, and `${service.host}` resolves to the
profile's module instance ID. Renderer-local service dependencies,
consumed contract fields, `${service.host}` resolves to the
profile's module instance ID (stable for name-prefix uses such as
backup volume names), and `${service.dns}` resolves to the Compose
DNS name the module's serving service answers to (the renderer's
compose service name for single-service modules, or when a namesake
compose service matches the module name such as `postgres` next to
its backup job containers; the instance ID for other multi-service
modules, which name their serving service via the contract's
`serviceName`). Renderer-local service dependencies,
volumes, build contexts, and service names are rewritten relative to
the module/profile and namespaced by module ID.
- Never resolve secret values into plans or rendered Compose. Profiles
Expand Down
23 changes: 23 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,29 @@ The format is based on Keep a Changelog.

### Fixed

- Fixed provided contract endpoints silently breaking when a module
instance id differs from the module's compose service name (issue
#1010, found in the review of #1006): contract resolution substituted
`${service.host}` with the instance id, while the renderer names each
compose service `<id>`/`<id>-<service>`, so profiles instantiating,
e.g., the S3-compatible object-storage module as
`id: object-storage` produced a contract endpoint no sibling container
could resolve. The interpolation vocabulary now separates the two
roles: `${service.host}` keeps resolving to the instance id (name
prefixes such as backup volume names), while the new
`${service.dns}` resolves through the same service-naming rule the
renderer uses -- the compose name of the module's single service, or
of its namesake service when auxiliary services exist (e.g.
`postgres` next to its backup job containers); the explicit
`serviceName` mechanism keeps covering other multi-service modules. The single-service modules advertising
network endpoints (KeyDB-compatible cache, identity provider, reverse
proxy, object storage, the Postgres warehouse, and the Superset BI
app) now build them from `${service.dns}`, and the
render cross-checks every provided `http-service`, `sql-database`,
`cache-service`, and `object-storage` contract's advertised network
host against the compose services it actually produced, failing with
`E150` instead of shipping a broken endpoint.

- Fixed the restic file backup module's restore path for multiple bound
sources (issue #1014, found in the post-merge review of #1011): the
scheduler took one separate restic snapshot per source, but the restore
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -788,6 +788,7 @@ Common errors from `cds validate`, `cds plan`, and `cds render`, and how to fix
| `[E134] ... Production backups must be encrypted ...` / `... must be declared required: true` | A production profile has no `spec.backup.encryption.keyFrom`, or the referenced secret alias is declared `required: false` so the key can be missing at runtime. | Declare the encryption key alias with `required: true` so a missing key fails closed at startup. |
| `[E135] ... Production backup policy "<class>" must declare restoreVerification` | A production backup policy has no `restoreVerification` block. | Add `restoreVerification.frequencyDays` to every production backup policy.|
| `[E136] ... Reverse-proxy route host ...` (unknown target / no resolvable http-service endpoint / backend address not a rendered compose service / entrypoint not declared by the proxy / undeclared certificate alias / duplicate route host) | A `config.routes` entry on a reverse-proxy provider names a module that is missing, does not provide an `http-service` contract with host/port (or a port outside 1-65535), advertises a backend address that matches no rendered compose service (multi-service modules must advertise `serviceName`), names a `<module-id>.<contract-name>` contract the target does not provide, names an entrypoint the proxy's `config.entryPoints` does not declare, references an undeclared certificate alias, or duplicates the host of an earlier route on the same proxy (#581). | Fix the route's `target`/`certificate`/`host`/`entryPoints` (a bare module id routes to its first http-service contract; `<module-id>.<contract-name>` picks a specific one; entrypoints must match the proxy's declared `config.entryPoints` keys), or advertise the serving compose service via `serviceName` in the target's http-service provide entry. |
| `[E150] ... Provided contract ... advertises network host ...` | A module's provided `http-service`, `sql-database`, `cache-service`, or `object-storage` contract advertises a network host (the `host` field, the `endpoint`'s hostname part, or a `serviceName` mapped through the renderer's service-naming rule) that matches no compose service the render produced for that module, so consumers would resolve a hostname nothing answers (#1010). | Instantiate the module with an id matching its compose service name, advertise the serving service via a `serviceName` field in the provide entry, or build the endpoint from `${service.dns}`. |
| `[E103] ... config.image.tag is required ... when config.image.source is "registry"` | A module instance sets `config.image.source: registry` without also setting `config.image.tag`. | Set `config.image.tag` to a version, or switch back to `config.image.source: build`. |
| `[E139] ... without an explicit host authentication advertisement` / `... host authentication method "trust"` | A `metadata.environment: production` profile runs a `sql-database` provider whose contract does not advertise a host authentication method, or advertises `trust` (issue #206). | Configure the provider's authentication explicitly (e.g. postgres `config.authentication.hostAuth: scram-sha-256`); trust authentication is prohibited in production profiles. |
| `[E140] ... configures webServer.mode "development"` | A production profile sets an HTTP-serving module's `config.webServer.mode` to `development`, selecting its built-in development web server (issue #216). | Set `config.webServer.mode` to `production` (or remove the override) so the service runs behind its production process manager; see `docs/serving-modes.md`. |
Expand Down
3 changes: 1 addition & 2 deletions cli/k8s_renderer.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@
from .diagnostics import Diagnostic
from .renderer import (
_build_context,
_compose_service_name,
_merge_init_db_env,
_resolve_expr,
_resolve_profile_dir,
Expand All @@ -45,7 +44,7 @@
declared_secret_env_names,
resolved_secret_values,
)
from .utils import _atomic_write
from .utils import _atomic_write, _compose_service_name
from .validator import validate_plan_marker_shapes

CHART_API_VERSION = "v2"
Expand Down
26 changes: 24 additions & 2 deletions cli/planner.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
secret_name_from_ref,
)
from .secrets import load_profile_secrets
from .utils import module_dns_host, module_name_from_source


@dataclass
Expand Down Expand Up @@ -616,11 +617,29 @@ def resolve_provided_contracts(
) -> dict[str, Any]:
"""
Resolve contracts provided by a module instance.

The `service` substitution context exposes two names (issue #1010):
`${service.host}` is the module instance id (stable for name-prefix
use such as backup volume names), while `${service.dns}` is the
Compose DNS name the module's serving service answers to -- the
renderer's compose service name (`<id>` when the service already
carries the id prefix, `<id>-<service>` otherwise) for a module with
a single compose service or a namesake service matching the module's
own name (e.g. `postgres` alongside its backup job containers), so
contract endpoints resolve on the Compose network even when the
instance id differs from the service name. Other multi-service
modules have no single DNS name; both placeholders keep the instance
id and their serving service is named explicitly via the contract's
`serviceName` field.
"""
secrets = secrets if secrets is not None else {}
provides = inst["module"].get("spec", {}).get("provides", [])
resolved: dict[str, Any] = {}
service_name = inst["id"]
implementation = (inst["module"].get("spec", {}) or {}).get("implementation", {}) or {}
compose_impl = implementation.get("compose") if isinstance(implementation, dict) else None
services = compose_impl.get("services") if isinstance(compose_impl, dict) else None
service_host = str(inst["id"])
module_name = str((inst["module"].get("metadata") or {}).get("name") or module_name_from_source(inst.get("source")))

for provided in provides:
provide_name = provided.get("name")
Expand All @@ -630,7 +649,10 @@ def resolve_provided_contracts(
contract,
context={
"config": inst["config"],
"service": {"host": service_name},
"service": {
"host": service_host,
"dns": module_dns_host(service_host, services, module_name),
},
"secrets": secrets,
"bindings": {},
},
Expand Down
121 changes: 112 additions & 9 deletions cli/renderer.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
from copy import deepcopy
from pathlib import Path
from typing import Any
from urllib.parse import urlsplit

import yaml

Expand All @@ -28,7 +29,12 @@
declared_secret_env_names,
resolved_secret_values,
)
from .utils import _atomic_write
from .utils import (
_atomic_write,
_compose_service_name,
module_dns_host,
module_name_from_source,
)
from .validator import validate_plan_marker_shapes


Expand Down Expand Up @@ -215,6 +221,8 @@ def render_compose(
if not compose["volumes"]:
compose.pop("volumes")

diagnostics.extend(_validate_provided_contract_hosts(plan, module_service_names))

_add_cross_module_dependencies(compose, plan, module_service_names)

output = yaml.safe_dump(compose, sort_keys=False)
Expand Down Expand Up @@ -252,6 +260,99 @@ def _is_valid_backend_port(value: Any) -> bool:
return isinstance(value, int) and 1 <= value <= 65535


_DNS_LABEL_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_-]*$")


def _endpoint_hostname(value: Any) -> str | None:
"""
Extract the network hostname a contract endpoint advertises.

Bare values are hostnames; URL-shaped endpoints contribute only their
hostname part. Anything else (external DNS names containing dots,
host:port pairs, empty values) is outside the Compose network's
namespace and cannot be cross-checked against rendered services.
"""
if not isinstance(value, str) or not value:
return None
if "://" in value:
try:
return urlsplit(value).hostname
except ValueError:
return None
return value


def _validate_provided_contract_hosts(
plan: dict[str, Any],
module_service_names: dict[str, list[str]],
) -> list[Diagnostic]:
"""
Cross-check every provided contract's advertised network host against
the compose services the render actually produced (issue #1010).

Mirrors the E136 route check: a contract advertising a host nothing
answers on the Compose network fails closed with E150 instead of
shipping a broken endpoint. Multi-service modules advertise their
serving service via `serviceName`; that name is mapped through the same
service-naming rule the renderer uses. Hosts outside the Compose
namespace (dots, ports, credentials) are not checked, and modules
whose services failed to render are left to their own diagnostics.
"""
diagnostics: list[Diagnostic] = []
for module in plan.get("modules", []):
if not isinstance(module, dict):
continue
module_id = str(module.get("id"))
rendered = module_service_names.get(module_id) or []
if not rendered:
continue
provides = module.get("provides")
if not isinstance(provides, dict):
continue
for provide_name, provided in provides.items():
if not isinstance(provided, dict):
continue
kind = provided.get("kind")
spec = provided.get("spec")
if not isinstance(spec, dict):
continue
candidates: list[tuple[str, str | None]] = []
if kind == "http-service":
advertised_service = spec.get("serviceName")
if isinstance(advertised_service, str) and advertised_service:
candidates.append(
("serviceName", _compose_service_name(module_id, advertised_service))
)
else:
candidates.append(("host", spec.get("host")))
elif kind in ("sql-database", "cache-service"):
candidates.append(("host", spec.get("host")))
elif kind == "object-storage":
candidates.append(("endpoint", _endpoint_hostname(spec.get("endpoint"))))
for field, host in candidates:
if not isinstance(host, str) or not _DNS_LABEL_RE.match(host):
continue
if host in rendered:
continue
diagnostics.append(Diagnostic(
level="error",
code="E150",
message=(
f'Provided contract "{provide_name}" of module "{module_id}" '
f'advertises network host "{host}", which is not a compose '
"service the render produced for that module (rendered "
f'services: {", ".join(rendered)}); consumers of the contract '
"would resolve a hostname nothing answers. Instantiate "
"the module with an id matching its compose service "
"name, advertise the serving service via a "
"serviceName field, or build the endpoint from "
"${service.dns}."
),
path=f"module:{module_id}.provides.{provide_name}.spec.{field}",
))
return diagnostics


def _generate_reverse_proxy_routes(
plan: dict[str, Any], module_service_names: dict[str, list[str]]
) -> tuple[dict[str, dict[str, Any]], list[Diagnostic]]:
Expand Down Expand Up @@ -803,10 +904,19 @@ def _build_context(
spec = contract.get("spec", {})
bindings[consume_name] = spec if isinstance(spec, dict) else {}

implementation = module.get("implementation") or {}
compose_impl = implementation.get("compose") if isinstance(implementation, dict) else None
services = compose_impl.get("services") if isinstance(compose_impl, dict) else None
module_id = str(module.get("id"))
module_name = module_name_from_source(module.get("source"))

return {
"config": module.get("config", {}),
"bindings": bindings,
"service": {"host": module.get("id")},
"service": {
"host": module_id,
"dns": module_dns_host(module_id, services, module_name),
},
"secrets": secrets,
"certificates": certificates,
}
Expand Down Expand Up @@ -1459,13 +1569,6 @@ def _is_named_volume(value: str) -> bool:
return "/" not in value and "\\" not in value


def _compose_service_name(module_id: str, service_name: str) -> str:
"""Normalize compose service names so module prefixes are not duplicated."""
if service_name == module_id or service_name.startswith(f"{module_id}-"):
return service_name
return f"{module_id}-{service_name}"


def _add_cross_module_dependencies(
compose: dict[str, Any],
plan: dict[str, Any],
Expand Down
3 changes: 2 additions & 1 deletion cli/security.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
from .diagnostics import Diagnostic
from .loader import load_yaml_file, resolve_module_file
from .planner import build_plan
from .renderer import _compose_service_name, render_compose
from .renderer import render_compose
from .resolver import parse_contract_ref
from .secret_leak import (
declared_secret_env_names,
Expand All @@ -31,6 +31,7 @@
)
from .secrets import load_secrets_from_env
from .security_common import SECRET_KEY_RE, SEVERITY_ORDER, infer_profile_class
from .utils import _compose_service_name

_PROFILE_SCOPES = {
"profile",
Expand Down
37 changes: 37 additions & 0 deletions cli/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import tempfile
from contextlib import suppress
from pathlib import Path
from typing import Any


def _atomic_write(path: Path, content: str, encoding: str = "utf-8", overwrite: bool = True) -> None:
Expand Down Expand Up @@ -33,3 +34,39 @@ def _atomic_write(path: Path, content: str, encoding: str = "utf-8", overwrite:
finally:
with suppress(OSError):
os.unlink(tmp_name)


def _compose_service_name(module_id: str, service_name: str) -> str:
"""Normalize compose service names so module prefixes are not duplicated."""
if service_name == module_id or service_name.startswith(f"{module_id}-"):
return service_name
return f"{module_id}-{service_name}"


def module_name_from_source(source: Any) -> str:
"""The module name a `source` path points at (its last path segment)."""
return str(source or "").rstrip("/").rsplit("/", 1)[-1]


def module_dns_host(module_id: str, compose_services: Any, module_name: str | None = None) -> str:
"""
The Compose DNS name a module instance's serving service answers to.

The renderer names each compose service `_compose_service_name(module_id,
service)`. A module whose compose services include exactly one service --
or a namesake service matching the module's own name (e.g. `postgres`
alongside its `postgres-backup`/`postgres-restore` job containers) --
therefore renders its serving service under that rule, and endpoints
advertised via `${service.dns}` resolve on the Compose network even when
the instance id differs from the service name (issue #1010). Other
multi-service modules have no single answer and keep the instance id;
they name their serving service explicitly via the contract's
`serviceName` field, which route generation and the E150 host check map
through the same naming rule.
"""
if isinstance(compose_services, dict):
if len(compose_services) == 1:
return _compose_service_name(module_id, next(iter(compose_services)))
if module_name and module_name in compose_services:
return _compose_service_name(module_id, module_name)
return module_id
Loading
Loading