Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,20 @@ AZURE_COMMUNICATION_SERVICE_CONNECTION_STRING=""
# ]
CLIENT_APPLICATION_DETAILS_FILE="path/to/client/applications/metadata/file.json"

# This file lists the origins that browser based applications, such as the IATI
# Dashboard, may call this API from. It should be a JSON list of origins. Each origin
# must be of the form scheme://host[:port] - lower case, with no trailing slash and no
# path - because the browser sends the Origin header in exactly that form and it is
# matched by string equality. Wildcards are not allowed.
# Example:
# [
# "https://dev-dashboard.iatistandard.org",
# "http://localhost:5173"
# ]
# If this variable is omitted the API allows no cross-origin requests at all, which is
# the correct setting for a deployment that only serves server-to-server clients.
# CORS_ALLOWED_ORIGINS_FILE="path/to/cors/allowed/origins/file.json"

DATA_REGISTRY_SUITECRM_API_URL = "https://base_url_of_suitecrm_instance.org"

DATA_REGISTRY_SUITECRM_CLIENT_ID = "RYD's client ID here"
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,5 @@ __pycache__/

# all key files
*.pem

/cors-allowed-origins.json
28 changes: 18 additions & 10 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

### Changed

### Deprecated

### Fixed

### Removed

### Security

## [0.3.9] - 2026-09-22

### Added

- Sentry error monitoring and request tracing, initialised in `src/main.py` before the
FastAPI application is created. Configured by the optional `SENTRY_DSN`,
`SENTRY_ENVIRONMENT`, and `SENTRY_TRACES_SAMPLE_RATE` environment variables; when no
Expand All @@ -19,16 +33,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
entirely, since its records are written at `CRITICAL` and carry the contents of the
`Authorization` header, and a failed startup is now logged rather than printed so that
it is reported as well.

### Changed

### Deprecated

### Fixed

### Removed

### Security
- CORS support, so that browser based applications such as the IATI Dashboard can call the
API. The origins that are allowed to do so are listed in a JSON file named by the new,
optional, `CORS_ALLOWED_ORIGINS_FILE` environment variable. If it is not set, no
cross-origin requests are allowed.

## [0.3.8] - 2026-07-01

Expand Down
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ The application is configured using a set of environment variables in a `.env` f
| `SENTRY_DSN` | DSN of the Sentry project to report errors to. **Optional** — leave empty or unset and Sentry is not initialised, and the application runs normally. |
| `SENTRY_ENVIRONMENT` | Environment name that events are tagged with in Sentry, e.g. `"dev"` or `"prod"`. Defaults to `"local-development"` rather than the SDK's own default of `"production"`, so that an unconfigured environment can never be mistaken for the live one. |
| `SENTRY_TRACES_SAMPLE_RATE` | Proportion of requests traced for performance monitoring, `0.0`–`1.0`. Defaults to `1.0`; lower it (e.g. `0.1`) if trace volume becomes a problem. |
| `CORS_ALLOWED_ORIGINS_FILE` | Path to a JSON file listing the origins that browser based applications, such as the IATI Dashboard, may call this API from. Omit it to allow no cross-origin requests at all. See `cors-allowed-origins.example.json` and the notes in `.env.example` for the required format. |

### Error monitoring with Sentry

Expand Down Expand Up @@ -169,6 +170,7 @@ Running the API in production can be run directly from its Docker container with
```
docker run \
--mount type=bind,source=.env,target=/api/.env,readonly \
--mount type=bind,source=cors-allowed-origins.json,target=/api/cors-allowed-origins.json,readonly \
--mount type=bind,source=./logs/,target=/api/logs/ \
--mount type=bind,source=./keys/,target=/api/keys/,readonly \
-p 8000:8000 \
Expand All @@ -178,12 +180,13 @@ docker run \

This performs the following setup:
* `--mount type=bind,source=.env,target=/api/.env,readonly`: shares `.env` configuration file on the host with the container.
* `--mount type=bind,source=cors-allowed-origins.json,target=/api/cors-allowed-origins.json,readonly`: shares the CORS allowed origins file on the host with the container. Mounting it, rather than baking it into the image, means origins can be added by editing the file and restarting the container. Omit this mount if the deployment does not serve any browser based clients.
* `--mount type=bind,source=./logs/,target=/api/logs/`: allows the container to write logs to the /logs directory on the host.
* `--mount type=bind,source=./keys/,target=/api/keys/,readonly`: shares the `/keys` directory on the host with the container so public and private keys can be used by the container.
* `-p 8000:8000`: shares port 8000 for API traffic.
* `-p 9000:9000`: shares port 9000 for Prometheus metrics (assuming that this is the port as specified by the `.env` file.)

Care should be taken to make sure that the `.env` variables match the log (`APP_LOG_PATH` and `AUDIT_LOG_PATH`) and key directories (`AUDIT_LOG_PUBLIC_KEY_PATH`) and the Prometheus metric port (`PROMETHEUS_PORT`).
Care should be taken to make sure that the `.env` variables match the log (`APP_LOG_PATH` and `AUDIT_LOG_PATH`) and key directories (`AUDIT_LOG_PUBLIC_KEY_PATH`) and the Prometheus metric port (`PROMETHEUS_PORT`). If CORS is in use, `CORS_ALLOWED_ORIGINS_FILE` must also match the target path of the mount above.

## Development

Expand Down
3 changes: 3 additions & 0 deletions cors-allowed-origins.example.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[
"https://dev-dashboard.iatistandard.org"
]
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "register-your-data-api"
version = "0.3.8"
version = "0.3.9"
requires-python = ">= 3.12.11"
readme = "README.md"
authors = [{name="IATI Secretariat", email="support@iatistandard.org"}]
Expand Down
23 changes: 23 additions & 0 deletions src/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,27 @@
import prometheus_client
from fastapi import FastAPI

import register_your_data_api.cors as cors
import register_your_data_api.exception_handlers
import register_your_data_api.util as util
from register_your_data_api.routers import datasets, discoverable_reporting_orgs, misc, reporting_orgs, users
from register_your_data_api.sentry import setup_sentry

# Assigned at the bottom of this module and acted on in prod_lifespan; the comment on the
# assignment explains why those are two different places.
_cors_configuration_error: RuntimeError | None = None


@contextlib.asynccontextmanager
async def prod_lifespan(app: FastAPI) -> AsyncIterator[None]:
if _cors_configuration_error is not None:
# Logged rather than printed, and with the exception attached, for the same reasons
# as the context failure below.
logging.getLogger(__name__).error(
"Could not initialise application - error configuring CORS", exc_info=_cors_configuration_error
)
sys.exit("Could not startup")

try:
context = util.Context()
context.setup()
Expand Down Expand Up @@ -53,4 +66,14 @@ def add_routers_and_general_exception_handling(app: FastAPI) -> None:

app = FastAPI(title="Register Your Data", lifespan=prod_lifespan, redirect_slashes=False)

# Middleware has to be registered before the application starts, so this runs at import.
# Exiting here would take down every importer of this module - which includes the whole
# test suite, via tests/helpers/mocking.py - and SystemExit during collection aborts pytest
# without reporting a cause. The failure is therefore carried into prod_lifespan, where
# the equivalent context failure is already reported.
try:
cors.add_cors_middleware(app, cors.load_allowed_origins())
except RuntimeError as err:
_cors_configuration_error = err

add_routers_and_general_exception_handling(app)
32 changes: 32 additions & 0 deletions src/register_your_data_api/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
"""Configuration read straight from the environment.

Both Sentry and CORS have to be configured before the FastAPI application object is
created, which is earlier than the ``Context`` (created in the application lifespan)
exists. They therefore read the environment here rather than going through ``Context``,
using the same precedence that ``Context`` uses: values from the ``.env`` file in the
current directory, overridden by real environment variables.
"""

import os

import dotenv


def get_environment_config(env_file: str = ".env") -> dict[str, str]:
"""Read configuration with the same precedence as Context: the env file, then os.environ.

Parameters
----------
env_file : str
Path to the environment file. A missing file is not an error, in which case only
os.environ is used.

Returns
-------
dict[str, str]
"""

env: dict[str, str] = {key: value for key, value in dotenv.dotenv_values(env_file).items() if value is not None}
env.update(os.environ)

return env
132 changes: 132 additions & 0 deletions src/register_your_data_api/cors.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
"""Cross-Origin Resource Sharing (CORS) configuration.

The allowed origins have to be read before the FastAPI app object is handed to uvicorn,
because Starlette refuses to add middleware once an application has started - and that
includes the lifespan in which the Context is built. This module therefore reads the
environment directly rather than going through Context.
"""

import json
import re
from typing import Final

from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware

from .config import get_environment_config

CORS_ALLOWED_ORIGINS_FILE: Final[str] = "CORS_ALLOWED_ORIGINS_FILE"

# The verbs the API serves, and no others.
ALLOWED_METHODS: Final[list[str]] = ["GET", "POST", "PATCH", "PUT", "DELETE"]

ALLOWED_HEADERS: Final[list[str]] = ["Authorization", "Content-Type"]

# Scheme, host and an optional port, and nothing else. Browsers send an Origin header in
# exactly this form and CORSMiddleware compares it by string equality, so an entry with a
# trailing slash, a path, or an upper case host would silently never match. A wildcard is
# rejected by this pattern too: a single "*" entry would switch CORSMiddleware into
# allow-all mode, which is precisely what we are avoiding.
_ORIGIN_PATTERN: Final[re.Pattern[str]] = re.compile(r"https?://[a-z0-9.-]+(:[0-9]+)?")


def load_allowed_origins(env: dict[str, str] | None = None) -> list[str]:
"""Load the origins the API will accept cross-origin requests from.

Parameters
----------
env : dict[str, str] | None
Environment variables to read the configuration from. Defaults to the values
returned by get_environment_config().

Returns
-------
list[str]
The configured origins, or an empty list if CORS_ALLOWED_ORIGINS_FILE is not set,
in which case no cross-origin requests are allowed at all.

Raises
------
RuntimeError
If CORS_ALLOWED_ORIGINS_FILE is set but the file it names cannot be read, is not
valid JSON, or does not contain a list of well formed origins.
"""

if env is None:
env = get_environment_config()

filename = env.get(CORS_ALLOWED_ORIGINS_FILE, "").strip()

if not filename:
return []

origins = _read_origins_file(filename)

if not isinstance(origins, list):
raise RuntimeError(f"CORS allowed origins file {filename} must contain a JSON list of origins")

invalid_origins = [origin for origin in origins if not _is_valid_origin(origin)]

if invalid_origins:
raise RuntimeError(
f"CORS allowed origins file {filename} contains invalid origins: {invalid_origins}. Each must be "
"of the form scheme://host[:port], lower case, with no trailing slash and no path. Wildcards are "
"not allowed."
)

return origins


def add_cors_middleware(app: FastAPI, allowed_origins: list[str]) -> None:
"""Add CORS middleware to a FastAPI app instance.

Preflight OPTIONS requests are answered by this middleware before the router, and so
before the per-endpoint Security() dependencies, which is what stops browsers being
given a 401 for a preflight.

Note that CORS is enforced by the browser, not by us: a request from an origin that is
not allowed is still served as normal, it just comes back without the headers the
browser needs in order to hand the response to the calling script.

An empty list registers no middleware at all, so a deployment that has not opted into
CORS is left exactly as it was. Registering the middleware with an empty allowlist
would not be equivalent: it would still intercept preflights, answering OPTIONS with a
400 where the router would previously have returned 405.

Parameters
----------
app : FastAPI
allowed_origins : list[str]
Origins to accept cross-origin requests from. An empty list disables CORS.
"""

if not allowed_origins:
return

app.add_middleware(
CORSMiddleware,
allow_origins=allowed_origins,
allow_credentials=False,
allow_methods=ALLOWED_METHODS,
allow_headers=ALLOWED_HEADERS,
)


def _read_origins_file(filename: str) -> object:
"""Read and parse the JSON origins file, translating any failure into a RuntimeError."""

try:
with open(filename, "r") as file:
return json.load(file)
except OSError as err:
raise RuntimeError(f"Could not read CORS allowed origins file {filename}: {err}") from err
except ValueError as err:
# Covers json.JSONDecodeError and the UnicodeDecodeError raised when the file is
# not readable as text; both are ValueError subclasses.
raise RuntimeError(f"CORS allowed origins file {filename} is not valid JSON: {err}") from err


def _is_valid_origin(origin: object) -> bool:
"""Check that a single entry from the origins file is an origin we can match against."""

return isinstance(origin, str) and _ORIGIN_PATTERN.fullmatch(origin) is not None
12 changes: 2 additions & 10 deletions src/register_your_data_api/sentry.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@
"""

import importlib.metadata
import os
from typing import Final

import dotenv
import sentry_sdk
from sentry_sdk.integrations.logging import ignore_logger, ignore_logger_for_sentry_logs
from sentry_sdk.scrubber import DEFAULT_DENYLIST, EventScrubber

from .config import get_environment_config

# Sentry's SDK sends no traces at all unless a sample rate is set, so a default is
# supplied here rather than leaving it to the SDK
DEFAULT_TRACES_SAMPLE_RATE: Final[float] = 1.0
Expand Down Expand Up @@ -52,14 +52,6 @@
]


def get_environment_config() -> dict[str, str]:
"""Reads configuration with the same precedence as ``Context``: ``.env`` then os.environ."""

env: dict[str, str] = {k: v for k, v in dotenv.dotenv_values(".env").items() if v is not None}
env.update(os.environ)
return env


def get_release() -> str | None:
"""Builds the Sentry release identifier from the installed package version."""

Expand Down
19 changes: 18 additions & 1 deletion tests/helpers/mocking.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
)
from register_your_data_api.auth.fga.models import FineGrainedAuthorisationRole
from register_your_data_api.client_application_details_provider import ClientApplicationDetails
from register_your_data_api.cors import add_cors_middleware
from tests.helpers.keys import KeyDict

from ..helpers import prom
Expand Down Expand Up @@ -233,7 +234,21 @@ def _create_test_key(self, key_name: str) -> None:
public_key,
)

def get_test_app(self) -> FastAPI:
def get_test_app(self, cors_allowed_origins: list[str] | None = None) -> FastAPI:
"""Build (or return the already built) test app.

Parameters
----------
cors_allowed_origins : list[str] | None
Origins to configure CORS middleware with. By default no CORS middleware is
added at all, matching an API deployed without CORS_ALLOWED_ORIGINS_FILE set.
The app is built once and then cached, so the first call wins - a later call
passing different origins has no effect.

Returns
-------
FastAPI
"""

@contextlib.asynccontextmanager
async def test_lifespan(app: FastAPI) -> AsyncIterator[None]:
Expand All @@ -247,6 +262,8 @@ async def test_lifespan(app: FastAPI) -> AsyncIterator[None]:

if not self._app_is_created:
self._app = FastAPI(title="Register Your Data", lifespan=test_lifespan)
if cors_allowed_origins is not None:
add_cors_middleware(self._app, cors_allowed_origins)
add_routers_and_general_exception_handling(self._app)
self._app_is_created = True

Expand Down
Loading
Loading