Add sentry error tracking

[arobson-ods]

Overview

Errors are now reported to [Sentry](https://sentry.io), covering both halves of #90: unexpected
exceptions and RuntimeErrors are streamed, and the ERROR cases raised in the SuiteCRM records whose reporting org does not exist are grouped so that repeat occurrences produce one issue rather than one per record.

What changed

Error reporting — src/config/sentry.py (new).
initialise_sentry is called from src/iati_bulk_data_service.py before anything else in startup which can fail, so failures during the rest of startup are reported. Every event carries the app version as its release, the environment, and a bds.operation tag naming which of checker / zipper / registry-changes-processor it came from, since the three run as separate containers reporting to one project.

Caught exceptions now reach Sentry: checker.py, registry_changes_processor.py.
The service loops deliberately swallow exceptions and retry. They logged the exception as an interpolated string and, in the checker, the traceback as a second log line. This means Sentry would have received two message-only events with no exception type and no stack frames, grouped on text which embedded the varying error message. They now use logger.exception, which yields one typed event with real frames.

Per-record SuiteCRM errors group by class of alert: iati_registry_suitecrm.py,src/config/sentry.py.
Sentry groups events made from log messages on their message text, and these messages name the record they are about, so fifty inconsistent records meant fifty issues and fifty emails. These errors are marked with extra={"bds_alert_group": "..."} and a before_send hook turns that into a fingerprint plus a bds.alert_group tag.

Credentials are withheld from events — src/config/sentry.py. See Architectural decisions.

Configuration — three variables (SENTRY_DSN, SENTRY_ENVIRONMENT,
SENTRY_TRACES_SAMPLE_RATE), classified in config.py, added to .env-example, the ARM template, the deploy workflow and the manual-deploy examples.

Tooling — [tool.ruff] line-length = 119 in pyproject.toml. Ruff is not project tooling, but the Ruff editor extension is widely installed and defaults to 88. This can cause churn in import statements which can be quite tricky to pin down.

Architectural decisions and scope

No Sentry API is called from application code. The only module which imports sentry_sdk is
src/config/sentry.py. Calls are expressed in standard-library terms - logger.exception(...) and extra={"bds_alert_group": ...} - and the adapter translates that into Sentry's model. This is why logger.exception was preferred over sentry_sdk.capture_exception(e): both produce identical events, and only one couples the app to a vendor.

Stack-frame variables are not sent at all (include_local_variables=False).
This is the significant decision in the PR and it fixes a real leak found in review: Sentry attaches a copy of every variable in scope in every frame, and psycopg.connect() assembles its keyword arguments into one string containing password=... held in a local named conninfo. So a database connection failure would send the password to Sentry in a variable unknown to the event scrubber. Setting this one option closes every form the credentials take, not just that one: the connection string built for yoyo, both Azure connection strings, and Azure's account_key, whose name is not in Sentry's own denylist either.

This is a trade-off with a significant cost: tracebacks keep every frame's file, line, function and source line, but lose the values of variables so a failure processing a dataset no longer shows the record. The log breadcrumbs attached to each event name the dataset being processed, which mitigates it.

Query strings of outgoing requests are withheld. Sentry records them with values intact. An Azure storage connection string using a SharedAccessSignature= rather than an AccountKey= puts that signature in the query string of every blob request, and the environments vary, so before_breadcrumb and before_send_transaction filter it either way. Method, base URL and status code are kept.

Database connection metadata in exception text is accepted, deliberately. psycopg's message names the host, port, user and database. Nothing can scrub exception messages. Those four are marked SECRET out of caution rather than because they are credentials, and suppressing them would make a database outage harder to diagnose. DB_PASS does not appear there. The README records this, along with the other things error reporting does not protect.

Out of scope

• Event volume — the registry processor's catch-all moved from warning to error inside a loop retrying every 15s, so a lasting fault is roughly 5,700 events a day. Better handled with a Sentry per-key rate limit than by reverting the level.
• iati_registry_suitecrm.py:85 passes 1 where libsuitecrm expects str. Harmless at runtime; unrelated.
• Reclassifying DB_HOST/DB_PORT/DB_USER/DB_NAME as LOGGABLE, so the policy matches the
  decision above. Not done here because it would also add them to the startup log.

Testing

Beyond the unit and integration tests, the behaviour was verified against the real SDK.

• A real error was confirmed in Sentry. A genuine unhandled exception raised through the app's own entry point was accepted by Sentry's ingest (HTTP 200) and confirmed in the UI.
• The password leak was reproduced and then shown fixed. Against a local endpoint standing in for Sentry, the canary password appeared at $.exception.values[0].stacktrace.frames[2].vars.conninfo; with the fix it appears nowhere.
• Every SECRET variable was checked with unique sentinel values. None reached the payload.
• The query-string filter was verified in both directions — without the hook the signature is transmitted verbatim, with it the value reads [Filtered] while method, URL and status remain.

Notes to reviewer

Things worth checking specifically:

1. The frame-variables trade-off. In order to mitigate risk of third-party libraries leaking secrets we lose potentially valuable debugging information. Considered scrubbing the payload of the secrets themselves rather than by variable name but a) SuiteCRM Oauth access token fetched at runtime b) secret may be encoded - requests-oauthlib sends the client id and secret as Basic base64(id:secret) c) felt inherently error-prone as a method. Possible enhancement: make include_local_variables configurable so at least we can get the full debugging info in the dev environment.
2. The four alert groups in iati_registry_suitecrm.py. The comment on Add Sentry error streaming #90 named one case; all
   four per-record errors interpolate a record identifier and so fragment identically, so all four are marked.
3. registry_changes_processor.py's catch-all moved from warning to error. Previously an unexpected error in that loop would not have reached Sentry at all. It will now create issues. The two transient handlers (ServiceBusConnectionError, MessagingEntityNotFoundError) are deliberately left at warning, being expected reconnection conditions rather than faults.
4. [tool.ruff] in pyproject.toml is config for a tool the project does not use and while it will be ignored if ruff isn't installed adding it is perhaps a bit questionable.

Before this is merged

• Add an alert rule in Sentry. bds.alert_group exists so a rule can target a class of alert rather than message text. Worth setting an interval, since a persistent fault re-reports every cycle.
• Also configure the server-side scrubbing rules the README describes, and consider a per-key rate limit given the volume note above.

[simon-20]

This is great @arobson-ods, all the right calls on the scrubbing I think. It'll be a huge improvement over what we have, and if the missing info really turns out to be problematic we can revisit.

The only very minor thing:

• I think we should remove the ruff configuration, to avoid confusion. If it's the ruff VS Code extension that this is designed to placate, then it is possible to turn that off per workspace.

[arobson-ods]

Many thanks @simonk. Of course you are right. I overreacted to trying to fix what looked like an isort problem and finding it was ruff.