Skip to content

feat: add optional message telemetry hooks for OpenTelemetry integration - #442

Open
gibson9583 wants to merge 5 commits into
OpenIntegrationEngine:mainfrom
gibson9583:feature/opentelemetry-minimal-engine-release
Open

gibson9583 wants to merge 5 commits into
OpenIntegrationEngine:mainfrom
gibson9583:feature/opentelemetry-minimal-engine-release

Conversation

@gibson9583

@gibson9583 gibson9583 commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds an optional message telemetry bridge to the engine so an extension can observe message processing and maintain trace context across destination-chain and JavaScript worker handoffs.

This provides the engine integration points needed by the OpenTelemetry plugin. SDK setup, exporters, sampling, configuration and durable trace-continuity policy remain extension responsibilities.

Closes #432

Changes

  • Add a MessageTelemetry.Provider contract with observations for source processing, destination attempts, filtering/transformation, sending and response transformation.
  • Transfer context across destination-chain and JavaScript executors, activating it only when work starts and restoring the worker's previous context when it finishes.
  • Add an ingress callback before the source map becomes read-only and is first persisted, allowing a provider to prepare private propagation metadata.
  • Carry optional dispatch context through native source-to-destination and destination-to-destination handoffs. The context uses a transient, owner-keyed slot on ConnectorMessage, with compare-and-set publication to prevent older or reentrant callbacks from overwriting a newer reservation.
  • Keep destination observations open through retry handling and engine cleanup, including DAO/queue cleanup attempts and status-lock release.
  • Document the provider contract, lifecycle boundaries and behavior matrix.

Compatibility and failure handling

  • No new runtime dependencies or database schema changes.
  • With no provider installed, observation hooks are no-ops and executor wrapping returns the original task.
  • Dispatch context is excluded from Java and XML serialization; the existing Java serialization identity is preserved. Providers must keep this context bounded, immutable and free of message content or live SDK resources.
  • Ordinary provider callback failures are isolated from message processing and produce one fixed warning per installation, without exception content. VM errors and ThreadDeath retain fatal handling.
  • Provider registration allows one active installation; closing an old registration cannot detach a newer provider.

Validation

Regression coverage includes synchronous and queued processing, parallel destinations, retries, persisted PENDING messages, filtering and transformation failures, callback failures, cleanup ordering, executor rejection/cancellation, and JavaScript context restoration.

Expose process, transform, send and response scopes with context
transfer at destination and JavaScript worker handoffs. Preserve
engine task execution, errors, cancellation and queue behavior.

Validate real channel paths, failures and fixture cleanup with
687 engine tests and independent adversarial review.

Signed-off-by: gibson9583 <cgibson@outlook.com>
Offer an optional resource-free source-map callback before the first
connector map is persisted, preserving dispatch and recovery behavior.

Signed-off-by: gibson9583 <cgibson@outlook.com>
Retain per-destination failure and retry context until engine cleanup
finishes. Preserve existing execution, first fatal evidence and callback
isolation across source chains and queued attempts.

Signed-off-by: gibson9583 <cgibson@outlook.com>
Let a registered telemetry provider prepare an opaque, bounded proof
before the source map is first persisted and receive it again when each
later stage starts. The proof lives in a transient, owner-keyed slot on
ConnectorMessage: it never retains a message, application map,
Throwable, SDK or provider, does not enter Java or XML serialization,
and cannot be read or forged through an unrelated key. Native dispatch
copies only that slot to destination messages in Channel and
DestinationChain. Providers that do not declare the capability keep
their original callbacks unchanged.

Cover ownership, replacement, reentrancy, failure retirement and
serialization exclusion with fourteen isolated dispatch tests.

Signed-off-by: gibson9583 <cgibson@outlook.com>
Replace the monitor on the ConnectorMessage telemetry slot with a
VarHandle over the existing transient volatile field. Reserve is a
plain volatile replacement; completion installs its value only when
the slot still holds that reservation. The pre-allocation clear is
dropped: an allocation failure on a message thread errors that message
and the object is never dispatched again, so a stale slot cannot be read.
No per-message allocation is added and the serial form is unchanged.

Signed-off-by: gibson9583 <cgibson@outlook.com>
@github-actions

Copy link
Copy Markdown

Test Results

132 files  + 9  132 suites  +9   3m 24s ⏱️ + 1m 35s
766 tests +76  766 ✅ +76  0 💤 ±0  0 ❌ ±0 
778 runs  +76  778 ✅ +76  0 💤 ±0  0 ❌ ±0 

Results for commit d5799f2. ± Comparison against base commit 9359d9a.

@jonbartels

Copy link
Copy Markdown
Contributor

@gibson9583 - I have not reviewed yet, but one of the first things that jumps out at me is that the PR needs links or docs for:

  • wheres the plugin?
  • Why can't the plugin add its own endpoints instead of modifying core OIE?
  • what data elements are exposed?
  • are there docker-compose files, integration tests, etc that would be useful for testing or demonstrating what this does?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[IDEA] Add support for telemetry plugins: a message lifecycle listener SPI in donkey

2 participants