diff --git a/CHANGELOG.md b/CHANGELOG.md index d9ddcd2..925cff2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,116 @@ All notable changes to this project are documented in this file. ## Unreleased +## 0.4.0 - 2026-09-01 + +- Closed three gaps found auditing Phases 1–3 against their own written + exit criteria (each had been marked "shipped" despite this): + - Phase 1: config loading from file/env. `load_config(dict)`, + `logger_from_file(path)` (JSON built in; YAML via the optional + `pip install logquill[yaml]`), and `logger_from_env(prefix= + "LOGQUILL_")` build a `Logger` from `{"name", "level", "transports": + [{"type"|"class", "options"}], "plugins": [...]}`. A small built-in + `"type"` registry covers the zero-dependency transports/plugins; + anything else (every cloud/SQL/NoSQL/queue transport, the alerting + plugins, framework adapters, your own subclass) goes through `"class"` + — a fully-qualified dotted path, resolved the same way `logging. + config.dictConfig` resolves one. `{prefix}LEVEL` in the environment + always overrides a config file's level. + - Phase 2: `FileTransport(encrypt_key=...)` encrypts each line with + `cryptography.fernet.Fernet` before writing — worth doing here + specifically because cloud transports typically already encrypt + server-side, but a local log file on disk usually doesn't. Optional + `pip install logquill[crypto]`, imported lazily. + - Phase 3: `SyslogTransport` — RFC 5424 messages over UDP (default) or + TCP, stdlib `socket` only, no dependency. Not a batching transport, + unlike the HTTP-API cloud transports: syslog is one-datagram/one- + message-per-call. +- Fixed a pre-existing crash the plugin-pipeline hypothesis property test + caught during this audit, unrelated to the three gaps above: any of + `Logger`'s message-taking methods (`.info()`, `.error()`, `.action()`, + ...) raised `TypeError: got multiple values for argument 'message'` if + the caller's `**meta` happened to contain a key literally named + `"message"` (and `.child()`/`.span()` had the same issue with `"name"`) + — exactly the kind of caller-crashing bug those hypothesis tests exist + to catch. `message`/`name` are now positional-only on every affected + method, so a `meta`/`fixed_meta` key with that exact name now flows + through as ordinary meta instead of colliding. +- Phase 5, trace correlation & agentic tracing, complete: + - `Logger.child(name, **fixed_meta)` — a namespaced logger sharing the + parent's transports, with its own plugin pipeline and optional fixed + context injected into every record. + - `.thought()/.action()/.observation()/.decision()` — `.info()` with + `meta.kind` pre-set, for tagging agent reasoning steps. + - `Logger.span(name)`, used as `with agent_log.span("call_llm"):` — + emits one record on exit carrying `meta.span_id`/`meta.duration_ms`; + every record logged inside the block (through any method) is + automatically stamped with `meta.parent_span_id`, so a full run + reconstructs its exact nesting by sorting on `span_id`/`parent_span_id`. + Still emits its record, at `ERROR` with `meta.error` set, if the block + raises — the exception itself propagates unchanged. + - `RunPlugin` — stamps `meta.run_id` (generated if not given) and an + incrementing `meta.step`; one instance scopes one run, so concurrent + runs never share a counter. + - `TraceContextPlugin` — stamps `meta.trace_id` for cross-service + correlation, distinct from `run_id`. Resolves an active OpenTelemetry + span's trace id first (best-effort, lazy import), then a W3C + `traceparent`/AWS X-Ray/GCP trace header propagated via the new + `set_traceparent()`/`reset_traceparent()` (a `contextvars`-based + per-thread/asyncio-task mechanism), and generates a fresh id only if + neither is available. + - `LogQuillAdapter` base class + `LangChainAdapter` + (`pip install logquill[langchain]`) — maps LangChain's + `BaseCallbackHandler` events onto the calls above; covers LangGraph + for free, since it shares LangChain's callback system. LangChain's own + `run_id`/`parent_run_id` are written directly onto + `meta.span_id`/`meta.parent_span_id`. `langchain-core` is never + imported unless `logquill.adapters.langchain` is imported explicitly. +- `CrewAIAdapter` (`pip install logquill[crewai]`) — a second + `LogQuillAdapter` implementation, ahead of the phase schedule (CrewAI was + listed as a Phase 5 follow-on, not required for that phase). Listens on + CrewAI's own event bus (`BaseEventListener`) rather than a single + callback handler; a crew kickoff and each task open/close a `.span()`, + while agent execution, tool usage, and LLM calls become `.action()`/ + `.observation()`/`.error()` pairs with `duration_ms`. Correlation reads + directly off CrewAI's own `event.parent_event_id`/`event.started_event_id` + (populated by CrewAI's own `contextvars`-backed scope stack) rather than + tracking anything independently — the same field-renaming approach + `LangChainAdapter` takes with LangChain's `run_id`/`parent_run_id`. + `crewai` is never imported unless `logquill.adapters.crewai` is imported + explicitly. +- `LlamaIndexAdapter` (`pip install logquill[llamaindex]`) — a third + `LogQuillAdapter` implementation. LlamaIndex's own instrumentation module + splits into two cooperating registrations on a shared dispatcher, so this + adapter holds one of each rather than being a handler itself: a span + handler for LlamaIndex's own method-level calls (`query()`, `chat()`, + `retrieve()`, ...), each becoming a `span_id`/`duration_ms` record with + `parent_span_id` set for a nested call; and an event handler for named + events fired *within* those calls, classified generically by class-name + suffix (`*StartEvent` -> `.action()`, `*EndEvent` -> `.observation()`, + `*ErrorEvent` -> `.error()`) rather than enumerated one by one, so a new + LlamaIndex event type needs no adapter change to show up correctly. + `llama-index-core` is never imported unless `logquill.adapters.llamaindex` + is imported explicitly. +- `AutoGenAdapter` (`pip install logquill[autogen]`) — a fourth + `LogQuillAdapter` implementation, rounding out every framework CLAUDE.md + names as a Phase 5 follow-on. Architecturally different from the other + three: (Microsoft) AutoGen's actual integration point is a stdlib + `logging.Handler` attached to `autogen_core.EVENT_LOGGER_NAME`, where + model clients and tools log structured event objects (not strings), so + the adapter is a `Handler` whose `emit()` unpacks that object rather than + a callback/event-bus registration. Each event becomes a flat + `.action()`/`.observation()`/`.error()` record; unlike the other three + adapters, AutoGen's structured events carry no call-level + `span_id`/`parent_span_id`-equivalent (only `agent_id`), so there's no + span tree to reconstruct here — documented as a real limitation, not + glossed over. Covers `autogen-core`/`autogen-agentchat` only — **not + AG2**, which forked from AutoGen and, as of its 2026 rewrite, moved onto + its own event-driven architecture sharing none of this (confirmed against + its source: zero references to `EVENT_LOGGER_NAME`); unlike LangGraph + sharing LangChain's callback system, this is a genuine divergence and AG2 + would need its own adapter. `autogen-core` is never imported unless + `logquill.adapters.autogen` is imported explicitly. + ## 0.3.0 - 2026-08-31 - Plugin pipeline, Phase 4 complete: `SamplingPlugin` gained tail-based diff --git a/README.md b/README.md index 01dcb33..d6df063 100644 --- a/README.md +++ b/README.md @@ -14,17 +14,19 @@ Sibling to [`logquill` on npm](https://www.npmjs.com/package/logquill) across a Python + Node stack. Status: pre-release, under active development. The core `Logger`, level -filtering, transports, and the plugin pipeline are implemented; -non-blocking async dispatch is not yet — see `CHANGELOG.md` for what's -landed so far. +filtering, transports, the plugin pipeline, and agentic/harness tracing are +implemented; non-blocking async dispatch is not yet — see `CHANGELOG.md` +for what's landed so far. ## Features - **Structured by default** — every call carries a `meta` dict, not just a message string - **Cross-language record shape** — identical JSON shape and level names/weights as [`logquill` on npm](https://www.npmjs.com/package/logquill) -- **Pluggable transports** — `ConsoleTransport` (colorized, stderr for errors), `FileTransport` (rotation), `HTTPTransport` (batched), plus SQL/NoSQL/message-queue/cloud-native sinks (see [Transports](#transports)); write your own by subclassing `Transport` +- **Pluggable transports** — `ConsoleTransport` (colorized, stderr for errors), `FileTransport` (rotation, optional encryption-at-rest), `HTTPTransport` (batched), `SyslogTransport` (RFC 5424, UDP/TCP), plus SQL/NoSQL/message-queue/cloud-native sinks (see [Transports](#transports)); write your own by subclassing `Transport` - **Pluggable formatters** — `JSONFormatter` out of the box; implement `format(record) -> str` for your own -- **Plugin pipeline** — `ContextPlugin`, `RedactPlugin` (by key), `PIIRedactPlugin` (by pattern), `SamplingPlugin` (with tail-based elevation), `TamperEvidentPlugin` (hash-chained logs), and `AlertingPlugin` (`SlackAlertPlugin`/`PagerDutyAlertPlugin`/`EmailAlertPlugin`, deduplicated) out of the box; a broken plugin can't crash logging; `.use()` also accepts a plain function, no subclassing required (see [Plugins](#plugins)) +- **Config from file/env** — `load_config(dict)`, `logger_from_file(path)` (JSON/YAML), `logger_from_env()` build a `Logger` from one config shape — see [Config](#config) +- **Plugin pipeline** — `ContextPlugin`, `RedactPlugin` (by key), `PIIRedactPlugin` (by pattern), `SamplingPlugin` (with tail-based elevation), `TamperEvidentPlugin` (hash-chained logs), `TraceContextPlugin` (cross-service trace correlation), and `AlertingPlugin` (`SlackAlertPlugin`/`PagerDutyAlertPlugin`/`EmailAlertPlugin`, deduplicated) out of the box; a broken plugin can't crash logging; `.use()` also accepts a plain function, no subclassing required (see [Plugins](#plugins)) +- **Agentic & harness tracing** — `.child()` loggers, `RunPlugin`, `.thought()/.action()/.observation()/.decision()`, `with agent_log.span(...)`, and framework adapters — `LangChainAdapter` (`pip install logquill[langchain]`, covers LangGraph for free), `CrewAIAdapter` (`pip install logquill[crewai]`), `LlamaIndexAdapter` (`pip install logquill[llamaindex]`), and `AutoGenAdapter` (`pip install logquill[autogen]`) — see [Agentic & harness tracing](#agentic--harness-tracing) - **Zero required runtime dependencies** — stdlib only; `aiohttp` is opt-in, for async HTTP - **Typed throughout** — `mypy --strict` clean on the public API - *(planned)* non-blocking async dispatch, `contextvars`-based context propagation — see `CHANGELOG.md` @@ -64,6 +66,61 @@ print(JSONFormatter().format(record)) # '{"timestamp":"2026-08-27T18:04:12.345Z","level":"INFO","logger":"app","message":"user signed up","meta":{"user_id":42,"plan":"pro"}}' ``` +## Config + +Build a `Logger` from a config dict, a JSON/YAML file, or the environment, +instead of wiring transports/plugins up by hand — the same shape across all +three: + +```python +from logquill import load_config + +logger = load_config({ + "name": "app", + "level": "INFO", + "transports": [{"type": "console"}], + "plugins": [{"type": "context", "options": {"service": "api", "env": "prod"}}], +}) +logger.info("ready") +``` + +```python +from logquill import logger_from_file + +logger = logger_from_file("config.json") # or config.yaml — needs `pip install logquill[yaml]` +``` + +```python +import os +from logquill import logger_from_env + +os.environ["LOGQUILL_CONFIG_FILE"] = "config.json" +os.environ["LOGQUILL_LEVEL"] = "DEBUG" # always overrides the file's level + +logger = logger_from_env() # prefix defaults to "LOGQUILL_" +``` + +A small built-in `"type"` registry covers the zero-dependency transports/ +plugins (`console`, `file`, `http`; `context`, `redact`, `sampling`, +`trace_context`, `run`, `pii_redact`, `tamper_evident`). Anything else — +every cloud/SQL/NoSQL/queue transport, the alerting plugins, a framework +adapter, or your own subclass — goes through `"class"` instead, a +fully-qualified dotted path resolved the same way `logging.config. +dictConfig` resolves one: + +```python +from logquill import load_config + +logger = load_config({ + "transports": [ + { + "class": "logquill.transports.cloud.datadog_transport.DatadogTransport", + "options": {"api_key": "..."}, + } + ], +}) +``` + ## Transports Attach transports to a `Logger` to actually write records somewhere. Each @@ -247,6 +304,48 @@ pauses sends until it elapses — dropping (not requeuing) any batch flushed during that window, since New Relic blocks the rest of that minute on a rate-limit breach anyway. +`SyslogTransport` sends each record as one RFC 5424 message over UDP +(default) or TCP — stdlib `socket` only, no dependency, and not a batching +transport (syslog is one-message-per-call, unlike the HTTP-API transports +above): + +```python +from logquill import Logger, SyslogTransport + +transport = SyslogTransport(host="syslog.internal", port=514, app_name="app") +logger = Logger("app", transports=[transport]) + +logger.error("payment webhook failed") +``` + +### Encryption-at-rest for file logs + +`FileTransport(encrypt_key=...)` encrypts each line with +`cryptography.fernet.Fernet` before writing — a local log file usually +isn't encrypted server-side the way a cloud sink already is: + +```python +from cryptography.fernet import Fernet +from logquill import FileTransport, Logger + +key = Fernet.generate_key() # store this somewhere safe — you need it to decrypt +transport = FileTransport("app.log", encrypt_key=key) +logger = Logger("app", transports=[transport]) + +logger.info("card charged", user_id=42) +logger.close() + +# decrypt back, one Fernet token per line +fernet = Fernet(key) +with open("app.log", "rb") as f: + for line in f: + print(fernet.decrypt(line.strip()).decode("utf-8")) +``` + +Needs the optional `cryptography` dependency (`pip install +logquill[crypto]`), imported lazily — `FileTransport` has zero +dependencies as long as `encrypt_key` stays unset. + ## Plugins Plugins hook into the pipeline around each log call: `before_log(record)` can @@ -390,6 +489,178 @@ Write your own destination by subclassing `AlertingPlugin` and implementing `send_alert(record, occurrences)`; thresholding, deduplication, and the never-block-the-caller behavior are all handled by the base class. +## Agentic & harness tracing + +`.child()` makes a namespaced logger that shares the parent's transports — +attach run-scoped plugins to it without touching the parent's pipeline. +`RunPlugin` stamps `meta.run_id` and an incrementing `meta.step`; the +`.thought()/.action()/.observation()/.decision()` convenience methods are +`.info()` with `meta.kind` pre-set, for tagging agent reasoning steps; and +`with agent_log.span(name):` stamps `meta.span_id`/`meta.duration_ms` on +exit, with every record logged inside the block automatically getting +`meta.parent_span_id` — so a full run reconstructs its exact order and +nesting by sorting on `run_id`/`step`/`span_id`/`parent_span_id`: + +```python +from logquill import CollectingTransport, Logger, RunPlugin + +sink = CollectingTransport() +log = Logger("app", transports=[sink]) +agent_log = log.child("agent").use(RunPlugin()) + +agent_log.thought("deciding what to do") +with agent_log.span("call_llm"): + agent_log.action("call the model") + agent_log.observation("got a response") +agent_log.decision("final answer ready") + +for record in sink.records: + print(record["meta"]["step"], record["meta"].get("kind"), record["message"]) +# 0 thought deciding what to do +# 1 action call the model +# 2 observation got a response +# 3 span call_llm +# 4 decision final answer ready +``` + +### Cross-service trace correlation + +`TraceContextPlugin` stamps `meta.trace_id` — distinct from `run_id`: +`trace_id` follows one request across services, `run_id` scopes one agent +run. It reads an active OpenTelemetry span's trace id first (if +`opentelemetry-api` is importable and a span is current), then an inbound +W3C `traceparent` / AWS X-Ray / GCP trace header — handed in via +`set_traceparent()` for the current thread/asyncio task, the way request +middleware would propagate one — and generates a fresh id only if neither +is available: + +```python +from logquill import Logger, TraceContextPlugin +from logquill.plugins.trace_context_plugin import reset_traceparent, set_traceparent + +# e.g. set once in HTTP middleware, from the inbound request's header +token = set_traceparent("00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01") +try: + logger = Logger("billing-service", plugins=[TraceContextPlugin()]) + record = logger.info("charged card") +finally: + reset_traceparent(token) + +assert record["meta"]["trace_id"] == "4bf92f3577b34da6a3ce929d0e0e4736" +``` + +### Framework adapters + +`LogQuillAdapter` is a thin base class for mapping a framework's own event +callbacks onto `.thought()/.action()/.observation()/.decision()` and +`.span()` — never a reimplementation of tracing logic per framework. +`LangChainAdapter` (covers LangGraph for free, since it shares LangChain's +callback system) ships behind the optional `langchain` extra: + +```bash +pip install logquill[langchain] +``` + +```python +from logquill import Logger, RunPlugin +from logquill.adapters.langchain import LangChainAdapter + +log = Logger("app") +handler = LangChainAdapter(log.child("agent").use(RunPlugin())) +llm = ChatOpenAI(callbacks=[handler]) # pass in like any other tracing handler +``` + +LangChain's own `run_id`/`parent_run_id` are written directly onto +`meta.span_id`/`meta.parent_span_id` — the shapes already match, so this is +field renaming, not translation. `langchain-core` is never imported unless +you import `logquill.adapters.langchain` yourself. + +`CrewAIAdapter` ships behind the optional `crewai` extra, listening on +CrewAI's own event bus rather than a single callback handler: + +```bash +pip install logquill[crewai] +``` + +```python +from logquill import Logger, RunPlugin +from logquill.adapters.crewai import CrewAIAdapter + +log = Logger("app") +listener = CrewAIAdapter(log.child("agent").use(RunPlugin())) # active as soon as it's constructed +crew = Crew(agents=[...], tasks=[...]) +crew.kickoff() +``` + +A crew kickoff and each task within it open/close a `.span()`; agent +execution, tool usage, and LLM calls become `.action()`/`.observation()`/ +`.error()` pairs carrying `duration_ms`. Same field-renaming approach as +`LangChainAdapter`: CrewAI's own event bus already threads +`event.parent_event_id` and, on every "ended" event, `event.started_event_id` +(the matching "started" event's id) through its own internal +`contextvars`-backed scope stack — those map directly onto +`meta.parent_span_id`/`meta.span_id`. `crewai` is never imported unless you +import `logquill.adapters.crewai` yourself. + +`LlamaIndexAdapter` ships behind the optional `llamaindex` extra: + +```bash +pip install logquill[llamaindex] +``` + +```python +from logquill import Logger, RunPlugin +from logquill.adapters.llamaindex import LlamaIndexAdapter + +log = Logger("app") +adapter = LlamaIndexAdapter(log.child("agent").use(RunPlugin())) # active as soon as it's constructed +index.as_query_engine().query("...") +``` + +LlamaIndex splits instrumentation into two cooperating pieces on its own +global dispatcher, so this adapter registers one of each rather than being a +handler itself: a span handler for LlamaIndex's own method-level calls +(`query()`, `chat()`, `retrieve()`, ...), which become `span_id`/ +`duration_ms` records with `parent_span_id` set for a nested call (e.g. +`retrieve()` inside `query()`); and an event handler for named events fired +*within* those calls (LLM calls, retrieval, synthesis, embedding, agent +steps), classified generically by name suffix (`*StartEvent` -> +`.action()`, `*EndEvent` -> `.observation()`, `*ErrorEvent` -> `.error()`) +rather than enumerated one by one, so a new LlamaIndex event type needs no +adapter change to show up correctly. `llama-index-core` is never imported +unless you import `logquill.adapters.llamaindex` yourself. + +`AutoGenAdapter` ships behind the optional `autogen` extra. Unlike the +other three, it's not a callback/event-bus registration — (Microsoft) +AutoGen's actual integration point is a stdlib `logging.Handler` attached +to `autogen_core.EVENT_LOGGER_NAME`, where model clients and tools log +structured event *objects* (not strings), so that's what this adapter is: + +```bash +pip install logquill[autogen] +``` + +```python +from logquill import Logger, RunPlugin +from logquill.adapters.autogen import AutoGenAdapter + +log = Logger("app") +adapter = AutoGenAdapter(log.child("agent").use(RunPlugin())) # active immediately +``` + +Each event becomes a flat `.action()`/`.observation()`/`.error()` record +carrying whatever fields AutoGen put on it (`agent_id`, token counts, tool +name/arguments/result, ...). Worth knowing before relying on it: unlike the +other three adapters, AutoGen's structured events carry no call-level +`span_id`/`parent_span_id`-equivalent, so there's no tree to reconstruct — +just per-event correlation via `agent_id`. **Covers (Microsoft) +`autogen-core`/`autogen-agentchat` only — not AG2.** AG2 forked from +AutoGen and, as of its 2026 rewrite, moved onto its own event-driven +architecture that no longer shares `EVENT_LOGGER_NAME` or any of these +event classes; that's a real divergence, not just a detail, so it needs its +own adapter rather than reusing this one. `autogen-core` is never imported +unless you import `logquill.adapters.autogen` yourself. + ## Development ```bash diff --git a/logquill/__init__.py b/logquill/__init__.py index d3799a7..c8e08f2 100644 --- a/logquill/__init__.py +++ b/logquill/__init__.py @@ -1,3 +1,5 @@ +from logquill.adapters.base import LogQuillAdapter +from logquill.config import load_config, logger_from_env, logger_from_file from logquill.formatter import Formatter, JSONFormatter from logquill.levels import Level, parse_level from logquill.logger import Logger @@ -8,9 +10,11 @@ from logquill.plugins.pii_redact_plugin import PIIRedactPlugin from logquill.plugins.plugin import FunctionPlugin, Plugin from logquill.plugins.redact_plugin import RedactPlugin +from logquill.plugins.run_plugin import RunPlugin from logquill.plugins.sampling_plugin import SamplingPlugin from logquill.plugins.slack_alert_plugin import SlackAlertPlugin from logquill.plugins.tamper_evident_plugin import TamperEvidentPlugin +from logquill.plugins.trace_context_plugin import TraceContextPlugin from logquill.records import LogRecord from logquill.transports.batching_transport import BatchingTransport from logquill.transports.cloud.app_insights_transport import AppInsightsTransport @@ -19,6 +23,7 @@ from logquill.transports.cloud.datadog_transport import DatadogTransport from logquill.transports.cloud.elasticsearch_transport import ElasticsearchTransport from logquill.transports.cloud.new_relic_transport import NewRelicTransport +from logquill.transports.cloud.syslog_transport import SyslogTransport from logquill.transports.console_transport import ConsoleTransport from logquill.transports.file_transport import FileTransport from logquill.transports.http_transport import HTTPTransport @@ -36,7 +41,7 @@ from logquill.transports.sql.sqlite_transport import SQLiteTransport from logquill.transports.transport import CollectingTransport, Transport -__version__ = "0.3.0" +__version__ = "0.4.0" __all__ = [ "AlertingPlugin", @@ -60,6 +65,7 @@ "JSONFormatter", "KafkaTransport", "Level", + "LogQuillAdapter", "LogRecord", "Logger", "MongoDBTransport", @@ -73,13 +79,19 @@ "RabbitMQTransport", "RedactPlugin", "RedisTransport", + "RunPlugin", "SQLLogRow", "SQLiteTransport", "SQSTransport", "SamplingPlugin", "SlackAlertPlugin", + "SyslogTransport", "TamperEvidentPlugin", + "TraceContextPlugin", "Transport", + "load_config", + "logger_from_env", + "logger_from_file", "parse_level", "__version__", ] diff --git a/logquill/adapters/__init__.py b/logquill/adapters/__init__.py new file mode 100644 index 0000000..455a8d1 --- /dev/null +++ b/logquill/adapters/__init__.py @@ -0,0 +1,3 @@ +from logquill.adapters.base import LogQuillAdapter + +__all__ = ["LogQuillAdapter"] diff --git a/logquill/adapters/autogen.py b/logquill/adapters/autogen.py new file mode 100644 index 0000000..87f9800 --- /dev/null +++ b/logquill/adapters/autogen.py @@ -0,0 +1,107 @@ +from __future__ import annotations + +import logging +from typing import Any + +try: + from autogen_core import EVENT_LOGGER_NAME # type: ignore[import-not-found] +except ImportError as exc: + raise ImportError( + "logquill.adapters.autogen requires the optional `autogen-core` " + "dependency — install with `pip install logquill[autogen]`." + ) from exc + +from logquill.adapters.base import LogQuillAdapter +from logquill.logger import Logger + +_ERROR_EVENT_NAMES = { + "MessageHandlerException": "message_handler_exception", + "AgentConstructionException": "agent_construction_exception", +} + + +class AutoGenAdapter(LogQuillAdapter, logging.Handler): + """Maps (Microsoft) AutoGen's structured event logging onto LogQuill calls. + + Deliberately not `LogQuillAdapter`-plus-a-callback-registry like + `LangChainAdapter`/`CrewAIAdapter`/`LlamaIndexAdapter` — AutoGen's + actual integration point for this is a stdlib `logging.Handler` + attached to `autogen_core.EVENT_LOGGER_NAME`: model clients and tools + log structured event *objects* (not strings) there via + `logging.getLogger(EVENT_LOGGER_NAME).info(SomeEvent(...))`, so this + adapter is a `Handler` whose `emit()` unpacks that object instead of + formatting it: + + from logquill import Logger, RunPlugin + from logquill.adapters.autogen import AutoGenAdapter + + log = Logger("app") + # active as soon as it's constructed + adapter = AutoGenAdapter(log.child("agent").use(RunPlugin())) + + **Only covers (Microsoft) `autogen-core`/`autogen-agentchat` — not + AG2.** AG2 forked from AutoGen and, as of its 2026 rewrite, moved onto + its own event-driven architecture (a "MemoryStream pub/sub event bus") + that no longer shares `autogen_core.EVENT_LOGGER_NAME` or any of the + event classes below (confirmed against the AG2 source: zero references + to `EVENT_LOGGER_NAME` in its repository). Unlike LangGraph sharing + LangChain's callback system, this is a real divergence, not a detail — + an AG2 adapter needs its own research and its own adapter, not this one + pointed at a different package name. + + **Weaker correlation than the other adapters, and worth knowing before + relying on it**: `autogen_core`'s structured events carry an `agent_id` + (or `sender`/`receiver` for message events) but no call-level + `span_id`/`parent_span_id`-equivalent — unlike LangChain's `run_id`/ + `parent_run_id`, CrewAI's `event_id`/`parent_event_id`, or LlamaIndex's + span ids. Each event here becomes a flat `.action()`/`.observation()`/ + `.error()` record carrying whatever fields AutoGen put on it; there's + no tree to reconstruct from `span_id`/`parent_span_id` the way there is + for the other three adapters. (AutoGen's *separate* native OpenTelemetry + tracing, via a `tracer_provider` passed to the agent runtime, does carry + real span hierarchy — pair it with `TraceContextPlugin`, which already + reads the active OTel span, if that's what you need; this adapter is + for the structured *event* stream specifically.) + + `autogen-core` is never imported unless you import + `logquill.adapters.autogen` yourself. + """ + + def __init__(self, agent_log: Logger) -> None: + LogQuillAdapter.__init__(self, agent_log) + logging.Handler.__init__(self) + self._event_logger = logging.getLogger(EVENT_LOGGER_NAME) + # AutoGen's own events are logged at INFO; a logger's effective + # level defaults to WARNING, which would otherwise filter every one + # of them out before `emit()` is ever called. + self._event_logger.setLevel(logging.INFO) + self._event_logger.addHandler(self) + + def close(self) -> None: + self._event_logger.removeHandler(self) + super().close() + + def emit(self, record: logging.LogRecord) -> None: + kwargs = getattr(record.msg, "kwargs", None) + if not isinstance(kwargs, dict): + return # not one of autogen_core.logging's structured event objects + event_type = kwargs.get("type") + meta: dict[str, Any] = {k: v for k, v in kwargs.items() if k != "type"} + + if event_type in _ERROR_EVENT_NAMES: + meta["error"] = meta.pop("exception", "") + self.log.error(_ERROR_EVENT_NAMES[event_type], **meta) + elif event_type == "LLMStreamStart": + self.log.action("llm_stream", **meta) + elif event_type == "LLMStreamEnd": + self.log.observation("llm_stream", **meta) + elif event_type == "LLMCall": + self.log.observation("llm_call", **meta) + elif event_type == "ToolCall": + self.log.observation("tool_call", **meta) + elif event_type == "Message": + self.log.action("message", **meta) + elif event_type == "MessageDropped": + self.log.observation("message_dropped", **meta) + else: + self.log.action(event_type or "autogen_event", **meta) diff --git a/logquill/adapters/base.py b/logquill/adapters/base.py new file mode 100644 index 0000000..6299241 --- /dev/null +++ b/logquill/adapters/base.py @@ -0,0 +1,21 @@ +from __future__ import annotations + +from logquill.logger import Logger + + +class LogQuillAdapter: + """Base class for framework tracing adapters. + + A concrete adapter subclasses this, holds a reference to the `Logger` to + forward events onto (`self.log`), and overrides only the events its + framework actually emits — translating them into `.thought()/.action()/ + .observation()/.decision()` and `.span()` calls. This is always meant to + be a thin mapping from the framework's native event shape onto + LogQuill's, never a reimplementation of tracing logic per framework. + + See `logquill.adapters.langchain.LangChainAdapter` for the reference + implementation. + """ + + def __init__(self, agent_log: Logger) -> None: + self.log = agent_log diff --git a/logquill/adapters/crewai.py b/logquill/adapters/crewai.py new file mode 100644 index 0000000..033a0f5 --- /dev/null +++ b/logquill/adapters/crewai.py @@ -0,0 +1,198 @@ +from __future__ import annotations + +from datetime import datetime +from typing import Any + +try: + from crewai.events import ( # type: ignore[import-not-found] + AgentExecutionCompletedEvent, + AgentExecutionErrorEvent, + AgentExecutionStartedEvent, + BaseEventListener, + CrewKickoffCompletedEvent, + CrewKickoffFailedEvent, + CrewKickoffStartedEvent, + LLMCallCompletedEvent, + LLMCallFailedEvent, + LLMCallStartedEvent, + TaskCompletedEvent, + TaskFailedEvent, + TaskStartedEvent, + ToolUsageErrorEvent, + ToolUsageFinishedEvent, + ToolUsageStartedEvent, + ) +except ImportError as exc: + raise ImportError( + "logquill.adapters.crewai requires the optional `crewai` dependency — " + "install with `pip install logquill[crewai]`." + ) from exc + +from logquill.adapters.base import LogQuillAdapter +from logquill.logger import Logger +from logquill.span import SpanContext + + +def _span_ids(event: Any) -> dict[str, Any]: + ids: dict[str, Any] = {"span_id": event.event_id} + if event.parent_event_id is not None: + ids["parent_span_id"] = event.parent_event_id + return ids + + +def _closing_ids(event: Any) -> dict[str, Any]: + ids: dict[str, Any] = {"span_id": event.started_event_id or event.event_id} + if event.parent_event_id is not None: + ids["parent_span_id"] = event.parent_event_id + return ids + + +# `type: ignore[misc]` — same reason as `LangChainAdapter`: `BaseEventListener` +# types as `Any` whenever `crewai` isn't installed in the environment running +# mypy (it's optional, never in this project's `dev` extra — see +# pyproject.toml), and mypy refuses to let a class subclass something typed +# `Any`. With the real package installed, this subclasses the genuine +# `BaseEventListener` and the ignore is inert. +class CrewAIAdapter(LogQuillAdapter, BaseEventListener): # type: ignore[misc] + """Maps CrewAI's event-bus events onto LogQuill calls. + + Unlike `LangChainAdapter`, correlation doesn't rely on any ambient state + this library tracks — CrewAI's own event bus already threads + `event.parent_event_id` (this event's logical parent) and, on every + "ended" event, `event.started_event_id` (the matching "started" event's + id) through a `contextvars`-backed scope stack internal to CrewAI. Those + are used directly as `span_id`/`parent_span_id`, the same way + `LangChainAdapter` uses LangChain's `run_id`/`parent_run_id` — field + renaming, not translation. + + A crew kickoff and each task within it open/close a `span()`; agent + execution, tool usage, and LLM calls are `.action()`/`.observation()`/ + `.error()` pairs carrying `duration_ms` — `ToolUsageFinishedEvent` + already carries its own `started_at`/`finished_at`, used directly; + agent execution and LLM calls fall back to timing this adapter records + itself at the matching start event. + + Instantiating this class registers its handlers immediately (that's + `BaseEventListener`'s own behavior) — keep a reference alive for as long + as you want it active, the same way any CrewAI custom listener works: + + from logquill import Logger, RunPlugin + from logquill.adapters.crewai import CrewAIAdapter + + log = Logger("app") + listener = CrewAIAdapter(log.child("agent").use(RunPlugin())) + crew = Crew(agents=[...], tasks=[...]) # listener is now active + crew.kickoff() + + If a listener is attached mid-run (so this adapter never saw the + matching "started" event for something already in progress), the + corresponding "ended" event is dropped rather than guessed at — the same + posture `SamplingPlugin`/`AlertingPlugin` take toward not fabricating + data they don't actually have. + """ + + def __init__(self, agent_log: Logger) -> None: + LogQuillAdapter.__init__(self, agent_log) + self._open_spans: dict[str, SpanContext] = {} + self._call_starts: dict[str, datetime] = {} + BaseEventListener.__init__(self) # registers handlers; needs self.log set first + + def _open_span(self, name: str, event: Any) -> None: + span = self.log.span(name, span_id=event.event_id, parent_span_id=event.parent_event_id) + span.__enter__() + self._open_spans[event.event_id] = span + + def _close_span(self, event: Any, error: BaseException | None = None) -> None: + span = self._open_spans.pop(event.started_event_id or "", None) + if span is None: + return + if error is not None: + span.__exit__(type(error), error, None) + else: + span.__exit__(None, None, None) + + def _step_start(self, name: str, event: Any) -> None: + self._call_starts[event.event_id] = event.timestamp + self.log.action(name, **_span_ids(event)) + + def _step_end(self, name: str, event: Any, *, error: str | None = None) -> None: + start = self._call_starts.pop(event.started_event_id or "", None) + ids = _closing_ids(event) + if start is not None: + ids["duration_ms"] = round((event.timestamp - start).total_seconds() * 1000, 3) + if error is not None: + self.log.error(name, error=error, **ids) + else: + self.log.observation(name, **ids) + + # Registered via plain calls rather than `@crewai_event_bus.on(...)` + # decorator syntax below — `crewai_event_bus` types as `Any` whenever + # `crewai` isn't installed (see the `type: ignore[misc]` note on the + # class itself), and mypy strict's `disallow_untyped_decorators` flags + # `@`-applying an `Any`-typed decorator even though the wrapped method + # itself is fully annotated. A plain call sidesteps that check. + def setup_listeners(self, crewai_event_bus: Any) -> None: + crewai_event_bus.on(CrewKickoffStartedEvent)(self._on_crew_started) + crewai_event_bus.on(CrewKickoffCompletedEvent)(self._on_crew_completed) + crewai_event_bus.on(CrewKickoffFailedEvent)(self._on_crew_failed) + crewai_event_bus.on(TaskStartedEvent)(self._on_task_started) + crewai_event_bus.on(TaskCompletedEvent)(self._on_task_completed) + crewai_event_bus.on(TaskFailedEvent)(self._on_task_failed) + crewai_event_bus.on(AgentExecutionStartedEvent)(self._on_agent_started) + crewai_event_bus.on(AgentExecutionCompletedEvent)(self._on_agent_completed) + crewai_event_bus.on(AgentExecutionErrorEvent)(self._on_agent_error) + crewai_event_bus.on(ToolUsageStartedEvent)(self._on_tool_started) + crewai_event_bus.on(ToolUsageFinishedEvent)(self._on_tool_finished) + crewai_event_bus.on(ToolUsageErrorEvent)(self._on_tool_error) + crewai_event_bus.on(LLMCallStartedEvent)(self._on_llm_started) + crewai_event_bus.on(LLMCallCompletedEvent)(self._on_llm_completed) + crewai_event_bus.on(LLMCallFailedEvent)(self._on_llm_failed) + + def _on_crew_started(self, source: Any, event: Any) -> None: + self._open_span(f"crew:{event.crew_name or 'crew'}", event) + + def _on_crew_completed(self, source: Any, event: Any) -> None: + self._close_span(event) + + def _on_crew_failed(self, source: Any, event: Any) -> None: + self._close_span(event, error=RuntimeError(event.error)) + + def _on_task_started(self, source: Any, event: Any) -> None: + self._open_span(f"task:{event.task_name or event.task_id or 'task'}", event) + + def _on_task_completed(self, source: Any, event: Any) -> None: + self._close_span(event) + + def _on_task_failed(self, source: Any, event: Any) -> None: + error_cls = event.error_type or RuntimeError + self._close_span(event, error=error_cls(event.error)) + + def _on_agent_started(self, source: Any, event: Any) -> None: + self._step_start(f"agent:{event.agent.role}", event) + + def _on_agent_completed(self, source: Any, event: Any) -> None: + self._step_end(f"agent:{event.agent.role}", event) + + def _on_agent_error(self, source: Any, event: Any) -> None: + self._step_end(f"agent:{event.agent.role}", event, error=event.error) + + def _on_tool_started(self, source: Any, event: Any) -> None: + self.log.action(event.tool_name, **_span_ids(event)) + + def _on_tool_finished(self, source: Any, event: Any) -> None: + duration_ms = (event.finished_at - event.started_at).total_seconds() * 1000 + self.log.observation( + event.tool_name, duration_ms=round(duration_ms, 3), **_closing_ids(event) + ) + + def _on_tool_error(self, source: Any, event: Any) -> None: + self.log.error(event.tool_name, error=str(event.error), **_closing_ids(event)) + + def _on_llm_started(self, source: Any, event: Any) -> None: + self._step_start("llm_call", event) + + def _on_llm_completed(self, source: Any, event: Any) -> None: + self._step_end("llm_call", event) + + def _on_llm_failed(self, source: Any, event: Any) -> None: + self._step_end("llm_call", event, error=event.error) diff --git a/logquill/adapters/langchain.py b/logquill/adapters/langchain.py new file mode 100644 index 0000000..0e1fba2 --- /dev/null +++ b/logquill/adapters/langchain.py @@ -0,0 +1,235 @@ +from __future__ import annotations + +import time +from typing import Any +from uuid import UUID + +try: + from langchain_core.callbacks import BaseCallbackHandler # type: ignore[import-not-found] +except ImportError as exc: + raise ImportError( + "logquill.adapters.langchain requires the optional `langchain-core` " + "dependency — install with `pip install logquill[langchain]`." + ) from exc + +from logquill.adapters.base import LogQuillAdapter +from logquill.logger import Logger +from logquill.span import SpanContext + + +def _span_ids(run_id: UUID, parent_run_id: UUID | None) -> dict[str, Any]: + ids: dict[str, Any] = {"span_id": str(run_id)} + if parent_run_id is not None: + ids["parent_span_id"] = str(parent_run_id) + return ids + + +def _tool_name(serialized: dict[str, Any] | None, fallback: str) -> str: + if isinstance(serialized, dict): + name = serialized.get("name") + if isinstance(name, str) and name: + return name + return fallback + + +# `type: ignore[misc]` — BaseCallbackHandler types as `Any` whenever +# langchain-core isn't installed in the environment running mypy (it's an +# optional dependency, never in this project's `dev` extra — see +# pyproject.toml), and mypy refuses to let a class subclass something typed +# `Any`. With the real package installed, this subclasses the genuine +# `BaseCallbackHandler` and the ignore is inert. +class LangChainAdapter(LogQuillAdapter, BaseCallbackHandler): # type: ignore[misc] + """Maps LangChain's `BaseCallbackHandler` events onto LogQuill calls — + LangGraph is covered for free, since it shares LangChain's callback + system. + + Pass an instance into a chain/agent invocation's `callbacks=[...]`, the + same way any other LangChain tracing handler (LangSmith, Langfuse, ...) + is wired in — no other instrumentation needed: + + from logquill.adapters.langchain import LangChainAdapter + from logquill.plugins.run_plugin import RunPlugin + + handler = LangChainAdapter(log.child("agent").use(RunPlugin())) + llm = ChatOpenAI(callbacks=[handler]) + + Event mapping: + + | LangChain callback | LogQuill call | + |--------------------------------------------------|-----------------------------------| + | `on_chain_start` / `on_chain_end` | opens/closes `span()` | + | `on_llm_start` / `on_llm_end` | `.action()` / `.observation()` | + | `on_agent_action` | `.action()` | + | `on_agent_finish` | `.decision()` | + | `on_tool_start`/`on_tool_end`/`on_tool_error` | `.action()`/`.observation()`/`.error()` | + + LangChain's own `run_id`/`parent_run_id` are written directly onto + `meta.span_id`/`meta.parent_span_id` — the shapes already match, so this + is field renaming, not translation. + """ + + def __init__(self, agent_log: Logger) -> None: + LogQuillAdapter.__init__(self, agent_log) + BaseCallbackHandler.__init__(self) + self._open_spans: dict[UUID, SpanContext] = {} + self._call_starts: dict[UUID, float] = {} + + def _duration_ms(self, run_id: UUID) -> float | None: + start = self._call_starts.pop(run_id, None) + if start is None: + return None + return round((time.monotonic() - start) * 1000, 3) + + # -- chains: each chain run opens/closes a span ---------------------- + + def on_chain_start( + self, + serialized: dict[str, Any], + inputs: dict[str, Any], + *, + run_id: UUID, + parent_run_id: UUID | None = None, + **kwargs: Any, + ) -> Any: + name = _tool_name(serialized, "chain") + span = self.log.span( + name, + span_id=str(run_id), + parent_span_id=str(parent_run_id) if parent_run_id is not None else None, + ) + span.__enter__() + self._open_spans[run_id] = span + + def on_chain_end( + self, + outputs: dict[str, Any], + *, + run_id: UUID, + parent_run_id: UUID | None = None, + **kwargs: Any, + ) -> Any: + span = self._open_spans.pop(run_id, None) + if span is not None: + span.__exit__(None, None, None) + + def on_chain_error( + self, + error: BaseException, + *, + run_id: UUID, + parent_run_id: UUID | None = None, + **kwargs: Any, + ) -> Any: + span = self._open_spans.pop(run_id, None) + if span is not None: + span.__exit__(type(error), error, error.__traceback__) + + # -- LLM calls: action (start) / observation (end) -------------------- + + def on_llm_start( + self, + serialized: dict[str, Any], + prompts: list[str], + *, + run_id: UUID, + parent_run_id: UUID | None = None, + **kwargs: Any, + ) -> Any: + self._call_starts[run_id] = time.monotonic() + self.log.action("llm_start", **_span_ids(run_id, parent_run_id)) + + def on_llm_end( + self, + response: Any, + *, + run_id: UUID, + parent_run_id: UUID | None = None, + **kwargs: Any, + ) -> Any: + duration_ms = self._duration_ms(run_id) + meta = _span_ids(run_id, parent_run_id) + if duration_ms is not None: + meta["duration_ms"] = duration_ms + self.log.observation("llm_end", **meta) + + def on_llm_error( + self, + error: BaseException, + *, + run_id: UUID, + parent_run_id: UUID | None = None, + **kwargs: Any, + ) -> Any: + self._duration_ms(run_id) + self.log.error("llm_error", error=str(error), **_span_ids(run_id, parent_run_id)) + + # -- agent-level events ------------------------------------------------- + + # `on_agent_action`/`on_agent_finish` carry the *enclosing* chain's own + # `run_id` (LangChain doesn't mint a fresh one for these events) — unlike + # `on_llm_start`/`on_tool_start`, mapping it onto `span_id` here would + # make the record its own parent, since that same id is already the + # active span pushed by the enclosing `on_chain_start`. Leaving span + # kwargs unset lets `Logger._log`'s ambient-span auto-stamp supply the + # correct `parent_span_id` instead. + + def on_agent_action( + self, + action: Any, + *, + run_id: UUID, + parent_run_id: UUID | None = None, + **kwargs: Any, + ) -> Any: + tool = getattr(action, "tool", "agent_action") + self.log.action(tool) + + def on_agent_finish( + self, + finish: Any, + *, + run_id: UUID, + parent_run_id: UUID | None = None, + **kwargs: Any, + ) -> Any: + self.log.decision("agent_finish") + + # -- tools: action (start) / observation (end) / error ------------------ + + def on_tool_start( + self, + serialized: dict[str, Any], + input_str: str, + *, + run_id: UUID, + parent_run_id: UUID | None = None, + **kwargs: Any, + ) -> Any: + name = _tool_name(serialized, "tool") + self._call_starts[run_id] = time.monotonic() + self.log.action(name, **_span_ids(run_id, parent_run_id)) + + def on_tool_end( + self, + output: Any, + *, + run_id: UUID, + parent_run_id: UUID | None = None, + **kwargs: Any, + ) -> Any: + duration_ms = self._duration_ms(run_id) + meta = _span_ids(run_id, parent_run_id) + if duration_ms is not None: + meta["duration_ms"] = duration_ms + self.log.observation("tool_end", **meta) + + def on_tool_error( + self, + error: BaseException, + *, + run_id: UUID, + parent_run_id: UUID | None = None, + **kwargs: Any, + ) -> Any: + self._duration_ms(run_id) + self.log.error("tool_error", error=str(error), **_span_ids(run_id, parent_run_id)) diff --git a/logquill/adapters/llamaindex.py b/logquill/adapters/llamaindex.py new file mode 100644 index 0000000..d5556b9 --- /dev/null +++ b/logquill/adapters/llamaindex.py @@ -0,0 +1,157 @@ +from __future__ import annotations + +from typing import Any + +try: + from llama_index.core.instrumentation import get_dispatcher # type: ignore[import-not-found] + from llama_index.core.instrumentation.event_handlers import ( # type: ignore[import-not-found] + BaseEventHandler, + ) + from llama_index.core.instrumentation.span_handlers import ( # type: ignore[import-not-found] + SimpleSpanHandler, + ) + from pydantic import ConfigDict # type: ignore[import-not-found] +except ImportError as exc: + raise ImportError( + "logquill.adapters.llamaindex requires the optional `llama-index-core` " + "dependency — install with `pip install logquill[llamaindex]`." + ) from exc + +from logquill.adapters.base import LogQuillAdapter +from logquill.logger import Logger + +# Event classes carrying no useful correlation id of their own beyond the +# span they fired in — every `XxxStartEvent`/`XxxEndEvent`/`XxxErrorEvent` +# pair shares its enclosing span's `span_id` rather than a call-specific id +# (confirmed against `llama_index/core/instrumentation/events/*.py` and +# `llama_index_instrumentation/base/event.py` upstream — `BaseEvent` has no +# "started_event_id"-equivalent, unlike CrewAI's events). So instead of +# enumerating every concrete event class (~25 across llm/retrieval/query/ +# synthesis/agent/embedding/chat_engine, and growing), events are classified +# generically by their `class_name()` suffix — `*StartEvent` -> `.action()`, +# `*EndEvent` -> `.observation()`, `*ErrorEvent` -> `.error()` — which also +# means a new event type LlamaIndex adds later needs no adapter change to +# show up correctly. +_SKIPPED_SUFFIXES = ("InProgressEvent", "DeltaReceivedEvent") + + +# `type: ignore[misc]` on both handler classes below — same reason as +# `LangChainAdapter`/`CrewAIAdapter`: their real bases type as `Any` whenever +# `llama-index-core` isn't installed in the environment running mypy (it's +# optional, never in this project's `dev` extra — see pyproject.toml), and +# mypy refuses to let a class subclass something typed `Any`. With the real +# package installed, these subclass the genuine base classes and the ignore +# is inert. +class _SpanLogger(SimpleSpanHandler): # type: ignore[misc] + """Logs LlamaIndex's own method-level spans (`query()`, `chat()`, + `retrieve()`, ...) — each one decorated internally with `@dispatcher.span` + — as they open and close. `SimpleSpanHandler` already tracks id/parent + id/duration for every span; this only adds logging on top of it.""" + + model_config = ConfigDict(arbitrary_types_allowed=True) + log: Logger + + def prepare_to_exit_span( + self, + id_: str, + bound_args: Any, + instance: Any = None, + result: Any = None, + **kwargs: Any, + ) -> Any: + span = super().prepare_to_exit_span(id_, bound_args, instance, result, **kwargs) + self._log_close(id_, span) + return span + + def prepare_to_drop_span( + self, + id_: str, + bound_args: Any, + instance: Any = None, + err: BaseException | None = None, + **kwargs: Any, + ) -> Any: + span = super().prepare_to_drop_span(id_, bound_args, instance, err, **kwargs) + self._log_close(id_, span, error=err) + return span + + def _log_close(self, id_: str, span: Any, error: BaseException | None = None) -> None: + # LlamaIndex names a span `f"{qualified_method_name}-{uuid}"` — the + # message drops the uuid suffix (`.partition` is a no-op, not an + # error, if `id_` happens to have none); the full `id_` stays the + # `span_id` so it's still unique for correlation. + name = id_.partition("-")[0] + meta: dict[str, Any] = {"span_id": id_, "kind": "span"} + if span is not None: + meta["duration_ms"] = round(span.duration * 1000, 3) + if span.parent_id is not None: + meta["parent_span_id"] = span.parent_id + if error is not None: + self.log.error(name, error=f"{type(error).__name__}: {error}", **meta) + else: + self.log.info(name, **meta) + + +class _EventLogger(BaseEventHandler): # type: ignore[misc] + """Logs LlamaIndex's named events (LLM calls, retrieval, synthesis, + embedding, agent steps, ...), nested under whichever span was open when + each one fired via `event.span_id` -> `meta.parent_span_id`.""" + + model_config = ConfigDict(arbitrary_types_allowed=True) + log: Logger + + def handle(self, event: Any, **kwargs: Any) -> Any: + name = event.class_name() + if name.endswith(_SKIPPED_SUFFIXES): + return None + + meta: dict[str, Any] = {} + span_id = getattr(event, "span_id", None) + if span_id is not None: + meta["parent_span_id"] = span_id + + if name.endswith("ErrorEvent"): + error = getattr(event, "exception", None) or getattr(event, "error", None) + self.log.error(name, error=str(error) if error is not None else "", **meta) + elif name.endswith("StartEvent"): + self.log.action(name, **meta) + elif name.endswith("EndEvent"): + self.log.observation(name, **meta) + else: + self.log.action(name, **meta) + return None + + +class LlamaIndexAdapter(LogQuillAdapter): + """Maps LlamaIndex's instrumentation module onto LogQuill calls. + + Unlike `LangChainAdapter`/`CrewAIAdapter`, LlamaIndex splits + instrumentation into two cooperating registrations on a shared global + dispatcher — a span handler (LlamaIndex's own internal method calls, + each already wrapped in a span by the framework) and an event handler + (named events fired *within* those spans) — so this adapter holds one + of each internally rather than being a handler itself: + + from logquill import Logger, RunPlugin + from logquill.adapters.llamaindex import LlamaIndexAdapter + + log = Logger("app") + adapter = LlamaIndexAdapter(log.child("agent").use(RunPlugin())) # active immediately + index.as_query_engine().query("...") + + A `query()`/`chat()`/`retrieve()` call becomes a `span_id`/`duration_ms` + record on completion (`meta.parent_span_id` set for a nested call, e.g. + `retrieve()` inside `query()`); LLM calls, retrieval, synthesis, + embedding, and agent-step events become `.action()`/`.observation()`/ + `.error()` records nested under whichever span was open via + `meta.parent_span_id`. `llama-index-core` is never imported unless you + import `logquill.adapters.llamaindex` yourself. + """ + + def __init__(self, agent_log: Logger) -> None: + super().__init__(agent_log) + self._span_handler = _SpanLogger(log=agent_log) + self._event_handler = _EventLogger(log=agent_log) + dispatcher = get_dispatcher() + dispatcher.add_span_handler(self._span_handler) + dispatcher.add_event_handler(self._event_handler) diff --git a/logquill/config.py b/logquill/config.py new file mode 100644 index 0000000..f53cda4 --- /dev/null +++ b/logquill/config.py @@ -0,0 +1,153 @@ +from __future__ import annotations + +import importlib +import json +import os +from pathlib import Path +from typing import Any + +from logquill.logger import Logger +from logquill.plugins.context_plugin import ContextPlugin +from logquill.plugins.pii_redact_plugin import PIIRedactPlugin +from logquill.plugins.plugin import Plugin +from logquill.plugins.redact_plugin import RedactPlugin +from logquill.plugins.run_plugin import RunPlugin +from logquill.plugins.sampling_plugin import SamplingPlugin +from logquill.plugins.tamper_evident_plugin import TamperEvidentPlugin +from logquill.plugins.trace_context_plugin import TraceContextPlugin +from logquill.transports.console_transport import ConsoleTransport +from logquill.transports.file_transport import FileTransport +from logquill.transports.http_transport import HTTPTransport +from logquill.transports.transport import Transport + +#: Shortcuts for the zero-required-dependency transports/plugins, so common +#: configs don't need a fully-qualified class path. Anything else — every +#: cloud/SQL/NoSQL/queue transport, the alerting plugins, framework +#: adapters, or your own subclass — goes through `"class"` instead (see +#: `load_config`). This is deliberately a small, curated list, not an +#: attempt to cover the whole catalog: the ones here are safe to construct +#: from plain config with no optional dependency surprising the caller. +_TRANSPORT_TYPES: dict[str, type[Transport]] = { + "console": ConsoleTransport, + "file": FileTransport, + "http": HTTPTransport, +} + +_PLUGIN_TYPES: dict[str, type[Plugin]] = { + "context": ContextPlugin, + "redact": RedactPlugin, + "sampling": SamplingPlugin, + "trace_context": TraceContextPlugin, + "run": RunPlugin, + "pii_redact": PIIRedactPlugin, + "tamper_evident": TamperEvidentPlugin, +} + + +def _resolve_class(entry: dict[str, Any], registry: dict[str, type]) -> type: + if "class" in entry: + dotted = entry["class"] + module_name, separator, class_name = dotted.rpartition(".") + if not separator: + raise ValueError( + f"'class' must be a fully-qualified dotted path (e.g. " + f"'logquill.transports.cloud.datadog_transport.DatadogTransport'), got {dotted!r}" + ) + module = importlib.import_module(module_name) + try: + return getattr(module, class_name) # type: ignore[no-any-return] + except AttributeError: + raise ValueError( + f"{dotted!r}: module {module_name!r} has no attribute {class_name!r}" + ) from None + if "type" in entry: + try: + return registry[entry["type"]] + except KeyError: + known = ", ".join(sorted(registry)) + raise ValueError( + f"Unknown type {entry['type']!r} — built-in types are: {known}. " + "Use 'class': '' for anything else." + ) from None + raise ValueError(f"Each transport/plugin entry needs a 'type' or 'class' key, got {entry!r}") + + +def _build(entries: list[dict[str, Any]] | None, registry: dict[str, type]) -> list[Any]: + built = [] + for entry in entries or []: + cls = _resolve_class(entry, registry) + built.append(cls(**entry.get("options", {}))) + return built + + +def load_config(data: dict[str, Any], *, name: str = "app") -> Logger: + """Build a `Logger` from an already-parsed config dict — the same + shape `logger_from_file`/`logger_from_env` parse JSON/YAML into: + + { + "name": "app", # optional, defaults to `name` + "level": "INFO", + "transports": [ + {"type": "console"}, + {"type": "file", "options": {"path": "app.log"}}, + {"class": "logquill.transports.cloud.datadog_transport.DatadogTransport", + "options": {"api_key": "..."}} + ], + "plugins": [ + {"type": "context", "options": {"service": "api"}}, + {"type": "sampling", "options": {"rate": 0.1}} + ] + } + + Each transport/plugin entry needs either `"type"` (a built-in shortcut — + see the module-level registries above) or `"class"` (a fully-qualified + dotted path, imported and instantiated the way `logging.config. + dictConfig` resolves a `class` key — the same trust boundary: only use + this with config you trust, the same as any other deployment config). + `"options"` becomes that class's constructor keyword arguments. + """ + logger_name = data.get("name", name) + level = data.get("level", "INFO") + transports = _build(data.get("transports"), _TRANSPORT_TYPES) + plugins = _build(data.get("plugins"), _PLUGIN_TYPES) + return Logger(logger_name, level=level, transports=transports, plugins=plugins) + + +def logger_from_file(path: str | Path, *, name: str = "app") -> Logger: + """Load a `Logger` from a `.json`/`.yaml`/`.yml` file — see `load_config` + for the shape. YAML needs the optional `PyYAML` dependency + (`pip install logquill[yaml]`); JSON needs nothing beyond the stdlib. + """ + file_path = Path(path) + text = file_path.read_text(encoding="utf-8") + if file_path.suffix in (".yaml", ".yml"): + try: + import yaml # type: ignore[import-untyped] + except ImportError as exc: + raise ImportError( + "logger_from_file(...) with a .yaml/.yml file requires the optional " + "`PyYAML` dependency — install with `pip install logquill[yaml]`." + ) from exc + data = yaml.safe_load(text) + else: + data = json.loads(text) + return load_config(data, name=name) + + +def logger_from_env(*, prefix: str = "LOGQUILL_", name: str = "app") -> Logger: + """Build a `Logger` from environment variables. + + `{prefix}CONFIG_FILE`, if set, is loaded via `logger_from_file` first — + the full transport/plugin config isn't practical to express as flat env + vars. `{prefix}LEVEL`, if set, is applied last and always wins, even + over a level set in the config file — the common convention of letting + an env var override a file for the one field ops teams actually reach + for at deploy time. + """ + config_file = os.environ.get(f"{prefix}CONFIG_FILE") + logger = logger_from_file(config_file, name=name) if config_file else Logger(name) + + level = os.environ.get(f"{prefix}LEVEL") + if level: + logger.set_level(level) + return logger diff --git a/logquill/logger.py b/logquill/logger.py index 7f75c5b..09d7c17 100644 --- a/logquill/logger.py +++ b/logquill/logger.py @@ -5,8 +5,10 @@ from typing import Any from logquill.levels import Level, parse_level +from logquill.plugins.context_plugin import ContextPlugin from logquill.plugins.plugin import FunctionPlugin, MiddlewareFunc, Plugin from logquill.records import LogRecord, create_record +from logquill.span import SpanContext, current_span_id from logquill.transports.transport import Transport _logger = logging.getLogger("logquill") @@ -47,6 +49,21 @@ def use(self, plugin: Plugin | MiddlewareFunc) -> Logger: self.plugins.append(plugin) return self + def child(self, name: str, /, **fixed_meta: Any) -> Logger: + """A namespaced logger under this one: `f"{self.name}.{name}"`. + + Shares this logger's transports (the same sink instances, so + `close()` on either flushes both) but starts with its own empty + plugin list — plugins are per-logger middleware, not inherited, so + a child can `.use(RunPlugin())` without attaching it to the + parent's pipeline too. Any `fixed_meta` given is injected into + every record the child produces, via an internal `ContextPlugin`. + """ + child_logger = Logger(f"{self.name}.{name}", level=self._level, transports=self.transports) + if fixed_meta: + child_logger.use(ContextPlugin(**fixed_meta)) + return child_logger + def close(self) -> None: """Close every attached transport. Call on shutdown to flush buffered writes.""" for transport in self.transports: @@ -62,6 +79,10 @@ def _log(self, level: Level, message: str, meta: dict[str, Any]) -> LogRecord | return None record = create_record(level=level, logger=self.name, message=message, meta=meta) + parent_span_id = current_span_id() + if parent_span_id is not None: + record["meta"].setdefault("parent_span_id", parent_span_id) + for plugin in self.plugins: try: result = plugin.before_log(record) @@ -88,20 +109,71 @@ def _log(self, level: Level, message: str, meta: dict[str, Any]) -> LogRecord | return record - def trace(self, message: str, **meta: Any) -> LogRecord | None: + # `message: str, /` (positional-only) on every method below: a caller + # passing `**meta` where `meta` happens to contain a `"message"` key + # (e.g. forwarding an adversarial or framework-supplied dict) would + # otherwise collide with the `message` parameter and raise + # `TypeError: got multiple values for argument 'message'`, crashing the + # caller — exactly what the plugin pipeline's hypothesis tests assert + # never happens (see `tests/test_plugin_pipeline_properties.py`). + def trace(self, message: str, /, **meta: Any) -> LogRecord | None: return self._log(Level.TRACE, message, meta) - def debug(self, message: str, **meta: Any) -> LogRecord | None: + def debug(self, message: str, /, **meta: Any) -> LogRecord | None: return self._log(Level.DEBUG, message, meta) - def info(self, message: str, **meta: Any) -> LogRecord | None: + def info(self, message: str, /, **meta: Any) -> LogRecord | None: return self._log(Level.INFO, message, meta) - def warn(self, message: str, **meta: Any) -> LogRecord | None: + def warn(self, message: str, /, **meta: Any) -> LogRecord | None: return self._log(Level.WARN, message, meta) - def error(self, message: str, **meta: Any) -> LogRecord | None: + def error(self, message: str, /, **meta: Any) -> LogRecord | None: return self._log(Level.ERROR, message, meta) - def fatal(self, message: str, **meta: Any) -> LogRecord | None: + def fatal(self, message: str, /, **meta: Any) -> LogRecord | None: return self._log(Level.FATAL, message, meta) + + def thought(self, message: str, /, **meta: Any) -> LogRecord | None: + """`.info()` tagged `meta.kind = "thought"` — an agent's internal + reasoning step, for harness/agentic tracing.""" + return self._log(Level.INFO, message, {"kind": "thought", **meta}) + + def action(self, message: str, /, **meta: Any) -> LogRecord | None: + """`.info()` tagged `meta.kind = "action"` — an agent taking an + action (a tool call, an LLM request), for harness/agentic tracing.""" + return self._log(Level.INFO, message, {"kind": "action", **meta}) + + def observation(self, message: str, /, **meta: Any) -> LogRecord | None: + """`.info()` tagged `meta.kind = "observation"` — the result an + agent observed from an action, for harness/agentic tracing.""" + return self._log(Level.INFO, message, {"kind": "observation", **meta}) + + def decision(self, message: str, /, **meta: Any) -> LogRecord | None: + """`.info()` tagged `meta.kind = "decision"` — an agent's concluding + decision for a step or run, for harness/agentic tracing.""" + return self._log(Level.INFO, message, {"kind": "decision", **meta}) + + def span( + self, + name: str, + /, + *, + span_id: str | None = None, + parent_span_id: str | None = None, + **meta: Any, + ) -> SpanContext: + """`with agent_log.span("call_llm"):` — on exit, emits one record + carrying `meta.span_id` and `meta.duration_ms`; every record logged + inside the block (through any method) is automatically stamped with + `meta.parent_span_id` pointing at this span, so nested/sub-agent + calls reconstruct their exact nesting when sorted by + `span_id`/`parent_span_id`. Still emits its record — at `ERROR`, + with `meta.error` set — if the block raises; the exception itself + propagates unchanged. + + `span_id`/`parent_span_id` normally auto-generate/auto-nest; pass + them explicitly to adopt an id handed in from elsewhere (see + `logquill.adapters.langchain.LangChainAdapter` for an example). + """ + return SpanContext(self, name, span_id=span_id, parent_span_id=parent_span_id, **meta) diff --git a/logquill/plugins/run_plugin.py b/logquill/plugins/run_plugin.py new file mode 100644 index 0000000..280b0c6 --- /dev/null +++ b/logquill/plugins/run_plugin.py @@ -0,0 +1,38 @@ +from __future__ import annotations + +import uuid + +from logquill.plugins.plugin import Plugin +from logquill.records import LogRecord + + +class RunPlugin(Plugin): + """Stamps `meta.run_id` — a stable id grouping every record from one + agent run — plus an incrementing `meta.step` counter, one per record + processed through this plugin instance. + + Distinct from `TraceContextPlugin`'s `trace_id`: `run_id` scopes one + agent run, `trace_id` follows one request across services. A run can + span multiple traces (e.g. an agent that calls several downstream + services); the two ids are independent. + + One instance is one run: attach a fresh `RunPlugin()` per run (typically + via `logger.child("agent").use(RunPlugin())`), never a process-wide + singleton shared across runs — otherwise concurrent runs would share + both the run id and the step counter. + + A record that already carries `meta["run_id"]` (e.g. propagated from an + upstream call) keeps its existing value; `meta["step"]` is always set + from this instance's own counter. + """ + + def __init__(self, run_id: str | None = None) -> None: + self.run_id = run_id or uuid.uuid4().hex + self._step = 0 + + def before_log(self, record: LogRecord) -> LogRecord | None: + meta = record["meta"] + meta.setdefault("run_id", self.run_id) + meta["step"] = self._step + self._step += 1 + return record diff --git a/logquill/plugins/trace_context_plugin.py b/logquill/plugins/trace_context_plugin.py new file mode 100644 index 0000000..b55cec8 --- /dev/null +++ b/logquill/plugins/trace_context_plugin.py @@ -0,0 +1,126 @@ +from __future__ import annotations + +import re +import secrets +from contextvars import ContextVar, Token + +from logquill.plugins.plugin import Plugin +from logquill.records import LogRecord + +_current_traceparent: ContextVar[str | None] = ContextVar("logquill_traceparent", default=None) + +# W3C Trace Context: "{version}-{trace-id}-{parent-id}-{trace-flags}", +# https://www.w3.org/TR/trace-context/#traceparent-header +_W3C_TRACEPARENT_RE = re.compile( + r"^[0-9a-f]{2}-(?P[0-9a-f]{32})-[0-9a-f]{16}-[0-9a-f]{2}$" +) +# AWS X-Ray: "Root=1-{8 hex}-{24 hex}[;Parent=...;Sampled=...]" +_XRAY_ROOT_RE = re.compile(r"Root=1-(?P