Skip to content

Expose colsUsed in table QueryContext #134

Description

@ei-grad

Problem

osquery core includes projection metadata in the table plugin request context. In TablePlugin::setRequestFromContext, it serializes colsUsed and colsUsedBitset alongside constraints. The Go binding currently parses only constraints into table.QueryContext, so Go table generators cannot tell which columns SQLite actually needs.

This matters for extension tables with expensive or large columns. For example, a table with message metadata plus large text / JSON payload columns can fail a metadata-only aggregate query:

SELECT source, session_id, role, content_kind, count(*) AS n
FROM ai_assistant_messages
WHERE source = 'codex'
  AND session_id = 'example-session'
GROUP BY source, session_id, role, content_kind;

The query does not need the large text/payload columns, but without colsUsed the Go extension has to conservatively generate complete rows. Those complete rows are sent over the extension Thrift boundary before SQLite applies aggregation, so large real-world sessions can hit MaxMessageSize reached even though the SQL result is small.

For plain SELECT count(*) FROM table, SQLite should not require any table column values. For grouped counts like the query above, the needed columns are the grouping/output columns, not every column in the table.

Expected API

Expose the projection metadata already sent by osquery core, for example:

type QueryContext struct {
    Constraints       map[string]ConstraintList
    ColumnsUsed       []string
    ColumnsUsedBitset *uint64
}

A helper such as IsColumnUsed(name string) could preserve compatibility by returning true when column-usage metadata is absent.

Compatibility

This should be backwards-compatible for existing generators if missing colsUsed means "assume all columns may be used".


Prepared with LM assistance — Codex CLI

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions