Skip to content

AsyncPGVectorStore.as_retriever filter in search_kwargs is not translated to target langchain_metadata column #276

Description

@cs051045

Hi,
I’m trying to understand how metadata filtering is intended to work when using AsyncPGVectorStore via as_retriever(), and I may be missing something.

I’m passing a filter through search_kwargs, for example:

vector_store : AsyncPGVectorStore = await get_store(table_name, embeddings)

search_kwargs = {
    "k": 50,
    "filter": {
        "some_metadata_key": {"$in": ["v1"]}
    }
}

retriever = vector_store.as_retriever(search_kwargs=search_kwargs)

results = await retriever.ainvoke(query)

Based on the documentation, I expected this filter to be applied to the configured metadata JSON column.

What I’m observing

Tracing the async execution path shows that the filter is eventually processed by:

_create_filter_clause(...) (langchain_postgres.v2.async_vectorstore.AsyncPGVectorStore._create_filter_clause)
_handle_field_filter(...)  (langchain_postgres.v2.async_vectorstore.AsyncPGVectorStore._handle_field_filter)

At this point, the filter field (some_metadata_key) is treated as a direct SQL column name, not as a key inside the metadata JSON column.

From reading the implementation, the generated SQL always follows patterns like:

some_metadata_key = ANY(:param)

There doesn’t appear to be any transformation such as:
langchain_metadata->> 'some_metadata_key'

or any logic that checks whether a filter should target the metadata JSON column instead of a physical table column.

As a result I get this error:

sqlalchemy.exc.ProgrammingError: (sqlalchemy.dialects.postgresql.asyncpg.ProgrammingError) <class 'asyncpg.exceptions.UndefinedColumnError'>: column "some_metadata_key" does not exist
[SQL: SELECT "id", "content", "embedding", "langchain_metadata", cosine_distance("embedding", $1) as distance
        FROM "rag_table" WHERE some_metadata_key = ANY($2) ORDER BY "embedding" <=> $1 LIMIT $3;
        ]

Is this behavior intentional (filters are only meant to apply to actual table columns)?

I don’t see where such a transformation would occur in:
_create_filter_clause
_handle_field_filter

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions