Hi,
I’m trying to understand how metadata filtering is intended to work when using AsyncPGVectorStore via as_retriever(), and I may be missing something.
I’m passing a filter through search_kwargs, for example:
vector_store : AsyncPGVectorStore = await get_store(table_name, embeddings)
search_kwargs = {
"k": 50,
"filter": {
"some_metadata_key": {"$in": ["v1"]}
}
}
retriever = vector_store.as_retriever(search_kwargs=search_kwargs)
results = await retriever.ainvoke(query)
Based on the documentation, I expected this filter to be applied to the configured metadata JSON column.
What I’m observing
Tracing the async execution path shows that the filter is eventually processed by:
_create_filter_clause(...) (langchain_postgres.v2.async_vectorstore.AsyncPGVectorStore._create_filter_clause)
_handle_field_filter(...) (langchain_postgres.v2.async_vectorstore.AsyncPGVectorStore._handle_field_filter)
At this point, the filter field (some_metadata_key) is treated as a direct SQL column name, not as a key inside the metadata JSON column.
From reading the implementation, the generated SQL always follows patterns like:
some_metadata_key = ANY(:param)
There doesn’t appear to be any transformation such as:
langchain_metadata->> 'some_metadata_key'
or any logic that checks whether a filter should target the metadata JSON column instead of a physical table column.
As a result I get this error:
sqlalchemy.exc.ProgrammingError: (sqlalchemy.dialects.postgresql.asyncpg.ProgrammingError) <class 'asyncpg.exceptions.UndefinedColumnError'>: column "some_metadata_key" does not exist
[SQL: SELECT "id", "content", "embedding", "langchain_metadata", cosine_distance("embedding", $1) as distance
FROM "rag_table" WHERE some_metadata_key = ANY($2) ORDER BY "embedding" <=> $1 LIMIT $3;
]
Is this behavior intentional (filters are only meant to apply to actual table columns)?
I don’t see where such a transformation would occur in:
_create_filter_clause
_handle_field_filter
Hi,
I’m trying to understand how metadata filtering is intended to work when using AsyncPGVectorStore via as_retriever(), and I may be missing something.
I’m passing a filter through search_kwargs, for example:
Based on the documentation, I expected this filter to be applied to the configured metadata JSON column.
What I’m observing
Tracing the async execution path shows that the filter is eventually processed by:
At this point, the filter field (some_metadata_key) is treated as a direct SQL column name, not as a key inside the metadata JSON column.
From reading the implementation, the generated SQL always follows patterns like:
some_metadata_key = ANY(:param)There doesn’t appear to be any transformation such as:
langchain_metadata->> 'some_metadata_key'
or any logic that checks whether a filter should target the metadata JSON column instead of a physical table column.
As a result I get this error:
Is this behavior intentional (filters are only meant to apply to actual table columns)?
I don’t see where such a transformation would occur in:
_create_filter_clause
_handle_field_filter