Filtered Retrieval Has Two Budgets

A permissioned retriever can return only authorized documents and still fail its user. Filtering may remove the candidates that approximate search found without finding the authorized evidence that exists elsewhere. Treat evidence demand and search effort as separate budgets, while keeping authorization outside the tuning loop.

Safe results can still be wrong

Imagine an internal assistant asked for a policy that exists in the requester's workspace. Its shared vector index returns nearby documents from other workspaces. The application removes those documents, receives an empty list, and tells the model there is no evidence. Nothing leaked. The retrieval result is nevertheless inadequate.

This is a hypothetical failure scenario, not an incident from an operated service. The mechanism is documented: pgvector's filtering guidance explains that approximate index scans apply filtering after scanning and can consequently return fewer results. Weaviate's filtering discussion describes the same candidate-starvation problem with post-filtering.

The engineering mistake is to make one number, top_k, carry two responsibilities: how much evidence the application wants and how much search work the backend may perform to find it. Those are different demands. A request for several authorized chunks says nothing about how many candidates the search must examine.

The central distinction is simple: permission is an invariant; search effort is a budget. An empty result should not automatically mean the authorized corpus contains no relevant material. It may mean the chosen search path stopped before finding it. That distinction belongs in retrieval evaluation and runtime control, not in an instruction asking the language model to try harder.

Where the candidates disappear

A metadata predicate describes which records qualify. It does not, by itself, explain how the search engine finds them. Two queries with similar-looking filters can spend their search work differently. PostgreSQL's EXPLAIN documentation is useful precisely because the execution plan, rather than the SQL's appearance, reveals the chosen access path.

In the pinned pgvector v0.8.1 README, approximate filtering can discard candidates after the index scan. Iterative scans can continue searching until enough results are found or configured limits are reached. Strict ordering controls the distance order of returned candidates. It does not turn approximate search into an exact-recall guarantee.

Other engines integrate filters differently. Weaviate passes an allow list into its HNSW search, restricting admission to the result set while graph traversal can still use other nodes. Elasticsearch distinguishes the kNN query's filter parameter from post-filters elsewhere in the Query DSL. Predicate placement is therefore a retrieval decision, not cosmetic query formatting.

Do not reduce these designs to a universal instruction to “filter first.” Logical eligibility, graph traversal, and final result admission are distinct operations. The actionable question is whether the implementation can continue finding eligible neighbors after ineligible candidates are excluded, within an acceptable cost envelope.

Nor is corpus-wide selectivity enough to choose that envelope. A small workspace might contain nearly all neighbors of one query and almost none of another. The relevant problem is candidate survival near this query, not just the percentage of records carrying a tenant label. Treat selectivity-based oversampling as a hypothesis to evaluate, not a completeness promise.

Separate evidence demand from search effort

A useful retrieval controller has two independent settings:

Budget	What it expresses	What reaching it means
Evidence demand	Desired authorized results and eventual context size	Enough material is available for the next stage, subject to relevance checks
Search effort	Candidate exploration, elapsed time, and permitted fallback work	Stop searching or return a degraded outcome, not broaden access

This is a proposed architecture, not a standardized vendor API. Backend-specific controls implement only parts of it. For example, pgvector's iterative scans have scan and memory limits; the documented tuple limit is approximate and does not affect the initial scan. An application deadline still needs its own enforcement.

The request path should establish identity and derive its authorization predicate before choosing a search strategy. A predicate supplied by a model or an untrusted caller is not proof of permission. Qdrant's filtering documentation defines conjunction and matching semantics; the application must still determine which conjunction the principal is entitled to use.

After bounded search, the controller can return enough eligible evidence, try an authorized fallback, or report insufficient evidence. It must never remove a tenant condition because a result set looks thin. It should also distinguish an operational failure from an ordinary no-evidence answer without exposing counts or document identities from unauthorized scopes.

This design keeps security and quality failures from being confused. A final authorization check can remain a defensive barrier, but it is not a mechanism for refilling a candidate pool. The component spending the search budget must understand the eligibility constraint. This also assumes the predicate reflects current access policy; a perfectly executed stale ACL filter can still be wrong. Policy freshness and revocation need their own consistency design.

A small counterexample you can run

The following standard-library Python program isolates the ordering problem. The distances and labels are deliberately synthetic. It is not an HNSW implementation, an ANN benchmark, or evidence about any vendor's latency. It demonstrates why selecting a global candidate prefix and then filtering is not equivalent to ranking inside the allowed set.

rows = [
    ("other-1", False, 0.01),
    ("other-2", False, 0.02),
    ("other-3", False, 0.03),
    ("allowed-1", True, 0.04),
    ("allowed-2", True, 0.05),
]
k, candidate_limit = 2, 3
ranked = sorted(rows, key=lambda row: (row[2], row[0]))
postfiltered = [row[0] for row in ranked[:candidate_limit] if row[1]][:k]
exact_allowed = [row[0] for row in ranked if row[1]][:k]
assert postfiltered == []
assert exact_allowed == ["allowed-1", "allowed-2"]
print("global candidates, then filter:", postfiltered)
print("exact ranking inside allowed set:", exact_allowed)

This program was executed with Python 3.13.5 during article preparation, and both assertions passed. That is the entire validation claim. No database benchmark or deployed authorization system was tested.

Increasing the candidate prefix would recover the allowed rows in this small list. It does not establish a universal production multiplier: the next query can push eligible neighbors farther down the ranking. The list example makes the failure inspectable without pretending to reproduce a graph index's traversal.

Use this example when reviewing a pipeline that fetches global candidates into application memory before checking scope. Ask which component is responsible for continuing the search after candidates are removed. If the answer is “none,” the pipeline has no explicit owner for filtered-result sufficiency.

Evaluate against the allowed corpus

The reference result should be an exact ranking over the same eligible records, not the global nearest neighbors with forbidden IDs removed afterward. pgvector recommends comparing approximate results with exact search. For permissioned retrieval, extend that comparison by holding principal, predicate, distance function, and corpus snapshot constant.

For each query, define the reference as the closest eligible records up to the requested result count. Authorized recall is the number of returned IDs in that reference divided by the reference's size. If fewer eligible records exist than requested, the denominator is consequently smaller than the requested count. If no eligible records exist, mark recall undefined and evaluate correct empty behavior separately. Resolve distance ties deterministically, or use a documented tie-aware policy.

Also measure result fill separately from recall. A system can return the requested number of authorized chunks while missing the best authorized neighbors. Conversely, an incomplete-looking response may contain every eligible record in a genuinely small scope. Neither metric replaces relevance judgments: exact nearest neighbors under an embedding distance are a retrieval reference, not proof that the evidence answers the question.

The baseline must use the application's actual database role. PostgreSQL row security normally does not constrain table owners, and superusers or roles with BYPASSRLS bypass it. FORCE ROW LEVEL SECURITY can subject the owner to policies; it does not remove those other bypass privileges. An administrator's successful query is therefore not an authorization test for the application.

Keep the comparison's visibility stable as well. Under PostgreSQL Read Committed isolation, successive statements can see different snapshots. Concurrent ingestion can make apparent recall drift a baseline mismatch. A controlled snapshot or suitable transaction isolation helps separate search approximation from corpus changes. That is an evaluation technique, not a promise about permission-revocation consistency.

Finally, inspect the actual plans. EXPLAIN ANALYZE executes the statement and reports observed rows and timing. Verify that the reference really uses the intended exact path, and run potentially expensive checks on controlled workloads. Do not assume a metadata index and a vector index compose into the plan imagined in a diagram.

Choose the fallback before the outage

The fallback should preserve the same eligibility predicate and change the search strategy. Three options are worth evaluating, depending on the workload:

Strategy	Appropriate question	Cost or limit to investigate
Exact ranking within a selective scope	Is the eligible subset small enough to search directly?	Vector distance work and concurrent request cost
Filter-aware ANN or bounded iterative search	Can the backend replenish eligible candidates efficiently?	Recall sensitivity, tail latency, memory, and stopping limits
Physical partitioning or partial indexes	Do stable access domains justify separate search structures?	Index maintenance, skew, domain changes, and operational overhead

These are alternatives, not a maturity ladder. pgvector's filtering guidance recommends starting with indexes on filter columns for selective exact search, and discusses partial indexes and partitioning for different filter distributions. The right choice requires testing the actual eligible-set sizes and query distribution.

Filter-aware search also has lifecycle costs. Qdrant's indexing documentation explains that payload indexes support cardinality estimation and search planning, while consuming resources. It also documents that filter-aware HNSW benefits depend on constructing the graph after the relevant payload indexes; adding those indexes later can require rebuilding HNSW to gain those benefits. “Add a filter” can therefore imply an indexing rollout, not just a query change.

Before deployment, evaluate small and large scopes, restrictive and permissive filters, empty scopes, and queries whose nearest global neighbors mostly belong elsewhere. Include denied-access tests independently of recall tests. Record strategy, returned count, bounded-work outcome, and policy version in protected telemetry, avoiding raw sensitive content by default.

If an authorized exact fallback would violate the deadline, return a constrained degraded outcome. Do not silently convert a search-budget failure into a claim that the document does not exist. The practical improvement is not unlimited search; it is an honest boundary between “nothing eligible was found” and “the authorized search was not sufficiently exhaustive.”

FAQ: the decision at the boundary

Is more oversampling enough?

It can recover candidates, but no fixed multiplier proves sufficiency across arbitrary filter distributions. pgvector's iterative scan limits make the stopping boundary explicit. Validate the chosen settings against an authorized exact baseline and an application deadline, rather than assuming a full result list establishes recall.

Does row-level security solve this?

It addresses row visibility for roles subject to the policies, not nearest-neighbor completeness. The PostgreSQL security documentation also makes role exemptions explicit. Test exclusion under the real application role, then test retrieval quality separately. Passing one test does not imply passing the other.

Should every empty result trigger exact search?

No. Exact fallback is an architectural choice whose cost must fit the workload. Weaviate's pre-filtering discussion shows why changing the search path can address candidate starvation, but it does not make all empty results bugs. True absence, strict relevance criteria, and exhausted search effort require different handling.

Engineering note based on public documentation and a synthetic executable counterexample. Prepared with Hermes assistance; no author-operated production incident or performance result is claimed.

Decision rule: when a permission filter produces too little evidence, change the authorized search strategy or report its limit. Never spend permission to buy recall.

References

1. pgvector v0.8.1 README
2. PostgreSQL 18 row security
3. PostgreSQL 18 EXPLAIN
4. PostgreSQL 18 transaction isolation
5. Qdrant filtering
6. Qdrant indexing
7. Weaviate filtering
8. Elasticsearch kNN query