Vector Stores Need Deletion Contracts

Deleting from a RAG system is not a vector database call. It is a contract that connects source identity, chunk lineage, metadata filters, asynchronous mutation state, cache invalidation, and retrieval verification so removed material cannot silently survive as context or citations.

The Engineering Problem

Most RAG systems start with a simple assumption: if a source document should disappear, delete its vectors. That assumption works in demos because the demo usually has one collection, one embedding pass, one retriever, and no downstream cache. Production systems are different. One uploaded PDF might become hundreds of chunks, multiple embeddings, a reranker cache, a citation table, and several application-level summaries. The vector store is only one layer in that chain.

The official database APIs expose real deletion primitives. Pinecone documents and vector records can be deleted from a namespace by ID, by metadata filter, or by deleting the namespace contents, and the document API requires exactly one of ids, filter, or delete_all in a request Pinecone docs. Chroma can delete by ID or by a where metadata filter, and its docs state that the embeddings, documents, and metadata associated with each item are deleted and cannot be undone Chroma docs. Weaviate client examples include deletion by object UUID Weaviate docs index.

Those APIs are necessary, but they are not the whole deletion story. A RAG system needs to answer a harder question: after deletion, can any query still retrieve, quote, summarize, or cite the removed source? If the system cannot answer that question, deletion is an unverified side effect, not an engineering guarantee.

Why the Obvious Solution Fails

The obvious solution is to call delete(ids=[...]) for every chunk ID derived from the source document. That is a good start, but it fails when the source-to-chunk mapping is incomplete. Chunk IDs are often generated during ingestion, not stored as a durable contract. If the ingestion job changes its chunking strategy, reprocesses the same source under a new revision, or writes multiple collections for different embedding models, an ID-only delete can miss old derived records.

The second obvious solution is metadata-filter deletion. Pinecone supports deletion by metadata filter for documents or records Pinecone docs, and Chroma supports deletion with a where filter Chroma docs. Filter deletes are useful when a team needs to remove every chunk where source_id = customer_123/manual.pdf or tenant_id = acme. They also introduce a new failure mode: the delete is only as correct as the metadata contract. If one ingestion path forgets source_id, writes a different tenant field, or stores a stale source revision, the filter looks broad but behaves narrow.

The third obvious solution is to delete the whole namespace or collection. That is operationally attractive for single-tenant indexes or hard reset workflows. It is usually too blunt for shared retrieval systems. It can remove unrelated records, trigger expensive reingestion, and create availability issues while the index is rebuilt. Pinecone names namespaces as the scope for many delete operations Pinecone docs, which is helpful only if the application designed namespace boundaries around deletion domains before the incident.

The pattern is the same in all three cases. Delete APIs remove records according to the address the application gives them. They do not prove that the application used the right address.

How Deletion Actually Works

A production delete has at least four layers.

First, there is the source layer. This is the durable object the business understands: a document, message thread, ticket, customer workspace, product manual, policy page, or code file. The source layer needs a stable source_id, a tenant boundary, a revision, and a deletion reason. Without that identity, the retrieval layer cannot know what all derived artifacts mean.

Second, there is the derived artifact layer. A single source can create chunks, embeddings, sparse vectors, summaries, citation spans, thumbnails, and prompt cache entries. Each derived artifact should carry the same source identity plus an artifact type and generation version. When the deletion request arrives, the system should know whether it is deleting only vector records or every artifact derived from the source.

Third, there is the storage mutation layer. Vector databases expose different mutation semantics. Qdrant documents that point modification operations are asynchronous, are first written to a write-ahead log, and that a wait=false or unspecified request returns an acknowledgement that does not mean the data is already available for retrieval Qdrant docs. Elasticsearch delete-by-query is also instructive: it uses a snapshot, can encounter version conflicts, processes matching documents in batches, retries rejected requests with exponential backoff, and supports request throttling Elasticsearch docs. Even outside vector databases, deletion is often a workflow, not a single instant.

Fourth, there is the retrieval behavior layer. The user does not care that a backend accepted a mutation if the application still retrieves stale context. A delete should not be considered complete until representative read-after-delete probes fail to retrieve the removed chunks, citation lookups no longer resolve to the deleted source, and cached answers that included the source have expired or been invalidated.

A Deletion Contract for RAG

A useful deletion contract is a small schema and a state machine.

deletion_request:
  deletion_id: del_2026_09_28_001
  tenant_id: acme
  source_id: acme_policy_handbook
  source_revision: rev_42
  scope:
    artifacts: [chunks, embeddings, citations, summaries, caches]
    collections: [rag_text_v3, rag_code_v2]
  selector:
    source_id: acme_policy_handbook
    source_revision_lte: rev_42
  requested_at: 2026-09-28T13:00:00Z
  verification:
    read_after_delete_queries: 12
    require_zero_source_hits: true

The schema does three things. It separates business identity from vector IDs. It names every derived artifact class that must be removed or invalidated. It records how the system will verify retrieval behavior after storage mutation.

The state machine can be simple.

requested -> targeted -> mutation_submitted -> mutation_applied
          -> caches_invalidated -> retrieval_verified -> complete
          -> failed

targeted means the system resolved the request into concrete IDs, filters, namespaces, collections, and cache keys. mutation_submitted means the vector store accepted the delete call. mutation_applied means the backend reports completion or the system has waited for the relevant consistency boundary. caches_invalidated means answer caches, reranker caches, and citation materialized views have been cleared or marked stale. retrieval_verified means probes no longer surface the deleted source.

This contract also gives operators a place to put failure evidence. If a Qdrant operation returns only an acknowledgement while wait is not set Qdrant docs, the state machine should not jump straight to complete. If an Elasticsearch delete-by-query reports version conflicts or partial failures, the state machine should preserve that evidence instead of pretending the delete was atomic Elasticsearch docs.

Failure Modes and Trade-offs

The first failure mode is orphaned chunks. This happens when the source table knows that a document was deleted, but the vector collection has chunks with no durable source reference. The fix is not a better delete script. The fix is to make source lineage mandatory at ingestion time.

The second failure mode is overbroad filter deletion. A filter like tenant_id = acme can be correct during tenant offboarding and catastrophic during a single-document delete. Filter deletes need dry-run counts, sampled targets, and operator-visible selectors. Pinecone and Chroma both expose filter-based deletion Pinecone docs Chroma docs, so the application layer has to decide when filters are safe enough.

The third failure mode is asynchronous completion drift. A delete acknowledgement is not always the same as query-visible deletion. Qdrant makes this distinction explicit by documenting acknowledgement behavior and eventual consistency when wait=false or no wait parameter is specified Qdrant docs. Systems that need strong user-facing deletion semantics should either request synchronous completion where the backend supports it or record the delete as pending until probes pass.

The fourth failure mode is stale generated output. A cached answer can keep quoting a deleted source even after the vector record is gone. This is why deletion scope must include answer caches, citation caches, and any generated summaries that were built from the source. If cache invalidation is too expensive, the system should at least attach source revision receipts to cached outputs so stale answers can be rejected at read time.

The fifth failure mode is audit blindness. If the only log line is delete succeeded, nobody can reconstruct what was targeted. A useful audit record includes the selector, target counts, backend responses, verification queries, timestamps, and final state. That record should not contain the sensitive content being deleted, but it should contain enough metadata to prove the system acted on the right objects.

Practical Implementation Considerations

Start by making ingestion produce a manifest. Each source revision should write a manifest that lists chunk IDs, embedding IDs, collection names, citation spans, and derived summaries. If your system cannot produce this manifest, deletion will depend on reverse-engineering metadata from the vector store during an incident.

Next, treat metadata as an API, not decoration. Required fields should include tenant_id, source_id, source_revision, artifact_type, chunk_index, ingestion_run_id, and embedding_model. The same field names should be used across collections. A vector database filter can only enforce what ingestion consistently writes.

Then add a deletion ledger. The ledger should be append-only and should track every state transition. This is especially useful when the backend has asynchronous or batched behavior. Elasticsearch delete-by-query documents batching, retries, throttling, and version-conflict behavior Elasticsearch docs. Those operational details belong in the deletion record because they affect whether the system can claim completion.

Finally, verify behavior with queries, not just API responses. For each delete, run a small set of probes that previously matched the source, plus metadata-filter probes against the source ID. The goal is not to prove the entire semantic space is clean. The goal is to catch obvious stale retrieval paths and prevent the system from marking deletion complete while known source-linked records still appear.

What I Would Do in Production

I would design deletion before choosing the vector database. The minimum production contract is a source manifest, mandatory lineage metadata, ID deletes for known artifacts, filter deletes as a fallback, cache invalidation, and read-after-delete verification. I would keep the delete state visible in operations dashboards until retrieval verification passes.

I would also separate deletion domains. If tenant-level deletion is common, use tenant-scoped namespaces, collections, or indexes where practical. If document-level deletion is common, do not rely only on namespace deletion. The storage boundary should match the most common deletion boundary.

For high-risk deletes, I would add a dry-run mode that counts candidate records and samples their metadata before mutation. For low-risk automated deletes, I would still log target counts and selectors. When a backend exposes cost or rate controls, such as Pinecone write-unit accounting for deletes Pinecone docs or Elasticsearch throttling Elasticsearch docs, I would make those controls part of the operational plan rather than discovering them during cleanup.

Most importantly, I would not expose a user-facing "deleted" state until the retrieval layer agrees. Storage accepted is not the same thing as retrieval clean.

FAQ

Why is deleting by vector ID not enough?

Vector IDs are derived artifacts. If every chunk ID is durably tied to the source revision, ID deletion can be precise. If the mapping is incomplete, ID deletion can miss old chunks, alternate embeddings, summaries, or cached citations. A deletion contract starts from source identity and resolves into vector IDs, not the other way around.

When should I use metadata-filter deletion?

Use filter deletion when the metadata contract is strong and the intended scope is naturally expressed as metadata, such as a source ID, tenant ID, or source revision. Because Pinecone and Chroma both support filter-based deletion Pinecone docs Chroma docs, the hard part is not API support. The hard part is proving every relevant record has correct metadata.

How should asynchronous deletes be verified?

Do not treat acknowledgement as completion unless the backend documents that behavior. Qdrant explicitly distinguishes accepted operations from query-visible availability when wait=false or wait is unspecified Qdrant docs. Track the delete as pending, wait for the relevant completion signal, then run retrieval probes against the deleted source identity.

What should a deletion audit record include?

Include the deletion ID, requester or system trigger, tenant ID, source ID, revision scope, selectors, target counts, backend responses, cache invalidation actions, verification probes, timestamps, and final state. Do not store the deleted content in the audit record. Store enough metadata to prove which derived artifacts were targeted and whether retrieval verification passed.

Key Takeaway

Vector stores give you deletion primitives. RAG systems need deletion contracts. The contract is what connects source identity to derived artifacts, async storage behavior, cache invalidation, and retrieval verification. Without it, a delete can succeed at the API layer while the AI system still remembers what it was supposed to forget.

References

1. Pinecone Docs, Delete data
2. Qdrant Docs, Points
3. Chroma Docs, Delete Data
4. Elasticsearch API documentation, Delete documents
5. Weaviate official docs index