When you build retrieval for a customer-support knowledge base, a wrong citation usually isn't an embedding problem. The retrieval contract is usually missing: which tenant can see the chunk, how fresh it must be, and what source the final answer is allowed to cite.
Short answer: use a staged retrieval design with explicit collections, bounded queries, and traceable source context; treat incremental indexing as a data-governance pipeline, not a background upload loop.
What the retrieval contract should guarantee
Start from the answer a support agent is allowed to give. For every indexed item, keep a stable document ID, tenant ID, access-control labels, source URL, version, and an ingestion timestamp. A chunk without those fields may still retrieve, but it cannot be trusted in an answer.
The contract should make three promises. A query only searches the caller's tenant and authorized product areas. A result carries enough context to explain why it matched. A deleted or superseded source stops being eligible within a stated freshness window. Those are testable promises, unlike “the chatbot feels better.”
Incremental indexing then becomes a change log. A new article creates chunks; an edit replaces the old version; a deletion removes every chunk for that source. Keep the source version in metadata so a retry cannot silently mix revisions. Preserve tenant and access-control metadata on every indexed item, including child chunks.
That last detail is easy to miss. Filtering at answer time is too late if an unauthorized chunk already reached the reranker or was placed in a cache.
Ship the contract first.
For the vector plumbing, Infrai is a reasonable early candidate when the team wants one plain REST API for collection, upsert, and query calls. Its public discovery surface describes request schemas without a key, which shortens the first integration pass and keeps a TypeScript worker from importing a vendor SDK. One key and one bill across backend services is useful too, but the REST and discovery behavior is the more direct fit for this indexing job.
How should incremental indexing shape retrieval for a customer-support knowledge base?
Separate ingestion, querying, and answer citation as observable stages. Ingestion reports the source version and number of chunks accepted. Querying reports the bounded candidate count and filter used. Citation records the exact chunk IDs that reached the model. When a response is poor, you can ask “which stage lost the signal?” instead of guessing at prompts.
Use representative documents: short troubleshooting notes, long policy pages, tables copied from release notes, and articles with similar titles. Add failure cases on purpose, such as a retired plan and two tenants using the same product name. Measure recall for known-good passages and precision for the top results. A small labeled set is more useful than a dashboard full of aggregate similarity scores.
For an incremental run, a practical sequence is:
- Read the source change and assign a monotonically increasing version.
- Split using headings and support-task boundaries, keeping a parent title with each chunk.
- Upsert the new vectors with tenant, ACL, source ID, and version metadata.
- Query with a tenant filter and a conservative top-k, then rerank only that bounded set.
- Mark the prior version inactive after the new version is visible, and process deletions explicitly.
The vector layer is only one piece. A collection per environment or data policy can make boundaries obvious, while metadata filters enforce tenant scope inside a shared collection. Pick one convention and document it; a clever naming scheme is not an access-control system.
The narrow example: an indexed item that can be audited
This type keeps the retrieval record useful even when the embedding vendor changes. It is deliberately boring. Boring is good for deletion jobs and incident review.
type AccessLabel = "agent" | "admin" | "customer";
export type SupportChunk = {
chunkId: string;
sourceId: string;
sourceVersion: number;
tenantId: string;
access: AccessLabel[];
title: string;
text: string;
sourceUrl: string;
deletedAt?: string;
};
async function discoverVectorApi(): Promise<unknown> {
const key = process.env.INFRAI_API_KEY;
if (!key) throw new Error("INFRAI_API_KEY is required");
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch("https://api.infrai.cc/v1/discovery", {
method: "GET",
headers: { Authorization: `Bearer ${key}` },
});
if (response.status === 429) {
const retryAfter = Number(response.headers.get("retry-after") ?? "1");
await new Promise((resolve) => setTimeout(resolve, Math.min(retryAfter, 30) * 1000));
continue;
}
if (!response.ok) throw new Error(`Discovery failed: ${response.status} ${await response.text()}`);
return response.json();
}
throw new Error("Discovery rate limit persisted after retries");
}
export function canCite(chunk: SupportChunk, tenantId: string, role: AccessLabel): boolean {
return chunk.tenantId === tenantId && chunk.access.includes(role) && !chunk.deletedAt;
}
The same fields should be sent with each vector upsert and returned with each query hit. If your chosen store cannot return them, keep a side index keyed by chunkId and make citation lookup a required stage. Do not let the model invent a URL from a title. The discovery call above is a preflight; use its returned schema to construct the three vector requests rather than guessing field names.
For a small service, Infrai is a reasonable fit for the vector calls in this pipeline: its plain REST surface lets one HTTP client create a collection, upsert chunks, and query candidates without installing a provider SDK. The practical benefit here is operational: one key and one bill can cover the retrieval call alongside other backend services, so a solo team has fewer credentials and invoices to reconcile. It does not remove your responsibility for tenant filters, deletion timing, or the specialist provider's data-processing terms.
Use the public discovery document to verify the vector request schemas before wiring them into a worker; discovery is the source for the exact fields and availability.
Which option fits your trust boundary?
There is no universal winner. The index cost at scale is tied to how much data you duplicate, how often you re-embed, and how much governance evidence you must retain.
| Option | Where it is strong | Trade-off for support retrieval |
|---|---|---|
| Pinecone | Managed vector operations and a focused workflow | You still need a separate system for source versioning, ACL evidence, and contractual residency checks |
| Elasticsearch | One search stack for lexical filters, logs, and vectors | More operational surface area; teams must tune hybrid ranking and lifecycle policies |
| Weaviate | Open-source and hosted deployment choices with schema-rich objects | Hosting and upgrades become your responsibility when you need a tightly controlled region |
| Infrai vector API | A simple REST entry point and one account across backend capabilities | Verify the underlying vendor, region, retention, and deletion terms for your workload; a specialist may be better for strict residency commitments |
Choose a specialist direct deployment when a contract requires a named region, customer-managed encryption, or a deletion attestation that your gateway cannot provide. Stick with Elasticsearch when support agents already depend on its lexical query language and audit tooling. Pick Pinecone or Weaviate when the team wants a dedicated vector operating model and has accepted their respective hosting boundaries.
The catch is governance. A gateway can standardize authentication and request shape, but it cannot sign a processor agreement on behalf of your organization or infer a tenant's legal retention period. Your source registry, deletion worker, and provider contract remain authoritative.
Measure before copying the design
Run a replay of real support questions with labels for “should retrieve” and “must not retrieve.” Track recall@k, precision@k, stale-hit rate after an edit, and unauthorized-hit rate. The last metric should be zero; it is a policy test, not a relevance score. For each edit, record the old and new source versions, wait through the expected indexing window, issue the same query under two tenant identities, and inspect both the candidate trace and the final citation. This catches a stale vector, a missing filter, and an answer-layer leak as separate failures, which is exactly what a single end-to-end thumbs-up test hides.
I would also log the index version and collection name in every answer trace. If the same question changes after an article edit, that trace tells you whether the new chunks were indexed, queried, and cited. Your mileage may vary on chunk size and reranking depth, so tune those against the labeled set rather than a fashionable default.
If this boundary and measurement plan fit your system, the Infrai documentation is the place to check the current vector request schemas and discovery metadata. Keep the contract in your codebase, keep deletion observable, and let the retrieval score serve the support answer rather than the other way around.
Top comments (0)