A vector database is the better starting point when the primary job is semantic retrieval over documents, products, tickets or other unstructured content. A knowledge graph is stronger when the answer depends on explicit entities, relationships, rules, provenance and multi-hop reasoning. Most serious enterprise AI systems eventually need a hybrid: vector retrieval finds relevant evidence while a graph resolves identity, relationships, authority and temporal context.
Do not choose from market fashion. Start with the questions users must answer, the evidence required and the consequences of a wrong result.
What a vector database actually does
A vector system converts content into embeddings and retrieves items with similar meaning. It handles paraphrases and fuzzy language better than keyword search and is useful for support, document discovery, recommendation and retrieval-augmented generation.
Its weakness is that similarity is not truth. A close passage may be obsolete, unauthorised or about a different entity. Metadata filters and reranking improve retrieval, but teams must design freshness, permissions and provenance.
What a knowledge graph actually does
A knowledge graph represents entities and typed relationships: a customer owns an account, a contract supersedes another contract, a component depends on a library, or a policy applies to a region. It supports traversals, rules and explainable paths.
Its weakness is modelling and maintenance. Entity resolution, ontology design and change governance require sustained data work. A graph does not automatically understand every document; it needs extraction and reconciliation.
The comparison dimensions
Use-case fit: vectors excel at discovery; graphs excel at relationships and constrained reasoning.
Data shape: vectors absorb unstructured content quickly; graphs reward stable entities and schemas.
Precision: vectors rank probable relevance; graphs can enforce explicit facts and rules.
Explainability: vector results need source citations; graph paths provide relationship-level explanation.
Freshness: both require pipelines, but graphs must manage entity and edge validity while vector indexes must handle replacement and deletion.
Permissions: vector metadata can filter access; graphs can model entitlements but require careful query enforcement.
Cost: vector pilots are usually faster; graph programmes often require greater upfront modelling but can serve many use cases.
When vector retrieval is enough
Choose vector-first for policy search, product documentation, case summarisation, semantic discovery and low-risk assistants where each answer can cite a small set of documents. Add keyword retrieval for exact identifiers, metadata filters for permissions and reranking for relevance.
A vector-only design becomes fragile when users ask who approved what, which contract governs a subsidiary, how two incidents are connected or what was true at a particular time.
When a knowledge graph earns its cost
A graph is justified when entity identity and relationships create business value across use cases. Examples include fraud networks, customer 360, supply-chain dependencies, life-sciences evidence, telecom assets, software dependencies and regulatory obligations.
The graph should have accountable domain owners and measurable decisions it improves. Building an enterprise ontology before a validated use case is a common expensive failure.
The hybrid architecture
Ingest documents and structured sources with identity, timestamps, permissions and provenance. Resolve entities into a graph. Chunk and embed narrative content into a vector index. At query time, classify intent, enforce identity and permission, retrieve semantic evidence, expand relevant graph neighbourhoods, rerank the combined context and generate an answer with citations.
Store the evidence path, retrieval version and policy decision for audit. Keep the system modular so retrieval or graph technology can change without rewriting the product.
Temporal context and conflicting truth
Enterprise data contains superseded contracts, draft policies and historical states. Add valid-from, valid-to and status semantics. Prefer authoritative sources by domain and expose conflicts instead of blending them silently.
A graph can represent version relationships, while the vector layer must prevent old chunks from outranking current approved material. The application should tell users when an answer is historical or uncertain.
Governance and security
Apply source-level and row-level access before retrieval, not after generation. Propagate deletions and consent changes into indexes and graphs. Track data lineage, embedding model, extraction model, schema version and query logs.
Test prompt injection in retrieved documents, cross-tenant leakage, graph traversal beyond entitlement and inference attacks. Sensitive relationships may reveal information even when individual nodes appear harmless.
Implementation roadmap
Start with twenty to fifty high-value questions and a scored evaluation set. Build the simplest vector baseline. Analyse failure modes. Add graph capability only where relationship, identity or rule failures are material. Pilot one bounded domain with named owners.
Measure answer correctness, citation support, retrieval recall, permission compliance, latency, cost and task completion. Expand the model through governed domain increments.
Build versus buy
Managed vector services reduce infrastructure work; database extensions may be sufficient when scale and workload are moderate. Managed graph platforms accelerate operations, while specialist graph databases offer richer traversal. The decision should consider data gravity, skills, portability, security and expected query patterns.
Avoid buying a graph product before proving ontology ownership, or a vector platform before understanding metadata and deletion requirements.
Common failure modes
Teams treat embeddings as a database of record. They build a graph without product questions. They ignore time and provenance. They create separate entity identities in every pipeline. They evaluate only demo questions. They add a graph to fix poor source data.
Architecture cannot compensate for missing ownership and unreliable data.
Reference architecture in detail
The ingestion layer should preserve source identifiers, timestamps, permissions and document status. Entity resolution should create durable identifiers and confidence scores rather than silently merging similar names. The graph layer stores governed relationships and temporal validity. The vector layer stores semantically searchable chunks linked back to source and entity IDs.
The query layer classifies the question, retrieves candidates, traverses relevant relationships, enforces access, reranks evidence and constructs a bounded context. The answer layer cites sources, expresses uncertainty and records the retrieval trace. Evaluation and observability span every layer.
Evaluation framework
Create question groups for lookup, relationship, comparison, temporal reasoning, aggregation and policy. For each, record expected evidence, allowed sources and unacceptable failure. Measure retrieval recall, entity-resolution precision, graph-path correctness, citation support, answer quality, permission enforcement, latency and cost.
Compare keyword, vector, graph and hybrid baselines. A hybrid architecture should earn its complexity through measurable gains on important questions.
Ontology and entity-governance model
Begin with the minimum concepts required by the product. Name a domain owner for each entity and relationship. Define identifiers, merge and split rules, cardinality, provenance, temporal semantics and quality thresholds.
Use change review for high-impact schema evolution and automate validation. Avoid a central team designing every domain without business ownership. Federated governance works when shared identity and interoperability rules remain explicit.
Operating and cost model
Vector cost includes embedding generation, storage, indexing, retrieval, reranking and refresh. Graph cost includes modelling, entity resolution, storage, traversal and stewardship. Hybrid cost adds orchestration and evaluation.
Model cost per governed answer and per updated source, not only infrastructure. Include correction and review. Managed services reduce operations but may increase data movement or lock-in; self-managed systems require specialist capability.
Industry examples
A bank can use vectors for policy passages and a graph for customer, account, beneficial-owner and rule relationships. A manufacturer can connect parts, suppliers, plants and incidents while retrieving manuals. A software company can connect services, owners, dependencies and vulnerabilities while searching runbooks. A healthcare organisation can link patient-safe concepts and evidence while enforcing strict access.
Each scenario depends on identity and relationship quality, not simply adding GraphRAG.
Detailed execution checklist
Question inventory
Collect real questions and classify whether they require similarity, exact lookup, relationships, temporal state, aggregation or rules. Architecture should be derived from this distribution.
Source authority
Rank systems of record and define conflict policy. A retrieved document and a graph edge need provenance and status so the system can prefer approved current evidence.
Entity resolution
Specify identifiers, matching features, confidence, review and correction. False merges create convincing wrong relationships; missed matches fragment the customer or asset view.
Chunking and embeddings
Choose boundaries that preserve meaning and citations. Test models on domain language, identifiers and multilingual content. Re-embedding strategy must include versioning and cost.
Graph modelling
Model only relationships required by validated decisions. Add temporal validity, direction and ownership. Avoid turning every noun into an entity before a product need exists.
Retrieval orchestration
Define when to use keyword, vector, structured query and graph traversal. Rerank combined evidence and cap context. Record why each item was selected.
Permission enforcement
Apply user and tenant policy before retrieval and traversal. Test indirect relationship leakage. Propagate source revocation and deletion into every derived store.
Answer evidence
Require citations that resolve to accessible source content. Show relationship paths for graph-derived conclusions and distinguish facts from model inference.
Freshness
Monitor ingestion delay, failed updates, stale embeddings and expired relationships. Give users a visible data timestamp for consequential decisions.
Fallback behaviour
When evidence conflicts or is insufficient, ask for clarification, show alternatives or escalate. The system should not fill a missing graph path with fluent speculation.
Observability
Trace query classification, retrieval, traversal, reranking, prompt, answer, cost and feedback. Protect logs while retaining enough evidence to diagnose failures.
Lifecycle governance
Assign owners for sources, ontology, evaluation and product decisions. Revalidate architecture as questions and data change.
Practitioner planning workbook
Use-case economics
For use-case economics, quantify decision value, user volume, error consequence and expected reuse. Record the current state, target state, accountable owner, dependencies and evidence required for approval. Test the assumption using production-like information, and define the failure signal that would stop or redirect the work.
Question taxonomy
The question taxonomy workstream should separate semantic lookup, exact lookup, relationships, time, aggregation and rules. Convert the decision into measurable acceptance criteria, an owner and a review date. Include normal operation, edge conditions and recovery, because a design that works only in the happy path is not production-ready.
Source inventory
Treat source inventory as an operating control: record authority, owner, freshness, access, format and deletion requirement. Preserve the supporting artefacts, unresolved questions and agreed exceptions. Recheck the control after launch so temporary migration choices do not become permanent unmanaged risk.
Vector baseline
For vector baseline, measure keyword plus vector retrieval before adding graph complexity. Record the current state, target state, accountable owner, dependencies and evidence required for approval. Test the assumption using production-like information, and define the failure signal that would stop or redirect the work.
Embedding choice
The embedding choice workstream should test domain language, multilingual data, identifiers and model change. Convert the decision into measurable acceptance criteria, an owner and a review date. Include normal operation, edge conditions and recovery, because a design that works only in the happy path is not production-ready.
Chunk design
Treat chunk design as an operating control: preserve meaning, citation, permissions and update boundaries. Preserve the supporting artefacts, unresolved questions and agreed exceptions. Recheck the control after launch so temporary migration choices do not become permanent unmanaged risk.
Metadata policy
For metadata policy, standardise source, entity, timestamp, status, tenant and authority filters. Record the current state, target state, accountable owner, dependencies and evidence required for approval. Test the assumption using production-like information, and define the failure signal that would stop or redirect the work.
Graph justification
The graph justification workstream should identify failures that require explicit entities and multi-hop relationships. Convert the decision into measurable acceptance criteria, an owner and a review date. Include normal operation, edge conditions and recovery, because a design that works only in the happy path is not production-ready.
Ontology scope
Treat ontology scope as an operating control: model the smallest set of governed concepts required by the product. Preserve the supporting artefacts, unresolved questions and agreed exceptions. Recheck the control after launch so temporary migration choices do not become permanent unmanaged risk.
Entity resolution
For entity resolution, evaluate merge, split, ambiguity and human-review workflows. Record the current state, target state, accountable owner, dependencies and evidence required for approval. Test the assumption using production-like information, and define the failure signal that would stop or redirect the work.
Temporal modelling
The temporal modelling workstream should represent superseded, draft, current and historical truth. Convert the decision into measurable acceptance criteria, an owner and a review date. Include normal operation, edge conditions and recovery, because a design that works only in the happy path is not production-ready.
Hybrid query
Treat hybrid query as an operating control: route, retrieve, traverse, rerank and assemble evidence under a bounded context. Preserve the supporting artefacts, unresolved questions and agreed exceptions. Recheck the control after launch so temporary migration choices do not become permanent unmanaged risk.
Permission testing
For permission testing, prevent cross-tenant, relationship and inference leakage. Record the current state, target state, accountable owner, dependencies and evidence required for approval. Test the assumption using production-like information, and define the failure signal that would stop or redirect the work.
Evaluation set
The evaluation set workstream should score retrieval, path, citation, answer, latency, cost and business completion. Convert the decision into measurable acceptance criteria, an owner and a review date. Include normal operation, edge conditions and recovery, because a design that works only in the happy path is not production-ready.
Correction flow
Treat correction flow as an operating control: allow authorised owners to fix entities, edges, sources and index state. Preserve the supporting artefacts, unresolved questions and agreed exceptions. Recheck the control after launch so temporary migration choices do not become permanent unmanaged risk.
Operating ownership
For operating ownership, assign data, ontology, retrieval, product and incident responsibilities. Record the current state, target state, accountable owner, dependencies and evidence required for approval. Test the assumption using production-like information, and define the failure signal that would stop or redirect the work.
Portability
The portability workstream should isolate storage choices behind services and retain exportable canonical data. Convert the decision into measurable acceptance criteria, an owner and a review date. Include normal operation, edge conditions and recovery, because a design that works only in the happy path is not production-ready.
Scale gate
Treat scale gate as an operating control: expand only when measurable outcome improvement exceeds added lifecycle cost. Preserve the supporting artefacts, unresolved questions and agreed exceptions. Recheck the control after launch so temporary migration choices do not become permanent unmanaged risk.
