When a Knowledge Graph Beats a Vector Database

A quality engineer comes to your desk with a question. A tantalum capacitor lot failed burn-in on Tuesday, and she wants to know which shipped assemblies contain parts traceable to the same sub-tier plating vendor. Your retrieval stack embeds the question, runs cosine similarity over eleven thousand chunks of supplier documentation and hands back seven passages about tantalum capacitors. Every one is relevant. Not one answers her. That gap is the whole knowledge graph vs vector database argument, and no embedding model closes it.

She asked what connects to this. The index only knows what looks like this. Those are different operations and only one of them is a search.

Similarity has a ceiling that re-ranking cannot raise

Approximate nearest neighbour search scores every candidate against the query independently. That independence is the trick that makes it fast, and it is also the ceiling. Nothing in the scoring function knows that document A carries a lot code that also appears in document B, which names a plating plant that appears in document C. Three documents, no shared vocabulary with the question, and the answer sits in the join.

A graph does the join because the join is the data structure. Traversal is the primitive, not something that emerges from better ranking.

The published evidence is more careful than the marketing around it. A Michigan State and Meta team ran plain RAG against four GraphRAG families under one protocol, identical chunking, identical embeddings and identical generation, with revisions running through March 2026. On single-hop Natural Questions, plain RAG scored 64.78 F1 against 60.04 for the best graph variant. On HotpotQA the order flipped, HippoRAG 2 at 63.01 against 60.04. The GraphRAG-Bench authors report the same shape from a separate corpus: near parity on simple fact retrieval, then roughly ten points for graphs on complex reasoning and thirteen on contextual summarisation.

Two independent groups, one pattern. Graphs tie or lose on lookup. They win on composition.

A vector index that misses something at least looks like it missed something. A graph that misses an edge looks finished.

Extraction is the cost centre, and it is where graph projects die

Retrieval quality is downstream of edge quality, and edge quality compounds badly. If each extracted edge is correct 90 percent of the time, a two-hop path is correct 81 percent of the time and a three-hop path 73 percent. No re-ranker repairs an edge that was never extracted.

Three separate jobs hide inside the word extraction. Finding entities. Finding relations between them. Deciding that 村田, Murata, MURATA MFG and the internal vendor code MRT-0041 are one node. The third job is the one that quietly ruins projects, because it is unglamorous, it never finishes and it has no benchmark.

I spend enough time in Huaqiangbei to have a strong opinion here. Walk into SEG Plaza with a part number and half the counters will not sell you that part. They will sell you a 替代型号, a substitute they swear is pin compatible. Often it is. The knowledge of which substitute is acceptable for which thermal profile lives in a salesman's head and a WeChat group, and it never enters your corpus. I have watched the same thing in Computer Village in Ikeja, where a trader can tell you which container a board arrived in and no document anywhere records it.

The token bill is the least interesting cost. Batch the extraction pass through vLLM against an open-weight model in the Qwen family and it stops mattering. What matters is the second and third re-extraction after the schema changes, the incremental indexing when documents update, the alias table that needs a named owner and the review interface domain experts use to correct edges, because they will not file tickets. Microsoft's own graphrag package, version 3.1.2 on PyPI as of 21 August 2026, still warns that indexing can be an expensive operation and still describes itself as a demonstration rather than an officially supported Microsoft offering. Read that twice before you standardise on it.

Schema first, or you get a hairball

Open extraction, where you hand a model a corpus and ask it to find whatever it finds, produces four hundred node types, nine synonyms for supplies and not one query you can write with confidence. The graph looks impressive in a force-directed layout. It answers nothing.

Work backwards from the questions instead. Write down twenty real ones, the kind people currently answer by phoning someone. Most industrial domains collapse to something like eight to fifteen node types and ten to twenty-five relation types. Part, lot, supplier, plant, process step, defect mode, assembly, shipment, document. Edges typed, directed and time-bounded, because a supplier relationship that ended in 2023 is not the same edge as one that started in 2025.

Every edge carries provenance: source document id, character span, extractor model version, confidence score. Without that you cannot debug a wrong answer and you cannot defend a right one to an auditor.

Property graph for engineering, RDF when it crosses the company fence

GQL became ISO/IEC 39075 on 11 April 2024, the first new ISO database language standard since SQL in 1987. Property graphs put attributes directly on nodes and edges, which fits engineering data where the edge genuinely has properties: quantity per assembly, effective date, drawing revision. Modelling that in triples means reification and pain.

RDF earns its place when the graph crosses an organisational boundary and needs global identifiers plus a vocabulary somebody else also implements. A regulatory product passport is exactly that shape. Know that the standard is mid-flight though. W3C moved RDF 1.2 Concepts and Semantics to Candidate Recommendation and invited implementations with comments due 5 May 2026, while the SPARQL 1.2 documents were still Working Drafts through mid-2026. Building production semantics on RDF-star right now means building on something still moving.

The pragmatic split most teams land on: property graph internally, RDF serialisation at the boundary.

A recall question that is genuinely graph-shaped

Manufacturing is the easy case for this argument because half the graph already exists and nobody had to extract it. The bill of materials is a graph. So is the routing, the supplier master and the lot genealogy sitting in the MES. Those are curated identifiers with referential integrity, which is a luxury.

Extraction earns its keep on the unstructured layer bolted to that spine: 8D reports, CAPA records, supplier audit PDFs, incoming inspection notes, field failure tickets, the email thread where an engineer mentions the plating supplier changed bath chemistry in March. The job is linking prose to identifiers that already exist. That is a far more tractable problem than inventing an ontology from nothing, and it is the version I would attempt first.

Two deadlines make this concrete. Regulation (EU) 2023/1542 entered into force on 17 August 2023 and makes a digital battery passport mandatory from 18 February 2027 for EV batteries, light means of transport batteries and industrial batteries above 2 kWh, covering carbon footprint, recycled content and supply chain due diligence. In the US, the FDA's Food Traceability Rule under FSMA 204 was pushed back 30 months by a Federal Register notice on 7 August 2025, moving compliance from 20 January 2026 to 20 July 2028, with Congress making the delay binding that November. Both require answers about upstream nodes you never buy from directly. That is a two-hop query with a legal deadline attached.

Vectors find the door, the graph walks the building

Hybrid retrieval is the part that works, and it is not complicated. Embed the question and run ANN with metadata filters to get seed entities. Expand one or two hops over an allowlist of relation types chosen per question class, with a degree cap so one hub node does not drag in the whole graph. Prune the resulting paths by hop count and edge confidence. Re-rank the union of chunk evidence and path evidence together. Synthesise with node ids in the citations so a human can open the graph and check.

One detail costs teams months. Do not trust embeddings for identifiers. Cosine similarity over C0805C104K5RACTU is close to meaningless, and lot codes are worse. Run an exact and lexical channel beside the dense one so identifiers hit by string match. Half the multi-hop failures I have seen were really identifier failures at hop zero.

Storage is a smaller decision than vendors suggest. Neo4j's vector index runs on Lucene HNSW, one to 4,096 dimensions, cosine or euclidean, with the Cypher SEARCH clause preferred from 2026.01 and in-index property filtering generally available in 2026.02, which removes the over-fetch-then-post-filter dance. Or keep vectors in pgvector, Qdrant or Milvus and the graph separately, joined on a stable entity id. The join key discipline matters more than the product.

For questions with no single answer node, such as what are the recurring themes across three years of supplier corrective actions, you need community structure rather than paths. Microsoft's GraphRAG clusters with the Leiden algorithm and pre-summarises each community. LazyGraphRAG defers that summarisation to query time and claims indexing cost identical to vector RAG, 0.1 percent of full GraphRAG and over 700 times lower query cost than global search. Worth reading. Also worth noting it was judged by LLM head-to-head comparisons on comprehensiveness, diversity and empowerment over 5,590 AP news articles and 100 synthetic queries, which is precisely the evaluation format now under attack. Ant Group's OpenSPG and its KAG framework take a different route, mutual indexing between graph and source text with logical-form-guided retrieval, and the Chinese-language write-ups are more candid about failure modes than the English ones.

The counter-argument I mostly agree with

Most teams asking me for a knowledge graph do not have a graph problem. They have a chunking problem wearing a graph costume. Fixed 512-token windows that split a table from its header. No document-level metadata, so you cannot filter to one plant or one quarter. No lexical channel. Top-k of 5 against a corpus that needs 40. Fix those and a good number of the failing questions start passing, in about a week, with no new database.

The stronger version of the objection is empirical. The GraphRAG-Bench authors state plainly that GraphRAG frequently underperforms vanilla RAG on real-world tasks and trace the cause to graph construction rather than retrieval. A separate paper on unbiased evaluation, revised in August 2026, documents position bias, length bias and trial bias in the LLM-judged comparisons that most GraphRAG results rest on, and finds gains that are much more moderate than reported previously once corrected. Treat every head-to-head chart with suspicion, including the ones from labs you like.

Here is the test that actually separates the two cases. Take those twenty real questions. Count how many require joining facts across documents that share no vocabulary with the question or with each other. Under a third, you have a chunking problem. Over a third, and particularly if the answers currently come from one person's memory, you have a graph problem.

And when it is genuinely a graph problem, do not choose. The Michigan State and Meta study found that running both retrievers and concatenating results gave a 6.4 percent improvement on MultiHop-RAG with Llama 3.1 70B, against 1.1 percent for routing each query to one system. Combining beat choosing by roughly six to one. Their own conclusion is that the two should not be treated as mutually exclusive.

The edges nobody writes down

The graphs that work are the ones where most of the structure already existed and the model's job was linking, not inventing. The moment you ask a model to derive the ontology and populate it and resolve the entities, quality falls off a cliff nobody measures until an auditor arrives.

There is a harder limit underneath that. The edges that matter most in a supply chain are the ones nobody documented. Two vendors that quietly share a plating line. The shift that runs hot on Friday nights. The distributor who relabels reels. None of it is in your corpus, so none of it enters your graph, and the graph will present its incomplete answer with exactly the same confidence as a complete one. A vector index that misses something at least looks like it missed something. A graph that misses an edge looks finished.

Tools referenced

pgvector, reviewed here: pgvector review.

Qdrant, reviewed here: Qdrant review.

Milvus, reviewed here: Milvus review.

LlamaIndex, reviewed here: LlamaIndex review.

LangGraph, reviewed here: LangGraph review.

RAGFlow, reviewed here: RAGFlow review.

Sources

Han et al., RAG vs. GraphRAG: A Systematic Evaluation and Key Insights (arXiv 2502.11371): https://arxiv.org/abs/2502.11371

Xiang et al., When to use Graphs in RAG / GraphRAG-Bench (arXiv 2506.05690): https://arxiv.org/abs/2506.05690

Zeng et al., An Unbiased Evaluation Framework for GraphRAG (arXiv 2506.06331): https://arxiv.org/abs/2506.06331

Microsoft Research, LazyGraphRAG sets a new standard for quality and cost: https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/

graphrag package on PyPI, version 3.1.2: https://pypi.org/project/graphrag/

Neo4j Cypher Manual, vector indexes and the SEARCH clause: https://neo4j.com/docs/cypher-manual/current/indexes/semantic-indexes/vector-indexes/

Federal Register, FSMA 204 Food Traceability Rule compliance date extension (7 August 2025): https://www.federalregister.gov/documents/2025/08/07/2025-14967/requirements-for-additional-traceability-records-for-certain-foods-compliance-date-extension

VentureBeat, Stop graphing everything: when GraphRAG actually beats vector RAG: https://venturebeat.com/orchestration/stop-graphing-everything-when-graphrag-actually-beats-vector-rag