Why RAG Isn't Enough for Legal AI
Chunk it, embed it, retrieve the nearest match, that's RAG. It works for support docs. It quietly breaks on law, and not because of the model.
Most AI products that touch legal text are built the same way: split the documents into chunks, embed them, and retrieve the nearest neighbors to a query at run time. That pattern works well for a support knowledge base or internal wiki, and it's why RAG became the default architecture for "make an LLM answer from our documents." Applied to law, it fails in ways that don't show up in a demo and only show up once someone relies on the answer.
The failure isn't the model, it's what got retrieved
A repealed article and the amendment that replaced it are often nearly identical in wording, sometimes differing by a single clause. To a vector embedding, they're neighbors. Nothing in a chunk of statutory text carries the information "this was struck down in 2021" or "superseded by Royal Decree M/132." The embedding only knows what the words say, not whether the words are still law. So a standard RAG pipeline retrieves whichever chunk scores highest on semantic similarity, hands it to the model as ground truth, and the model dutifully cites something that stopped being enforceable years ago. This is exactly what we see in due diligence workflows against the newer Saudi Companies Law: the 2015 statute's language is close enough to the current one that plain retrieval surfaces the wrong version almost as often as the right one.
A vector index knows what a sentence says. It has no way to know whether the sentence is still true.
Chunking breaks the structure the law actually depends on
Splitting a document into fixed-size chunks assumes each chunk is a self-contained unit of meaning. Legal text isn't written that way. An article that caps liquidated damages only makes sense next to the article that defines when a court can intervene to reduce them, two provisions that a naive chunker will happily split across two separate embeddings with no link between them. A judicial principle from a Cassation Court ruling means one thing on its own and something else entirely once you know the General Assembly later overturned it, a relationship that lives in a completely different document a flat chunk store has no way to surface alongside the original.
Exact citation lookup is a search problem RAG wasn't built for
A huge share of real legal queries aren't semantic at all, they're exact: "what does Article 224 say right now," not "find me something about damages." Vector search is built to find things that mean roughly the same thing as the query, which makes it a poor fit for a question that has exactly one correct, retrievable answer. Hybrid retrieval, keyword and exact-match alongside semantic search, isn't an optimization here. It's the difference between answering the question that was asked and answering an approximately similar one.
What this actually requires
None of this is a criticism of any particular model. It's an architecture problem: a flat store of embedded chunks has no concept of legal validity over time, no representation of the relationships between a statute, its amendments, its implementing regulations, and the case law that interprets it, and no reliable path to an exact citation. We built Dike as a citation graph specifically because these relationships need to be first-class, queryable structure, not something a retriever hopes to reconstruct from proximity in vector space. When an agent asks for an article, it gets the current version, a flag if it's been amended or repealed, and links to what interprets it, not just whatever chunk happened to score highest.
RAG isn't wrong for every problem. It's wrong for the specific claim "this text is still the law," and that's the one claim a legal AI product can't afford to get wrong.