# Dike Model: Legal Document Typing That Knows When to Ask a Human

> A compact 278M-parameter model that types legal documents across five jurisdictions in real time, matches a model roughly 100 times its size where it answers, and sends everything else to review.

**Type:** Model Report | **Track:** Model Engineering | **Jurisdictions:** EG, SA, AE, GCC | **Date:** September 2026 | **Author:** Dike Research Team | **Read time:** 7 min read

## Abstract

Dike now has a small, calibrated model that reads a legal document and answers two questions: what kind of document is this, and which area of law does it belong to. It is 278M parameters, answers 845 documents per second on one GPU, and does one thing most classifiers skip: it tells you when it is not sure. On the 62.4% of held-out documents it answers on its own, it agrees with a reference model roughly 100 times its size 99.24% of the time. The rest go to a ranked review queue instead of into the index with a guess attached.

## Key Stats

- **Model parameters:** 278M
- **Documents per second:** 845/s
- **Agreement on answered:** 99.24%
- **Wrong types auto-accepted:** 0 of 2,704

## Key Findings

- The Dike model reproduces the domain labels of a reference model roughly 100 times its size, with 99.24% agreement on the 62.4% of held-out documents it chooses to answer (1,480 documents, confidence bar 0.975). The rest go to a ranked review queue instead of getting a guess.
- Document type is close to solved for known sources: a source prefix in the document ID determines the type for all but a handful of documents. The model is the safety net for new sources: on 2,704 unseen documents it was right 99.8% of the time, auto-accepted 97.3% of them, and made zero errors among the ones it auto-accepted.
- It is fast enough to sit in the ingestion path: 845 documents per second on one GPU, 30 per second on a CPU alone, with identical answers on both. That is roughly 170 times faster than the large reference model that labeled our training data.
- Every number here is agreement with a large reference model, not accuracy against lawyers. No human-labeled reference set exists yet, so we built the model to make its uncertainty visible and its abstentions cheap, and every response carries a provisional warning until that changes.

## 1. Metadata decides whether legal AI can be trusted

A legal search API is only as good as its metadata. Whether a record is a statute, a regulation, a court ruling or a treaty, and which area of law it belongs to, decides what a query retrieves and what an AI agent is allowed to cite. Across our corpus that metadata was uneven: some sources were clean, others had never been typed, and the corpus keeps growing. A large language model can type documents well, but it is slow and expensive at this scale. We wanted its judgment at the cost of a small model, in a shape a product can depend on: fast, typed, calibrated, and willing to say "I am not sure".

## 2. Big-model judgment at small-model speed

The Dike model gives two typed answers instead of free text: a domain (12 labels, from financial to criminal) and a document type (6 labels). To build it, a large open model, Qwen3.8-27B, first labeled tens of thousands of documents. Each document was asked several ways, so that the order of the answer choices could not sway the result, and answers that did not hold up were re-checked and given less weight. We then trained a compact 278M-parameter multilingual encoder, XLM-RoBERTa base, to reproduce those judgments. Nearly half of the labeling effort went to the documents the model found hardest, not to easy ones. The result keeps most of the large model's judgment at a fraction of its cost.

**Where the 82,025 training labels came from**

| Item | Value |
| --- | --- |
| Random and balanced sample | 30,000 docs |
| Hard cases, Egypt and Saudi Arabia (first passes) | 27,577 docs |
| Kuwait, Qatar and UAE | 12,332 docs |
| Hard cases, Egypt and Saudi Arabia (later pass) | 12,116 docs |

_A random sample keeps the baseline honest; the two hard-case groups spend the labeling effort where the model was weakest. Kuwait, Qatar and the UAE were labeled separately because they had the least data._

> **Cleaning the labels paid off** (finding): Where the large model wavered on a document (14,050 of them), we asked it again in six different ways and weighted each label by how many of those answers agreed. Together with 12,116 more labels on hard cases, this raised the share of documents the Dike model can answer automatically by 1.4 points, at the same agreement on the ones it answers.

## 3. Four guardrails that make it safe to automate

Accuracy alone does not make a classifier safe to automate. What matters is what happens on the documents it gets wrong, so much of the engineering went into the guardrails around the model rather than the model itself.

**How often the domain answer changes when the text is degraded**

| Item | Value |
| --- | --- |
| Title removed | 5.1% |
| Cut to the first 500 characters | 3.5% |
| 2% OCR-style character noise | 2.8% |
| Diacritics stripped | 1.1% |
| Western digits instead of Arabic-Indic | 0.6% |
| Whitespace collapsed | 0% |

_1,000 documents per test. Document type changed in at most 0.3% of documents (title removed) and in none of the other cases._

> **The model's answer is never the last word on a conflict** (note): If the model is at least 90% sure a document is a different type than its source says, the answer is not overwritten. It is flagged for review with the reason attached. That is how we found real treaties and founding deeds stored as legislation.

| Guardrail | What it does | Why it matters |
| --- | --- | --- |
| Calibrated confidence | Confidence is fitted on held-out data (expected calibration error 0.020) | Thresholds mean something, and drift is measurable |
| Three actions | Every answer is accept, review or escalate | Only confident answers are ever automatic |
| ID rule cross-check | A source prefix fixes the document type; the model can only disagree loudly | Misfiled treaties and fatwas go to a person, not the index |
| "Unclear" is never auto-accepted | A dedicated label for documents that fit nothing | The model cannot hide a bad guess behind a category |

## 4. What it does on documents it never saw

The model rows below use held-out documents that were excluded from training; the ID-rule row covers the whole corpus. Agreement is measured on the documents the model answers on its own, and coverage is the share it chose to answer.

**What happens to held-out documents (domain, bar 0.975)**

| Item | Value |
| --- | --- |
| Answered automatically | 62.4% |
| Sent to review or escalated | 37.6% |

_Answered automatically means the model was confident enough to skip review. On that group it agrees with the reference model 99.24% of the time. Based on 1,480 held-out documents._

**Share of domain answers accepted automatically, by jurisdiction**

| Item | Value |
| --- | --- |
| Saudi Arabia | 88.5% |
| Egypt | 59.9% |
| Qatar | 57.3% |
| Kuwait | 48.9% |
| UAE | 30.3% |

_From a full-corpus shadow run in which nothing was written back. A lower rate means more documents go to human review, which is the intended behavior where the model is less sure. Kuwait, Qatar and the UAE had the least training data._

**Coverage against agreement as the confidence bar rises**

| Item | Value |
| --- | --- |
| 0.50 (98.3% answered) | 92.7% |
| 0.80 (83.4% answered) | 96.4% |
| 0.90 (75% answered) | 98% |
| 0.95 (68% answered) | 98.8% |
| 0.975 (62.4% answered) | 99.2% |
| 0.99 (56.4% answered) | 99.5% |

_Each point is a confidence bar, from 0.50 (right) to 0.99 (left). Raising the bar buys agreement at the cost of coverage. We run at 0.975. Based on 1,480 held-out documents._

> **What abstaining buys you** (note): Across the full corpus the model auto-accepts about 60% of domain answers and sends the rest to a review queue ranked by how uncertain it is. A reviewer's time goes to the documents where a wrong label is most likely, not to the ones the model is sure about.

| Answer | Documents | Answered automatically | Agreement on answered |
| --- | --- | --- | --- |
| Domain (bar 0.975) | 1,480 held out | 62.4% | 99.24% with the reference model |
| Document type, model alone | 2,704 held out | 97.3% | 99.8% overall, 0 wrong among accepted |
| Document type, ID rule | Whole corpus | All but a handful | By construction, from the source |

## 5. What an answer looks like

The API returns the same shape a language-model call would, so it slots into agent code without special handling. For each question it returns a choice, a calibrated confidence, the full probability distribution and an action. It also names the model, counts tokens, and, while the calibration is provisional, says so in a warning on every response.

| Field | Example | Meaning |
| --- | --- | --- |
| choice | financial | The most probable label |
| confidence | 0.991 | Calibrated against the reference model |
| action | accept | accept, review or escalate |
| source | id_rule / model | Which mechanism produced a document type |
| warnings | provisional | Present until refitted on human labels |

## 6. Small, fast and boring to run

The Dike model is a 1.1 GB file. It answers 845 documents per second on one GPU and 30 per second on a CPU, with the same answer on both. One document takes under 10 ms on a GPU and about 30 ms on a CPU. In our parity test, 100 of 100 documents got the same answer on both. We ran the real HTTP service through a 20-check release test on 500 documents it had never seen, and it passed all 20: malformed input gets a clean error instead of a crash, the same request always returns the same answer, results do not change with batch size, and the server log never contains document text. Under 16 parallel clients, while the GPU was also busy with a training job, it held 71 requests per second at a 95th percentile of 248 ms.

**Throughput, documents per second**

| Item | Value |
| --- | --- |
| Qwen3.8-27B | 5 docs/s |
| Dike model on a CPU | 30 docs/s |
| Dike model on one GPU | 845 docs/s |

_The Qwen figure is approximate (about 5 documents per second with option shuffling) and was measured on the same GPU while labeling. The Dike figures come from our own benchmark._

**Release test, 20 of 20 checks passed**

| Item | Value |
| --- | --- |
| Bad input and HTTP handling | 7 checks |
| Repeatable answers, batch and CPU-GPU parity | 3 checks |
| Rules every answer must obey | 6 checks |
| Load from parallel clients | 2 checks |
| Logs contain no document text | 2 checks |

_The real HTTP service, run against 500 documents it had never seen. Every check passed; the chart shows how they split by area._

## 7. What this unlocks for Dike users

Typed metadata turns search from "find text like this" into "find this kind of authority". It lets queries filter by document type and legal domain across every jurisdiction on one shared taxonomy, and lets agents cite only answers marked accept while treating review and escalate as suggestions. New sources are onboarded by checking a small verified sample first, so the model is never trusted alone on something it has not seen. Because the model runs on ordinary hardware and answers in milliseconds, it is built to type documents as they are ingested rather than in a nightly batch, which keeps metadata current as the corpus grows. We are rolling it out in shadow mode first: it logs its answers without writing them anywhere.

## 8. Where the next gains come from

More labels stopped helping. Agreement with the reference model plateaued around 92%, and the large model itself gave the same answer across differently ordered questions only about 89% of the time. A model cannot be more certain than the labels it learns from, so the next gains come from better labels: human review of the documents where the large model wavers, not a bigger model. Extra pretraining on hundreds of thousands of our own documents added 0.6 points of agreement and 1.0 point of coverage. We measured it and kept it as an option rather than adopting it.

**What each improvement added**

| Item | Value |
| --- | --- |
| Cleaner labels and more hard cases | 1.4 pts |
| Pretraining on our own documents | 1 pts |

_Extra share of documents the model can answer automatically across the whole corpus, at the same agreement on the ones it answers. The pretraining step also added 0.6 points of agreement._

## 9. Limitations and what comes next

Every domain number is agreement with a large reference model. No lawyer has labeled a reference set yet, so we cannot state the model's accuracy against people, and we do not. The stated confidence is calibrated against the reference model and is marked provisional in every response until it is refitted on human labels. The model is weaker than an earlier version at spotting a treaty or a fatwa filed under legislation, which is why the ID cross-check and the review queue stay in place. Kuwait, Qatar and the UAE have less training data than Egypt and Saudi Arabia. The next step is a human-labeled reference set: it replaces the large model as the yardstick, refits the confidence and removes the provisional warning.

## Citation

```bibtex

```