Letra da Lei
Back to blog

AI + Law

Why ChatGPT gets Brazilian law wrong

Frontier models produce text that sounds legal — but they hallucinate article numbers, effective dates, and entire case summaries. The problem isn't the model. It's the context.

·3 min read·Letra da Lei team

Ask ChatGPT about the statute of limitations for hidden defects under Brazil's Consumer Protection Code. The answer comes back fast, in confident prose, citing an article number that doesn't exist — or does exist, but covers something else entirely.

It isn't an isolated case. In tests we ran with Claude, GPT-4, and Gemini, frontier models invent Brazilian legal citations in an alarming share of queries. The problem isn't the model's intelligence. It's the grounding.

What grounding means, and why Brazilian law lacks it

Generative models are trained on huge corpora from the open web. For US law — with public bases like Cornell LII, Justia, court opinions on PACER — there's signal everywhere. For Brazilian law, signal is scarce: fragmented across Official Gazette PDFs, locked behind commercial platform paywalls, or diluted across legal blog commentary.

Three symptoms every lawyer has seen

  1. Swapped article numbers. The model cites "Art. 47 of the CDC" when it meant "Art. 27". The semantic content is close — the number is not.
  2. Invented case summaries. "STJ Summary 312" becomes a confident citation. Check the official base: it doesn't exist.
  3. Repealed provisions treated as in force. The model doesn't know that article was amended by Law 14.181/2021. It keeps answering with the old wording.

Why just adding PDFs doesn't fix it

The first instinct is: "fine, drop the Constitution into a RAG and ship it." It's not that simple.

Brazilian law has native structure. Caput, paragraphs, incisos, alíneas. Effective dates per provision, not per document. Cross-citations between laws. Stacking a PDF into a vector store ignores all of that and produces chunks that cut an inciso in half.

The article is the smallest citable unit of Brazilian law. The AI needs to receive the law the way a lawyer reads it — not as a chopped-up PDF.

— Letra da Lei, internal manifesto

What Letra da Lei will do differently

The approach we're building: instead of stacking files, we'll index article by article. Each chunk carries its source metadata:

{
  "law": "Law 8.078/90 (CDC)",
  "article": "27",
  "effective_date": "1990-09-12",
  "caput": "...",
  "paragraphs": ["§ 1 — ...", "§ 2 — ..."]
}

Exposed via MCP (Model Context Protocol), the standard Anthropic created and the industry adopted. The idea: when you ask Claude about CDC prescription, it will decide to consult Letra da Lei automatically, receive the relevant articles with up-to-date effective dates, and answer with the real citation.

The takeaway for lawyers

You don't have to wait for the next generation of models. Claude is already capable — what's missing is the law on its side. That's what we're building: you keep your model, we deliver the letter of the law.

The waitlist is open. Sign up now to get early access when we launch.