The Hallucination Problem in Technical Domains
Large language models (LLMs) have revolutionized how professionals access information, but they remain notoriously unreliable when dealing with precise, regulated content. Nowhere is this more critical than in construction codes—where a single hallucinated reference can lead to structural failures, legal liabilities, or safety hazards. A recent investigation published on Habr Source reveals a stark reality: standard LLMs can fabricate up to 120 fake references in a single query about building regulations, while agentic search systems reduce that number to just 8. This article examines what agentic search does differently and why it matters for industries where accuracy is non-negotiable.
The Scale of the Problem: 120 Fabricated Citations
In the study, researchers tested several LLMs (including GPT-4, Claude, and open-source alternatives) on queries related to international building codes (IBC), fire safety standards (NFPA), and European construction norms (Eurocodes). The results were alarming:
| Metric | Standard LLM (no retrieval) | Agentic search system |
|---|---|---|
| Average fabricated references per query | 120 | 8 |
| Accuracy of cited regulations | ~12% | ~94% |
| Time to generate response | 2–4 seconds | 8–15 seconds |
Standard LLMs, when asked to "list fire safety requirements for a 10-story commercial building under NFPA 101," confidently generated paragraph-long answers with citations to sections that simply do not exist. One model invented an entire subsection "NFPA 101:2021 Section 15.3.2.1"—which does not appear in any official document. The agentic search system, by contrast, retrieved actual sections from the current NFPA 101 edition and only cited verifiable text.
How Agentic Search Works: From Guesswork to Grounded Retrieval
The core difference lies in architecture. Standard LLMs operate as closed systems: they predict the next token based on training data, which may include outdated, contradictory, or hallucinated information. Agentic search systems, as described in the source article, combine three components:
- Query decomposition – The system breaks a complex question into sub-questions (e.g., "What is the occupancy classification?" → "What are the egress requirements for that classification?").
- Retrieval-augmented generation (RAG) – Instead of relying on internal knowledge, the agent queries a curated database of official building codes, standards, and amendments in real time. Each retrieved chunk includes metadata (edition, section number, date).
- Cross-verification – The agent performs a second pass: it checks that each cited section exists in the original document. If a citation is missing or ambiguous, the agent flags it rather than fabricating.
For example, when asked about "maximum occupancy load for a restaurant under IBC 2021," the agentic system:
- Extracts the key parameters (IBC 2021, Table 1004.1.2, "Restaurant").
- Queries a vector database containing the full IBC text.
- Returns the exact value (15 sq ft per person for standing space) with a direct link to the section.
- If the section is not found, it returns "No matching regulation found" instead of inventing one.
Practical Implications for Construction Professionals
For architects, engineers, and contractors, the difference between 120 and 8 hallucinated references is not just academic—it has real-world consequences:
- Compliance risk: A false citation could lead to design errors that violate fire codes, resulting in failed inspections or, worse, unsafe buildings.
- Legal liability: If a project relies on an LLM-generated reference that doesn't exist, the professional is still responsible for the error.
- Time wasted: Verifying 120 fake citations takes hours. With agentic search, the 8 remaining hallucinations are easier to spot and correct.
The source article highlights a case where a standard LLM recommended a fire-resistant wall assembly based on a nonexistent ASTM standard. An engineer who trusted the output would have specified the wrong materials, leading to a costly retrofit.
Limitations and Next Steps
Agentic search is not a silver bullet. The study found that the system still hallucinated 8 references per query, often due to:
- Incomplete database coverage (e.g., state-level amendments not included).
- Ambiguous queries (e.g., "latest version" of a code that has multiple editions in the same year).
- Retrieval errors where the agent picks a similar but incorrect section.
However, the reduction from 120 to 8 represents an 85% decrease in hallucination rate. The researchers suggest that combining agentic search with human-in-the-loop validation could bring that number close to zero. For instance, an agent could flag any retrieved section that has been revised within the last 12 months, prompting the user to check the latest amendment.
Conclusion
The construction industry cannot afford to trust LLMs blindly. The Habr investigation demonstrates that standard models fabricate references at an alarming rate—120 per query—while agentic search systems cut that to 8 by grounding responses in real-time retrieval and cross-verification. As building codes become more complex and frequently updated, the shift from closed-model guessing to open, verifiable search is not just a technical improvement; it is a safety imperative. Professionals should consider integrating agentic search tools into their workflows, while remaining vigilant: even 8 hallucinations are 8 too many when lives and structures are at stake.
Comments