Working notes on the parts of AI that break under institutional load. Retrieval, citation faithfulness, multi-hop reasoning, knowledge conflicts. Cited from the literature, refined in production.
Even when an agent executes the stated task, uses the available tools, follows a plan, and produces a plausible result, the outcome may still fail the user's actual intent. This note introduces the Specification Gap: the structural gap between what the user means and what the agent operationalizes as a task.
Agentic AI · Specification Gaming · Intent Alignment · LLM
External calls introduce signals, but they do not automatically introduce grounding. A tool call grounds an agent only if the selected tool is appropriate, the arguments are correct, the tool interface is reliable, the data is fresh, the output is valid, and downstream steps preserve the signal.
Agentic AI · Tool Use · LLM · Grounding
A verifier that shares the same information boundary, model priors, and error surface as the system it evaluates may increase confidence without increasing reliability. The most dangerous verification failure is not the one that looks broken—it is the one that looks fine.
Agentic AI · Verification · LLM · Multi-Agent Systems
Agentic AI systems fail differently from single-step interactions. This note examines how minor early errors compound across workflow steps, why pass^k reliability collapses across repeated executions, and why the most dangerous aspect of agentic drift is that outputs remain fluent.
Agentic AI · Trajectory Integrity · LLM · Reliability
These systems can appear to succeed while failing at the level of evidence and reasoning. The gap between a system that works and one that merely appears to work is not an academic distinction.
AI · RAG · Opinion · Enterprise AI
RAG evaluation suffers from Diagnostic Collapse: single scalar scores cannot distinguish retrieval failure from grounding failure.
RAG · Evaluation · LLM · Grounding
Multi-hop RAG fails even when correct evidence is retrieved. In 73-84% of errors, the model had the right context but failed to reason over it—not a retrieval problem.
RAG · Multi-Hop · Reasoning · LLM
RAG systems exhibit Temporal-Semantic Mismatch from multi-layer structural degradation. RAG mitigates but does not resolve the dynamic knowledge problem.
RAG · Temporal Knowledge · LLM · Time-Sensitive
RAG assumes models prioritize retrieved evidence over parametric memory. Empirical studies show this fails under knowledge conflict in dynamic domains.
RAG · Knowledge Conflicts · Parametric Memory · LLM
Citation-based RAG aims to improve groundedness, yet citations do not guarantee claims are supported. This note explains why RAG still hallucinates.
RAG · Hallucination · Citation · LLM
Do long-context LLMs replace retrieval? Evidence shows complementary roles in knowledge-intensive AI.
RAG · Long Context · LLM · Retrieval