by Yassin HafidMar 26, 20264 min read

When Retrieved Evidence Conflicts with Model Memory: Knowledge Conflicts in RAG Systems

Abstract. Retrieval-augmented generation assumes that models will prioritize retrieved evidence over their internal parametric memory. Empirical studies show this assumption can fail under knowledge conflict; indeed, models may exhibit confirmation bias, selectively absorb evidence that aligns with prior beliefs, and paradoxically resist updating knowledge that changes most often across studied model families. This failure is especially concerning in high-dynamicity settings, and may be particularly relevant in domains such as financial markets, regulatory reporting, news monitoring, and other time-sensitive applications where retrieved evidence is most likely to conflict with outdated parametric priors. This note examines why knowledge conflicts between retrieved context and parametric memory undermine the reliability of RAG systems, even when retrieval is successful.

§1. The Question

If a RAG system retrieves a passage that contradicts what the model already "knows," which source wins? And does the model signal when a conflict has occurred?

§2. Scope and Definitions

Parametric Knowledge: Facts encoded in model weights during pretraining.

Contextual Knowledge: Information supplied at inference time via retrieved passages.

Knowledge Conflict: A discrepancy between parametric knowledge and retrieved contextual knowledge.

Memorization Ratio: The proportion of responses in which a model defaults to its parametric answer despite conflicting retrieved evidence 2.

Confirmation Bias: The tendency of a model to accept external evidence more readily when it partially aligns with parametric memory, even when the overall evidence conflicts 1.

This note focuses on context-memory conflicts in RAG settings where retrieved passages are factually inconsistent with model priors.

§3. Key Findings

LLMs exhibit contradictory behavior under conflict: They can be receptive to conflicting evidence when it is coherent and convincing; however, they show strong confirmation bias when external evidence partially aligns with parametric memory while other parts conflict 1.
Context-receptivity dominates single-evidence scenarios: Memorization ratios remain below 50% across twelve LLMs from four model series; this indicates that models are highly receptive to external evidence when it is the only evidence available, even when it conflicts with their parametric memory. However, this receptivity is not uniform; indeed, all models exhibit lower memorization ratios in temporal and semantic conflicts than in misinformation conflicts, indicating higher sensitivity to these types of external conflicting knowledge 2.
Synthetic benchmarks overstate context utilisation: Compared with real-world retrieved contexts, synthetic datasets can yield inflated ACU-based context-utilisation results. In DRUID, models show reduced ACU scores for supporting evidence, and context-repulsion is rare. While some models exhibit negative ACU scores (i.e., actively moving away from the evidence), synthetic data exaggerates this effect 3.
Fact dynamicity is a stronger negative indicator than fact popularity: Fact dynamicity is the strongest, most consistent negative indicator of model persuasion across models; it outperforms fact popularity. Facts that change regularly are less likely to be updated with context-retrieval, whereas more static facts are more easily persuaded 4.
Fragility under knowledge shift: Most RAG systems suffer a substantial performance drop when retrieved source documents contain hypothetical knowledge updates that conflict with the model's parametric knowledge. Questions requiring the integration of contextual knowledge with parametric knowledge pose a significant challenge to current LLMs, particularly in the Distant Implicit setting; this highlights a fundamental fragility in how models combine their parametric knowledge with shifting external evidence 5.

§4. Technical Deep Dive: Why Conflicts Go Unresolved

§A. Confirmation Bias and Coherence Sensitivity

Model behavior under conflict is not uniform. Evidence presented coherently and convincingly, rather than through simple entity substitution, is more likely to override parametric memory 1. This suggests a practical implication for deployed RAG systems: low-quality or poorly phrased retrieved passages may fail to displace incorrect parametric priors even when the underlying evidence is factually correct.

§B. The Paradox of Dynamic Facts

Counter-intuitively, models are most resistant to updating the very facts that change most often. Fact dynamicity is the strongest, most consistent negative indicator of model persuasion across models, outperforming fact popularity 4. One plausible explanation is intra-memory conflict: dynamic facts may appear in multiple conflicting states during pre-training, making their internal representations harder to displace than those of more static facts.

§C. Benchmark Inflation and Real-World Performance

Most prior studies of knowledge conflict used synthetic datasets that do not reflect the complexity and diversity of real retrieved contexts. When evaluated on real-world retrieval contexts, models show reduced ACU scores for supporting evidence, and no single context characteristic reliably predicts when a model will or will not follow the evidence 3. This suggests that conflict-resolution performance reported in controlled settings may not transfer cleanly to deployed systems.

§5. Practical Taxonomy of Conflict Failure Modes

Coherence-Gated Override: Model ignores retrieved evidence when it is not presented coherently enough, even if factually correct 1.
Confirmation Absorption: Model accepts evidence that partially supports prior beliefs while discarding the conflicting portion 1.
Dynamic Fact Resistance: Model fails to update knowledge about temporally or contextually variable facts, precisely the facts where retrieval is most needed 4.
Knowledge Shift Collapse: RAG performance degrades substantially when retrieved documents reflect updated knowledge that contradicts training-time priors 5.
Benchmark-Reality Gap: Context utilisation measured on synthetic conflict data does not generalize to real-world retrieval; models appear more robust in controlled settings than in deployment 3.

§6. Implications for AI System Design

Do not assume conflict detection is implicit: Models do not reliably signal when retrieved evidence conflicts with parametric memory; explicit conflict detection mechanisms are needed.
Coherence of retrieved passages matters beyond relevance: A retrieved passage may be topically relevant but insufficiently coherent to override prior beliefs; retrieval systems should optimize for coherence as well as relevance 1.
Evaluate on real-world retrieval contexts: Synthetic conflict benchmarks overstate context utilisation; system evaluation should use real retrieved data with realistic stance diversity 3.
Prioritize temporal knowledge as a high-risk conflict category: Dynamic and temporally variable facts represent the highest-risk knowledge conflict scenario and warrant targeted retrieval and verification strategies 4.

§7. Open Questions

Conflict detection: Can models be trained or prompted to reliably detect when retrieved evidence conflicts with parametric memory, and signal that conflict or uncertainty explicitly?
Resolution criteria: When should a model defer to retrieved evidence and when to parametric memory? No principled resolution criterion currently exists that generalizes across conflict types 2.
Coherence and trust: Beyond topical relevance, what properties of retrieved passages determine whether a model treats them as credible and follows them under conflict? 1 3
Dynamic knowledge: How should RAG systems be designed for domains with high fact dynamicity, where retrieved evidence may frequently conflict with outdated internal knowledge? 4 5
Conflict propagation in reasoning: In multi-step reasoning, does early acceptance of a retrieved conflicting fact persist, weaken, or collapse as the reasoning chain becomes more implicit?