Information Seeking Behavior in LLM-Based RAG Mental Models and Missing Information

Open Access
Authors
Publication date 2026
Book title SIGIR '26
Book subtitle Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval : July 20-24, 2026, Melbourne, Australia
ISBN (electronic)
  • 9798400725999
Event 49th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2026
Pages (from-to) 2684-2695
Number of pages 12
Publisher New York, NY: Association for Computing Machinery
Organisations
  • Interfacultary Research - Institute for Logic, Language and Computation (ILLC)
Abstract

Retrieval-augmented generation (RAG) systems are rapidly becoming a primary entry point to information, replacing ranked lists with synthesized, citation-backed answers. While recent work has focused on improving RAG architectures and measuring answer correctness, much less is known about how users conceptualize these systems and how such mental models reshape information seeking. In this Perspectives Paper, we argue that RAG interfaces compress the visible stages of the Information Search Process: exploration, comparison, and synthesis migrate inside the system, while the user sees only a conversational request-response loop. We ground this argument in a qualitative mixed-methods study of nine participants using an LLM-based ''chat with documents'' interface for exploratory, known-item, list-based, and absent-item tasks. Most participants did not describe the system in terms of selective retrieval and ranking. Instead, they imported an analyst-centric mental model, assuming exhaustive reading, persistent memory, and near-perfect recall. These assumptions systematically reconfigured behavior: participants engaged in minimal query reformulation, delegated sense-making to the system, rarely inspected sources, and treated ''not found'' responses as strong evidence of nonexistence. A minority of participants articulated IR-consistent models and exhibited more iterative, verification-oriented behavior. We use these findings to question implicit behavioral assumptions in interactive IR and RAG evaluation and to outline design implications for handling missing information, retrieval visibility, and verification scaffolding. Our perspective is that understanding users' mental models of both system operation and their own role is central to assessing the epistemic consequences of generative search.

Document type Conference contribution
Language English
Published at
https://doi.org/10.1145/3805712.3808537 (Final published version)
Other links
Downloads
3805712.3808537 (Final published version)
Permalink to this page
Back