Information Seeking Behavior in LLM-Based RAG Mental Models and Missing Information
| Authors | |
|---|---|
| Publication date | 2026 |
| Book title | SIGIR '26 |
| Book subtitle | Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval : July 20-24, 2026, Melbourne, Australia |
| ISBN (electronic) |
|
| Event | 49th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2026 |
| Pages (from-to) | 2684-2695 |
| Number of pages | 12 |
| Publisher | New York, NY: Association for Computing Machinery |
| Organisations |
|
| Abstract |
Retrieval-augmented generation (RAG) systems are rapidly becoming a primary entry point to information, replacing ranked lists with synthesized, citation-backed answers. While recent work has focused on improving RAG architectures and measuring answer correctness, much less is known about how users conceptualize these systems and how such mental models reshape information seeking. In this Perspectives Paper, we argue that RAG interfaces compress the visible stages of the Information Search Process: exploration, comparison, and synthesis migrate inside the system, while the user sees only a conversational request-response loop. We ground this argument in a qualitative mixed-methods study of nine participants using an LLM-based ''chat with documents'' interface for exploratory, known-item, list-based, and absent-item tasks. Most participants did not describe the system in terms of selective retrieval and ranking. Instead, they imported an analyst-centric mental model, assuming exhaustive reading, persistent memory, and near-perfect recall. These assumptions systematically reconfigured behavior: participants engaged in minimal query reformulation, delegated sense-making to the system, rarely inspected sources, and treated ''not found'' responses as strong evidence of nonexistence. A minority of participants articulated IR-consistent models and exhibited more iterative, verification-oriented behavior. We use these findings to question implicit behavioral assumptions in interactive IR and RAG evaluation and to outline design implications for handling missing information, retrieval visibility, and verification scaffolding. Our perspective is that understanding users' mental models of both system operation and their own role is central to assessing the epistemic consequences of generative search. |
| Document type | Conference contribution |
| Language | English |
| Published at |
https://doi.org/10.1145/3805712.3808537
(Final published version)
|
| Other links | |
| Downloads |
3805712.3808537
(Final published version)
|
| Permalink to this page | |
