Optimizing Agentic Workflows for Information Access

Information access systems have been embedded into the capillaries of human society, serving as essential tools for connecting people to information. Many information access systems rely on static workflows, which follow a fixed execution process for all user queries. However, this “one-size-fits-all” approach limits the ability of information access systems to address real-world user queries in complex scenarios that demand case-by-case handling. To overcome the constraints, prior work has explored agentic workflows for information access, in which one or multiple autonomous agents dynamically adjust execution paths to each user query. This thesis targets optimizing agentic workflows for information access by addressing limitations in three core components: mixed-initiative strategy planning, ranking strategy planning, and ranking result reflection. The first part of the thesis focuses on optimizing mixed-initiative strategy planning. This part comprises one chapter that focuses on resolving the issue of a narrow scope in system-initiative actions for predicting the timing of system initiative-taking. To solve this issue, this chapter broadens the scope of system-initiative actions by defining and modeling a new task, system initiative prediction (SIP). The second part of this thesis aims to optimize ranking strategy planning. This part consists of one chapter that focuses on dynamic per-query re-ranking depth prediction, an important task in ranking strategy planning. Given the limited prior research on this task in the context of large language model (LLM)-based re-ranking, this chapter explores dynamic per-query re-ranking depth prediction in this new context. The third part of the thesis aims to optimize ranking result reflection. This part comprises two chapters that focus on query performance prediction (QPP), a long-standing methodology in ranking result reflection. Despite its importance, little research explores QPP in the area of conversational search, and little research explores improving QPP accuracy by using the capabilities of LLMs. The first chapter in this part adapts QPP methods, originally designed for ad-hoc search, to conversational search scenarios. The second chapter enhances QPP accuracy by leveraging LLMs' capabilities.