Conversational Search with Tail Entities

Conversational search faces incomplete and informal follow-up questions. Prior works address these by contextualizing user utterances with cues derived from the previous turns of the conversation. This approach works well when the conversation centers on prominent entities, for which knowledge bases (KBs) or language models (LMs) can provide rich background. This work addresses the unexplored direction where user questions are about tail entities, not featured in KBs and sparsely covered by LMs. We devise a new method, called CONSENT, for selectively contextualizing a user utterance with turns, KB-linkable entities, and mentions of tail and out-of-KB (OKB) entities. CONSENT derives relatedness weights from Sentence-BERT similarities and employs an integer linear program (ILP) for judiciously selecting the best context cues for a given set of candidate answers. This method couples the contextualization and answer-ranking stages, and jointly infers the best choices for both.