University of Amsterdam at the CLEF 2025 Eloquent Track Evaluating the Influence of Stylistic Prompt Variations on Semantic Interpretation

This paper reports on the University of Amsterdam’s participation in the CLEF 2025 Eloquent Track’s Robustness and Consistency Task. Our overall goal is to evaluate the influence of stylistic prompt variations on semantic interpretation. Our specific focus is to investigate how variations in prompt tone, structure, and persona affect the consistency and robustness of responses generated by large language models (LLMs). We approach this through two complementary methods. First, we use a model-as-judge setup to quantify semantic consistency: each stylistic variant prompt is compared to its original base prompt using GPT-4.1 to rate the similarity of the generated responses on a 0–5 scale. Second, we conduct an inductive qualitative analysis on a selected prompt to closely examine how different stylistic framings influence content shifts in model outputs. Our results suggest that prompt reformulations can lead to variations in output, informational content, and tone.