Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It

{ “meta”: “This paper investigates how phrasing affects LLMs’ ability to navigate user beliefs and factual information, revealing significant performance variability.”, “body”: “Problem — This preprint addresses the gap in understanding how large language models (LLMs) handle user beliefs intertwined with factual information. Prior research indicated that LLMs struggle to acknowledge user beliefs based on incorrect information, but the specific impact of phrasing on this capability had not been thoroughly evaluated across multiple models.\n\nMethod — The authors evaluate 10 LLMs using 18 different epistemic expressions to assess how the phrasing of beliefs influences the models’ performance in distinguishing between factual and false information. They identify that the accuracy gap varies significantly depending on the verb used to express the belief, with a range from +50% accuracy on “