Expectation management in humans and LLMs
Mirative markers, such as "surprisingly", explicitly encode a violation of expectations. Such markers are used for expectation management during communication. Sensitivity to mirative markers relies on two abilities: i) updating expectations upon recognizing a mirative marker, and ii) identifying expectation violations warranting the use of a mirative marker. In this study we compared sensitivity to mirative markers in humans and large language models (LLMs). In part 1, we used a sentence-completion task, where humans and LLMs were presented with sentence fragments and asked to continue them. Results show that for both humans and LLMs, the presence of a mirative marker significantly increased response entropy and decreased top response probability, in line with theoretical proposals for mirative markers. In part 2, we created a novel task of mirative polarity selection where humans and LLMs are presented with a sentence pair and asked to select whether it was connected by a mirative marker ("surprisingly") or an anti-mirative marker ("unsurprisingly"). Results show that LLMs perform at an impressive human level. We conclude that both humans and LLMs use mirative markers as cues for calibrating their subsequent expectations during sentence processing.