Publication
Using LLMs as sentiment analyzers to predict review helpfulness: first insights to open the black box
Christian G. H. Winter; Nicolas A. Zacharias; Mattis Hartwig
In: Marketing Letters (Mark Lett), Vol. 37, No. 1, Page 30, Springer Nature, 5/2026.
Abstract
This study examines the potential of large language models for sentiment analysis
in marketing. Using the empirical setting of online customer reviews, we further
explore implications for prediction of review helpfulness. Relying on a dataset of
28,900 product reviews from a consumer platform and an experiment with N = 1063
participants, we find that the LLM’s accuracy in assessing intended meaning (as in
the star-rating) in customer-written text depends on the degree of emotionality, as
in purchases of hedonic (vs. utilitarian) goods. We further demonstrate that deviations
between LLM classification and original star rating predict lower review helpfulness.
This effect is mediated by the deviation of human readers’ assumption on
the intended star rating from the actual star rating and moderated by the degree of
information asymmetry before the purchase; that is, a greater deviation between the
LLM classification and the original star rating indicates lower review helpfulness for
search goods than for experience goods.