Four best practices for measuring news sentiment using ‘off-the-shelf’ dictionaries: a large-scale p-hacking experiment
PDF
Code
Online Appendix

Keywords

sentiment analysis
p-hacking
news sentiment
agenda setting
text-as-data
validity

How to Cite

Chan, C.- hong, Bajjalieh, J., Auvil, L., Wessler, H., Althaus, S., Welbers, K., van Atteveldt, W., & Jungblut, M. (2021). Four best practices for measuring news sentiment using ‘off-the-shelf’ dictionaries: a large-scale p-hacking experiment. Computational Communication Research, 3(1), 1-27. Retrieved from https://computationalcommunication.org/ccr/article/view/40

Abstract

We examined the validity of 37 sentiment indicators based on dictionary-based methods using a large news corpus and demonstrate the risk of generating a spectrum of results with different levels of statistical significance by presenting an analysis of relationships between news sentiment and U.S. presidential approval. We summarize our findings into four best practices: 1) use a theory-informed sentiment dictionary; 2) do not assume that the validity and reliability of the dictionary is ‘built-in’; 3) check for the influence of content length and 4) do not use multiple dictionaries to test the same statistical hypothesis.

PDF
Code
Online Appendix
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.