Introduction to the Special Issue on Images as Data

Abstract

Visual information (primarily still images and videos) is crucial for the study of many current communications, political, and social phenomena. Yet research leveraging large corpora of visuals to answer social science questions is still scarce, especially relative to the explosion of research using “big data” text-as-data methods. This special issue fosters innovative theoretical and methodological research in the area of images as data. The featured articles use computational methods to analyze, from a social science perspective, large quantities of images as well as videos. Three overarching motivations fueled the creation of this special issue on “Images as Data” at Computational Communication Research.

First, visuals are becoming a more frequent form of communication, especially online. Social media is a good reflection of this new communication paradigm. Not only have social media platforms become more central to people’s lives (more people, and more often, use social media to interact with each other and to learn about what is going on in the World), but the platforms have become more visual. In the last few years, we note an increase in visuals on messages posted on the more “traditional” social media platforms such as Twitter, Facebook and Weibo. And “younger” platforms such as Instagram, Tik Tok, and Douyin are even more specifically designed around visual communication. Other communication platforms have followed this visual trend. For example, online news outlets have also increased the amount of visuals they attach to news stories. Posting images or videos online comes at a tiny fraction of the cost of printing images in a newspaper; and in return, images and videos often help news media get people’s attention, attract readership, and frame the news story. In sum, images and videos are a central part of many phenomena of interest to social scientists today (e.g. news values, framing, agenda-setting, dis/mis-information, mobilization, etc.), and we should do more to include visuals in our analyses.

Second, a growing amount of visual data is available to social scientists. As communications become more visual, researchers interested in studying these phenomena have also had increased access to images and videos for analysis. Repositories of visual data from public communications are gradually flourishing. The TV News Archive¹ from the Internet Archive (mostly focused on news from English-speaking countries; see Dietrich and Ko (2022) in this issue) and the media collection of the Institute for Sound and Vision in the Netherlands are good examples, as is the Wesleyan Media Project, which collects political advertising in the US (see Neumann et al. (2022) in this issue). Visuals can also fill the research needs of scholars who are not necessarily interested in studying communications. As some examples, open-source satellite images are on the rise, which can provide measures of economic development in areas for which little data is available;² and many art institutions are digitizing artwork and creating open data sets of historical imagery that can, for example, facilitate the study gender roles across time and places.³ In sum, the growing volume of visual data can help social scientists answer both new and long-standing questions in the literature.

Third, new computational methods are available for automated image analysis. Fueled by the deep learning revolution, the last few years have seen major advancements in “computer vision,” the subfield of computer science (and related disciplines) concerned with automatically analyzing images. There have been key improvements in crucial research tasks, such as object recognition, image classification, face recognition, facial traits analysis, etc (see Steinert-Threlkeld and Joo (2022), in this issue, for an introduction to these tasks). These new methods allow social scientists to automatically analyze large quantities of images. With new tools we can more efficiently study how different groups or news organizations frame particular issues; how people process and learn particular information; the presence of representation biases in public communications; and the correlates of economic development, among many other topics. However, in order to take full advantage of these methods, social scientists first need to learn about how these methods work and for what research purposes, and under what conditions, they can be useful.