Citation in APA format: Kosinski, M. (2025). Using big data. In D. T. Gilbert, S. T. Fiske, E. J. Finkel, & W. B. Mendes (Eds.), The handbook of social psychology (6th ed.). Situational Press. https://doi.org/10.70400/AWLY9440

Using Big Data 

Michal Kosinski, Stanford University 

Big Data is everywhere. Globally, billions of people use online platforms and digital devices to communicate, socialize, study, shop, and work. In the process, they generate a vast trove of digital footprints, documenting their thoughts, feelings, actions, and interactions. Social exchanges, for example, are captured in emails, instant messages, and social networking platforms. Expressions of attitudes and biases manifest in blog articles, online profiles, and social media posts. Online job markets and retail platforms capture our occupational and economic activities. Collaborative behaviors are traceable through corporate project management tools and massive multiplayer online games. Emotional states and well-being leave traces in language and digital images. Intimate attributes like sexual preferences and romantic choices are exposed on online dating sites. Moreover, even offline behaviors and social interactions are increasingly monitored by digital sensors in smartphones, wearable devices, smart buildings, and vehicles. 

These digital footprints of thought and behavior are commonly captured and recorded, creating extensive datasets widely referred to as Big Data. This chapter demonstrates that Big Data—characterized by its vast size, rapid accumulation, diversity, and complexity—is an invaluable resource for researchers interested in the human mind and behavior. Although not a silver bullet, Big Data presents an opportunity to overcome some of the significant challenges in social science research. When used correctly, it can provide new insights; enhance the diversity of research samples; assist in accessing under-researched populations; and yield more reliable, replicable, and ecologically valid results. Conversely, in the wrong hands, this data could be misused to invade privacy or manipulate the opinions of millions of people. 

CHAPTER OUTLINE 

The following pages delve into the world of Big Data, using a selection of seminal studies to explore its benefits and challenges. The chapter begins with a pivotal case study on the role of Big Data in the 2016 U.S. presidential election and the controversial tactics employed by Cambridge Analytica. Following this, Big Data is defined, highlighting its key characteristics—volume, velocity, veracity, and variety—in the context of psychological studies conducted on X (formerly Twitter), one of the popular sources of Big Data among social scientists. The surprising range of insights that can be derived from Big Data is illustrated through a review of how geolocation history, a seemingly superficial type of data, can be transformed into psychologically relevant information that reveals individual habits, interactions, preferences, and psychological traits. The four types of Big Data widely used in psychological science—usage logs, language data, audiovisual data, and data from mobile sensors—are then outlined. The potential of Big Data studies to address the shortage of statistical power affecting psychological science and to boost its ecological validity, diversity, and representativeness is discussed. Several popular strategies for collecting Big Data are presented, including employing secondary data, recording data from online platforms, developing research apps and collaborating with existing ones, and employing pre-trained models. The chapter also delves into how Big Data facilitates both theory- and data-driven research, such as experiments and observational studies, including natural experiments and longitudinal studies. The improvement of psychological science’s predictive validity through Big Data is examined. Finally, the ethical challenges faced by researchers employing Big Data in their work are scrutinized. 

THE DATA THAT TURNED THE WORLD UPSIDE DOWN 

The societal importance of Big Data has steadily grown since the advent of the digital revolution in the early 2000s, driven by the burgeoning societal impact of the internet, social media, and mobile technology. However, the 2016 U.S. presidential election served as a pivotal moment in raising public awareness about the ramifications of Big Data analytics (Grassegger & Krogerus, 2017). 

On the morning of November 9, Hillary Clinton addressed her supporters at a Manhattan hotel, stating, “We must accept this result ... Donald Trump is going to be our president ... we owe him an open mind and a chance to lead.” The world recoiled in shock. Clinton had a decisive lead in the polls leading up to the election, and the election was not even supposed to be close. People were frantically trying to understand what had happened. 

Around the same time, Cambridge Analytica—a then lesser-known company—released a statement saying, “We are thrilled that our revolutionary approach to data-driven communication has played such an integral part in President-elect Trump’s extraordinary win” (Cambridge Analytica, 2016). They had previously employed the same Big Data approach to help elect Thom Tillis to the U.S. Senate in 2014; to persuade British voters to support Brexit; and to boost Ted Cruz, a then relatively unknown Texas senator, to a surprising second place in the Republican primaries in May 2016 (Davies, 2015). After Cruz’s defeat, Trump swiftly hired Cambridge Analytica. 

Few were familiar with Cambridge Analytica’s Big-Data-driven “revolutionary approach.” In fact, this approach was not even theirs, as it was originally developed and patented by Facebook (Nowak & Eckles, 2012). It comprised three distinct steps. 

In the first step, psychologists working for Cambridge Analytica created a machine learning model allowing them to predict individuals’ Big Five personality profiles from their Facebook Likes. The conventional approach to personality measurement involves collecting participants’ responses to a personality questionnaire. However, personality can also be measured by observing individuals’ behaviors or the footprints of such behaviors. For instance, as outlined in the seminal paper “A Room with a Cue: Personality Judgments Based on Offices and Bedrooms,” by Sam Gosling and colleagues (2002), one can assess an individual’s personality by inspecting their personal spaces. It turns out that similar judgments can also be derived from the footprints of behaviors left in virtual environments, such as Facebook Likes (Kosinski et al., 2013a). 

Superficially, Facebook Likes are a trivial type of Big Data. People “Like” pictures of their friends’ babies, funny goat videos, or Facebook pages of their favorite local bars and restaurants. If asked, Facebook users would likely claim that their “Likes” are not particularly indicative of their actual preferences, as many are given just to validate a Facebook friend. However, as advanced by Facebook in its patent (Nowak & Eckles, 2012)—and as research confirms (e.g., Kosinski et al., 2013b)—Facebook Likes are strongly linked with one’s personality, political orientation, religiosity, happiness, intelligence, sexual orientation, race, gender, and more. For example, extroversion is linked with liking “Beer Pong,” “Dance,” “Theatre,” and “Socializing,” whereas disagreeability is revealed by liking “Prada,” “Friedrich Nietzsche,” “Timmy South Park,” and “Satanism.” Notably, the predictive accuracy of personality assessments based on Facebook Likes has been shown to exceed that of evaluations made by an individual’s friends or family (Youyou et al., 2015). This underlines the potent informative value that even seemingly trivial types of Big Data can hold in modeling and understanding human psychology. 

To obtain a suitable sample to train their model, Cambridge Analytica contracted Qualtrics, an online survey platform (Levy, 2020). They recruited about 200,000 participants from the U.S. who were paid $4 to complete a Big Five personality questionnaire and share their Facebook profiles. Given such a dataset, training accurate Likes-based models predicting users’ personality or other psychological traits is trivial. Such models can be trained by executing merely 15 lines of code written in R language on a standard, off-the-shelf laptop (Kosinski et al., 2016). 

In the second step, Cambridge Analytica applied its model to infer the personality profiles of hundreds of millions of unsuspecting Americans. To do so, they needed to obtain their Facebook Likes. Unfortunately, such data was and still is surprisingly easily accessible: For example, it can be bought in online data marketplaces, extracted from leaked databases circulating on the dark web, or obtained directly from Facebook. Moreover, at the time, Facebook allowed its users to share not only their profile data but also the data of all their Facebook friends. This implied that by convincing—or duping—a single Facebook user to share their data, one could gain access to the data of potentially hundreds of their friends. Take, for example, the 200,000 Facebook users recruited to train Cambridge Analytica’s predictive models. To receive their payment, they had to give Cambridge Analytica access not only to their own Facebook profile data, but also to the Facebook profile data of all their friends. Cambridge Analytica obtained the Facebook profiles of over 100 million Americans from this single source. 

Facebook Likes obtained from Qualtrics and other sources were entered into the predictive model trained in the first step and converted into accurate personality profiles. As stated by Alexander Nix, the CEO of Cambridge Analytica, “By having hundreds and hundreds of thousands of Americans undertake this survey, we were able to form a model to predict the personality of every single adult in the USA” (2016). Many of them had likely never completed a personality test. None of them were aware that their personality profiles were now in the possession of a political marketing firm. 

In the third and final step, Cambridge Analytica used voters’ inferred personality profiles to target them with carefully personalized political ads. These ads were disseminated through social media, email, or direct canvassing. Consider, for example, the ads they used to promote Thom Tillis in his 2014 senate campaign in North Carolina (Talbot, 2016). As shown in Figure 1, voters profiled as diligent, hard-working, and responsible (i.e., highly Conscientious) received ads promising that Tillis would restore common sense in Washington. Those profiled as anxious (i.e., highly Neurotic) were promised more safety. Messages shown to prosocial and altruistic voters (i.e., highly Agreeable) emphasized Tillis’s experience in improving the economy—presumably for everyone’s benefit. As Nix elaborated: “If you know the personality of the people you are targeting, you can nuance your message to resonate more effectively” (2016). 

Figure 1: Cambridge Analytica’s Ads Promoting Thom Tillis in His 2014 Senate Campaign in North Carolina Aimed at Voters Predicted to Be Conscientious, Neurotic, and Agreeable

Note: These ads were recreated from low-resolution screenshots from Talbot (2016)

Could Cambridge Analytica’s Big-Data-driven “revolutionary approach” have been a decisive factor in Trump’s unexpected victory, Brexit, and Ted Cruz’s remarkable success in the Republican primaries? It is impossible to be sure. Elections are shaped by myriad complex factors: economic conditions, the candidates’ policies and personalities, and various campaign strategies. Isolating the effect of any given factor, such as targeted advertising campaigns by one company, requires a level of evidence that is hard to obtain outside of controlled experimental settings. However, one can use psychological science to approximate the effectiveness of Cambridge Analytica’s approach. 

Case Study: Evaluating the Effectiveness of Cambridge Analytica’s Approach 

One of the challenges of studying the effectiveness of ethically questionable political marketing practices is collecting appropriate data. Companies like Facebook and Cambridge Analytica have it but are unlikely to share it. Manipulating participants’ political views in an experiment would be unethical. One can, however, attempt to approximate the effectiveness of such campaigns by replacing ads promoting politicians with ads promoting services or products. 

This was the approach adopted by Sandra Matz and colleagues, who conducted a series of field experiments to understand the efficiency of Cambridge Analytica’s approach (Matz et al., 2017). The design of their experiments closely followed the one employed by Cambridge Analytica. They used Facebook Advertising Platform[1] to post ads that resonated with different personality types. Instead of promoting politicians, they promoted products: a line of cosmetics, a crossword app, and a smartphone game (see Figure 2). For instance, an ad promoting cosmetics to extroverts featured an image of a woman dancing at a party, with the caption: “Dance like no one’s watching (but they totally are).” An ad aimed at introverts featured a lone person putting on makeup, accompanied by the slogan: “Beauty doesn’t have to shout.” 

Facebook’s advertising platform does not permit targeting users based on explicit personality traits. However, advertisers can achieve a similar effect by targeting people based on other characteristics correlated with personality, such as their jobs, education, interests, or what they “Like” on Facebook. Accordingly, the researchers targeted their ads at people who “Liked” Facebook pages indicative of different personality profiles. For instance, users who “Liked” Facebook pages for “Stargate” and “Computers” were classified as introverts, whereas those who “Liked” “Making People Laugh” or “Beer Pong” were classified as extroverts. 

The campaign reached over 3.5 million users, generating nearly 15,000 clicks, and resulted in 390 cosmetic purchases and over 2,300 game and app installations. The findings confirmed the high efficacy of psychological targeting: Ads that aligned with the users’ personalities boosted sales by about 50%. For example, the users who followed pages indicative of extroversion were 77% more likely to purchase a cosmetic product when exposed to the ad tailored for extroverts. 

Figure 2: Facebook Ads Tailored to Appeal to Extroverts (Left) and Introverts (Right) Employed in the Field Experiment on the Facebook Ads Platform (Matz et al., 2017) 

Although these results do not allow for the direct estimation of the effects of Cambridge Analytica’s campaigns, they demonstrate the remarkable efficiency of their personality targeting approach. In tightly contested scenarios like the 2016 U.S. presidential election or the Brexit referendum, persuading even a few thousand voters to change their stance—or abstain from voting altogether—could alter the outcome. For example, in the 2000 U.S. presidential election, George Bush won Florida (and thus the election) by 537 votes, or less than 0.01% of votes cast in that state (Bond et al., 2012). 

Donald Trump’s unexpected victory in the 2016 presidential election served as a startling wake-up call to scholars, policymakers, and voters worldwide. Although people were aware of the existence of Big Data and its marketing applications, its full potential impact remained unclear—beyond, perhaps, that one may be shown dating app ads after ending their relationship via Facebook message. Trump’s victory made it clear that much more is likely at stake. Irrespective of one’s political stance, Trump’s presidency ushered in profound social and political changes in the U.S. and internationally. This pivotal historical moment was likely catalyzed by a small political marketing firm’s strategic use of Big Data to influence voter behavior. The role of Big Data in these events demonstrated its importance and power. 

A comprehensive description of the Cambridge Analytica saga—replete with unforeseen plot twists, much Big Data, corporate greed, and rogue academics—can be found in the “Clown Show” chapter in Steven Levy’s (2020) excellent book, Facebook: Inside Story. 

WHAT IS BIG DATA? 

As the name suggests, Big Data tends to be big. However, its size—or its volume, as it is often referred to in scholarly literature—is not all that sets it apart from other types of data. Beyond volume, Big Data is characterized by three other attributes, all conveniently starting with the letter “v”: Velocity refers to the high-speed rate at which Big Data accumulates; (low) veracity signifies the messy character of Big Data; and variety refers to its diverse and unstructured nature. This section delves into these four Vs in the context of Big Data laboriously generated by users of X (formerly known as Twitter), one of the most utilized sources of Big Data by social scientists. (A Google Scholar search for “Twitter Psychology” in May 2024 returned over three million documents.) 

Volume and Velocity 

In 2024, over 550 million people visited X at least once a month to write and read posts (formerly Tweets), in which they shared news and ideas, commented on current events; or boasted about their achievements. Together, X users generated about 500 million posts per day—about 6,000 posts per second. Moreover, posts are just one of many types of data generated by X users. Apart from posting, X users can view, like, retweet, or comment on others’ posts; follow accounts they find interesting; and exchange private messages. These activities generate much data—up to about four million events or 11 gigabytes per second (Zhang & Malife, 2021). In other words, X data is not only massive (high volume) but also accrues at a breakneck speed (high velocity). For that reason, data scientists often refer to this ceaseless torrent of data as “Twitter Firehose.” 

Big Data, with its enormous volume and velocity, presents challenges in collection, storage, and analysis. However, these barriers continually decrease due to the growing accessibility of computing power and storage in personal computers and the cloud. As of 2024, a standard laptop can effectively analyze a complex dataset involving millions of participants.

Successfully overcoming these challenges allows researchers to study phenomena that may be too subtle to be detectable in smaller conventional datasets. Moreover, Big Data’s velocity facilitates studying events as they unfold in real time. A prime example illustrating these advantages of Big Data is provided by the work of Sharath Guntuku and colleagues (2020), who monitored X posts for reports of physiological symptoms and indicators of psychological well-being during the COVID-19 pandemic. 

Case Study: Using X Firehose to Track the Psychological Impact of the COVID-19 Pandemic 

Detecting and monitoring the spread of infectious diseases is, literally, a matter of life and death. However, as highlighted by the COVID-19 pandemic, this task is challenging, and many countries fell short in effectively managing it. Initially, only a tiny segment of the population may be affected, making early detection difficult and resulting in the loss of critical response time. Furthermore, pandemics can escalate so rapidly that they outpace conventional monitoring systems, such as national registers of hospitalization data. Days could pass between when a person contracts the virus and when they seek hospital care, and even more time before reports from individual hospitals are collated into central databases. 

Moreover, certain groups, such as undocumented immigrants or uninsured citizens, may be largely overlooked by formal healthcare systems, rendering their health status virtually invisible. More effective methods, like wastewater testing for pathogens, exist; however, they are typically expensive and unattainable for many developing countries. Moreover, the psychological effects of illness and prolonged quarantine rarely receive any attention at all. 

Big Data offers a chance to address these and other limitations. Guntuku and colleagues (2020) recorded a tiny fraction of the data gushing daily from the Twitter firehose—a random sample of five million posts, or roughly 1% of the total. These posts were automatically processed to extract words indicative of psychological states such as stress, anxiety, and loneliness, as well as physiological symptoms related to COVID-19. Their frequencies were compared with those observed in the same period in the pre-pandemic year. The results were visualized on an interactive dashboard that provided real-time insights into these indicators’ temporal and geographic patterns juxtaposed with historical data. 

Their dashboard, illustrated in Figure 3, facilitated the detection of areas experiencing deteriorations not only in physical but also in mental health. It offered critical insights into communities typically excluded from the standard healthcare system that may be present on X, such as uninsured citizens. Moreover, the system revealed several novel COVID-19 symptoms ahead of their official acknowledgment by the Centers for Disease Control and Prevention (CDC). All this was achieved swiftly, inexpensively, and without further strain on the existing healthcare infrastructure. Although their dashboard is no longer active, it provides an elegant proof of concept. Similar systems could act as early-warning mechanisms for pandemics and other public emergencies, enabling real-time tracking of their evolution and efficiently allocating resources to where they are most needed. 

Guntuku and colleagues’ study illustrates two essential advantages of Big Data studies. First, it shows how Big Data collected for one objective can be repurposed to achieve other goals. X was not designed to monitor pandemics or assess its users’ psychological wellness. Nevertheless, X data shows much potential for achieving these and, as discussed further in this chapter, many other goals. Second, Big Data’s immense volume and velocity mean that even infrequent events can be reliably detected. Although only a minuscule portion of X posts may include language indicative of psychological well-being or describing physiological symptoms, it appeared with sufficient frequency to be reliably detected. In fact, it was possible to achieve this goal by monitoring a tiny subset of all X posts—as mentioned before, their results were derived from only about 1% of all X daily posts. 

Figure 3: Dashboard Visualizing the Results of the Analyses Conducted by Sharath Guntuku and Colleagues (2020) 

(Low) Veracity 

The next “v” stands for veracity, which refers to the often messy, inconsistent, and incomplete nature of Big Data. For instance, X posts may be poorly written, inaccurate, or employ offensive language. The purported location from which a post originates might be deliberately falsified by the user, imprecise due to issues with the Global Positioning System (GPS) sensors, or missing entirely because of privacy settings. Furthermore, user profiles and bios can be fake, and some may not be humans at all—bots represent a non-trivial portion of users on this and other online platforms. 

The low veracity of X posts may seem problematic, particularly for social scientists accustomed to neat variables collected in carefully planned lab experiments and psychological surveys. However, this low veracity is a by-product of one of Big Data’s most substantial advantages: its high ecological validity. As discussed later in this chapter, X posts are generated by individuals going about their daily activities within their natural environment, providing an excellent opportunity to study spontaneous behavior in real-world contexts. In fact, even the fake profiles and their posts are of interest to social scientists, as they constitute an essential part of today’s digital environment. 

Furthermore, the seemingly chaotic nature of Big Data—its messiness, inconsistency, and gaps—can be highly valuable from a scientific viewpoint. Often, it is the absence of information, spelling mistakes, or profane language that can provide useful insights into psychological phenomena that would be difficult to examine using conventional research methods. Consider the study by Eichstaedt and colleagues (2015), which demonstrated the potential of analyzing conversations on X to study cardiovascular health issues. 

Case Study: Foul Language and Heart Problems 

Atherosclerotic heart disease (AHD), commonly known as coronary artery disease, results from plaque accumulation within the arteries that nourish the heart muscle. This condition can precipitate heart attacks, heart failure, and arrhythmias. It stands as one of the foremost causes of death in the United States, claiming a life every 83 seconds. Although the main risk factors for AHD include smoking, diabetes, obesity, lack of exercise, and poor diet, psychological components like prolonged stress and a hostile social environment are also significant contributors. Eichstaedt and colleagues (2015) designed an elegant study to investigate these associations using Big Data collected on X. 

The research team gathered a sample of 826 million publicly accessible X posts from a period stretching from June 2009 to March 2010. As many X users include their geographic locations in their profiles, the researchers could link their posts with specific U.S. counties. This process yielded 148 million county-associated posts spanning 1,347 counties. The team then analyzed these posts, extracting each county’s relative prevalence of various words and expressions. This linguistic data was then merged with county-level statistics—including age-adjusted AHD mortality rates and factors such as smoking prevalence, educational attainment, median income, and demographic composition—gathered from the CDC and the U.S. Census Bureau. 

Anyone familiar with X knows that many of its users do not care much about proper spelling and restraint in language. Consequently, the 148 million X posts examined by the researchers were replete with informal, colloquial, and offensive language rarely found in the more polished conventional datasets collected through surveys and questionnaires. This included outright expletives like “fuck”; informal expressions such as “grrrr”; and various misspellings and colloquial spellings like “ya’ll” or the elongated “sooooooo bad.” 

A social scientist might instinctively want to clean up the “messy” X data by eliminating profanities, correcting misspelled words, and translating colloquialisms into their formal equivalents. Yet, that strips away layers of valuable information. Rather than sanitizing their data, Eichstaedt and colleagues embraced its messy nature, exposing notable positive correlations between AHD mortality and language indicative of hostility, aggression, hate, interpersonal strife, boredom, and fatigue (see Figure 4). 

The strength of these correlations was remarkable. In fact, when employed to predict AHD mortality, models based on language in X posts outperformed those based on common demographic, socioeconomic, and physical health risk factors, including rates of smoking, diabetes, hypertension, and obesity. Notably, the most predictive words and phrases were the same expletives, colloquialisms, and misspellings that would typically be absent from data collected using conventional methods. 

Figure 4: Word Clouds Representing Nine Topics Positively Correlated with County-Level AHD Mortality in Eichstaedt et al. (2015)

Note: Topics are automatically extracted and can be thought of as clusters of semantically related words (words can belong to multiple topics). The size of each word is indicative of its prevalence (importance). This figure grouped topics into three broad categories (by row): hostility and aggression, hate and interpersonal tension, and boredom and fatigue; and their bivariate correlation with heart disease mortality is given.

Variety 

The last “v” stands for Big Data’s exceptional variety. Conventional data used in psychological studies typically includes variables such as participant age, gender, reaction time on some tasks, or responses on a five-point Likert scale ranging from “strongly disagree” (1) to “strongly agree” (5). Such variables are usually arranged in neatly organized tables and come with a codebook or a key to understanding their meaning. In other words, conventional data tends to be highly structured. Big Data, on the other hand, is highly unstructured—or contains highly diverse and complex variables that are much less organized. 

Consider the data points associated with a single X post in Figure 5. Posts’ content can include text, hashtags, hyperlinks, polls, photos, videos, and audio clips. Apart from their content, posts include the date and location of their publication and detailed information about their authors, such as their name, profile image, bio, and the link to their homepage. X posts are also described by the users’ replies or reposts, and the lists of people who viewed and liked them. Overall, the diverse set of data points associated with a single X post can be larger than many of the datasets employed in conventional studies.  

To add to the challenge, most of these data points are unstructured, open-ended, and dynamic. Post text can vary in length, nature, and language; users can invent new hashtags and abandon well-established ones; and new comments and Likes can appear at any point, including after the data has been collected. Furthermore, posts are just one of many elements of X containing data that could interest a psychologist. Interesting information could also be found in users’ profiles; their private messages; or the lists of accounts a given user follows, blocks, or is followed by. Although it may not always be immediately apparent, these elements hold valuable insights into people’s thoughts, emotions, actions, and interactions. 

Analyzing the diverse and unstructured data generated on X can be challenging, given that it does not come with an interpretive guide for researchers. To unlock its potential, one typically needs to convert it into structured, understandable variables fit for statistical analysis. This was elegantly demonstrated by Ryan Ritter and colleagues (2014) who used Big Data collected on X to study the association between religious belief and happiness. 

Figure 5: An Example X Post and a Few Selected Pieces of Data and Metadata Describing It 

Case Study: The Opium of the People 

Karl Marx famously quipped that religion is “the opium of the people,” offering solace to individuals during challenging times but at a tremendous long-term cost to themselves and society (Marx, 1844). This inspired much research on the links between religion and the well-being of believers. Ryan Ritter and colleagues set out to explore this question using Big Data collected on X (2014). 

X was not designed to measure users’ happiness or religiosity any more than it was designed to track the psychological effects of the COVID-19 pandemic. Nonetheless, these and other variables can be inferred from X’s data with a dose of ingenuity. To differentiate between religious and non-religious users, the researchers identified accounts likely to attract devout Christian followers (e.g., the profile of Pope Benedict XVI) and those likely to appeal to staunch atheists (e.g., Richard Dawkins or Sam Harris). About 8,000 followers were sampled from each of these groups, and their X posts were recorded—about two million in total. 

Next, the authors set out to measure the happiness of X users in these two groups. Traditionally, psychologists measure happiness using carefully crafted and validated questionnaires like the Satisfaction with Life Scale (Diener et al., 1985). However, a growing number of tools can estimate psycho-demographic traits, interests, or preferences from unstructured text, such as the language used in X posts. The authors employed one of the most widely used and rigorously validated tools: James Pennebaker’s Linguistic Inquiry and Word Count (LIWC; Pennebaker et al., 2022). 

LIWC is designed to analyze a sample of text written by an individual to extract insights into their psychological states, social dynamics, and language style. The software includes an extensive dictionary of words categorized into over 60 categories, indicative of the text author experiencing different psychological states, engaging in various types of cognitive and social processes, or having particular interests or preferences. Conveniently, in the context of Ritter and colleagues’ study, LIWC includes dictionaries aimed at positive emotions (e.g., “love” and “nice”), negative emotions (e.g., “hurt” and “nasty”), and religiosity (e.g., “prayer” and “God”). As its dictionaries were developed by experts, LIWC exemplifies a supervised approach to data analysis. This contrasts with the unsupervised approach employed by Eichstaedt and colleagues (2015) to study the links between language, cardiovascular disease, and county-level mortality (see the previous section). There, an algorithm was employed to automatically identify clusters of words rather than some predefined categories in the data.  

Figure 6: The Top 30 Differences in Word Use Between Atheists and Christians Identified Using LIWC by Ritter et al. (2014)

Note: Circle size indicates the overall word popularity. Their color and position on the X-axis convey words’ relative overrepresentation among Christians and atheists. The values represent the overrepresentation of a given word (per 100,000) among Christians and atheists. Figure reprinted from Ritter et al.(2014). 

The two million X posts of Christians and atheists were processed using LIWC to predict their authors’ religiosity and their propensity to experience positive and negative emotions. The results confirmed that the former scored significantly higher on LIWC’s religious belief dimension, validating the approach to classifying religious belief based on the accounts that people followed. Ritter and colleagues’ findings support results previously derived from far smaller samples gathered in controlled laboratory settings or through self-reports: Christians’ tended to use more words indicating positive emotions and fewer words indicating negative emotions. The authors also explored the differences in thinking styles between these groups (see Figure 6). Christians were more likely to use words indicating certainty and emotion (e.g., “know” and “feel”), whereas atheists tended to use words expressing skepticism and analysis (e.g., “thought” and “reason”). 

Ritter and colleagues’ dataset was collected and analyzed at a minimum monetary and time cost to researchers and participants. The data used in this study was publicly available and could be recorded without burdening (or paying) the participants. Thanks to automated processing methods, the investigators could perform their analysis swiftly and efficiently, with considerably less effort than required by traditional study designs. 

THE TIP OF THE ICEBERG 

The proliferation of Big Data has ushered in a unique conundrum for psychological researchers: Data originally amassed for purposes as diverse as e-commerce, social interaction, and digital content consumption are now being repurposed to glean psychological insights. The crux of the challenge lies in transforming this data—from incidental digital footprints into meaningful psychological constructs. 

Superficially, data from social media platforms, search engine queries, and online transaction logs can seem like an unwieldy trove of information, replete with noise and devoid of psychological intent. For example, search queries may seem to be mainly relevant to engineers looking to refine search algorithms or marketers interested in ad targeting, like suggesting hair loss solutions to someone googling “hairstyles for men over 40.” However, such search queries carry valuable information that can be leveraged to study critical psychological phenomena.  

To unlock these insights, researchers can employ a combination of computational techniques and psychological theory. For example, natural language processing methods can be used to analyze sentiment and emotional tone in text (Yaden et al., 2017) or detect words and phrases correlating with psychological traits (Park et al., 2014); the frequency and timing of social media posts may serve as indicators of social engagement or personality (Kosinski et al., 2013a), and purchase histories might provide a window into personal values and well-being (Matz et al., 2016). 

In other words, the information visible on the surface is often just the tip of the iceberg of the insights that can be derived from Big Data with some ingenuity. This is vividly demonstrated by geolocation history. 

Case Study: Geolocation Data 

Geolocation data is widely collected by various apps and devices, including smartphones, smartwatches, and modern cars. It typically includes four variables: latitude, longitude, altitude, and the time stamp of each location record. At first glance, the relevance of these variables to psychology may not be obvious. However, a growing body of research demonstrates how geolocation history can be transformed into psychologically relevant information that reveals individual habits, interactions, preferences, and psychological traits (Hinds et al., 2022; Raento et al., 2009). 

Investigating longitudinal patterns in geolocation data can reveal insights into individuals’ habitual routines and actions, such as the spatial and temporal patterns of their social engagements, work, rest periods, and commuting behaviors. For example, Marta González and colleagues (2008) analyzed the movements of 100,000 mobile phone users and discovered that, contrary to theoretical predictions, human mobility often follows simple and predictable patterns. This holds even in the wake of major disruptions and disasters. Geolocation data from the mobile phones of 1.9 million users recorded around the time of the destructive 2010 Haiti earthquake revealed that their mobility patterns post-disaster mirrored those from normal times. In fact, they became slightly more predictable (X. Lu et al., 2012). These findings were closely mirrored by the analysis of the mobility of people affected by the 2012 Hurricane Sandy (Q. Wang & Taylor, 2014).  

Despite their high predictability, mobility patterns are also highly personal. Drawing from geolocation data of 1.5 million individuals, Yves-Alexandre de Montjoye and colleagues (2013) found that a mere four randomly selected spatiotemporal data points could identify 95% of the participants. A study by Chaoming Song et al. (2010) further corroborated these results. They found extreme regularity in human mobility, with most people sticking to a small, recurrent set of locations. 

Analyzing temporal properties of geolocation data—which includes factors like distance traveled, speed of movement, and variations in speed—can yield insights into participants’ activities, transportation methods, or exercise habits (Zheng et al., 2008; Reddy et al., 2010). For instance, Allesio Rossi et al. (2018) gathered geolocation data that detailed professional soccer players’ training sessions. They translated this data into identifiable activities, such as high-speed running or sudden accelerations and decelerations, and then crafted statistical models to predict injuries. Similarly, Sohrab Saeb et al. (2016) studied college students’ mobility patterns to predict the seriousness of their depression symptoms. 

Combining raw geolocation data with other datasets can greatly enrich the information available. Tools like Google Maps[2] or Open Street Map[3] can be used to convert latitude and longitude coordinates into the names and descriptions of visited locations. This process can reveal where individuals live, work, study, eat, shop, engage in religious activities, and socialize. For instance, Thomas Louail and colleagues (2014) examined geolocation data from 200,000 mobile phone users to study the social hotspots across 31 Spanish cities. Similarly, Justin Cranshaw and colleagues (2021) used geo-localized check-ins from a social media platform to divide Pittsburg into clusters characterized by different social dynamics and activities. Location details can also enable inferences about people’s careers, income brackets, education levels, socioeconomic status, religious preferences, and other personal attributes. For example, Joshua Blumenstock and colleagues (2015) analyzed mobile phone data from 1.5 million Rwandans to reconstruct the distribution of wealth at the level of microregions, some composed of just a few households (see Figure 7).   

Figure 7: Maps of Poverty and Wealth in Rwanda Derived from Mobile Data (Blumenstock et al., 2015)

Note: The cells are shaded according to the average predicted wealth of mobile subscribers in that cell. The bottom right inset represents an enlargement of a one square kilometer region, with cells representing groups as small as five to 15 individuals. Figure reprinted from Blumenstock et al. (2015). 

Another layer of insights can be obtained by combining the geolocation data of many individuals. Observing that certain individuals often spend time together in the same location can reveal romantic, professional, or personal relationships. The duration and frequency of these encounters can serve as measures of the relationship’s nature and strength. As a result, geolocation data can be used to construct detailed social network profiles. For example, Nathan Eagle and colleagues (2009) showed that friendship dyads demonstrate distinctive temporal and spatial patterns in their physical proximity and thus can be accurately detected from people’s geolocation.  

In essence, geolocation—which may initially appear to be a rather mundane and superficial type of data—can serve as a rich source of information related to individuals’ traits, behaviors, and the intricacies of their social networks.[4]

TYPES OF BIG DATA 

The previous section highlighted the diverse insights that can be garnered from one type of Big Data: individuals’ geolocation history. Numerous other types of Big Data can offer a similar breadth of insights. These can be categorized into four main groups: mobile sensors, usage logs, language data, and images and audiovisual data. This section briefly delves into each of these categories. 

Mobile Sensors 

Mobile devices—including smartwatches, fitness trackers, and smartphones—are equipped with precise sensors that monitor their owners’ behavior and surroundings. GPS sensors record the time-stamped longitude and latitude coordinates of a device. That can be employed to track users’ mobility patterns (see the previous section). Microphones can be used to monitor events like sneezing, coughing, smoking, or brushing teeth (e.g., Amoh & Odame, 2015). Accelerometers record the acceleration of a device in a three-dimensional space and can be used to track users’ activity (e.g., Althoff et al., 2017). More intricate behaviors can be discerned by synthesizing data from multiple sensors. Combining cartographic information with GPS data can identify locations such as a user’s workplace, home, or shopping center, as well as the paths taken to reach these sites (Raento et al., 2009). Combining accelerometer and microphone data can pinpoint activities like clapping, vacuuming, or disposing trash (H. Lu et al., 2009). Integrating phone usage logs with data from ambient light sensors, microphones, or accelerometers offers insights into users’ sleep patterns (Hao et al., 2013) or levels of alertness (Murnane et al., 2016). 

There is a growing availability of tools that can be used to collect and analyze data from mobile sensors. For example, an open-source Beiwe Research Platform (Torous et al., 2016) enables researchers to gather behavioral data from participants’ phones, including GPS, accelerometer, phone call and text message logs, and survey data. The Google Maps Platform can convert latitude and longitude coordinates into a given location’s address and its detailed description. The Google Activity Recognition[5] translates accelerometer data into types of physical activities of device users, such as sitting, walking, or cycling. 

Big Data collected from mobile sensors offers an unprecedented opportunity to study humans in their natural environments. For instance, Amie Gordon and Wendy Mendes (2021) used a smartphone’s built-in optic sensor to assess the heart rate and blood pressure of over 20,000 participants over three weeks. By matching these records with self-reported stress levels and emotions, they could study physiologic responses linked to stress and emotion. Olivia Walch and colleagues (2016) used data collected from mobile phones to study sleep patterns across different geographies, seasons, and demographic groups, advancing our knowledge of typical sleep patterns. They found, for example, that people exposed to outdoor light tend to go to sleep earlier and sleep longer than those exposed to indoor light. Tim Althoff and colleagues (2017) used data from smartphones with built-in accelerometers to study the physical activity of over 700,000 people across 111 countries. They found that inequality in activity distribution within countries and between genders better predicts obesity prevalence in the population than average activity volume. 

The comprehensive “Handbook of Mobile Sensing in Psychology: Methods and Applications” by Matthias Mehl and colleagues (2023) is an excellent resource for those planning to analyze data from mobile sensors. 

Usage Logs 

The behaviors of individuals in both online and offline settings are extensively monitored and recorded by digital products and services. These records often resemble diary-like logs, filled with time-stamped entries, such as “John Doe listened to ‘Poker Face’ by Lady Gaga at 3:00 PM on January 1, 2024.” Typical examples of usage logs include search queries; web-browsing history; activity logs from e-learning platforms, streaming services, and games; lists of Likes or reposts on platforms such as Instagram, Facebook, or X; geo-location history; or purchase logs. Usage logs can be evaluated using relatively straightforward statistical tools, and their processing generally requires fewer computational resources than other data types, such as language, images, or audiovisual data. 

The simplicity and widespread availability of usage logs make them popular among scientists interested in studying human behavior. For example, Adam Fourney and colleagues (2015) used search logs of about 13,000 pregnant women to study the evolution of their interests, concerns, and emotional states during pregnancy and the first few months of caring for their newborns (see Figure 8). Márton Mestyán and colleagues (2013) found that movies’ popularity can be reliably predicted—before their release—by analyzing activity logs of editors and viewers on these movies’ Wikipedia entries. Sandra Matz and colleagues (2016) studied the links between consumers’ well-being, personality, and financial transaction logs extracted from their online bank accounts. They found that money buys happiness, but only when spending fits consumers’ personality profiles.  

Figure 8: Histograms for Terms in Search Queries Reflecting Potential Emotional States Over the 40 Gestational Weeks of Pregnancy

Note: Figure reprinted from Fourney et al. (2015). 

Digital footprints, such as social media usage logs, offer profound insights into individual psychology, as evidenced by their correlation with diverse psychological constructs. For instance, the records of what people like on Facebook can reveal a person’s political and religious beliefs, relationship status, substance use habits, ethnic background, sexual orientation, the nature of their social networks, as well as indicators of their well-being, personality traits, and cognitive abilities (Kosinski et al., 2013b). Consequently, these digital traces have become invaluable tools for psychological assessment and research, providing a window into personal characteristics that were previously difficult to measure (Youyou et al., 2017). However, exploiting such data also raises significant ethical concerns, particularly regarding privacy and consent. Notably, the Cambridge Analytica scandal highlighted the risks associated with the unauthorized use of Facebook data to infer personality profiles and target individuals with tailored political advertising (Levy, 2020). 

Moreover, instances like Target’s analysis of customer purchases to infer pregnancy status and send targeted product catalogs (Hill, 2012) illustrate the potential for such insights to lead to uncomfortable or even dangerous situations: In too many cultures, women still face severe social repercussions or even death if they are found to conceive a child outside of wedlock. These examples underscore the dual-edged nature of leveraging digital footprints in psychological research, a topic that will be explored in greater depth later in this chapter.  

Those interested in incorporating usage logs in their studies may consider Eric Chen and Sean Wojcik’s “A practical guide to Big Data research in psychology” (2016) and Michal Kosinski and colleagues’ “Mining Big Data to Extract Patterns and Predict Real-Life Outcomes” (2016). 

Language Data 

Beginning with Sigmund Freud (1891/1953), many psychologists have recognized the strong connection between an individual’s use of language and psychological phenomena. However, the study of language due to the challenges of capturing it in natural settings and the absence of efficient analytical tools. Technological progress has resolved both issues. First, as much of human communication and information exchange is now mediated by digital technologies, language is one of the most common types of Big Data. Those Big Data sources include social media posts, instant messages, emails, text documents, e-books, online encyclopedias, conference calls, and YouTube videos (which can now be easily transcribed using voice-to-text technology). Second, the widespread availability of linguistic data, coupled with the advancement of computing power, has given rise to powerful and user-friendly tools for language analysis, as discussed in more detail in the accessible tutorials by Nickolas Jones and colleagues (2016) and by Brendan Kennedy and colleagues (2022).  

A growing number of psychological studies employ Big Language Data. For example, Andrew Schwartz and colleagues (2013) studied the relationship between the language in Facebook status updates from over 75,000 participants and their personality, gender, and age. Among many patterns observed in their study, some were aligned with theoretical predictions and previous findings (e.g., neurotic individuals used terms like “sick of” and “depressed” more frequently). Yet, other patterns suggested novel hypotheses (e.g., the link between the language suggesting active lifestyles and participants’ emotional stability; see Figure 9). Cristian Danescu-Niculescu-Mizil and colleagues (2012) studied how language can reveal power relationships in two settings: discussions among Wikipedia editors and arguments before the U.S. Supreme Court. They found that individuals with less power were more likely to adapt their language to match the linguistic style of their more powerful counterparts. This pattern was less pronounced or even reversed when the power dynamics were flipped. Stevie Chancellor and colleagues (2019) studied how individuals discuss and share experiences about opioid use recovery on Reddit, an online forum, and identified common themes, sentiments, and discussions related to alternative treatments for opioid addiction. The study revealed that Reddit users employ a wide range of nonconventional treatments, including both pharmacological options, like Kratom and cannabidiol (CBD), and non-pharmacological approaches, such as exercise, meditation, and dietary changes. 

Figure 9: Words, Phrases, and Topics Extracted from Facebook Status Updates That Are Most Distinguishing Between Neurotic and Emotionally Stable Individuals (Schwartz et al., 2013) 

Images and Audiovisual Data 

Digital environments are widely used to create, share, publish, and consume multimedia content such as images, videos, and audio clips. Millions of individuals upload videos to YouTube, share photos and videos on Instagram and Facebook, and communicate through video calls. Other sources of image and audiovisual data include video blogs, podcasts, voicemail records, and voice samples captured by smart-home assistants like Amazon’s Alexa or Google Home. Moreover, people’s voices and appearances are increasingly often captured without their consent or awareness: We are surrounded by a growing number of closed-circuit television (CCTV) cameras, we appear in the background of photos taken by others, and we are captured by cameras monitoring the surroundings of self-driving cars. 

These images and audiovisual recordings offer a wealth of data on behavior, thoughts, and interactions. In the past, their use in psychological research was limited by the effort required to code the visual content or transcribe the audio recordings. However, there is now a burgeoning array of analytical tools designed to automatically extract such information, thereby saving considerable time and effort. Speech-to-text software packages[6] enable transcribing audio and video files; visual object recognition algorithms can identify and categorize objects in images;[7] and advanced facial-detection software, such as Microsoft Cognitive Services,[8] can detect faces and interpret emotional expressions (but see Lisa Barrett and colleagues’ 2019 article on the challenges of using facial expressions as a measure of emotion). 

As a result, recent years have witnessed a surge in psychological research harnessing the power of images and audiovisual data. For example, Alan Cowen and colleagues (2021) extracted faces from six million videos from 144 countries and used a deep neural network to detect and organize facial expressions into 16 clusters. The videos were also coded for context, such as “wedding” or “dance.” The team found a 70% overlap in the associations between context and facial expression across diverse geographical regions, suggesting substantial universality in human emotional expression. Andrew Reece and Christopher Danforth (2017) analyzed 43,950 Instagram photos from 166 users to identify markers of depression. By employing machine learning for color analysis and facial detection, among other techniques, they identified that certain features, such as color saturation and the presence of faces, could serve as predictors of depression. Their models outperformed general practitioners’ diagnostic success rates, even when analyzing posts made before participants were diagnosed with depression. Michal Kosinski (2021) applied a facial recognition algorithm to social media profile images of over one million individuals. The results showed that by comparing the similarity of an individual’s face to liberal and conservative others, it was possible to distinguish between liberals and conservatives in over 70% of cases. This high level of precision held steady across variables like age, gender, and ethnicity, and was consistent in various countries, including the U.S., Canada, and the UK. 

WHY “BIG” IN “BIG DATA” MATTERS 

An inconvenient truth about psychological science is that many of its claims and theories are based on studies employing inadequately small samples (Fraley & Vazire, 2014). This is highly problematic, as small sample studies are characterized by low statistical power.  

Statistical power, also known as sensitivity, represents the probability of committing a Type II error, or missing an actual effect (e.g., false negative). Consequently, literature replete with underpowered studies is likely overlooking many significant effects. Moreover, although it may seem paradoxical, underpowered studies tend to exaggerate the magnitude of real effects and detect non-existent effects (this is referred to as false positive or Type I error; Button et al., 2013). This is because only the most extreme or anomalous effects can achieve statistical significance in low-powered studies.  

In a perfect world, an occasional false positive would be counterbalanced by the larger body of studies finding no effect. However, due to publication bias—also known as the file-drawer problem—studies showing no effect are rarely published (Rosenthal, 1979). Similarly, in such a perfect world, an occasional exaggerated effect would be counterbalanced by the understated reports. However, underpowered studies simply cannot detect small effects. In other words, the positive results of small-sample studies should be treated with great skepticism, even if they have been repeatedly replicated in other small samples (Fraley & Vazire, 2014). As powerfully argued by Tal Yarkoni and Jacob Westfall (2017), small samples are one of the main culprits of the failure to replicate many classic psychology studies, referred to as the replicability crisis. 

Big Data can help to address this issue. Its immense volume and velocity result in samples comprising tens of thousands to millions of individuals, offering statistical power unmatched in conventional studies. To fully appreciate the urgent need to boost the sample size in psychological science—and thus the opportunity provided by Big Data—it is crucial to consider the magnitude of effects that researchers should expect to detect in psychological science. 

Typical Effect Sizes in Psychological Science 

To determine the required sample size for testing their hypotheses, researchers initially assess the potential impact of the effect they aim to uncover. It is common in psychology to reference Jacob Cohen (1988), who designated effects equivalent to the Pearson product-moment correlation of r = .10, .30, and .50 as thresholds for small, medium, and large effects, respectively. Although Cohen later regretted suggesting these thresholds (Funder & Ozer, 2019), they have gained widespread acceptance and are regularly leveraged to disregard effects under r = .30 as minor and trivial. 

In reality, only the most extraordinary—or most exaggerated—psychological effects can pass Cohen’s threshold for a medium effect (r = .3). Dan Richard and colleagues’ (2003) review of more than 25,000 psychological studies encompassing eight million participants found an average effect size of r = .21, a finding replicated by Chris Fraley and Michael Marks (2007). More recently, Gilles Gignac and Eva Szodorai (2016) observed an even lower average reported effect of r = .19, with the weakest 25% of effects falling below r = .11, and the strongest 25% of effects being above r = .29. In other words, just a quarter of reported effects surpassed Cohen’s threshold for a medium effect.  

Additionally, many well-established and consequential psychological effects are well below the average (Richard et al., 2003). For example, the scarcity principle, or the cognitive bias boosting the perceived value of scarce resources, manifests at r = .12. The self-serving bias, or a tendency to blame external factors rather than oneself for failures, comes in at r = .10. The correlation between a communicator’s perceived credibility and their ability to persuade others equals r = .10. And let us not forget that those estimates are based on the literature filled with underpowered studies that simply cannot reliably detect small effects. In other words, they are likely inflated. Additionally, psychology is not the only field in which effects tend to be in this range (Funder & Ozer, 2019). For example, the pain-relieving effect of a nonsteroidal anti-inflammatory drug, such as Ibuprofen, is approximately r = .14. The use of antihistamines to prevent the effects of allergies, such as runny nose or sneezing, is around r = .11. While smoking nicotine remains one of the leading preventable causes of deaths globally, killing over eight million people per year, its association with lung cancer equals r = .08.  

Finally, even the seemingly small effects can have tremendous practical implications. Consider the 1987 study conducted on 22,071 participants examining the links between taking aspirin and heart attacks (Rosenthal, 1990). The early results showed that taking aspirin reduces the chances of heart attack with an effect size of r = .034. Although it is far below Cohen's small effect threshold, this effect translated into a dramatic 45% drop in heart attacks among those taking aspirin (Table 1). Its importance is underscored by the fact that the study was immediately terminated, as the researchers decided it would be unethical to continue giving half of the participants a placebo. 

Table 1: Effects of Aspirin on Heart Attacks Among 22,071 Physicians

Note: Table adapted from Rosenthal (1990).

In other words, when estimating the expected magnitude of effects, one should consider both their size and their implications. Many consequential and well-established psychological effects are around or below r = .10. Conversely, the effects that are now commonly referred to as medium or large (e.g., r > .3) are, in fact, extraordinarily large and should be treated as such: with healthy skepticism. 

How Big Is Big Enough? 

Consequently, when planning their studies, psychologists should not only expect to find effects of r = .1 and below, but also recognize that effects as small as r = .03 could have immense implications (Rosenthal, 1990). What sample sizes are necessary to secure statistical power sufficient to detect such effects? A commonly recommended power level in psychological research is 80% (Cohen, 1988), which translates into an 80% probability of detecting an effect of a given size if it genuinely exists in the population. The sample size necessary to detect a correlation of r = .10 (e.g., self-serving bias) with the power of 80% equals n = 780. The sample necessary to detect an effect of r = .03, such as the life-saving effect of aspirin, must include over 8,700 participants. Moreover, those numbers assume no attrition of participants throughout the study, the threshold for statistical significance at α= .05, and the most straightforward research design involving a single statistical comparison. In practice, samples that are several times larger are necessary to study effects of this magnitude.  

The samples used in conventional psychological research tend to be much smaller partly due to the high cost of collecting data. Expenses like participant compensation for time and travel, research assistant salaries, and laboratory equipment often add up to hundreds of dollars per participant. Online samples recruited via platforms such as Prolific.com or Amazon Mechanical Turk tend to be less expensive, but their costs are still significant. For example, in 2024, the cost of recruiting 1,000 participants for a 30-minute online study on Prolific.com was approximately $8,000.[9] As a result, the median sample size in studies published in the four leading social psychology journals equaled ñ = 81 in 2009 and 164 in 2018 (calculated using data published by Kai Sassenberg and Lara Ditrich, 2019). Samples of such size are characterized by statistical power sufficient to detect the effects of r = .31 in 2009 and r = .22 in 2018. In other words, over half of the studies published in 2018 were too small to detect an average effect in psychological science.  

Big Data samples provide a cost-efficient alternative. Instead of paying a few hundred or a few thousand participants to self-report their preferences in surveys or try to behave naturally in artificial experimental setups, researchers can observe the actual preferences and behaviors of hundreds of thousands or millions of participants going about their lives in their natural environment. The resulting statistical power allows for detecting effects too subtle to be captured in conventional studies and reduces the risk of detecting false positives. The benefits provided by the volume of Big Data have led to the popular saying that “more data beats a cleverer algorithm” (Domingos, 2012, p. 84). It is often the case that the most effective way to enhance the reliability and validity of research findings is by collecting more data.  

The potential of Big Data to address the challenges of low statistical power, the file-drawer problem, and the prohibitive cost of collecting data is well illustrated by the research on the links between antisocial behavioral tendencies and facial width-to-height ratio (fWHR; Kosinski, 2017).

Case Study: Are Broad-Faced People Antisocial? 

The results of many conventional studies show that facial width correlates positively with antisocial and dominant behaviors, particularly in men. Broader-faced men were found to be more likely to engage in inter-group competition, cheating, lying, aggressive, and deceptive negotiation strategies, and to be more racially prejudiced (e.g., Hehman et al., 2013). The reported effects were enormous. For example, the facial width of hockey players was correlated with their penalty minutes—a surrogate for aggression—at the level of r = .54 (Carré & McCormick, 2008). If this estimate were accurate, it would stand as one of the most substantial effects in psychology. To put this in perspective, consider a well-established finding that the relationship between aging and cognitive decline is approximately r ≈ .5 (Verhaeghen & Salthouse, 1997). 

However, as is typical of many areas of psychological research, conventional fWHR studies predominantly employ inadequately small samples. About half included fewer than 25 participants, with an average sample size of around 40 (Kosinski, 2017). For example, the study by Justin Carré and Cheryl McCormick (2008) was based on a sample of just 21 hockey players. Consequently, the effect sizes reported in fWHR had to be very large. A study including 40 participants cannot detect a correlation below r = .43. 

Inadequately small samples in fWHR studies suggest that their findings may be exaggerated and include many false positives. To test this hypothesis, Michal Kosinski (2017) obtained data from 137,000 participants, including their scores on a range of psychological questionnaires and their Facebook profile images. The images were analyzed using a facial recognition algorithm to automatically delineate facial contours and estimate their facial width and height ratio (see Figure 10). The results revealed that although there was a correlation between fWHR and certain self-reported behaviors, the observed effects were not only smaller but also contrary to the ones typically reported in the fWHR literature. For instance, facial width showed a positive correlation with agreeableness—a trait indicating a propensity for prosocial, sympathetic, trusting, and cooperative behavior—in both men (r = .04) and women (r = .07). Additionally, individuals with wider faces demonstrated a somewhat lower interest in drug use, weapons, and body modifications such as piercings and tattoos. 

Naturally, the debate surrounding fWHR is far from settled: One should be cautious when drawing conclusions from any single study, no matter how large its sample may be. However, the results of this Big Data study offer some counterweight to those observed in past small-sample studies, illustrating the potential of Big Data to help tackle the problem of Type I errors. Moreover, this study underscores the second advantage of ample statistical power typical of Big Data studies: its potential to reduce the risk of Type II errors (or missing the actual effects). With 137,000 subjects, it was possible to detect an effect size as tiny as r = .008, with a statistical power of 80%. Several such novel effects were detected. For example, fWHR correlated with the Satisfaction with Life Scale (Diener et al., 1985) for both men (r=.074) and women (r=.085).

Figure 10: Facial Landmarks Placed Automatically on a Facial Image by Facial Recognition Software (Kosinski, 2017)

Note: The fWHR was estimated by dividing the distance between the vertical lines marking the cheekbones by the distance between the horizontal lines marking the mid-brow and upper lip.  

This study illustrates yet another advantage of Big Data samples: their affordability. Collecting data from 137,000 participants using conventional methods would be extremely costly and challenging. First, it is difficult to access participant pools of this size. For example, the entire participant pool on Prolific.com, one of the most popular online participant recruitment platforms, included 120,000 participants in 2024. Second, the associated costs would be staggering. Given the time required for participants to complete questionnaires employed in this study—about 60 minutes—the cost of collecting this data on a platform such as Prolific would exceed $2 million in 2024.[10] Instead, data used in this project was obtained at no cost from the MyPersonality database, which was originally collected for a different purpose (Stillwell & Kosinski, 2015). 

GETTING OUT OF THE LAB 

Psychological labs are wonderful. They provide researchers with a controlled environment where variables can be carefully managed to isolate factors of interest and minimize external influences that could skew the results. By providing a consistent setting for all participants, labs improve the reliability of results. They also allow for precise and comprehensive observation of participant behaviors and interactions, yielding data of exceptional quality. 

However, throughout the field’s history, there have been constant calls for psychologists to get out of their labs to study people in their natural environments (Barker & Wright, 1951). This is because the advantages of brick-and-mortar and virtual psychological labs are matched by several major drawbacks. First, lab samples tend to be of low diversity, predominantly including university students. Second, ethical and practical constraints cap the range of variables that can be manipulated in a lab environment, thereby narrowing the breadth of hypotheses that can be rigorously tested. As a result, some of the most important psychological phenomena, such as response to severe stress or anxiety, cannot be easily studied in the lab. Third, participants’ awareness of participating in a study introduces many biases. These include demand characteristics, where participants unconsciously alter their behavior due to their assumptions of the study’s purpose (Nichols & Maner, 2008); experimenter expectancy effects, where the experimenters’ beliefs affect participants’ behaviors (Rosenthal, 1967); and the Hawthorne effect, where participants change their behavior—typically in a socially desirable way—simply because they are aware that they are being observed (Sedgwick & Greenwood, 2015). Fourth, lab experiments usually involve artificial situations deviating significantly from real-life scenarios. All these disadvantages greatly reduce the generalizability of lab-based findings to broader, real-world situations. Finally, conducting research in a lab can be prohibitively expensive. 

Big Data samples offer a chance to address some of the shortcomings of lab-based studies (Rust et al., 2020). They are generated by people going about their business in their natural environments rather than in the artificial setting of a psychological lab or an online testing platform. This allows researchers to observe spontaneous and authentic behaviors, including in situations that would be difficult or unethical to simulate in the laboratory setting. By relying on automatically recorded behavior rather than participants’ self-reports, researchers reduce the role of memory biases, such as recency effects and social desirability. Moreover, the data can be collected retrospectively—days, weeks, or even years after it was generated. As a result, participants are—at the time when their behavior is recorded—unaware that they are generating research data. This reduces the role of participant biases, such as demand characteristics or the Hawthorne effect. 

These benefits of Big Data are illustrated in the context of research on an important and consequential research question: What drives people’s romantic choices? 

Case Study: Online Dating 

Historically, research into romantic preferences has primarily relied on self-reported data (for a comprehensive review, see Eli Finkel’s chapter in this handbook, 2024). In a typical study, participants might be asked to describe their ideal partner and rate the importance of attributes such as “physical attractiveness” or “generosity.” Commonly, findings from these studies indicate that both men and women prioritize qualities like integrity, dependability, and intelligence, assigning less importance to wealth, fitness, or physical attractiveness (Christensen, 1947; Gerlach, 2019). 

Can we trust these results? They sound almost too good to be true. How can we be sure that the participants know what they are attracted to? Also, even if they are aware of their preferences, are they willing to report them honestly? Decades of research on response biases suggest caution (Rust et al., 2020). Self-reports typically reflect explicit, conscious, and motivated opinions rather than actual behaviors and preferences. For example, participants may be unwilling or unable to admit even to themselves their own socially undesirable preferences, such as for physical attractiveness or race (e.g., “I am not shallow enough to prioritize physical beauty!”). Alternatively, they might generalize their preferences from readily available examples (e.g., “My current partner is exceptionally kind, so kindness must have been crucial in my attraction.”). These self-imposed narrative fallacies can skew the data away from genuine unconscious or instinctive preferences and toward a socially accepted or personally idealized version of attraction. 

Relationship scholars have long recognized the shortcomings of self-reported data in capturing the nuances of romantic preferences. However, historically, there were few alternatives. One approach involves speed-dating experiments, in which participants engage in brief interactions with a series of potential romantic partners under the watchful eye of the experimenters. While there have been many excellent speed-dating studies, they have significant limitations (Finkel et al., 2007). Speed-dating events are challenging to set up, may attract highly atypical participants, and constitute a rather artificial context to meet others. Another solution to collecting such data might involve physically following single individuals to document their actual romantic choices rather than relying on their self-reports. However, such a methodology would be exorbitantly costly, labor-intensive, and highly intrusive. The very act of being observed by researchers, especially in the context of intimate behaviors like flirting, is likely to alter the participants’ natural conduct significantly. 

Fortunately (at least for researchers), dating is increasingly mediated by online dating platforms. As of 2019, about half of all new relationships are arranged through dating platforms (Rosenfeld et al., 2019). Investigating how individuals use these platforms provides an opportunity to bypass the inherent biases of self-reported data, offering a clearer view of their actual romantic preferences and choices. The average user dedicates over 50 minutes daily to swiping and messaging on these platforms, creating an extensive digital trail of hundreds of choices and interactions (Badoo, 2016). Even the most enthusiastic participants in traditional studies would be unlikely to have enough commitment, time, and attention to generate so much research data. 

Dating platforms record who initiates contact; their chances of receiving a response; and much personal information about their users, including their age, location, education, income, political views, religiosity, age, and income. Although users often showcase themselves in the most favorable light and may occasionally tweak the truth, research indicates that these deviations are minor (Sharabi & Caughlin, 2019). Similarly, although profile images are carefully selected to present the users in the most flattering way, they are a good proxy for their physical attractiveness, ethnicity, and other demographic attributes. Additionally, the behavior of users following the initial in-person date, which could entail both parties deactivating their accounts or setting up dates with other users, can reveal whether the offline date was successful. Finally, the longevity of the relationship that initiated on a dating website can be studied by matching partners’ details with marriage and divorce records, which are public in most U.S. states. 

As a result, relationship scholars rushed to collect data from online dating platforms. Some have enlisted volunteers willing to share their profile details, others have designed field experiments, and a few have gone straight to the source—obtaining data directly from the platform administrators. These methods have resulted in not just larger pools of participants but also a richer cache of data on individual preferences and behaviors. Their results offer a more comprehensive, albeit less flattering, portrayal of human romantic preferences. Physical attractiveness emerges as the most important factor for both genders, although men place about twice as much emphasis on it. Income ranks as the second most important aspect for both genders, although women tend to value it about twice as much as men (Hitsch et al., 2010). More disturbingly, the dating website data analysis reveals race’s significant impact on individuals’ choices (Table 2). For example, the results presented by Christian Rudder (2019) show that when Black men contact White women on OkCupid, a dating platform, they experience a reply rate of 21%. In contrast, White men receive responses at a rate of 29%. Black women, when reaching out to Black men, are replied to 37% of the time; in contrast, White women communicating with Black men see a response rate of 51%. 

Table 2: Probability of Getting a Response to a Message Sent on Okcupid Given Users’ Race and Gender

Note: Table adapted from Rudder (2019). 

Some of these insights might be specific to the realm of online dating. For instance, the prominent role of physical beauty could be partially attributed to the challenge of assessing character from a dating profile. Users of dating websites may value character more than the data suggests but can only discern it after meeting offline. Other insights likely generalize beyond online dating. For example, individuals may genuinely believe they do not discriminate against others based on their race (or are too embarrassed to admit it). However, their online behavior tells a different story. 

That online dating studies facilitate comparisons between racial groups is a symptom of yet another major advantage of Big Data: They typically include a more diverse and more representative population than conventional studies. The latter are often too small to facilitate intragroup comparisons, even just between genders, let alone between multiple racial or ethnic groups. In contrast, Rudder’s analysis (2019) of racial biases in message responses involved a dataset of over one million individuals from a broad demographic range, including more than 67,000 people who identified as Hispanic, 31,000 as Native American, and approximately 12,000 as Pacific Islander. 

The Dynamic Nature of Digital Environments 

The research on people’s romantic choices elegantly illustrates the advantages of the naturalistic character of digital environments, where much of Big Data originates. However, this naturalistic character is also a source of major challenges. One of them relates to the dynamic character of such environments: They are continuously and frequently updated, usually without warning. Such updates can interrupt data collection, add noise to collected data, or change users’ behavior in ways that may be mistaken for an effect. Consider the curious example of Facebook status updates. 

Case Study: The Evolution of Facebook Status Updates  

In Facebook’s early days, the status update textbox was prefaced by a fixed “[User’s name] is” prompt, nudging the early Facebook users to write self-centered posts using third-person self-references (e.g., “Mary is really enjoying Britney’s new no-hair style.”) Toward the end of November of 2007, the fixed prompt was replaced with a placeholder text asking, “What are you doing right now?” which disappeared when the user started typing, enabling the users to broaden the range of topics discussed in their posts (e.g., “Just got my copy of Harry Potter and the Deathly Hallows!”). At the beginning of 2009, Facebook updated the placeholder to state “What’s on your mind?” encouraging users to shift from writing about themselves to sharing their thoughts on current events, politics, and societal issues (e.g., “Yes we can!”). 

A researcher unaware of these changes in Facebook’s user interface might erroneously attribute the resulting change in user language to some external factors or events. The 2009 update, for example, coincided with Barack Obama’s inauguration for the U. S. presidency. This event likely left a mark in the status updates of the predominantly young, liberal, and educated demographic that formed the early core of Facebook’s user base. However, to study it, one would need to control for a much more trivial (and potentially much more impactful) factor: the change of the status update placeholder text from “What are you doing right now?” to “What’s on your mind?”

Similar changes in the functioning of digital platforms and devices are common, often occur unannounced, and are rarely as evident as the changes to the Facebook status update feature.

DIVERSE AND REPRESENTATIVE POPULATIONS 

Psychology literature frequently highlights the importance of ensuring that the populations under study are not only diverse but also representative of the general population. It further emphasizes the necessity of acknowledging, respecting, and examining the differences in psychological mechanisms within these groups. However, a closer look reveals that many of psychology’s claims and conclusions are derived from studies employing convenience samples of undergraduate psychology students or—more recently—professional test takers recruited on platforms such as Prolific or Amazon Mechanical Turk (Hanel & Vione, 2016; Landers & Behrend, 2015). This is because gathering diverse and representative samples is much easier said than done. Many researchers face constraints in resources, time, and expertise, which are necessary to venture beyond the walls of their universities or an online survey platform. Additionally, some demographic groups can be hard to approach; they might be wary of researchers or may not have the inclination or time to engage in studies (Ellard-Gray, 2015). 

Relying on nondiverse samples compromises the credibility of psychological science. Policymakers, business leaders, clinical practitioners, and scholars from different disciplines might justifiably question the validity and representativeness of conclusions derived from professional test takers earning $12 per hour or from college students who are disproportionately young, female, and mainly from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) cultures, as pointed out by Joseph Henrich and colleagues (2010). 

Big Data offers a chance to address this problem and greatly increase the diversity and representativeness of samples used in psychological research. Big Data sources, such as social media platforms or search engines, have a global reach, facilitating cross-cultural comparisons and increasing the external validity of findings. Moreover, Big Data often captures information from a broad spectrum of individuals, spanning various ages, ethnicities, socioeconomic statuses, educational backgrounds, subcultures, and more (Gosling & Mason, 2015). Many Big Data samples approach the size of the studied population and thus are nearly perfectly representative. These advantages of Big Data are well illustrated by the research employing one of the most universally used digital environments: Google Search. 

Case Study: Tracking Suicidal Ideation Using Google Trends 

As the Pew Research Center (2021) reported, internet use has become nearly ubiquitous among North Americans aged 18 to 64, with roughly 98% being online and about 92% using Google Search. Consequently, data generated by Google Search users reflects the behaviors, preferences, and interests of the vast majority of U.S. adults. Amazingly, Google makes this invaluable data freely available, in an aggregate form, for anyone to study. Meet Google Trends.[11]

Google Trends allows anyone to examine the frequency of keywords, topics, and phrases searched for on Google over various time frames. It offers capabilities to benchmark the search popularity of different terms, discover related searches, and investigate the geographical spread of search interests. Powered by the billions of queries Google processes daily, Google Trends offers a lens into collective behaviors and sentiments on a massive scale. As it provides near-instantaneous data, researchers and analysts can use it to promptly spot and evaluate evolving trends, capturing shifts in public interest and concern as they unfold. Researchers can also explore the data retrospectively to examine how the search trends evolved over the last decades. 

Aggregated search queries are a gold mine for understanding human behavior and psychology (Stephens-Davidowitz, 2017). People are unusually honest when they search. Unlike traditional surveys, which are susceptible to social desirability bias, search queries tend to reflect individuals’ genuine interests, concerns, or curiosities, offering a more candid glimpse into minds. Additionally, the privacy of individuals is inherently safeguarded when using Google Trends, as researchers access only aggregated data rather than individual search histories. 

The advantages of using Google Trends are well illustrated by the study conducted by John Ayers and colleagues (2017). On March 31, 2017, Netflix released 13 Reasons Why, a controversial television series revolving around topics such as suicide, sexual assault, bullying, substance abuse, and mental health. The first season included a three-minute-long suicide scene that generated significant controversy and concern from mental health professionals, educators, and parents who believed it might pose a risk for vulnerable viewers, inspire copycat incidents, and trigger those with suicidal ideation. 

Were those fears justified? However, collecting the necessary conventional data would be tremendously challenging, as people are reluctant to discuss suicidal ideation with others (including with, or perhaps predominantly with, researchers). Furthermore, some populations that could have been especially affected by 13 Reasons Why may be particularly difficult to enroll in a psychological study. These populations include socially isolated groups, such as people who are facing economic or educational disadvantages, people who have psychological disorders, and adolescents who are rarely included in research samples due to their vulnerable status. Although these populations may be hard to reach through conventional recruitment methods, many are active users of Google Search, and thus, their behaviors are captured and aggregated by Google Trends. 

Ayers and colleagues used Google Trends to measure the prevalence of suicide-related search terms before and after the release of 13 Reasons Why (see Figure 11). They found that the release coincided with a 19% increase in the overall number of concerning search terms, such as “how to commit suicide.” Moreover, there was an increase in searches for suicide prevention and helplines (e.g., “suicide hotline”), indicating that some viewers might have been seeking help. Importantly, those results likely understate the effect that this show had on its viewers. Google Trends captures trends across all Google Search users, yet the immediate effects of the show were likely limited only to the show’s viewers. Thus, had the analysis been limited to the show’s audience, the observed effects might have been even more substantial.  

Figure 11: Daily Trends for All Google Searches Including the Term “Suicide” but Excluding “Suicide Squad” Before and after the Release of 13 Reasons Why on March 31st, 2017 (Ayers et al., 2017)

Note: Search volume is relative to its peak popularity in the studied time frame. 

Their results emphasize the significant influence that media, especially popular shows, can have on public mental health. Although an increase in search volumes does not necessarily translate to a rise in actual suicidal behaviors, it underscores the potential harms and benefits related to how content creators handle sensitive topics. Ultimately, in July 2019, Netflix decided to edit out the graphic scene, added viewer discretion warnings, and created a website with crisis helplines for viewers who might be affected by the content (Marshall, 2019). 

Many other psychological studies employed Google Trends to study psychological phenomena. A quick search on Google Scholar for “Google Trends psychology” returns over 14,000 results, with papers titles ranging from “Trendy Plots – A Google Trends-based Inquiry on the Social Determinants of Conspiracy Theories” (Cafiero et al., 2020) to “The impact of COVID-19 Pandemic on Pornography Habits: A Global Analysis of Google Trends” (Zattoni et al., 2021). For a more comprehensive discussion of how search data can be used in psychological research, consider Seth Stephens-Davidowitz’s insightful book Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are (2017), which discusses a broad array of analyses employing search data to study sexuality, race and prejudice, mental health, child-rearing, predicting election outcomes, violence and aggression, and many other issues that tend to be challenging to study using conventional methods. 

COLLECTING BIG DATA 

The examples of studies highlighted thus far in this chapter demonstrate that collecting Big Data is often more straightforward than acquiring conventional data. This section briefly overviews the strategies most widely employed to collect Big Data. 

Preregistration 

The vast availability of pre-collected Big Data sets and the large number of variables they typically contain make them particularly prone to p-hacking (i.e., reanalyzing until an effect is found; Simmons et al., 2011) and HARKing (i.e., hypothesizing after results are known; Kerr, 1998). Consequently, preregistration of Big Data studies becomes an indispensable step. Preregistration helps confine the scope of p-hacking and HARKing by restricting researchers’ degrees of freedom. It also enhances transparency, improves study planning, and helps reduce publication bias (Weston et al., 2019). Brian Nosek and colleagues’ (2018) paper “The preregistration revolution” and Van den Akker and colleagues’ (2021) tutorial for preregistering secondary data analysis are valuable resources for researchers planning to pre-register secondary analyses of Big Data. 

Secondary Data 

Before collecting new data, it is prudent to first check if the needed data already exists, which can save time, reduce costs, and minimize the burdens placed on study participants. Notable directories of data sources include the “List of datasets for machine-learning research” on Wikipedia;[12] Cameron Brick and colleagues’ “Directory of free, open psychological datasets” (2024);[13] the list of data sources hosted by the Inter-university Consortium for Political and Social Research;[14] Kaggle.com,[15] the online platform for data scientists; and two major governmental repositories: Data.Gov,[16] including over 300,000 datasets, and European Union Data,[17] hosting over 1.5 million datasets. Many of these datasets are published by businesses. Facebook, for example, shared aggregate mobility data from approximately 29 million users to support social distancing research during the COVID-19 crisis.[18] Google created Dataset Search,[19] a hosting platform and search engine for data published by Google and others. Additionally, a growing number of scholarly papers are now accompanied by data that can be used to reproduce their results. Such data can often be leveraged to explore research questions extending beyond the purpose for which it was originally collected.  

Several important factors should be evaluated when searching for and considering secondary data sources (Johnston, 2014): Was the data collected legally and ethically? Do the owners have the right to share it? Can it be freely used for the planned research? (Some data, for example, can be used only to replicate the original study. In other cases, the publication of the results requires data owners’ permission.) Is it clear how the data was originally generated? In what ways was it altered before it was published? Does it comply with relevant privacy rules? In what ways should the data source be cited or attributed? Often, the answers to these questions can be found alongside the data. In other cases, contacting the owner may be necessary: They may be willing to provide missing information or alter the data to allow for it to be reanalyzed (e.g., by de-identifying it to comply with privacy regulations). 

Application Programming Interfaces (Apis) 

An alternative method for obtaining data is to collect it directly from the platform where it was generated. Many platforms enable researchers and other external parties to access their data through specialized Application Programming Interfaces (APIs). APIs allow software applications to interact with each other conveniently. Thanks to APIs, one can write a single line of code in R or Python to send a request and receive a response from applications hosted by others.[20]

One API frequently used in psychological research is provided by X, offering access to users’ profiles and their posts. Figure 12 illustrates the appearance of a post to X platform users, alongside the representation of that post when retrieved via the API. To access X’s API, one must first register on the X developer platform[21] and then connect with the API using one of the many libraries available for R, Python, and other programming languages. For those new to programming or data science, Sean Murphy’s excellent, highly recommended tutorial (2017) provides a step-by-step guide. Given that APIs tend to be similar, learning to use a given API enables one to collect Big Data from many other APIs. 

Figure 12: An X Post as Seen by X Users Alongside the Same Post Retrieved from X’s API and Stored in an R Data Frame

Note: Figure adapted from Murphy (2017). 

Data Scraping 

APIs are the most convenient method for collecting data from digital platforms whenever available. They are specifically designed for data exchange and usually provide the most dependable and efficient means of access. If an API is unavailable, researchers might resort to “scraping” data by using computer code to traverse the digital platform to collect the displayed data. Although this approach may initially seem daunting, numerous user-friendly libraries in R and Python simplify the process. 

Richard Landers and colleagues’ (2016) tutorial tailored for psychologists, provides a good starting point for those considering collecting data using scraping. It includes sample code and a case study highlighting the advantages and challenges of scraping data. Notably, it estimates the time investment required to learn how to scrape. Initially, each author completed a 13-hour Python course available for free on Codecademy.[22] Next, they dedicated between five and 20 hours (depending on their prior coding experience) to mastering Scrapy,[23] an open-source Python package for web scraping. Finally, they spent about four additional hours developing Python code to collect forum posts from Healthboards.com. This example demonstrates that an individual with no previous programming experience might need less than 35 hours to acquire the necessary skills and an additional four hours to apply them to a specific project. (It is worth noting that collecting data via APIs is generally more straightforward and can be learned even more quickly.) 

When preparing to gather data through APIs or web scraping, it is essential to consider the legal, practical, and ethical considerations. Two points are of particular interest in the context of web scraping. First, if a platform lacks an API, it may be because the owners prefer to restrict data downloading. Therefore, reviewing and abiding by the platform’s Terms of Use before scraping is crucial (but see Case Study: Reverse-Engineering Censorship in China section). Second, researchers must ensure that their activities do not overload the platform with excessive download requests and thus negatively impact the platform’s functioning. For instance, Landers et al. (2017) deliberately designed their web-scraping script to pause for two seconds after capturing each post to avoid disrupting or slowing down the website. 

Creating Your Apps 

Some researchers set out to create apps and platforms to collect research data. A few successful projects of this kind exist, including MyBPLab (Gordon & Mendes, 2021), trackyourhappiness.org (Killingsworth & Gilbert, 2010), and myPersonality.org (Stillwell & Kosinski, 2015). However, creating a new app is a risky endeavor. Many researchers invested considerable talent, time, and money into such projects, but many failed to attract more than a few hundred participants. This is because academic projects must compete for users’ attention with highly engaging apps developed by startups and tech companies supported with huge budgets, professional software development teams, and nimble marketing. Moreover, developing software for deployment on users’ computers or smartphones is considerably more complex than writing code for data analysis. Apps need to meet stringent security standards to avoid errors or data leakage. Also, they need to be regularly updated to remain compatible with the rapidly evolving operating systems, browsers, and platform policies. Thus, a team of well-paid expert software engineers may be necessary not only to build but also to maintain an app.  

When designing apps to collect data, it is crucial to plan how to keep users engaged and motivate them to share their data (Kosinski et al., 2015). Financial incentives, commonly used to recruit participants in conventional studies, become too expensive as the sample size grows. Moreover, although financial rewards may prompt participation, they do not necessarily encourage honest or natural responses. Instead, they can elicit deceitful or random responses, or attract professional participants or scammers creating fake profiles to extort compensation. An app that provides a gratifying experience or insightful feedback can better align the interests of participants and researchers. Accordingly, apps such as MyBPlab, trackyourhappiness.org, or myPersonality rewarded participants with detailed feedback on their results. Opting for feedback over monetary rewards not only made these projects affordable but also increased data validity. Dishonest participants had little to gain from participating in these studies. Their only reward was feedback, which would be invalid if they had faked their data or responses.  

Case Study: Mypersonality.Org 

In the summer of 2007, David Stillwell, now a professor of Computational Social Science at the University of Cambridge, had just finished his undergraduate studies and was preparing to embark on his PhD in Psychology at the University of Nottingham. Eager to start collecting data for his research but lacking a budget to recruit participants, Stillwell decided to combine his interest in research with a desire to learn how to code. He devoted the next few weeks to developing a simple web app enabling users to complete Lew Goldberg’s 100-item Five Factor Model of Personality questionnaire (1992) and, in return, receive feedback on their scores (see Figure 13). Participants were given the choice to opt in to donate their questionnaire responses and Facebook profile information for research. Stillwell called his app “myPersonality” (Stillwell & Kosinski, 2012). 

Stillwell shared myPersonality with his Facebook friends, hoping to collect a few dozen responses. To his astonishment, it went viral—the friends of his friends, and then their friends, began to participate. Soon, it started attracting tens of thousands of daily users captivated by the chance to receive feedback on their personality and discuss their results with their Facebook friends. Recognizing the app’s potential, Stillwell kept adding questionnaires, including an intelligence test and a Satisfaction with Life questionnaire (Diener et al., 1985). By the time he discontinued this project in 2012, it had attracted over six million participants, with more than 40% choosing to donate their data for research purposes. 

Figure 13: Mypersonality App on Facebook 

Stillwell’s successful data collection was not fueled by extensive funding—he had none—but by the compelling design of his app, which was inherently valuable and entertaining to its users. This approach not only saved him (and other scholars) research funds but also produced a sample that was more robust and diverse than those commonly found in social science studies. Many of the myPersonality participants might not have been motivated by financial incentives typically offered in psychological studies. Instead, they were drawn to the opportunity to gain insights from the feedback on their questionnaire results, and subsequently, they voluntarily donated their data to further scientific research. Moreover, given that their only motivation was to obtain feedback on their scores, they were likely to answer honestly and attentively, producing data of exceptional quality (Kosinski et al., 2015).  

Stillwell used only a tiny portion of the myPersonality database for his PhD research (Hagger-Johnson et al., 2011). However, wishing to honor the generosity of the millions of volunteers who had donated their data, he began sharing the anonymized data with fellow researchers. This enabled an extraordinary range of studies. As of today, a Google Scholar query for “myPersonality” yields over 1,700 peer-reviewed academic papers. These include studies on the relationship between language and depression (Schwartz et al., 2014), psychological aspects of music preferences (Nave et al., 2018), and the relationship between personality and environmental pathogen prevalence (Brown et al., 2023). 

Notably, although myPersonality proved to be a success, it was an exception rather than the rule. It was launched in the early days of Facebook when competition for user engagement was significantly lower. Also, illustrating the challenges related to app maintenance, it was discontinued in 2012 because Stillwell could not keep updating the app to stay abreast of Facebook’s changing requirements. 

Collaborating with Existing Apps 

Considering the effort and considerable risks associated with creating an app, partnering with an existing one is often more advantageous. Numerous apps capture highly ecologically valid digital footprints from thousands or millions of users. As evidenced by many studies mentioned in this chapter, app owners are frequently willing to share user data with academic researchers or collaborate on experimental studies. 

When exploring collaborative opportunities, several vital considerations stand out. First, established companies are often bureaucratic, closely guard their data, and may worry about the PR risks of publishing results based on their data. On the other hand, collaboration with startups tends to be smoother, as they are generally more flexible and stand to gain more from such partnerships, potentially leading to app improvements, increased credibility, or the opportunity to hire researchers from the project. Second, app owners often proceed slowly or change their minds mid-process. Thus, it may be prudent to expand the pool of potential data sources instead of relying on just one. Lastly, during negotiations, one should reject any terms limiting academic freedom. Although it is routine to share manuscripts with app owners before publication or to discuss whether they would like to be acknowledged as a data source, it is essential to maintain control over the publication and presentation of the research findings. 

Pre-Trained Models 

Most of the studies described in this chapter, and most psychology studies in general, involve collecting data, analyzing data to build a statistical model, and interpreting the model to obtain psychological insights. Recently, alternative methods have emerged that leverage models pre-trained on datasets that are either proprietary or massive enough to be virtually inaccessible to academic researchers. Such pre-trained models include the Google Trends platform discussed in the section “Diverse and Representative Populations,” large language models like GPT-4,[24] text-to-image models such as MidJourney,[25] or Google Search Autocomplete. These models enable researchers to skip data collection and model training, and focus instead on interpreting pre-trained models to produce insights (see Figure 14). 

Figure 14: Conventional Big Data Research Pipeline and the Alternative Approach Employing Pre-Trained Models 

Using pre-trained models allows researchers to leverage diverse and large datasets that could otherwise be outside their reach, saving time, money, and also energy, thus reducing the environmental impact of their research. Additionally, pre-trained models typically pose fewer privacy risks, as they do not involve direct interaction with individuals or their sensitive data. However, there are also significant drawbacks. The information about the data collection, preprocessing, and training processes is often incomplete, which can obscure biases and hinder models’ interpretation.  

Research on perceptions of public figures’ personalities well illustrates the potential of employing pre-trained models in psychological research. 

Case Study: Using GPT to Predict the Personality of Public Figures 

Traditionally, peoples’ perceptions of famous individuals’ personalities are gauged via surveys—panels of individuals would be asked to evaluate the personality of a figure such as Barack Obama. As the public sphere moves into the digital realm, people increasingly express their opinions and perceptions in blog posts, social media updates, Wikipedia entries, newspaper articles, and books. This transition opens an alternative approach to measuring people’s perceptions. 

Instead of surveying human judges, Xubo Cao and Michal Kosinski (2024a; 2024b) found that ChatGPT-4 can reproduce human judges’ perceptions of public figures’ personalities with surprising accuracy, ranging from r = .8 to .9. OpenAI pre-trained this model using a massive language sample extracted from billions of web pages, social media, English Wikipedia, YouTube videos, and thousands of books. Compiling such an extensive dataset would be beyond reach for most academic researchers and would have a massive environmental impact. The training of ChatGPT-4 reportedly cost $100 million and resulted in tens of thousands of tons of CO2 emissions (Knight, 2023).

THEORY- AND DATA-DRIVEN BIG DATA STUDIES  

Research methods are often divided into the following: theory- or data-driven. Theory-driven or deductive research is defined as the testing and refinement of theoretical propositions through the collection and analysis of data. It starts with a hypothesis or a theory, which is used to formulate specific predictions, and then designing studies to test these predictions empirically. In data-driven or inductive research, researchers collect and analyze datasets to identify patterns, correlations, or trends, which can then lead to the generation of new hypotheses or theories. 

Big Data studies tend to be classified as data-driven research. This is misleading, as Big Data is an excellent resource in the context of both theory- and data-driven research. Big Data can be employed to empirically test theoretical predictions (e.g., Youyou et al., 2017) and search for previously unknown patterns (e.g., Schwartz et al., 2013). Both theory- and data-driven research benefit from Big Data’s high diversity and ecological validity. Its size offers ample statistical power to validate theoretical predictions and discover subtle effects. Its retrospective character reduces response bias and enables the study of phenomena that would be difficult or unethical to simulate in laboratory settings.  

Moreover, theory- and data-driven research are intertwined in practice, and the boundaries between them are blurry. Theorizing is typically informed by formally collected data or intuitions derived from casual observation of others or oneself (e.g., through introspection). Sometimes, theories are inspired by unexpected patterns observed in data collected to validate other theories. Consider Pavlov’s classical conditioning. Pavlov was studying digestion in dogs, and his objective was to measure the amount of saliva produced in response to food. When analyzing the data, Pavlov noticed that the dogs began to salivate not only when they tasted food but also when they saw the lab technician who normally fed them or even when they heard their footsteps. Intrigued, Pavlov conducted a series of experiments where he paired food presentation with a neutral stimulus, such as a bell. Soon, the dogs began to salivate in response to the bell alone (Tarpy, 1975). As Big Data samples typically include a broad range of variables, they can facilitate discoveries of unexpected patterns and mechanisms that would be more difficult to spot in conventional studies.  

On the other hand, theory plays a crucial role in data-driven research, serving as a guiding framework that shapes and informs every stage of the research process. It helps formulate relevant research questions and provides direction for what to look for in the data. It informs the study’s design, including the selection of variables, the nature of the relationships to be explored, and the methodology to be employed. Theoretical frameworks provide a basis for interpreting patterns that may emerge in the data and for positioning such findings within the broader context of existing knowledge. The interplay between existing theories and new observations can lead to refining existing theories or developing new ones. Finally, novel data-driven hypotheses are typically validated using a theory-driven approach. In other words, both data (including Big Data) and theory play a pivotal role in both data-driven and theory-driven research. 

The following sections outline several examples of Big Data studies that clearly illustrate the intertwined character of data- and theory-driven research. 

Randomized Experiments 

Randomized experiments involve assigning participants randomly to control and experimental groups to control for confounding variables that could sway the results. Despite being regarded as the one of the best techniques for gathering data, the adoption of randomized experiments in psychological science has been limited due to their prohibitive costs and the difficulty of recruiting participants. Fortunately, these limitations are much less severe in the relatively easily modifiable digital environments populated with billions of people whose behaviors are, by default, scrupulously recorded. 

Academic researchers, industry professionals, and government organizations have embraced this emerging opportunity. Particularly active are technology firms deploying thousands of randomized experiments, often called A/B tests, to optimize website design, marketing materials, and user experiences. In a typical A/B test, the users are randomly assigned to be shown one of two variants (A or B) of a given feature or service. Their behavior is analyzed to see which variant led to a more favorable outcome, typically measured by metrics like click-through rates, conversion rates, or engagement levels. In 2020 alone, tech giants like Google, LinkedIn, and Microsoft conducted approximately 20,000 A/B tests (Kohavi et al., 2020). 

Industry-conducted randomized experiments often focus on maximizing profits or enhancing customer satisfaction, but some hold significant social and scientific value. A prime example is the randomized experiment conducted by Robert Bond from UC San Diego in collaboration with Facebook’s Data Science team (2012). 

Case Study: A 61-Million-Person Experiment in Social Influence and Political Mobilization 

Robert Bond and colleagues (2012) were interested in understanding the degree to which people influence each other online, focusing on voting in the national elections. On November 2, 2010, the day of the U.S. congressional elections, 61 million U.S. Facebook users were divided into three groups. The first group received a pro-voting message, a link to find local polling places, a clickable button reading “I Voted.” This message was accompanied by a counter of Facebook users who had already reported voting and six randomly selected profile pictures of their Facebook friends who had already clicked the “I Voted” button (see Figure 15). The second group received the same message but without the friend images. The third, control, group received no message. 

Figure 15: The Message Shown to Facebook Users in the First Group

Note: Figure reprinted from Bond et al. (2012).

The impact of these treatments was measured in three ways. The clicks on the “I Voted” button were used as a proxy for political self-expression. (Importantly, clicking the “I Voted” button does not prove that one had voted.) The clicks on the polling-location search link were used to capture people’s desire to seek information about the election and their intent to vote. Moreover, users’ data was cross-referenced with public voter records published after the election to measure the actual voting behavior.  

The results showed that including friends’ pictures increased the likelihood of voting by .4%. Although such an increase may seem small, it has enormous practical implications in the closely contested U.S. elections. For example, George Bush won Florida—and thus the presidential race—by 537 votes, or less than 0.01% of the state’s total. 

Moreover, the researchers were interested in how social influence spreads across the social network. Their results showed that the propensity to vote spread to participants’ Facebook friends. This contagion effect was positively correlated with the strength of the relationship between a given pair of friends, approximated by the frequency of their Facebook interactions. Those results suggest that strong ties are instrumental in spreading both online and real-world behaviors in human social networks.  

Case Study: Boosting Fine Collection 

Social scientists are also increasingly deploying randomized experiments in collaboration with government organizations. A notable example is the work of the United Kingdom Cabinet Office’s Behavioral Insights Team, also known as the “Nudge Unit.” As Laura Haynes and colleagues (2012) outlined, the Nudge Unit used randomized experiments to evaluate the effectiveness of various government policies. 

In one of their projects, the Nudge Unit explored the impact of sending text reminders to collect court fines. Individuals with outstanding fines were randomly divided into five groups. The control group received no text, while the other groups were sent either a standard reminder or one of three personalized messages, including the owed amount, the recipient’s name, or both. The results revealed that the reminders remarkably increased the payment rates, from 5% for those who received no message to 23% for those who received the standard message and 33% for those who received the message including their name. 

The studies led by Bond (2012) and Haynes (2012) highlight several benefits of conducting randomized experiments in real-world digital environments. First, both studies were cost-effective, straightforward to implement, and leveraged existing digital platforms; it would be virtually impossible to conduct sufficiently high-powered studies of such a nature using conventional methods. Second, they both benefited from the “Big” in Big Data. Only a vast sample can provide the statistical power necessary to detect a change of .4% in the probability of voting. Third, the samples were highly representative. The study of court fines repayment engaged the very individuals with pending fines. Thanks to Facebook’s massive user base, the study of voting behaviors included a fifth of the entire U.S. population. Additionally, although Facebook users tend to be younger, they still include a representative cross-section of U.S. society (Kosinski et al., 2015). Fourth, both studies boasted high ecological validity, recording actual voting and fines repayment. This contrasts with many conventional studies, which often employ undergraduate psychology students or professional test-takers hired on Prolific who are asked to imagine that they owe fines or vote in an election (Hanel & Vione, 2016; Landers & Behrend, 2015). Finally, the findings of these two studies have a high societal impact and are directly applicable. The Court Service integrated Haynes and colleagues’ (2012) results study to save £30 million and prevent 150,000 bailiff interventions annually. Similarly, Facebook modified its user interface based on Bond and colleagues’ (2012) findings, leading to an estimated increase in voter turnout by over 300,000. 

Case Study: Online Dating Study 

The randomized experiments led by Bond (2012) and Haynes (2012) share a common characteristic: They were carried out by researchers wielding complete control over the experimental setup. Many digital platforms, ranging from gaming sites to dating apps, are open to partnering with academic researchers, particularly when they perceive potential benefits to their products or broader societal contributions. Most of these platforms are experienced in a/B testing and equipped with the necessary technology. 

However, conducting randomized experiments in the digital wild does not always require collaboration with major players like Facebook or the UK Government. The key is to devise a method for randomly assigning participants to different experimental conditions within the constraints of the chosen digital platform. Take, for instance, a randomized experiment by Johan Egebark and colleagues (2021), which investigated the influence of attractiveness and education level in online dating. After obtaining permission from their institutional review board (IRB), the researchers created 12 fake profiles on a dating site, varying in gender (male or female), educational background (high school or university degree), and the attractiveness of the profile picture. They then identified all active heterosexual users on the site—1,550 men and 1,117 women—and randomly assigned each to one of the fictitious profiles of the opposite gender. The team sent an invitation for a real-life meeting and tracked the number of users who opened the message, the number of positive responses received, and their response time. (Users who responded were sent a polite message indicating that the profile owner had already found someone else.) The researchers recorded each participant’s profile photo, education level, area of residence, age, and gender. Human judges estimated the participants’ attractiveness based on their profile photos.[26]

Egebark and colleagues’ findings revealed a nuanced pattern: Although attractiveness was universally desired, men with higher education preferred women with less education, whereas highly educated women preferred equally educated men. Moreover, both genders preferred attractive others, regardless of their own attractiveness. 

Case Study: Reducing the Spread of COVID 

Online advertising platforms, like those offered by Facebook or Google, provide an alternative avenue for conducting randomized experiments without a formal collaboration with a given platform. These platforms frequently offer tools for a/B testing, including random assignment to control and experimental conditions.[27] Although they are intended for benchmarking and increasing the efficacy of advertising campaigns, they can also be used for scientific research. Moreover, even when random assignment is not available, it is possible to split participants quasi-randomly based on their demographics or location. For instance, one could split participants based on their birth years, assigning those born in odd years to the control condition and those born in even years to the experimental condition. The participants could also be split based on their zip codes, which can be randomly assigned to control and experimental conditions. 

The latter approach was employed by Breza and colleagues (2021) to evaluate the impact of different approaches to promoting behaviors reducing the spread of COVID-19. In this randomized experiment, U.S. zip codes were randomly split into experimental and control groups. Researchers used the Facebook Advertising Platform[28] to target users in the experimental zip codes with videos urging them to stay home during Thanksgiving and Christmas. Users located in the control zip codes received no messages. The campaign reached over 35 million individuals. Its effectiveness was assessed using COVID-19 infection data published by the state governments and data on Facebook users’ geographical mobility published by Facebook. The findings indicated that the campaign reduced the distance traveled and decreased COVID-19 infections by approximately 3.5%. 

The studies led by Egebark (2021) and Breza (2021) were conducted without privileged access to the digital platforms involved. Anyone can create an account on a dating platform and study its users’ behavior (after obtaining IRB approval). Anyone with a credit card and a modest budget can launch an online advertising campaign. Moreover, as demonstrated in the latter study, it is often unnecessary to gather individuals’ data directly. Breza and colleagues did not have access to personal mobility records or individual COVID-19 infection statuses, and they did not directly monitor who viewed their posts. Instead, they used aggregated data from external sources, such as Facebook and local government agencies. This approach of using aggregated and indirectly collected data proved sufficient for assessing the effectiveness of their interventions. Moreover, it greatly reduced the cost of the study and the privacy risks faced by the participants.  

Natural Experiments 

In many contexts, conducting randomized experiments is prohibitively difficult or impossible. Some research questions involve phenomena that are impossible or too costly to control, such as large-scale policy changes or natural disasters. In other cases, experimental manipulation is impossible due to ethical constraints. For instance, assigning participants to conditions that might cause them harm or distress would be unethical. Furthermore, randomized experiments are impractical in studying long-term effects or rare events, such as pandemics and other natural and man-made disasters. In these cases, researchers often turn to natural experiments, a type of observational study. Here, natural variations or external changes that affect some groups but not others serve as a proxy for experimental manipulation.  

Many advantages of Big Data discussed in this chapter render it particularly well suited to conducting natural experiments. Its retrospective nature means that researchers can study rare events that occurred in the past instead of waiting for them to happen again or trying to simulate them artificially in a controlled setting. Its longitudinal character enables researchers to study how the effects unfold over time. Also, its vast size facilitates the study of population-wide effects and the detection of subtle or rare effects that would be likely invisible in smaller samples. In other words, Big Data is a gold mine for researchers using natural experiments to study policy changes or human-made or natural disasters. Consider, for example, hurricanes. 

Case Study: Social Impacts of Hurricane Ike 

Annually, one or two hurricanes strike the Atlantic Coast of the United States, causing widespread destruction to property, communities, and individual lives. Hurricanes can change course suddenly, and their trajectories are notoriously unpredictable, making it impossible accurately to forecast their path. Whether a specific community situated within its potential path is affected or not can seem almost random. This unpredictability allows for studying the impact of hurricanes through natural experiments. 

Tuan Phan and Edoardo Airoldi (2015) capitalized on the quasi-random nature of hurricane impacts to explore how such events influence people’s social networks. Their study focused on Hurricane Ike, which struck in September 2008, causing extensive damage from Louisiana to Texas and then moving north through the Midwest before dissipating in Michigan. The researchers gathered Facebook data from about 65,000 students at five universities heavily impacted by the hurricane and compared it with data from students at 10 similar universities that Ike missed. Their analysis encompassed Facebook posts, messages, and the dynamics of the students’ Facebook friendship networks, both before and after the hurricane. The findings indicated that directly impacted students developed stronger social connections after the disaster than their unaffected counterparts. 

Crucially, the researchers did not have to continuously track social networking activities while waiting for a hurricane to strike. Because Big Data is continuously recorded, observations can be made retrospectively, simplifying the study of natural experiments. 

Longitudinal Studies 

The next advantage of observing behavior through the lens of Big Data relates to the ease of conducting longitudinal studies. Collecting longitudinal data in conventional studies is notably challenging (Menard, 2007). Tracking participants over the years demands ongoing funding and consistent effort from researchers. Participants tend to drop out due to issues such as loss of interest, relocation, or death. This can lead to a reduced sample size and biases in the results. Moreover, repeatedly using the same measures over long periods reduces their validity due to exposure effects. To circumvent these difficulties, some psychologists employ data tracked by governments or NGOs, like child mortality, unemployment, or economic output. However, those datasets are less likely to include variables relevant to psychology, such as social connections, opinions, or well-being. 

The advent of Big Data has revolutionized this landscape, offering new avenues for conducting longitudinal studies. With the ability to retrospectively record data, it is now more feasible to carry out backward-looking longitudinal studies using pre-existing data. Also, the scope of what can be studied has expanded remarkably. Social media platforms, online forums, and other digital platforms provide a rich repository of psychologically significant variables. These range from individuals’ social networking behaviors to traces of their thoughts, feelings, and opinions, presenting a diverse and comprehensive dataset. This new era of Big Data not only simplifies the logistics of longitudinal research but also enhances its breadth and depth, allowing for a more nuanced understanding of behavior and social dynamics over time. 

Case Study: Tracking Covid-19 Through Amazon Reviews 

Anosmia, or loss of smell, is one of the early symptoms of COVID-19, useful for differentiating it from other viral infections. As such, it could be used to detect and track current and past COVID-19 infections. The problem is that directly testing people’s sense of smell could be expensive and cumbersome. 

Nick Beauchamp (2022) suggested a clever alternative: He hypothesized that the loss of smell should affect people’s everyday experiences and thus leave traces in their digital footprints. He focused on the Amazon product reviews of Yankee Candles, a popular brand of scented candles and home fragrance products. He scraped 9,827 Amazon reviews from the four best-selling Yankee Candle scented products from September 2018 to December 2021. The data was aggregated by week to compute the percentage of reviews that mentioned “no smell” or “no scent.” The mean number of reviews per week was 57, including an average of 1.7 reviews mentioning the lack of smell. 

Additionally, he obtained the CDC’s weekly counts of new COVID-19 cases. The results (see Figure 16) showed that the number of COVID-19 cases in a given week positively correlated with the number of “no smell” reviews in the subsequent week. He calculated that each additional 1,000,000 COVID-19 cases per week was linked with a 2.5 percentage points increase in "no smell" reviews. Data sets that  

Beauchamp’s study elegantly illustrates several characteristics typical of Big Data studies. First, it demonstrates that large datasets offer sufficient statistical power to study highly rare events. For instance, although only about 3% of the reviewers noted the lack of scent, recording 250 reviews per month on average provided sufficient statistical power to detect the trend. Second, this study underscores the benefits of retrospectively collecting Big Data. The odds of the onset of a pandemic coinciding with an ongoing longitudinal study tracking changes in smell perception are relatively low. It is much more likely that one could identify a method to retrospectively extract traces of changing smell perception from one of the existing Big Data sets. Third, it highlights the cost-effectiveness of such research. Beauchamp recorded his data retrospectively using a simple script. Contrast this with the expense and complexity required to continuously test the sense of smell in thousands of participants over a prolonged period. Fourth, the study brings to light one of the challenges of gathering data in uncontrolled environments: data contamination. As reported by the author (2022), a social media post reporting on the preliminary results of his study went viral, leading to an influx of Yankee candle reviews explicitly referencing it, potentially confounding future research that might use similar data. Fifth, as in many other studies discussed in this chapter, the researchers leveraged data generated by people going about their daily lives without imposing additional burdens on the participants.

Figure 16: Weekly COVID Cases Versus Scented Product Reviews Mentioning the Lack of Smell (Beauchamp, 2022) 

USING BIG DATA TO BOOST THE PREDICTIVE VALIDITY OF PSYCHOLOGICAL SCIENCE 

Psychological science has historically been predominantly concerned with developing, testing, and refining theories explaining human behavior (Yarkoni & Westfall, 2017). Journal editors, reviewers, hiring committees, and funding agencies tend to reward theoretical contributions, directing jobs, funds, and journal space to scholars who excel at explaining behavior in elegant ways. Unfortunately, this sometimes happens without regard to whether these theories predict real-world behavior. 

The field’s preoccupation with explanation makes it easy to forget that outside of academia—in policymaking, clinical and occupational psychology, marketing, or management—the ability to make accurate behavior predictions tends to be much more critical. In fact, behavioral predictions can be exceptionally useful, even if the underlying causal mechanisms are poorly understood. For example, although the causal underpinnings of the placebo effect are poorly understood, it is widely leveraged by scientists, healthcare professionals, and policymakers to improve the quality of research, enhance treatment outcomes, and craft policy (Oken, 2008). Other examples of psychological phenomena that are widely and beneficially applied in practice despite their poorly understood causal underpinnings include confirmation bias (Nickerson, 1998), prospect theory (Tversky & Kahneman,1992), and personality theory (Roberts & Yoon, 2022). 

Prioritizing explanation at the expense of prediction has significant consequences for research practices and contributes to some of the field’s most pressing issues (Giner-Sorolla, 2024). Data incongruent with theory is commonly discarded (the file drawer problem; Rosenthal, 1979) or reanalyzed until it confirms a theory (p-hacking; Simmons et al., 2011), or theories are modified post hoc to fit the data (HARKing; Kerr, 1998). Such practices often lead to overfitting, or creating models that explain specific datasets well but fail to predict new observations accurately. As a result, many psychological theories cannot meaningfully predict future behavior, likely contributing to the difficulty in replicating their findings (Open Science Collaboration, 2015). The pitfalls of prioritizing explanation over predictions are epitomized by the famous quote from Daryl Bem, one of the most famous social psychologists: “if you looked at all my past experiments, they were always rhetorical devices. I gathered data to show how my point would be made. I used data as a point of persuasion, and I never really worried about, ‘Will this replicate or will this not?’” (Engber, 2017). 

Importantly, theories that do not predict behavior are not merely useless; they can be outright dangerous. Decisions made by governments, organizations, and individuals are often theory-driven; if these theories do not predict behavior, the consequences can be disastrously far-reaching. Perhaps the starkest example is provided by the tragedy of communism.  

Communist ideology heavily relied on the psychological theories of Karl Marx and other social scientists. These theories maintained that individuals would be happier and more productive working for communal welfare rather than personal gain (Marx, 2000). Drawing on Locke’s tabula rasa theory of the human mind, they postulated that interpersonal conflict, competitiveness, greed, and selfishness were not inherent to human nature and could be eliminated by removing socioeconomic divides and private property. These theories were not just academic talk. At the height of communism in the latter half of the 20th century, around a third of the world’s population lived in countries governed by communist regimes that derived their policies from these theories. 

Unfortunately, as history has tragically shown, Marx’s and others’ psychological theories did not predict real-life behavior well. The social and economic reforms derived from these theories failed to eradicate greed, selfishness, or competitiveness. Instead, they led to social upheaval, economic hardship, and environmental destruction. A tragic illustration of such policies is provided by collectivization, which transitioned private farms and businesses to collective or state ownership. The collectivization of agriculture in the Soviet Union led to famine that cost the lives of about 11 million people in 1932–33 (Rosefielde, 1996). Similar reform in Mao’s China claimed at least 45 million deaths (Dikötter, 2010). These lives were lost because well-meaning policymakers instilled reforms derived from theories that did not predict behavior. It is much better to admit that we do not know how something works than to advance untested theories. 

Big Data will not eradicate bad policies or famine (although it is increasingly employed in agricultural research to improve farms’ outputs; Kamilaris et al., 2017). However, it can help with conducting science that is better at both: explaining and predicting behavior. First, some conventional research methods are ill-suited to test whether a given theory predicts real-life behavior. For example, that an experimental manipulation alters participants’ behavior in a highly controlled experimental setting does not guarantee that the same would apply in real-world scenarios. Big Data studies, on the other hand, are typically conducted in naturalistic settings and involve highly representative samples, boosting their predictive and ecological validities.  

Second, traditional research methods are fine-tuned for testing predictions of well-defined hypotheses involving a few variables. As such, they are unlikely to discover mechanisms not hypothesized by the researchers designing the study. Serendipitous discoveries happen, such as Pavlov’s discovery of classical conditioning, but by accident rather than by design. As a result, many factors and causal links are missing from current theories, limiting both their predictive power and the understanding of the underlying phenomena. This problem is aggravated by small sample sizes typical of conventional research, leading to false negatives, false positives, and exaggerated effects. Conversely, Big Data sets are typically composed of many observations and variables, supporting the discovery of novel and unexpected mechanisms and correlations.   

Third, traditional psychological models are typically evaluated based on their fit to existing data, using metrics such as the R-squared coefficient or the Akaike information criterion (Akaike, 1974). This approach naturally favors models that most accurately explain the current dataset. In contrast, Big Data models are typically evaluated on their ability to predict new, unseen data. This is demonstrated through cross-validated prediction metrics and classification measures, such as the root mean square error (RMSE) or the area under the receiver operating characteristic curve (AUC; Hastie et al., 2009). This leads to models prioritizing accurate predictions of future behavior over fitting past data. 

Predictive Models Employing Big Data  

Big Data sets composed of many observations and variables combined with modern computational methods enable the development of complex models predicting human traits and behavior with uncanny accuracy. Studies have found that such models can predict the likelihood of criminal behavior from the court records (Kleinberg et al., 2018), pregnancy from purchasing patterns (Hill, 2012), personality from facial images (Kachur et al., 2020), substance use and sexual orientation from Facebook Likes (Kosinski et al., 2013b), age and gender from search logs (Bi et al., 2013), social relationships from smartphone data (Reinhardt et al., 2015), ethnicity and race from music listening habits (Krismayer et al., 2017), educational level from Wi-Fi traffic (H. Li et al., 2017), social class from the online social network structure (Filho et al., 2014), income from call logs (Aarthi et al., 2011), religious and political views from the list of apps installed on a smartphone (Seneviratne et al., 2014), occupation from geo-location data (Solomon et al., 2018), or health from the smartphone’s accelerometer data (Weiss & Lockhart, 2011). 

These models are so pervasive and embedded in our lives that their existence is easy to overlook. For example, social media platforms like Facebook, X, tiktok, or Instagram use algorithms to predict user interests and preferences to tailor their news feed and advertisements. The platforms use the records of a user’s past behaviors and interactions—such as likes, shares, comments, or the amount of time spent looking at a post—to estimate which pieces of available content a user will find most attractive (Anandhan et al., 2018). The accuracy of these predictions is so high that most people have trouble turning away from social media newsfeeds, leading to a growing problem with social media addiction (Qin et al., 2022). 

Similar algorithms are employed to predict behavior by e-commerce platforms, such as Amazon or eBay, to recommend products to users based on their search history and past purchases. Streaming services, including Netflix or Spotify, predict what movies, TV shows, or music users might enjoy. Search engines like Google and Bing predict the most relevant search results based on a user’s search query, search history, and past browsing behavior. Ride-sharing apps like Uber and Lyft predict where and when people will need rides and where they will likely go to determine the prices and reposition their idle drives. Email services like Gmail employ algorithms to predict emails that a user will likely find important, try to anticipate their responses, and filter spam by estimating the likelihood that a given email was written by a human. 

Predictive models derived from Big Data are often so complex that humans cannot easily interpret them. Even the most basic predictive models, like linear or logistic regression, become challenging to interpret when they include more than a handful of variables (Hastie et al., 2009). This issue is further aggravated in the context of much more complex models, such as artificial neural networks typically composed of hundreds of millions of neurons arranged in dozens of layers (Lecun et al., 2015). Furthermore, predictive models often employ variables that are non-interpretable for humans. Facial recognition algorithms, for example, convert facial images into hundreds or thousands of numerical variables (Q. Cao et al., 2018). These variables efficiently subsume the information about facial appearance, as evidenced by these algorithms’ ability to recognize faces with uncanny accuracy. As these variables are not neatly aligned with human-interpretable facial features such as skin tone, facial hair, or expression, understanding how these models work is challenging. 

Interpreting Black Box Models 

The low interpretability of Big Data predictive models has earned them the pejorative black box label. However, there are many ways in which black box models can be interpreted to help understand human behavior. Despite their complexity, it is typically possible to employ black box models to gain insights into the causal mechanisms driving the phenomena that a model is capturing. This is evident from science’s success in interpreting one notably intricate predictive model and the prototypical black box: the human brain. Despite its complexity, psychological science has been rather successful at explaining the mechanisms underlying its functioning. Interestingly, many methods that psychologists use to study the black box of the human brain, such as presenting participants with various stimuli or assessing their performance across varying contexts, can also be applied to decipher the workings of algorithmic black boxes. 

Research on the links between facial appearance and psychological traits provides a good illustration of the strategies that can be employed to interpret the functioning of black box models.  

Case Study: The Links Between Facial Appearance and Psychological Traits 

The human face might be the most accessible source of information about our demographics and psychological states: Gender, age, race, and emotions are prominently displayed on individuals’ faces and can be easily perceived by others. However, decades of research on human perceptions show that people can also judge others’ psychological traits—such as honesty, personality, intelligence, sexual orientation, political orientation, and violent tendencies—from their faces (Zebrowitz et al., 2002). Such judgments are not very accurate but are instant and spontaneous. Moreover, low human accuracy does not seem to constitute the limit of what is possible: Facial recognition algorithms trained on Big Data have outperformed humans at face-based judgments of personality (Pearson’s r = .25 vs. .12; Kachur et al., 2020; Segalin et al., 2017), political orientation (r = .38 vs. .09; Joo et al., 2015; Kosinski, 2021; Rasmussen et al., 2023), and sexual orientation (r = .45 vs. .10; D. Wang, 2022; Y. Wang & Kosinski, 2018).  

The results of studies employing humans and algorithms indicate that two types of black box predictive models—human brains and facial recognition algorithms—can predict psychological traits from facial images. What remains controversial is the extent to which such judgments are enabled by facial features (e.g., facial morphology and wrinkle patterns) versus other factors, such as self-presentation (e.g., makeup, facial hair style, or head orientation), facial expression, targets’ demographics (e.g., age, gender, and ethnicity), or properties of the facial image (e.g., resolution and sharpness). Researchers employed several methods to investigate which of these factors predict psychological traits.  

Examining the extreme cases

Black box models typically convert data, such as images, into non-interpretable variables. Although this benefits their accuracy, it also makes them non-interpretable. One approach to circumvent this issue is examining the cases that trigger extreme predictions. For example, Yilun Wang and Michal Kosinski (2018) studied the links between facial images and sexual orientation in a sample of 7,894 men and 6,882 women self-identifying as gay (50%) and straight (50%). They employed VGG2 (Q. Cao et al., 2018), a popular facial recognition algorithm, to extract 4,096 variables from facial images. Next, they showed that these variables sufficiently differed between gay and straight individuals to allow for accurate classification. A cross-validated logistic regression, controlling for age, achieved a classification accuracy of 71% for women and 81% for men, powerfully illustrating privacy risks posed by widely used facial recognition technology.[29]

To identify facial features distinguishing between gay and straight individuals, the authors selected 100 images classified as most likely to represent gay versus straight men and women. These facial images were aggregated to create composite images and average outlines, amplifying facial features that were most distinguishing, according to the classifier, between straight and gay individuals. The inspection of the results presented in Figure 17 suggests that the classifier utilized facial hair (shaved men were classified as more likely to be gay), glasses (men wearing them were classified as more likely to be gay), and facial expressions (smiling women were classified as more likely to be straight).  

Figure 17: Composite Faces and the Average Facial Outlines of Men and Women Classified as Most and Least Likely to Be Gay

Note: Each image was created by averaging 100 facial images. Figure reprinted from Y. Wang and Kosinski (2018).

Extracting interpretable variables

Another approach to interpreting black box models is to build simpler models employing several interpretable images. Such models are less accurate but offer a glimpse into the links between predictors and predicted variables and thus hint toward what signal may be employed by the black box model. Dawei Wang (2022), for example, replicated the results of Y. Wang and Kosinski (2018) using an independent sample and obtained similar classification accuracy: 70% for women and 66% for men. He extended the original research, attempting to identify interpretable facial features linked with sexual orientation.  

Dawei Wang used Face++ API[30] to extract 20 interpretable variables from facial images. These included perceived emotional expressions, such as happiness or sadness; head orientation (i.e., pitch, yaw, and roll); and the presence of glasses. The combined predictive power of these interpretable variables was below the accuracy of the black box model: 61% for women and 55% for men. However, it allowed the identification of several interpretable differences between the facial images of gay and straight individuals: Gay men and women, for example, were less likely to wear glasses. Although not guaranteed, it is likely that the non-interpretable variables employed by the black box model also encoded these features.  

Artificially altering features

A model’s functioning can also be approximated by altering the predictors and observing the change in the predicted outcomes. Tweaking a given variable while leaving other variables constant can allow isolating the effect of such a variable on the outcome of interest. (This is similar to traditional experimental manipulation.) For example, Michael Stirrat and David Perrret (2010) took 12 images of male faces and manipulated them to decrease and increase their facial width. Next, these images were shown to 285 human judges who were asked to assess the trustworthiness of the person depicted in the image. The results showed that the judges were likely to choose images with narrower faces as being more trustworthy. 

Controlling for Features

Although an individual predictor’s role in a complex model can be challenging to interpret, its relative importance can be deduced from the change in the model’s accuracy after a given predictor is excluded. Many studies showed that both humans and algorithms can predict others’ political orientation from their naturalistic facial images, such as those downloaded from their social media profiles (Kosinski, 2021). However, it was unclear whether such judgments are enabled by facial features or other factors, such as self-presentation, facial expression, targets’ demographics, background, or image properties.  

To address this limitation, Kosinski and colleagues (2024) photographed the faces of 591 participants in the lab. All images were taken with the same camera and on the same neutral background. Participants removed all jewelry and make-up, and shaved facial hair. Their hair was hidden. The images were cropped in a tight rectangle from the top of the forehead (the start of the hairline) to the bottom of the chin. Participants also completed a political orientation questionnaire. Participants’ images were shown to human raters and a facial recognition algorithm while controlling for age, gender, and ethnicity. The accuracy achieved by humans (r = .21) and the algorithm (r = .22) was comparable and relatively high. (in fact, such an effect is on par with how well job interviews predict job success or alcohol drives aggressiveness; Meyer et al., 2001.) The algorithm’s predictive accuracy was even higher (r = .31) when it could leverage information on participants’ age, gender, and ethnicity. By controlling for self-presentation, facial expression, demographics, background, and image properties, this study provided evidence suggesting that political orientation is linked with innate facial features.  

This paper included one more analysis involving excluding predictors. To examine which parts of a facial image were linked with political orientation, portions of the images were iteratively masked, and images were re-entered into the political orientation classifier. The results presented in Figure 18 reveal that the facial areas employed by the classifier included the philtrum, eyebrows and eyes, nasal bridge, and mouth corners.  

Figure 18: Heat Map Representing the Average Absolute Change in Predicted Political Orientation Score Resulting from Masking a Given Image Area

Note: The color scale ranges from transparent (no change) to dark green (a change of .09 standard deviation). Figure reprinted from Kosinski (2021).

Predictive accuracy

Finally, predictive accuracy alone can offer valuable insights. That variable a predicts variable B implies that a influences B, B influences a, and/or one or more latent variables C influence both a and B. Take, for example, the results indicating that both human judges and facial recognition algorithms can predict political orientation from facial images, even when controlling for self-presentation, facial expression, demographics, background, and image properties (Kosinski et al., 2024). These results indicate that facial appearance and political orientation are linked. The causal mechanisms underlying this link have yet to be understood, but there are only three possible explanations (see Figure 19).  

First, political orientation could affect facial appearance. For example, conservatives tend to be more self-disciplined and are thus healthier, consume less alcohol and tobacco, and have a better diet (Chan, 2019). These behavioral differences are known to be reflected in facial features, such as skin health or the amount and distribution of subcutaneous fat (Richmond et al., 2018). Second, facial appearance could affect political orientation. for example, people largely agree when judging others’ political orientation from faces (Todorov et al., 2015). Regardless of the accuracy of such judgments, people perceived as having a particular attribute are treated accordingly; internalize such attributions; and, over time, may engage in behaviors consistent with others’ perceptions (i.e., self-fulfilling prophecy effect; Slepian & Ames, 2016). Third, many factors likely shape both psychological traits and facial appearance. Twin studies, for example, have found that genes are responsible for over 50% of the variation in facial features (Richmond et al., 2018) and political orientation (Alford et al., 2005).  

Figure 19: Three Possible Causal Pathways Linking Facial Features and Political Orientation

Note: Figure reprinted from Kosinski et al. (2024).

Beyond Explanation 

Traditional scientific methods involve collecting data, which is then analyzed using statistical tools to gain insights into underlying phenomena and encode them into theories. However, given that human brains struggle to capture and represent the relationships between more than a handful of variables (Miller, 1956), psychological theories typically include just a few predictors and a single outcome. Unsurprisingly, such oversimplified theories are usually characterized by a limited ability to predict real-world behavior (Oberauer & Lewandowsky, 2019).  

Complex, data-driven models tend to decisively outperform classical theories when predicting human traits and behavior, indicating that they can capture underlying mechanisms with greater detail. Although we strive to interpret and extract insights from such models, we should remember that there exists some theoretical boundary of the complexity of a model that a human can comprehend (Simon, 1957). Thus, we may have to accept that many mechanisms that can be modeled using algorithms may be too complex for humans to understand. Like the brains of other animals, the human brain’s cognitive capacity is limited, also in terms of the complexity of mechanisms that it can comprehend. Although modern technology and education can greatly expand human comprehension abilities, they are, nonetheless, finite and bound by the speed of biological and cultural evolution.  

In other words, much of the future progress in social psychology and other sciences may be driven by increasingly powerful predictive models that may not be explainable to humans. Importantly, this does not mean that such models cannot be used for human benefit. In fact, for a long time now, that is precisely what has been happening. We do not fully understand the functioning of the neural networks in the human brain. Nevertheless, we have been applying these neural networks to make decisions and predict the future with spectacular success. 

Psychological Measurement 

Another application of black box models is in psychological measurement. In traditional settings, psychological attributes like personality, cognitive skills, well-being, or mood are typically gauged through self-report methods like tests and questionnaires. Extensive research and real-world applications have established the reliability, practicality, and accuracy of these well-constructed self-reports. They remain widely used in various settings, including recruitment processes, high-stakes educational evaluations, and clinical practices (Rust et al., 2020). 

Nevertheless, self-reports have many limitations (Rust et al., 2020). First, they are typically conducted in controlled environments like assessment centers or online platforms, which may not accurately reflect real-life situations. Second, these methods capture only a momentary glimpse of a participant’s state, potentially influenced by transient factors like the testing context, stress, or fatigue. Third, self-reports mainly capture explicit, conscious responses, making them susceptible to dishonesty and misrepresentation, especially when outcomes depend heavily on the results. This is evident when using platforms such as Mechanical Turk or Prolific, where participants may misrepresent their traits to qualify for more lucrative studies (Kan & Drummey, 2018). Misrepresentation can also arise from unconscious biases. Therefore, even the most well-established and validated self-report assessments are far from perfect predictors of practical, real-life outcomes, such as job performance, well-being, or physical activity. 

Big Data provides the means to overcome some of the limitations of self-reports, increasingly serving as both an alternative and a complement to traditional methods. for example, research has demonstrated that personality, intelligence, and attitudes such as political views and religiosity can be inferred from Facebook Likes (Kosinski et al., 2013b). Mood, life satisfaction, and personality can be inferred from social media posts (Schwartz et al., 2016; Park et al., 2014). Psychometric scores derived from Big Data can be estimated for many participants simultaneously while saving time and costs for both participants and researchers. They are also less prone to misrepresentation. Although it is relatively easy to modify responses on self-reported questionnaires, it is far more difficult to consistently misrepresent one’s behavior over an extended period, such as weeks, months, or even years.  

Additionally, as Big Data can be gathered retrospectively, respondents would need to anticipate the benefit or necessity of such misrepresentation well before an assessment takes place. For instance, an introverted job applicant could easily tailor their responses to appear more extroverted on a screening questionnaire. However, fabricating a digital footprint that convincingly depicts an extroverted lifestyle over a long period would be substantially more challenging. Likewise, although a sufficiently motivated and unprincipled individual might cheat on an aptitude test, it is considerably more challenging to consistently feign higher performance levels in real-life tasks that depend on a specific ability and do so over an extended period of time. 

As a result, psychometric scores derived from Big Data are characterized by relatively high quality (Park et al., 2014; Youyou et al., 2015). for instance, Big-Data-based personality scores show high external validity (i.e., ability to predict real-life outcomes and other psychological traits); high test-retest reliability (i.e., consistency between a participant’s scores recorded at different times); and high concurrent validity (i.e., high correlation between scores derived from different types of Big Data). Furthermore, personality scores derived from Big Data strongly correlate with traditional questionnaire-based scores (r=.56; Youyou et al., 2015). 

Predictive models are also used to operationalize psycho-demographic constructs in more robust ways. For example, psychologists have long argued that gender could be more accurately represented as a spectrum of masculinity–femininity rather than the male–female dichotomy. Although there are tools that can be used to measure individuals’ position on such a spectrum (e.g., Bem Sex Role Inventory; Bem, 1974), they are cumbersome and thus infrequently used. Consequently, researchers typically record gender as a binary variable, limiting the accuracy of their results and reinforcing the binary understanding of gender.  

Models predicting gender offer a chance to overcome this constraint. Predictive algorithms do not simply assign male or female labels; they return probabilistic scores reflecting an individual’s position on the masculinity–femininity spectrum. Such a score can—but does not have to be—dichotomized. Predictive models could also be used to offer a more nuanced measurement of other constructs traditionally represented as binary or categorical, such as ethnicity, political orientation, or religiosity (Kosinski et al., 2016; Preoţiuc-Pietro et al., 2017). 

Furthermore, Big Data models enable the assessment of psychological traits in scenarios where conventional methods are impractical or unfeasible. For example, it is unlikely that shoppers would complete a psychological survey upon entering a store, whether online or physical. however, psychological profiles are now routinely inferred from Big Data samples containing purchase histories or web-browsing logs to tailor product recommendations (Dhelim et al., 2020). Similarly, Big Data models are used by music and movie recommendation systems like Spotify and YouTube to predict user music preferences (Nave et al., 2018). Dating websites employ analogous algorithms for matchmaking (Tomita et al., 2022). Facebook leverages user profile data to infer their preferences and interests and serve them with fitting content and advertisements (Naumov et al., 2019). 

Finally, predictive models can be used to replace or complement human raters. In many studies, researchers employ human raters to gauge others’ traits or states. Raters may be shown a fragment of a political speech and asked to judge the authors’ political orientation or video recording of a social interaction to judge the participants’ emotions or attitudes toward each other. Predictive models are now used to replace human raters on tasks ranging from judging emotion and political orientation from language (e.g., Preoţiuc-Pietro et al., 2017; Yaden et al., 2017) to detecting deception from facial images (Delmas et al., 2024) and judging the personality of public figures (X. Cao & Kosinski, 2024b). 

Case Study: Birds of Feather Flock Together 

The strengths of psychological measures employing Big Data are elegantly demonstrated in a study by Wu Youyou and colleagues (2017), which focused on the significance of personality similarity in forging social relationships. Although traditional research has repeatedly found that friends (and romantic partners) are often similar in age, education, race, religion, attitudes, and intelligence, these studies typically failed to detect similarity in their personality traits. This is surprising as personality affects many important life choices, from choosing university majors and careers to picking sports, hobbies, and even holiday destinations. If people are surrounded by those with similar personalities at work, school, and the gym, how is it possible that they do not end up forging friendships and partnerships with the same people? 

Youyou and colleagues hypothesized that conventional studies’ failure to show personality similarities among friends and romantic partners might be due to the limitations of traditional personality questionnaires. In typical self-report items, such as “I am well organized,” respondents tend to answer by comparing themselves with a salient comparison group rather than in absolute terms. For example, a well-organized person with an even better-organized spouse might see themselves as not particularly organized. This phenomenon, known as the reference group effect, can obscure true personality similarities in self-report measures. To address this issue, the authors collected Facebook profile data from over 50,000 pairs of friends and romantic couples. They estimated their Big Five personality traits (openness to experience, conscientiousness, extroversion, agreeableness, and emotional stability) using traditional questionnaires and two predictive models based on Facebook Likes and language used in status updates. The findings presented in Figure 20 revealed that friends and romantic partners do indeed have similar personalities, provided these traits are gauged using their actual behaviors and not self-reports.  

This study underscores how measures based on Big Data not only bypass the biases inherent in traditional self-reports but also offer a convenient, unobtrusive, and cost-effective way to analyze large populations. Gathering self-reports from over 50,000 pairs of friends and romantic partners took years. However, estimating their personalities using Big Data took a mere few hours.  

Importantly, although measures based on Big Data have many advantage—including convenience, a non-intrusive nature, and affordability—they are also prone to misuse, as well illustrated by the Cambridge Analytica scandal discussed at the beginning of this chapter.  

Figure 20: Radar Charts Presenting the Similarity in the Big Five Personality Traits Between Romantic Partners and Between Friends

Note: Results are shown separately for analyses based on Facebook Likes, Facebook language use, the combination of these two, and self-report questionnaires. Figure reprinted from Youyou et al. (2017). 

Bias 

The impressive accuracy of predictive algorithms must not overshadow the potential for bias in their predictions. Algorithms are not to be blamed for this; the datasets they learn from have been typically generated by humans, who are subject to prejudices, limited self-awareness, and perspectives shaped by societal norms and individual motivations. for instance, a predictive model trained on a standard internet corpus learned to associate “woman” with arts- and care-related occupations while linking “man” with math and science professions (Caliskan et al., 2017). Similarly, an algorithm was found to correlate European American names with more favorable terms than African American ones, reflecting biases documented through the Implicit Association Test (Greenwald et al., 1998). Such instances highlight the roots of algorithmic bias. If these algorithms are employed in decision-making tools, such as content recommendation or search query autocompletion, they risk perpetuating—not diminishing—existing social disparities. 

As predictive modeling is applied to increasingly high-stakes decisions, such as determining bail eligibility for defendants (Kleinberg et al., 2018), it is crucial to actively mitigate the extent to which these algorithms perpetuate human biases. To this end, social scientists, engineers, and policymakers must recognize the risks associated with algorithmic bias in developing models and assessing AI systems’ effects on societal behavior once they are in widespread use. 

When considering black box models for practical applications, it is essential to have realistic expectations regarding accuracy and bias—neither perfect accuracy nor complete freedom from bias is attainable. Instead, these models should be compared to the current alternatives, including human decision-makers, traditional psychometric tests, and clinical assessments. Research as far back as the mid-20th century indicates that even simple algorithms can outperform humans in predicting outcomes and making decisions (Sletto & Meehl, 1955). Algorithms have only grown more precise and adaptable since. Excessive fearmongering (e.g., O’Neil, 2016), which hinders the advancement and integration of these algorithms into decision-making processes, carries a high cost for both individuals and society. This cost manifests in various forms: the fallout of preventable crimes, the societal and personal ramifications of unnecessary incarcerations, the adversity faced by those with undiagnosed illnesses, and the discontent of people pursuing ill-suited careers. Often, these burdens fall disproportionately on disadvantaged groups and individuals. A Big-Data algorithm capable of diagnosing—and soon also treating—psychological disorders may not seem critical to an affluent Western European living in a large city, privileged with easy access to affordable and high-quality mental health care. However, the great majority of humans are deprived of such access, and such an algorithm could prove to be lifesaving. 

PROTECTION OF HUMAN SUBJECTS 

Researchers conducting Big Data research must adhere to the same fundamental principles of respect for persons, beneficence, and justice governing all human subject research. However, there are several aspects in which Big Data studies tend to diverge from conventional research. First, Big Data is often generated in naturalistic environments that were not designed for research purposes and it is thus often collected without without an appropriate consent process. Second, the robust character of Big Data makes it challenging to de-identify participants. Third, the collection and use of Big Data are limited by privacy and consumer protection laws, which are evolving and differ between jurisdictions. Fourth, digital platforms where Big Data is generated are governed by their own terms of service, which add another layer of regulation to conducting studies and collecting data.  

This section discusses selected issues pertaining to human subject research employing Big Data. It focuses on the U.S. regulations for protecting human subjects, commonly referred to as the 2018 Common Rule, or Title 45, Part 46 of the Code of Federal Regulations (CFR; Protection of Human Subjects, 2018). the most recent regulations, their interpretations, and detailed guidelines can be checked on the website of the Office for Human Research Protections of the U.S. Department of Health and Human Services.[31] Importantly, research outside the U.S. falls under other regulatory regimes. For example, those employing data collected in the European Union should review their regulations, such as the General Data Protection Regulation (GDPR). Useful guidelines can also be found on the website[32] of the British Psychological Society (GDPR was retained in domestic law after Britain left the European Union).  

The starting point for the discussion in this section is provided by the definition of the “human subject” employed by the common rule (45 CFR 46.102(e)): 

“Human subject means a living individual about whom an investigator (whether professional or student) conducting research: 

(i) Obtains information or biospecimens through intervention or interaction with the individual, and uses, studies, or analyzes the information or biospecimens; or 

(ii) Obtains, uses, studies, analyzes, or generates identifiable private information or identifiable biospecimens.” 

Private Versus Public Information 

Big Data research often involves analyzing data generated for non-research purposes, such as product reviews on an e-commerce platform or geolocation logs obtained from a smartphone app. As it is often stripped of identifiable private information, its analysis may fall outside the regulatory definition of human subject research and not require an IRB review. Unfortunately, deciding whether a Big Data set contains identifiable and private information is often complex.  

How do we decide whether a study employs participants’ private information and thus constitutes human subject research? According to the Common Rule (2018), private information is defined as “information about behavior that occurs in a context in which an individual can reasonably expect that no observation or recording is taking place, and information that has been provided for specific purposes by an individual and that the individual can reasonably expect will not be made public (e.g., a medical record)” (45 CFR 46.102(e)). The first part of this definition cites a reasonable expectation that no observation or recording is taking place. Given that most digitally mediated behaviors are now recorded, it could be argued that having such an expectation while using online platforms and digital products would be unreasonable. The second part of the definition cites a reasonable expectation that information provided for a specific purpose will not be made public. When is it reasonable to expect that one’s information will remain private? Would it be reasonable to assume that one’s product reviews are private if they are only accessible to other registered users? What if anyone can register as a user and obtain access to such data? What if a given platform is intended for a particular group of individuals, such as accountants, football fans, or alcoholics? Would it be reasonable to expect that information shared on such platforms would not be available to outsiders, such as researchers? In other words, that data can be easily or freely accessed does not automatically make it public.  

When considering whether information recorded on a given platform is public, one should first consult its terms of service, which typically clarify whether the data is public and how it can be used. In the absence of terms of service or when it is unclear on this issue, one should consider whether the participation on the platform must be authorized by the community or a moderator. Platforms where anyone can participate, even if it requires creating an account or paying a fee, could be considered public. Finally, one should consider the explicit or implicit norms governing a given venue. The community norms posted on a publicly accessible discussion forum, for example, may openly stipulate that the community members expect their discussions to remain private. Such a document may not be binding in a legal sense. However, the principle of beneficence—one of the three core principles outlined in the Belmont Report (National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, 1979)—requires that the well-being of participants be placed at the forefront of any research project. The social norms of online dating provide an interesting example. Friends and coworkers regularly encounter each other on dating websites and are thus exposed to information about each other’s preferences, often in intimate detail. Yet, people commonly pretend not to know facts about each other that were gleaned from dating website profiles, thus implicitly treating this quasi-publicly available information as personal. 

Finally, the mere fact that information is public does not imply that its use should automatically be considered exempt from the requirement of IRB review. Some publicly available data may have been made public illegally or without an individual’s consent. Examples include leaked documents or data that was scraped from private forums. Additionally, per 45 CFR 46.104(d), if the disclosure of information—even if it is public—could “reasonably place the subjects at risk of criminal or civil liability or be damaging to the subjects’ financial standing, employability, or reputation,” then research must be reviewed and approved by the IRB.  

Identifiable Versus Non-Identifiable Information 

How do we decide whether a study employs participants’ identifiable information and thus constitutes human subject research? According to the Common Rule (2018), private information is defined as “information for which the identity of the subject is or may readily be ascertained by the investigator or associated with the information” (45 CFR 46.102(e)). At first glance, Big Data sets are typically de-identified or stripped of details, allowing identification of the participants. For example, a bank would typically carefully remove identifiable details—such as names, email addresses, and account numbers—before sharing the transaction data with the researchers. Moreover, even when data is collected directly by the researchers, they often avoid recording identifiable information or remove it immediately after data collection is completed.  

Yet, the challenge in the context of many de-identified Big Data sets is that—with some effort—it may be possible to re-identify some individuals. Many digital footprints—such as communication patterns, geolocation history, web browsing logs, lists of Facebook Likes, or social media posts—are virtually guaranteed to be unique. For example, de Montjoye and colleagues (2013) found that four random spatiotemporal data points could identify 95% of the participants in a sample of 1.5 million people. Moreover, individuals can be identified by combining their superficially non-identifiable characteristics, such as birth dates, genders, and ZIP codes. Research has shown that such information can identify nearly 90% of individuals in the U.S. when cross-referenced with data in publicly accessible databases (Sweeney, 1997). This issue is exacerbated by improving research practices in social sciences: as more scientists publicly share their datasets, there is a growing risk of someone attempting to re-identify one of them. 

Several methods were developed to prevent de-identification of the data. Those include k-anonymity (Sweeney, 2002), ℓ-diversity (Machanavajjhala et al., 2007), t-closeness (N. Li et al., 2006), and differential privacy (Dwork, 2006). however, even the most careful de-identification may not provide complete protection from de-identification (Sweeney et al., 2018).  

In other words, a motivated researcher could, with some effort, reidentify some of the individuals in most Big Data sets. Thus, when investigators and IRBs assess whether a given Big Data set should be considered de-identified, it may not be practical to demand perfection. Instead, it might be more pragmatic to ask how “readily” such a dataset could be re-identified and how much risk this presents to the privacy and welfare of affected individuals. Furthermore, exercising caution in handling, storing, and sharing data—regardless of its de-identified status—is essential. This is well illustrated by the case study conducted by Latanya Sweeney (2015).  

Case Study: Re-Identifying Hospital Patient Data 

In 2012, Latanya Sweeney, a professor at Harvard Kennedy School, paid $50 to purchase a copy of Washington State’s publicly accessible hospital dataset (Sweeney, 2015). The dataset encompassed all hospitalizations within the state throughout the year and included information on patient demographics, diagnoses, procedures, attending physicians, hospital names, a summary of charges, and payment details. Notably absent were patient names and addresses, with only ZIP codes provided. 

Sweeney found 81 news stories published in the same year in the local newspapers in Washington containing the term “hospitalized” and the identities of the hospitalized persons. About 40% of these stories (35) could be easily matched with a record in the hospital dataset (see Figure 21). The matches included famous politicians, athletes, and business figures. Some database entries contained sensitive details, such as information about substance use and sexually transmitted diseases.  

In essence, Sweeney demonstrated that highly sensitive data could be easily re-identified using publicly available information. Further hospitalization records could likely be re-identified using other data sources, such as prescription records, geolocation history, or credit card statements. Moreover, people with direct knowledge of the incident and the hospitalized individual’s identity (e.g., employers, neighbors, or acquaintances) could easily re-identify a record by simply looking it up in the database. Re-identification has become even easier today, with the increased availability of information on accidents and health-related matters on social media platforms. Additionally, technological advancement could make de-identification much quicker; instead of manually cross-referencing data with news articles or other databases, one could use ChatGPT-4 or another large language model to perform such operations automatically. 

Figure 21: Matching Publicly Available Medical Information to News Stories to Re-Identify Patients

Note: Figure reprinted from Sweeney (2015).

Data Protection 

The problem of re-identification is amplified by the fact that a data breach or improper use involving Big Data encompassing thousands or millions of individuals can have more severe consequences than compromising a smaller conventional dataset. Moreover, Big Data often contains more sensitive personal details than conventional data. Those may include information about individuals’ health, finances, sexuality, political and religious affiliations, or information posing potential reputational risks.  

Additionally, because Big Data can contain hundreds of thousands of entries, participants cannot easily review and redact sensitive information before sharing their data with researchers. Even if participants could manually review their data, they would likely underestimate the depth of information disclosed. Sharing one’s Facebook Likes may seem benign, but such data can be used to predict individuals’ intimate traits such as their personality, IQ, political preferences, religion, sexual orientation, and others (Kosinski et al., 2013b). Similarly, someone sharing their location history likely understands that it reveals places they visited and their movement patterns. however, this data may also reveal their social connections, occupation, political beliefs, and other intimate attributes (Hinds et al., 2022). These challenges not only threaten participant privacy but also create ethical and legal liabilities for researchers, who might unwittingly receive sensitive or legally protected information. 

As a result, it is of the utmost importance to employ state-of-the-art technology to protect Big Data. It should be encrypted in transit and storage and protected by technology that allows for remotely locking devices and destroying the data if the device is lost or stolen. Moreover, personal identifiers and IDs should be removed and destroyed as soon as they are no longer needed (Secretary’s Advisory Committee on Human Research Protections, 2015). 

Consent 

Informed consent has been the gold standard for data use and collection, and is one of the main foundations of ethical research practices. Unfortunately, it is inherently incompatible with some of the Big Data research practices (Froomkin, 2019). For example, one of the principles of informed consent is to fully and carefully explain the study goals to participants. However, one of the central promises of Big Data is to uncover unexpected correlations in data, implying that it is often impossible to precisely outline the study goals to the subjects and thus seek their informed consent in the traditional sense. In response, the Revised Common Rule (2018) introduced a novel form of “broad consent,” where human subjects can agree to broad and varied forms of data use and reuse.  

Another challenge is that Big Data is often collected by businesses or organizations without any informed consent process at all. Yet, if it contains identifiable private information or could readily be re-identified, its analysis constitutes human subject research and thus requires consent. In some cases, an IRB can waive the requirement for consent.  

Yet another challenge pertains to the vague boundary of participant consent. Participants may have consented to their data, such as their Facebook profile, being recorded. However, in digital environments, the boundaries between data belonging to different people are often blurred. A Facebook profile, for example, usually contains other people’s photos, as well as their comments or messages. The extent to which such information can be ethically studied without consent from all involved parties is often ambiguous. Can one use messages sent by participants to others? What about those that participants received from others? What if a participant re-posted somebody else’s message? 

The Cambridge Analytica scandal illustrates this dilemma well. As discussed earlier, Cambridge Analytica used the Facebook Likes of millions of Americans to infer their personality profiles and subsequently targeted them with personalized ads. The firm acquired people’s Likes by paying $4 to several hundred thousand individuals for their Facebook profile data. This data included not only the users’ Facebook Likes but also the names, Facebook identification numbers, and Likes of their friends, who were oblivious to the fact that their data had been collected and used. Was it appropriate for these people to share—and for Cambridge Analytica to collect—their friends’ lists of Likes? At that time, Facebook’s Terms of Service allowed its users to share their friends’ data with apps. However, as illustrated by the widespread public and regulatory backlash, this data collection effort clearly contravened the privacy norms and ran afoul of legal frameworks in the US, the UK, and several other countries (Levy, 2020). 

Terms of Service 

Another challenge pertains to the terms of service governing the operation of digital platforms and the collection and use of their data. Just because a research practice is allowed by the terms of service does not automatically render it legal or ethical. For example, Cambridge Analytica’s data collection was largely within Facebook’s Terms of Service. 

Conversely, research that breaches terms of service might nonetheless be appropriate and ethically sound. Platform administrators often claim that their terms of service safeguard their users. Yet, their motives primarily appear to be protecting their profits and reducing public scrutiny of their activities. As a result, in exceptional cases, an IRB can approve research employing data obtained in violation of a platform’s terms of service. Take, for instance, the notable study on Chinese censorship by King and colleagues (2014). 

Case Study: Reverse-Engineering Censorship in China 

Censorship in China has a long history, stretching from Emperor Qin’s burning of Confucian texts in the third century BCE to the Communist Party’s regulation of traditional broadcast media (Xu & Albert, 2017). Today, censorship extends to social media platforms that over 1.3 billion Chinese people use to express their opinions. This censorship operates on two levels. First, tens of thousands of censors within social media companies and government agencies scrutinize user posts for potential removal. Second, algorithmic content filters block posts automatically, and these posts are then evaluated by censors, who make the final decision on whether to allow a given post or not. 

Gary King and colleagues (2014) conducted three studies to better understand the inner workings of the Chinese social media censorship apparatus (see Figure 22). In the first study, they implemented software to rapidly download new posts before censors could review them. By observing which of these posts were later taken down, they could reverse-engineer the rules followed by the censors who manually review the posts. Second, they created a vast number of fake social media accounts across china and shared a range of posts intended to trigger the automated content filters. By observing which of these posts were later manually unblocked by the censors, they deduced the rules guiding decisions followed by the censors controlling the automatic content filtering. Third, to gain further insights, they started a fake social media site in china to obtain access to software, documentation, and procedures that Chinese social media companies use to censor their users’ posts. 

Their findings showed that criticism of the state, its leaders, and its policies was tolerated, provided it did not call for collective action. Posts supporting or opposing a protest and posts calling for a rally in support of a policy or a leader were removed. This suggested that Chinese censorship aimed to prevent any collective action, whether opposing, supporting, or unrelated to the government. 

Importantly, King and colleagues’ studies violated not only these social platforms’ terms of service but also Chinese law. However, Harvard’s IRB pre-approved them, recognizing their societal importance. 

Figure 22: The Chinese Censorship Decision-Making Process

Note: Figure reprinted from King et al. (2014).

CONCLUSION 

The transformative impact of Big Data on the field of social psychology is both profound and multifaceted. This chapter has highlighted the vast potential of Big Data to revolutionize our understanding of human behavior, providing unprecedented insights and opportunities for research. However, with these opportunities come significant challenges and responsibilities. 

Big data, with its sheer volume, velocity, variety, and veracity, offers an unparalleled resource for studying the human mind and behavior. It enables researchers to analyze large, diverse, and ecologically valid samples, overcoming many limitations of traditional research methods. Studies utilizing Big Data can capture the nuances of human behavior in real-world contexts, enhancing the robustness and replicability of research findings. Moreover, Big Data facilitates the examination of previously inaccessible populations and phenomena, contributing to a more inclusive and comprehensive understanding of social psychology. 

However, the use of Big Data also raises critical ethical, legal, and methodological issues. Privacy concerns are paramount, as the re-identification of individuals from ostensibly anonymized datasets remains a significant risk. Researchers must navigate the complexities of informed consent in a landscape where data is often collected without participants’ explicit awareness or agreement. The potential for algorithmic bias, reflecting and perpetuating existing social inequalities, necessitates rigorous scrutiny and mitigation efforts. Additionally, the reliance on data generated by digital platforms introduces challenges related to the dynamic nature of these environments and the influence of corporate interests. 

To harness the full potential of Big Data while addressing these challenges, social psychologists must adopt a balanced approach. This involves embracing innovative methods and technologies while adhering to the highest ethical standards. It also requires fostering interdisciplinary collaboration, integrating insights from computer science, data ethics, and law to develop robust frameworks for responsible data use. 

In conclusion, Big Data presents a paradigm shift for social psychology, offering both remarkable opportunities and formidable challenges. By leveraging the power of Big Data, researchers can gain deeper insights into human behavior, contributing to the advancement of knowledge and the betterment of society. However, this must be done with a steadfast commitment to ethical principles, ensuring that the benefits of Big Data are realized without compromising the rights and well-being of individuals. The future of social psychology in the era of Big Data is bright, but it demands careful navigation, thoughtful reflection, and unwavering dedication to ethical research practices. 

REFERENCES 

Aarthi, S., Bharanidharan, S., Saravanan, M., & Anand, V. (2011). Predicting Customer Demographics in a Mobile Social Network. In International Conference on Advances in Social Networks Analysis and Mining, 553–554. https://doi.org/10.1109/ASONAM.2011.13

Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. https://doi.org/10.1109/TAC.1974.1100705

Alford, J. R., Funk, C. L., & Hibbing, J. R. (2005). Are Political Orientations Genetically Transmitted? The American Political Science Review, 99(2), 153–167. https://doi.org/10.1017/S0003055405051579

Althoff, T., Sosič, R., Hicks, J. L., King, A. C., Delp, S. L., & Leskovec, J. (2017). Large-scale physical activity data reveal worldwide activity inequality. Nature, 547(7663), 336–339. https://doi.org/10.1038/nature23018

Amoh, J., & Odame, K. M. (2015). DeepCough: A deep convolutional neural network in a wearable cough detection system. In Proceedings of the IEEE Biomedical Circuits and Systems Conference, 1–4. https://doi.org/10.1109/BioCAS.2015.7348395

Anandhan, A., Shuib, L., Ismail, M. A., & Mujtaba, G. (2018). Social Media Recommender Systems: Review and Open Research Issues. IEEE Access, 6, 15608–15628. https://doi.org/10.1109/ACCESS.2018.2810062

Ayers, J. W., Althouse, B. M., Leas, E., Dredze, M., & Allem, J. P. (2017). Internet Searches for Suicide Following the Release of 13 Reasons Why. JAMA Internal Medicine 177(10), 1527–1529. https://doi.org/10.1001/jamainternmed.2017.3333

Badoo (2016). Badoo Survey Examines Dating Behaviors Across the World. https://badoo.com/team/press/91

Barker, R. G., & Wright, H. F. (1951). One boy’s day. A specimen record of behavior. New York: Harper & Row.

Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements. Psychological Science in the Public Interest, 20(1), 1–68. https://doi.org/10.1177/1529100619832930

Beauchamp, N. (2022). “This Candle Has No Smell”: Detecting the Effect of COVID Anosmia on Amazon Reviews Using Bayesian Vector Autoregression. In Proceedings of the International AAAI Conference on Web and Social Media 16, 1363–1367. https://doi.org/10.1609/icwsm.v16i1.19388

Bem, S. L. (1974). The measurement of psychological androgyny. Journal of Consulting and Clinical Psychology 42(2), 155–162. https://dx.doi.org/10.1037/h0036215

Bi, B., Shokouhi, M., Kosinski, M., & Graepel, T. (2013). Inferring the demographics of search users: social data meets search queries. In Proceedings of the 22nd International Conference on World Wide Web. https://doi.org/10.1145/2488388.2488401

Blumenstock, J., Cadamuro, G., & On, R. (2015). Predicting poverty and wealth from mobile phone metadata. Science, 350(6264), 1073–1076. https://doi.org/10.1126/science.aac4420

Bond, R. M., Fariss, C. J., Jones, J. J., Kramer, A. D., Marlow, C., Settle, J. E., & Fowler, J. H. (2012). A 61-million-person experiment in social influence and political mobilization. Nature, 489(7415), 295–298. https://doi.org/10.1038/nature11421

Breza, E., Stanford, F. C., Alsan, M., Alsan, B., Banerjee, A., Chandrasekhar, A. G., Eichmeyer, S., Glushko, T., Goldsmith-Pinkham, P., Holland, K., Hoppe, E., Karnani, M., Liegl, S., Loisel, T., Ogbu-Nwobodo, L., Olken, B. A., Torres, C., Vautrey, P.-L., Warner, E. T., Wootton, S., & Duflo, E. (2021). Effects of a large-scale social media advertising campaign on holiday travel and COVID-19 infections: a cluster randomized controlled trial. Nature medicine, 27(9), 1622–1628. https://doi.org/10.1038/s41591-021-01487-3

Brick, C., Botzet, L., Costello, C., Batruch, A., Arslan, R. C., Struhl, M. K., Sommet, N., Green, J., Nuijten, M. B., Conley, M. A., Richardson, T., Sorhagen, N., Olsson-Collentine, A., Feldman, G., Feingold, F., Manley, H., Mullarkey, M. C., & Dienlin, T. (2024). Directory of free, open psychological datasets. https://doi.org/10.17605/OSF.IO/TH8EW

Brown, G., Walasek, L., Mullett, T., Quispe-Torreblanca, E., Fincher, C., Kosinski, M., & Stillwell, D. J. (2023). Political Attitudes and Disease Threat: Regional Pathogen Stress Is Associated With Conservative Ideology Only for Older Individuals. Personality and Social Psychology Bulletin. https://doi.org/10.1177/01461672231183199

Button, K., Ioannidis, J., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience 14, 365–376. https://doi.org/10.1038/nrn3475

Cafiero, F., Cordonier, L., & Bronner, G. (2020). Trendy plots: A Google Trends-based inquiry on the social determinants of conspiracy theories. In Proceedings of the International Conference on Computational Social Science (IC2S2).

Caliskan, A., Bryson, J. J., & Narayanan, A. (2017). Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334), 183–186. https://doi.org/10.1126/science.aal4230

Cambridge Analytica. (2016, November 9). Cambridge Analytica Congratulates President-elect Donald Trump and Vice President-elect Mike Pence [Press release]. https://www.prnewswire.com/news-releases/cambridge-analytica-congratulates-president-elect-donald-trump-and-vice-president-elect-mike-pence-300359987.html

Cao, Q., Shen, L., Xie, W., Parkhi, O. M., & Zisserman, A. (2018). VGGFace2: A Dataset for Recognising Faces across Pose and Age. IEEE International Conference on Automatic Face & Gesture Recognition, 67–74. https://doi.org/10.1109/FG.2018.00020

Cao, X., & Kosinski, M. (2024a). Large language models know how the personality of public figures is perceived by the general public. Scientific Reports, 14(1), 6735. https://doi.org/10.1038/s41598-024-57271-z

Cao, X., & Kosinski, M. (2024b). Large language models and humans converge in judging public figures’ personalities. PNAS Nexus, 3(10). https://doi.org/10.1093/pnasnexus/pgae418

Carré, J. M., & McCormick, C. M. (2008). In your face: facial metrics predict aggressive behaviour in the laboratory and in varsity and professional hockey players. Proceedings of the Royal Society B: Biological Sciences, 275(1651), 2651–2656. https://doi.org/10.1098/rspb.2008.0873

Chan, E. Y. (2019). Political orientation and physical health: The role of personal responsibility. Personality and Individual Differences, 141, 117–122. https://doi.org/10.1016/j.paid.2019.01.005

Chancellor, S., Nitzburg, G., Hu, A., Zampieri, F., & De Choudhury, M. (2019). Discovering Alternative Treatments for Opioid Use Recovery Using Social Media. In Proceedings of the Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300354

Chen, E. E., & Wojcik, S. P. (2016). A practical guide to Big Data research in psychology. Psychological Methods, 21(4), 458. https://doi.org/10.1037/met0000111

Christensen, H. T. (1947). Student Views on Mate Selection. Marriage & Family Living, 9, 85–88. https://doi.org/10.2307/347505

Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.

Cowen, A. S., Keltner, D., Schroff, F., Jou B., Adam, H., & Prasad, G. (2021). Sixteen facial expressions occur in similar contexts worldwide. Nature 589, 251–257. https://doi.org/10.1038/s41586-020-3037-7

Cranshaw, J., Schwartz, R., Hong, J., & Sadeh, N. (2021). The Livehoods Project: Utilizing Social Media to Understand the Dynamics of a City. In Proceedings of the International AAAI Conference on Web and Social Media, 6(1), 58–65. https://doi.org/10.1609/icwsm.v6i1.14278

Davies, H. (2015). Ted Cruz using firm that harvested data on millions of unwitting Facebook users. The Guardian. https://www.theguardian.com/us-news/2015/dec/11/senator-ted-cruz-president-campaign-facebook-user-data

Danescu-Niculescu-Mizil, C., Lee, L., Pang, B., & Kleinberg, J. (2012). Echoes of power: Language effects and power differences in social interaction. In Proceedings of the 21st International Conference on World Wide Web. 699–708. https://doi.org/10.1145/2187836.2187931

de Montjoye, Y. A., Hidalgo, C., Verleysen, M., & Blondel V. D. (2013). Unique in the Crowd: The privacy bounds of human mobility. Sci Rep 3, 1376. https://doi.org/10.1038/srep01376

Delmas, H., Denault, V., Burgoon, J. K., & Dunbar, N. E. (2024). A Review of Automatic Lie Detection from Facial Features. Journal of Nonverbal Behavior, 1–44. https://doi.org/10.1007/s10919-024-00451-2

Dhelim, S., Ning, H., Aung, N., Huang, R., & Ma, J. (2020). Personality-Aware Product Recommendation System Based on User Interests Mining and Metapath Discovery. IEEE Transactions on Computational Social Systems, 8(1), 86–98. https://doi.org/10.1109/TCSS.2020.3037040

Diener, E. D., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The Satisfaction With Life Scale. Journal of Personality Assessment, 49(1), 71–75. https://doi.org/10.1207/s15327752jpa4901_13

Dikötter, F. (2010). Mao’s great famine: the history of China’s most devastating catastrophe, 1958-62. A&C Black.

Domingos, P. (2012). A few useful things to know about machine learning. Communications of the ACM, 55(10), 78. https://doi.org/10.1145/2347736.2347755

Dwork, C. (2006). Differential Privacy. International colloquium on automata, languages, and programming, 1–12. Berlin, Heidelberg: Springer Berlin Heidelberg. https://doi.org/10.1007/11787006_1

Eagle, N., Pentland, A., & Lazer, D. (2009). Inferring friendship network structure by using mobile phone data. Proceedings of the National Academy of Sciences, 106(36), 15274–15278. https://doi.org/10.1073/pnas.0900282106

Egebark, J., Ekström, M., Plug, E., & Van Praag, M. (2021). Brains or beauty? Causal evidence on the returns to education and attractiveness in the online dating market. Journal of Public Economics, 196, 104372. https://doi.org/10.1016/j.jpubeco.2021.104372

Eichstaedt, J. C., Schwartz, H. A., Kern, M. L., Park, G., Labarthe, D. R., Merchant, R. M., Jha, S., Agrawal, M., Dziurzynski, L. A., Sap, M., Weeg, C., Larson, E. E., Ungar, L. H., & Seligman, M. E. P. (2015). Psychological Language on Twitter Predicts County-Level Heart Disease mortality. Psychological Science, 26(2), 159–169. https://doi.org/10.1177/0956797614557867

Ellard-Gray, A., Jeffrey, N. K., Choubak, M., & Crann, S. E. (2015). Finding the Hidden Participant: Solutions for Recruiting Hidden, Hard-to-Reach, and Vulnerable Populations. International Journal of Qualitative Methods, 14(5), 1609406915621420. https://doi.org/10.1177/1609406915621420

Engber, D. (2017, June 7). Daryl Bem Proved ESP Is Real: Which means science is broken. Slate. https://slate.com/health-and-science/2017/06/daryl-bem-proved-esp-is-real-showed-science-is-broken.html

Filho, R. M., Borges, G. R., Almeida, J. M., & Pappa, G. L. (2014). Inferring User Social Class in Online Social Networks. In Proceedings of the 8th Workshop on Social Network Mining and Analysis, 1–5. https://doi.org/10.1145/2659480.2659502

Finkel, E. J., Eastwick, P. W., & Matthews, J. (2007). Speed‐dating as an invaluable tool for studying romantic attraction: A methodological primer. Personal Relationships, 14(1), 149–166. https://psycnet.apa.org/doi/10.1111/j.1475-6811.2006.00146.x

Finkel, E. J. (2024). Romantic relationships. In: D. T. Gilbert, S. T. Fiske, E. J. Finkel, & W. B. Mendes (Eds). Handbook of Social Psychology, 6th Ed. Situational Press.

Fourney, A., White, R. W., & Horvitz, E. (2015, April). Exploring Time-Dependent Concerns about Pregnancy and Childbirth from Search Logs. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 737–746. https://doi.org/10.1145/2702123.2702427

Fraley, R. C., & Marks, M. J. (2007). The null hypothesis significance-testing debate and its implications for personality research. In R. W. Robins, R. C. Fraley, & R. F. Krueger (Eds.), Handbook of Research Methods in Personality Psychology, 149–169. The Guilford Press.

Fraley, R. C., & Vazire, S. (2014). The N-Pact Factor: Evaluating the Quality of Empirical Journals with Respect to Sample Size and Statistical Power. PLOS ONE, 9(10), e109019. https://doi.org/10.1371/journal.pone.0109019

Freud, S. (1953). On aphasia; a critical study. International Universities Press. (Original work published in 1891)

Froomkin, A. M. (2019). Big Data: Destroyer of Informed Consent. Yale JL & Tech., 21, 27. https://ssrn.com/abstract=3405482

Funder, D. C., & Ozer, D. J. (2019). Evaluating Effect Size in Psychological Research: Sense and Nonsense. Advances in Methods and Practices in Psychological Science 2(2):156–168. https://doi.org/10.1177/2515245919847202

Gerlach, T. (2019). Ideal Partner Survey. https://assets.ctfassets.net/juauvlea4rbf/1kmtOU2RRXrAB9Jz1JRmwe/20ee3375a5ba9f2d31fcbf9fb5a2e541/191105_Ideal_partner_survey.pdf

Gignac, G. E., & Szodorai, E. T. (2016). Effect size guidelines for individual differences researchers. Personality and Individual Differences, 102, 74–78. https://doi.org/10.1016/j.paid.2016.06.069

Giner-Sorolla, R. (2024). Changing practices and priorities in social psychological research methods and reporting. Handbook of Social Psychology, 6th Ed. In: D. T. Gilbert, S. T. Fiske, E. J. Finkel, and W. B. Mendes (Eds). Situational Press.

Goldberg, L. R. (1992). The development of markers for the Big-Five factor structure. Psychological Assessment, 4, 26–42. https://psycnet.apa.org/doi/10.1037/1040-3590.4.1.26

González, M., Hidalgo, C. & Barabási, A. L. (2008) Understanding individual human mobility patterns. Nature 453, 779–782. https://doi.org/10.1038/nature06958

Gordon, A. M., & Mendes, W. B. (2021). A large-scale study of stress, emotions, and blood pressure in daily life using a digital platform. Proceedings of the National Academy of Sciences, 118(31), e2105573118. https://doi.org/10.1073/pnas.2105573118

Gosling, S. D., & Mason, W. (2015). Internet Research in Psychology. Annual Review of Psychology, 66, 877–902. https://doi.org/10.1146/annurev-psych-010814-015321

Gosling, S. D., Ko, S. J., Mannarelli, T., & Morris, M. E. (2002). A room with a cue: personality judgments based on offices and bedrooms. Journal of Personality and Social Psychology, 82(3), 379. https://doi.org/10.1037//0022-3514.82.3.379

Grassegger, H., & Krogerus, M. (2017). The data that turned the world upside down. Vice Motherboard, 28.

Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74(6), 1464–1480. https://doi.org/10.1037/0022-3514.74.6.1464

Guntuku, S. C., Sherman, G., Stokes, D. C., Agarwal, A. K., Seltzer, E., Merchant, R. M., & Ungar, L. H. (2020). Tracking Mental Health and Symptom Mentions on Twitter During COVID-19. Journal of General Internal Medicine, 35(9), 2798–2800. https://doi.org/10.1007/s11606-020-05988-8

Hagger-Johnson, G., Egan, V. & Stillwell, D. J. (2011). Are social networking profiles reliable indicators of sensational interests? Journal of Research in Personality. https://doi.org/10.1016/j.jrp.2010.11.013

Hao, T., Xing, G., & Zhou, G. (2013). iSleep: unobtrusive sleep quality monitoring using smartphones. In Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems (SenSys ‘13), 1–14. https://doi.org/10.1145/2517351.2517359

Hastie, T., Tibshirani, R., Friedman, J. H., & Friedman, J. H. (2009). The elements of statistical learning: data mining, inference, and prediction. New York: Springer.

Hanel, P. H. P., & Vione, K. C. (2016). Do Student Samples Provide an Accurate Estimate of the General Public? PLOS ONE, 11(12), Article e0168354. https://doi.org/10.1371/journal.pone.0168354

Haynes, L., Service, O., Goldacre, B., & Torgerson, D. (2012). Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials. SSRN. https://dx.doi.org/10.2139/ssrn.2131581

Hehman, E., Leitner, J. B., Deegan, M. P., & Gaertner, S. L. (2013). Facial Structure Is Indicative of Explicit Support for Prejudicial Beliefs. Psychological Science, 24(3), 289–296. https://doi.org/10.1177/0956797612451467

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). Most people are not WEIRD. Nature, 466(7302), 29. https://doi.org/10.1038/466029a

Hill, K. (2012, February 16). How Target Figured Out a Teen Girl Was Pregnant Before Her Father Did. Forbes. https://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/

Hinds, J., Brown, O., Smith, L. G. E., Piwek, L., Ellis, D. A., & Joinson, A. N. (2022). Integrating Insights About Human Movement Patterns From Digital Data Into Psychological Science. Current Directions in Psychological Science, 31(1), 88–95. https://doi.org/10.1177/09637214211042324

Hitsch, G. J., Hortaçsu, A., & Ariely, D. (2010). Matching and Sorting in Online Dating. American Economic Review, 100(1), 130–63. https://doi.org/10.1257/aer.100.1.130

Jones, N. M., Wojcik, S. P., Sweeting, J., & Silver, R. C. (2016). Tweeting negative emotion: An investigation of Twitter data in the aftermath of violence on college campuses. Psychological Methods, 21(4), 526–541. https://doi.org/10.1037/met0000099

Johnston, M. P. (2014). Secondary data analysis: A method of which the time has come. Qualitative and Quantitative Methods in Libraries, 3(3), 619–626.

Joo, J., Steen, F. F., & Zhu, S.-C. (2015). Automated Facial Trait Judgment and Election Outcome Prediction: Social Dimensions of Face. IEEE International Conference on Computer Vision, 3712–3720. https://doi.org/10.1109/ICCV.2015.423

Kan, I. P., & Drummey, A. B. (2018). Do imposters threaten data quality? An examination of worker misrepresentation and downstream consequences in Amazon’s Mechanical Turk workforce. Computers in Human Behavior, 83, 243–253. https://doi.org/10.1016/j.chb.2018.02.005

Kachur, A., Osin, E., Davydov, D., Shutilov, K., & Novokshonov, A. (2020). Assessing the Big Five personality traits using real-life static facial images. Scientific Reports 10, 8487. https://doi.org/10.1038/s41598-020-65358-6

Kamilaris, A., Kartakoullis, A., & Prenafeta-Boldú, F. X. (2017). A review on the practice of Big Data analysis in agriculture. Computers and Electronics in Agriculture, 143, 23–37. https://doi.org/10.1016/j.compag.2017.09.037

Kennedy, B., Ashokkumar, A., Boyd, R. L., & Dehghani, M. (2022). Text Analysis for Psychology: Methods, Principles, and Practices. In M. Dehghani & R. L. Boyd (Eds.), The Handbook of Language Analysis in Psychology, 3–62. The Guilford Press. https://doi.org/10.31234/osf.io/h2b8t

Kerr, N. L. (1998). HARKing: Hypothesizing After the Results are Known. Personality and Social Psychology Review, 2(3), 196–217. https://doi.org/10.1207/s15327957pspr0203_4

Killingsworth, M. A., & Gilbert, D. T. (2010). A Wandering Mind Is an Unhappy Mind. Science, 330(6006), 932–932. https://doi.org/10.1126/science.1192439

King, G., Pan, J., & Roberts, M. E. (2014). Reverse-engineering censorship in China: Randomized experimentation and participant observation. Science, 345(6199), 1251722. https://doi.org/10.1126/science.1251722

Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018). Human Decisions and Machine Predictions. The Quarterly Journal of Economics, 133(1), 237–293. https://dx.doi.org/10.1093/qje/qjx032

Knight, W. (2023). OpenAI’s CEO Says the Age of Giant AI Models Is Already Over. Wired. https://www.wired.com/story/openai-ceo-sam-altman-the-age-of-giant-ai-models-is-already-over/

Kohavi, R., Tang, D., Xu, Y., Hemkens, L. G., & Ioannidis, J. (2020). Online randomized controlled experiments at scale: lessons and extensions to medicine. Trials, 21(1), 1–9. https://doi.org/10.1186/s13063-020-4084-y

Kosinski, M. (2017). Facial Width-to-Height Ratio Does Not Predict Self-Reported Behavioral Tendencies. Psychological Science, 28(11), 1675–1682. https://doi.org/10.1177/0956797617716929

Kosinski, M. (2021). Facial recognition technology can expose political orientation from naturalistic facial images. Scientific Reports, 11(1), 100. https://doi.org/10.1038/s41598-020-79310-1

Kosinski, M., Bachrach, Y., Kohli, P., Stillwell, D. J., & Graepel, T. (2013a). Manifestations of User Personality in Website Choice and Behaviour on Online Social Networks. Machine Learning. https://doi.org/10.1007/s10994-013-5415-y

Kosinski, M., Khambatta, P., & Wang Y. (2024). Facial Recognition Technology and Human Raters Can Predict Political Orientation From Images of Expressionless Faces Even When Controlling for Demographics and Self-Presentation. American Psychologist. https://doi.org/10.1037/amp0001295

Kosinski, M., Matz, S. C., Gosling, S. D., Popov, V., & Stillwell, D. J. (2015). Facebook as a research tool for the social sciences: Opportunities, challenges, ethical considerations, and practical guidelines. American Psychologist, 70(6), 543. https://doi.org/10.1037/a0039210

Kosinski, M., Stillwell, D. J., & Graepel, T. (2013b). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences, 110(15), 5802-5805. https://doi.org/10.1073/pnas.121877211

Kosinski, M., Wang, Y., Lakkaraju, H., & Leskovec, J. (2016). Mining Big Data to Extract Patterns and Predict Real-Life Outcomes. Psychological Methods. https://doi.org/10.1037/met0000105

Krismayer, T., Schedl, M., Knees, P., & Rabiser, R. (2017). Prediction of user demographics from music listening habits. In Proceedings of the 15th International Workshop on Content-Based Multimedia Indexing, 1–7. https://doi.org/10.1007/s11042-018-5980-y

Landers, R. N., & Behrend, T. (2015). An Inconvenient Truth: Arbitrary Distinctions Between Organizational, Mechanical Turk, and Other Convenience Samples. Industrial and Organizational Psychology, 8(2), 142–164. https://doi.org/10.1017/iop.2015.13

Landers, R. N., Brusso, R. C., Cavanaugh, K. J., & Collmus, A. B. (2016). A primer on theory-driven web scraping: Automatic extraction of Big Data from the internet for use in psychological research. Psychological Methods, 21(4). https://doi.org/10.1037/met0000081

Lecun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539

Levy, S. (2020). Facebook: The inside story. Penguin; UK.

Li, N., Li, T., & Venkatasubramanian, S. (2006). t-Closeness: Privacy Beyond k-Anonymity and l-Diversity. In IEEE 23rd International Conference on Data Engineering, 106–115. https://doi.org/10.1109/ICDE.2007.367856

Li, H., Zhu, H., & Ma, D. (2017). Demographic Information Inference through Meta-Data Analysis of Wi-Fi Traffic. IEEE Transactions on Mobile Computing, 17(5), 1033–1047. https://doi.org/10.1109/TMC.2017.2753244

Saeed, J., & Abdulazeez, A. M. (2021). Facial Beauty Prediction and Analysis based on Deep Convolutional Neural Network: A Review. Journal of Soft Computing and Data Mining, 2(1), 1–12.

Louail, T., Lenormand, M., Cantu Ros, O., Picornell, M., Herranz, R., Frias-Martinez, E., Ramasco, J. J., & Barthelemy, M. (2014). From mobile phone data to the spatial structure of cities. Sci Rep 4, 5276. https://doi.org/10.1038/srep05276

Lu, X., Bengtsson, L., & Holme, P. (2012). Predictability of population displacement after the 2010 Haiti earthquake. Proceedings of the National Academy of Sciences, 109(29), 11576–11581. https://doi.org/10.1073/pnas.1203882109

Lu, H., Pan, W., Lane, N. D., Choudhury, T., & Campbell, A. T. (2009). SoundSense: Scalable sound sensing for people-centric applications on mobile phones. In Proceedings of the 7th International Conference on Mobile Systems, Applications, and Services, 165–178. https://doi.org/10.1145/1555816.1555834

Machanavajjhala, A., Kifer, D., Gehrke, J., & Venkitasubramaniam, M. (2007). l-diversity: Privacy beyond k-anonymity. ACM Transactions on Knowledge Discovery from Data (TKDD), 1(1), 3-es. https://doi.org/10.1109/ICDE.2006.1

Marshall, A (2019, July 16). Netflix Deletes 13 Reasons Why Suicide Scene. The New York Times. https://www.nytimes.com/2019/07/16/arts/television/netflix-deleted-13-reasons-why-suicide-scene.html

Marx, K. (1844). Introduction to A Contribution to the Critique of Hegel’s Philosophy of Right. Deutsch-Französische Jahrbücher, 7, 261–71.

Marx, K. (2000). Karl Marx: Selected Writings, 2nd edition, David McLellan (ed.), Oxford: Oxford University Press.

Matz, S. C., Gladstone, J. J., & Stillwell, D. J. (2016). Money Buys Happiness When Spending Fits Our Personality. Psychological Science, 27(5), 715–725. https://doi.org/10.1177/0956797616635200

Matz, S. C., Kosinski, M., Nave, G., & Stillwell, D. J. (2017). Psychological targeting as an effective approach to digital mass persuasion. Proceedings of the National Academy of Sciences, 114(48), 12714–12719. https://doi.org/10.1073/pnas.1710966114

Mehl, M. R., Eid, M., Wrzus, C., Harari, G. M., & Ebner-Priemer, U. (Eds.) (2023). Handbook of mobile sensing in psychology: Methods and applications. Guilford Press: New York, NY.

Menard, S. (Ed.). (2007). Handbook of longitudinal research: Design, measurement, and analysis. Elsevier.

Mestyán, M., Yasseri, T., & Kertész, J. (2013). Early Prediction of Movie Box Office Success Based on Wikipedia Activity Big Data. PLOS ONE, 8(8), e71226. https://doi.org/10.1371/journal.pone.0071226

Meyer, G. J., Finn, S. E., Eyde, L. D., Kay, G. G., Moreland, K. L., Dies, R. R., Eisman, E. J., Kubiszyn, T. W., & Reed, G. M. (2001). Psychological testing and psychological assessment: A review of evidence and issues. American Psychologist, 56(2), 128–165. https://psycnet.apa.org/doi/10.1037/0003-066X.56.2.128

Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2), 81. https://doi.org/10.1037/h0043158

Müller, S. R., Bayer, J. B., Ross, M. Q., Mount, J., Stachl, C., Harari, G. M., Chang, Y., & Le, H. T. K. (2022). Analyzing GPS Data for Psychological Research: A Tutorial. Advances in Methods and Practices in Psychological Science 5(2). https://doi.org/10.1177/25152459221082680

Murnane, E. L., Abdullah, S., Matthews, M., Kay, M., Kientz, J. A., Choudhury, T., Gay, G., & Cosley, D. (2016). Mobile Manifestations of Alertness: Connecting Biological Rhythms with Patterns of Smartphone App Use. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services, 465–477. https://doi.org/10.1145/2935334.2935383

Murphy, S. C. (2017). A Hands-On Guide to Conducting Psychological Research on Twitter. Social Psychological and Personality Science, 8 (4), 396–412. https://doi.org/10.1177/1948550617697178

Naumov, M., Mudigere, D., Shi, H.-J. M., Huang, J., Sundaraman, N., Park, J., Wang, X., Gupta, U., Wu, C.-J., Azzolini, A. G., Dzhulgakov, D., Mallevich, A., Cherniavskii, I., Lu, Y., Krishnamoorthi, R., Yu, A., Kondratenko, V., Pereira, S., Chen, X., Chen, W., Rao, V., Jia, B., Xiong, L., & Smelyanskiy, M. (2019). Deep Learning Recommendation Model for Personalization and Recommendation Systems. arXiv preprint arXiv:1906.00091. https://doi.org/10.48550/arXiv.1906.00091

National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. (1979). The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research. U.S. Department of Health and Human Services. https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/read-the-belmont-report/index.html

Nave, G., Minxha, J., Greenberg, D. M., Kosinski, M., Stillwell, D. J. & Rentfrow, J. (2018). Musical Preferences Predict Personality: Evidence From Active Listening and Facebook Likes. Psychological Science. https://doi.org/10.1177/0956797618761659

Nichols, A. L., & Maner, J. K. (2008). The Good-Subject Effect: Investigating Participant Demand Characteristics. The Journal of General Psychology, 135(2), 151–166. https://doi.org/10.3200/GENP.135.2.151-166

Nickerson, R. S. (1998). Confirmation Bias: A Ubiquitous Phenomenon in Many Guises. Review of General Psychology, 2(2), 175–220. https://doi.org/10.1037/1089-2680.2.2.175

Nix, A. (2016). The Power of Big Data and Psychographics. Concordia Summit. https://www.youtube.com/watch?v=n8Dd5aVXLCc

Nowak, M., & Eckles, D. (2012). U.S. Patent No. 8,825,764. Washington, DC: U.S. Patent and Trademark Office. https://patents.google.com/patent/US8825764B2/en

Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114

Oberauer, K. & Lewandowsky, S. (2019). Addressing the theory crisis in psychology. Psychonomic Bulletin & Review, 26(5), 1596–1618. https://doi.org/10.3758/s13423-019-01645-2

Oken, B. S. (2008). Placebo effects: clinical aspects and neurobiology. Brain, 131(11), 2812–2823. https://doi.org/10.1093/brain/awn116

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251). https://doi.org/10.1126/science.aac4716

O’Neil, C. (2016). Weapons of Math Destruction. Crown.

Park, G., Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Kosinski, M., Stillwell, D. J., Ungar, L. H., & Seligman, M. E. P. (2015). Automatic personality assessment through social media language. Journal of Personality and Social Psychology, 108(6), 934–952. https://doi.org/10.1037/pspp0000020

Pennebaker, J. W., Boyd, R. L., Booth, R. J., Ashokkumar, A., & Francis, M. E. (2022). Linguistic Inquiry and Word Count: LIWC-22. Pennebaker Conglomerates. https://www.liwc.app

Pew Research Center (2021, April 7). Internet/Broadband Fact Sheet. https://www.pewresearch.org/internet/fact-sheet/internet-broadband/

Phan, T. Q., & Airoldi, E. M. (2015). A natural experiment of social network formation and dynamics. Proceedings of the National Academy of Sciences, 112(21), 6595–6600. https://doi.org/10.1073/pnas.1404770112

Preoţiuc-Pietro, D., Liu, Y., Hopkins, D., & Ungar, L. (2017). Beyond Binary Labels: Political Ideology Prediction of Twitter Users. In Proceedings of the 55th annual meeting of the Association for Computational Linguistics 1, 729–740. https://doi.org/10.18653/v1/P17-1068

Protection of Human Subjects, 45 C.F.R. § 46 (2018). https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/index.html

Qin, Y., Omar, B., & Musetti, A. (2022). The addiction behavior of short-form video app TikTok: The information quality and system quality perspective. Frontiers in Psychology, 13, 932805. https://doi.org/10.3389/fpsyg.2022.932805

Raento, M., Oulasvirta, A., & Eagle, N. (2009). Smartphones: An Emerging Tool for Social Scientists. Sociological Methods & Research, 37(3), 426–454. https://doi.org/10.1177/0049124108330005

Rasmussen, S. H. R., Ludeke, S. G., & Klemmensen, R. (2023). Using deep learning to predict ideology from facial photographs: expressions, beauty, and extra-facial information. Scientific Reports, 13(1). https://doi.org/10.1038/s41598-023-31796-1

Reddy, S., Mun, M., Burke, J., Estrin, D., Hansen, M., & Srivastava, M. (2010). Using mobile phones to determine transportation modes. ACM Transactions on Sensor Networks (TOSN), 6(2), 1–27. https://doi.org/10.1145/1689239.1689243

Reece, A. G., & Danforth, C. M. (2017). Instagram photos reveal predictive markers of depression. EPJ Data Science 6, 15. https://doi.org/10.1140/epjds/s13688-017-0110-z

Reinhardt, D., Engelmann, F., Moerov, A., & Hollick, M. (2015). Show me your phone, I will tell you who your friends are: analyzing smartphone data to identify social relationships. In Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia, 75–83. https://doi.org/10.1145/2836041.2836048

Richard, F. D., Bond, C. F., & Stokes-Zoota, J. J. (2003). One Hundred Years of Social Psychology Quantitatively Described. Review of General Psychology, 7(4), 331–363. https://doi.org/10.1037/1089-2680.7.4.331

Richmond, S., Howe, L. J., Lewis, S., Stergiakouli, E., & Zhurov, A. (2018). Facial Genetics: A Brief Overview. Frontiers in Genetics, p. 9, 462. https://doi.org/10.3389/fgene.2018.00462

Ritter, R. S., Preston, J. L., & Hernandez, I. (2014). Happy Tweets: Christians Are Happier, More Socially Connected, and Less Analytical Than Atheists on Twitter. Social Psychological and Personality Science, 5(2), 243–249. https://doi.org/10.1177/1948550613492345

Roberts, B. W., & Yoon, H. J. (2022). Personality Psychology. Annual Review of Psychology, 73, 489–516. https://doi.org/10.1146/annurev-psych-020821-114927

Rosenfeld, M. J., Thomas, R. J., & Hausen, S. (2019). Disintermediating your friends: How online dating in the United States displaces other ways of meeting. Proceedings of the National Academy of Sciences, 116(36), 17753–17758. https://doi.org/10.1073/pnas.1908630116

Rosefielde, S. (1996). Stalinism in Post-Communist Perspective: New Evidence on Killings, Forced Labour and Economic Growth in the 1930s. Europe-Asia Studies. 48 (6): 959–987. https://doi.org/10.1080/09668139608412393

Rosenthal, R. (1967). Covert communication in the psychological experiment. Psychological Bulletin, 67(5), 356–367. https://doi.org/10.1037/h0024529

Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638

Rosenthal, R. (1990). How are we doing in soft psychology? American Psychologist, 45(6), 775–777. https://doi.org/10.1037/0003-066X.45.6.775

Rossi, A., Pappalardo, L., Cintia, P., Iaia, F. M., Fernàndez, J., & Medina, D. (2018). Effective injury forecasting in soccer with GPS training data and machine learning. PLOS ONE 13(7): e0201264. https://doi.org/10.1371/journal.pone.0201264

Rudder, C. (2009). How Your Race Affects The Messages You Get. OK Cupid Blog. https://www.gwern.net/docs/psychology/okcupid/howyourraceaffectsthemessagesyouget.html

Rust, J., Kosinski, M., & Stillwell, D. (2020). Modern Psychometrics: The Science of Psychological Assessment (4th ed.). Routledge. https://doi.org/10.4324/9781315637686

Saeb, S., Lattie, E. G., Schueller, S. M., Kording, K. P., Mohr, D. C. (2016). The relationship between mobile phone location sensor data and depressive symptom severity. PeerJ. 4:e2537. https://doi.org/10.7717/peerj.2537

Sassenberg, K, & Ditrich, L. (2019). Research in Social Psychology Changed Between 2011 and 2016: Larger Sample Sizes, More Self-Report Measures, and More Online Studies. Advances in Methods and Practices in Psychological Science 2(2):107–114. https://doi.org/10.1177/2515245919838781

Secretary’s Advisory Committee on Human Research Protections (2015). Attachment A: Human Subjects Research Implications of “Big Data” Studies. U.S. Department of Health and Human Services. https://www.hhs.gov/ohrp/sachrp-committee/recommendations/2015-april-24-attachment-a/index.html

Sedgwick, P., & Greenwood, N. (2015). Understanding the Hawthorne effect. Bmj, 351. https://doi.org/10.1136/bmj.h4672

Segalin, C., Celli, F., Polonio, L., Kosinski, M., Stillwell, D. J., Sebe, N., Cristani, M., & Lepri, B. (2017). What your Facebook Profile Picture Reveals about your Personality. In Proceedings of the 2017 ACM on Multimedia Conference, 460–468. https://doi.org/10.1145/3123266.3123331

Seneviratne, S., Seneviratne, A., Mohapatra, P., & Mahanti, A. (2014). Predicting user traits from a snapshot of apps installed on a smartphone. ACM SIGMOBILE Mobile Computing and Communications Review, 18(2), 1–8. https://dx.doi.org/10.1145/2636242.2636244

Sharabi, L. L., & Caughlin, J. P. (2019). Deception in online dating: Significance and implications for the first offline date. New Media & Society, 21(1), 229–247. https://doi.org/10.1177/1461444818792425

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632

Simon, H. A. (1957). Models of Man, New York: John Wiley.

Schwartz, H. A., Sap, M., Kern, M. L., Eichstaedt, J. C., Kapelner, A., Agrawal, M., Blanco, E., Dziurzynski, L., Park, G., Stillwell, D. J., Kosinski, M., Seligman, M.E.P., & Ungar, L. H. (2016). Predicting Individual Well-Being Through the Language of Social Media. Pacific Symposium on Biocomputing. https://doi.org/10.1142/9789814749411_0047

Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., Kosinski, M., Stillwell, D. J., Seligman, M. E. P., & Ungar, L. H. (2013). Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach. PLOS ONE. https://doi.org/10.1371/journal.pone.0073791

Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Park, G., Sap, M., Stillwell, D. J., Kosinski, M., & Ungar, L. H. (2014). Towards Assessing Changes in Degree of Depression through Facebook. In Proceedings of Conference of the Association for Computational Linguistics (ACL). https://doi.org/10.3115/v1/W14-3214

Slepian, M. L., & Ames, D. R. (2016). Internalized Impressions: The Link Between Apparent Facial Trustworthiness and Deceptive Behavior Is Mediated by Targets’ Expectations of How They Will Be Judged. Psychological Science, 27(2), 282–288. https://doi.org/10.1177/0956797615594897

Sletto, R. F., & Meehl, P. E. (1955). Clinical versus Statistical Prediction. American Sociological Review, 20(4), 482–483. https://doi.org/10.2307/2092752

Song, C., Qu, Z., Blumm, N., & Barabási, A. L. (2010). Limits of Predictability in Human Mobility. Science, 327(5968), 1018–1021. https://doi.org/10.1126/science.1177170

Solomon, A., Bar, A., Yanai, C., Shapira, B., & Rokach, L. (2018). Predict Demographic Information Using Word2vec on Spatial Trajectories. In Proceedings of the 26th Conference on User Modeling, Adaptation and Personalization, 331–339. https://doi.org/10.1145/3209219.3209224

Stephens-Davidowitz, S. (2017). Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are. Dey Street Books.

Stillwell, D. J., & Kosinski, M. (2015). myPersonality Project website. www.mypersonality.org

Stirrat, M., & Perrett, D. I. (2010). Valid Facial Cues to Cooperation and Trust: Male Facial Width and Trustworthiness. Psychological Science, 21(3), 349–354. https://doi.org/10.1177/0956797610362647

Sweeney, L. (1997). Weaving Technology and Policy Together to Maintain Confidentiality. Journal of Law, Medicine & Ethics, 25(2–3), 98–110. https://doi.org/10.1111/j.1748-720x.1997.tb01885.x

Sweeney, L. (2002). k-anonymity: A model for protecting privacy. International Journal of Uncertainty, Fuzziness and Knowledge-based Systems, 10(05), 557–570. https://doi.org/10.1142/S0218488502001648

Sweeney, L. (2015). Only You, Your Doctor, and Many Others May Know. Technology Science. 2015092903. https://techscience.org/a/2015092903/

Sweeney, L, von Loewenfeldt, M, & Perry, M. (2018). Saying it’s Anonymous Doesn’t Make It So: Re-identifications of “anonymized” law school data. Technology Science. 2018111301. https://techscience.org/a/2018111301/

Talbot, D. (2016, April 15). How Political Candidates Know If You’re Neurotic. MIT Technology Review. https://www.technologyreview.com/2016/04/15/160556/

Tarpy, R. M. (1975). Basic Principles of Learning. Glenview, IL: Scott, Foresman and Company, 15–17.

Todorov, A., Olivola, C. Y., Dotsch, R., & Mende-Siedlecki, P. (2015). Social Attributions from Faces: Determinants, Consequences, Accuracy, and Functional Significance. Annual Review of Psychology, 66(1), 519–545. https://doi.org/10.1146/annurev-psych-113011-143831

Tomita, Y., Togashi, R., & Moriwaki, D. (2022). Matching Theory-based Recommender Systems in Online Dating. In Proceedings of the 16th ACM Conference on Recommender Systems (pp. 538–541). https://doi.org/10.1145/3523227.3547406

Torous, J., Kiang, M. V., Lorme, J., & Onnela, J. P. (2016). New Tools for New Research in Psychiatry: A Scalable and Customizable Platform to Empower Data Driven Smartphone Research. JMIR mental health, 3(2), e5165. https://doi.org/10.2196/mental.5165

Tversky, A., & Kahneman, D. (1992). Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and Uncertainty, 5, 297–323. https://doi.org/10.1007/BF00122574

Van den Akker, O., Weston, S., Campbell, L., Chopik, B., Damian, R., Davis-Kean, P., Hall, A., Kosie, J., Kruse, E., Olsen, J., Ritchie, S., Valentine, K. D., Veer, A., & Bakker, M. (2021). Preregistration of secondary data analysis: A template and tutorial. Meta-psychology, 5, 2625.

Verhaeghen, P., & Salthouse, T. A. (1997). Meta-analyses of age-cognition relations in adulthood: estimates of linear and nonlinear age effects and structural models. Psychological Bulletin, 122(3), 231–249. https://doi.org/10.1037/0033-2909.122.3.231

Walch, O. J., Cochran, A., & Forger, D. B. (2016). A global quantification of "normal" sleep schedules using smartphone data. Science Advances, 2(5), e1501705. https://doi.org/10.1126/sciadv.1501705

Wang, D. (2022). Presentation in self-posted facial images can expose sexual orientation: Implications for research and privacy. Journal of Personality and Social Psychology, 122(5), 806–824. https://doi.org/10.1037/pspa0000294

Wang, Q., & Taylor, J. E. (2014). Quantifying Human Mobility Perturbation and Resilience in Hurricane Sandy. PLOS ONE, 9(11): e112608. https://doi.org/10.1371/journal.pone.0112608

Wang, Y., & Kosinski, M. (2018). Deep neural networks are more accurate than humans at detecting sexual orientation from facial images. Journal of Personality and Social Psychology (JPSP), 144(2), 246–257. https://doi.org/10.1037/pspa0000098

Weiss, G. M., & Lockhart, J. W. (2011). Identifying user traits by mining smartphone accelerometer data. In Proceedings of the Fifth International Workshop on Knowledge Discovery from Sensor Data, 61–69. https://doi.org/10.1145/2003653.2003660

Weston, S. J., Ritchie, S. J., Rohrer, J. M., & Przybylski, A. K. (2019). Recommendations for Increasing the Transparency of Analysis of Preexisting Data Sets. Advances in Methods and Practices in Psychological Science. 2(3):214–227. https://doi.org/10.1177/2515245919848684

Xu, B., & Albert, E. (2017). Media Censorship in China. Council on Foreign Relations. https://www.cfr.org/backgrounder/media-censorship-china

Yaden, D., Eichstaedt, J. C., Kern, M.L., Smith, L. Bufonne, A., Stillwell, D. J., Kosinski, M., Ungar, L., Seligman, M. E. P., Schwartz, H. A. (2017). The Language of Religious Affiliation: Social, Emotional, and Cognitive Difference. Social Psychological and Personality Science. https://doi.org/10.1177/1948550617711228

Yarkoni, T., & Westfall, J. (2017). Choosing Prediction Over Explanation in Psychology: Lessons from Machine Learning. Perspectives on Psychological Science, 12(6), 1100–1122. https://doi.org/10.1177/1745691617693393

Youyou, W., Kosinski, M., & Stillwell, D. J. (2015). Computer-based personality judgments are more accurate than those made by humans. Proceedings of the National Academy of Sciences, 112(4), 1036–1040. https://doi.org/10.1073/pnas.1418680112

Youyou, W., Stillwell, D. J., Schwartz, A. H., & Kosinski, M. (2017). Birds of a Feather Do Flock Together: Behavior-Based Personality-Assessment Method Reveals Personality Similarity Among Couples and Friends. Psychological Science, 28(3), 276–284. https://doi.org/10.1093/nq/s3-IX.218.176-b

Zattoni, F., Gül, M., Soligo, M., Morlacco, A., Motterle, G., Collavino, J., Barneschi, A. C., Moschini M., & Moro, F. D. (2021). The impact of COVID-19 pandemic on pornography habits: a global analysis of Google Trends. International Journal of Impotence Research, 33(8), 824–831. https://doi.org/10.1038/s41443-020-00380-w

Zebrowitz, L. A., Hall, J. A., Murphy, N. A., & Rhodes, G. (2002). Looking Smart and Looking Good: Facial Cues to Intelligence and their Origins. Personality and Social Psychology Bulletin. 28, 238–249. https://psycnet.apa.org/doi/10.1177/0146167202282009

Zhang, L. & Malife, C. (2021, October 22). Processing billions of events in real time at Twitter. X Blog. https://blog.twitter.com/engineering/en_us/topics/infrastructure/2021/processing-billions-of-events-in-real-time-at-twitter-

Zheng, Y., Liu, L., Wang, L., & Xie, X. (2008). Learning transportation mode from raw GPS data for geographic applications on the web. In Proceedings of the 17th International Conference on World Wide Web, 247–256. https://doi.org/10.1145/1367497.1367532

ENDNOTES

The Handbook of Social Psychology, 6th edition © 2025 by Situational Press is licensed under Creative Commons- Attribution-NonCommercial-NoDerivatives 4.0 International. This work may be copied and distributed only in unmodified form, for noncommercial purposes, and with attribution.

https://www.facebook.com/business/ads ↑

http://developers.google.com/maps ↑

http://www.openstreetmap.org ↑

 Those interested in using geolocation data in psychological studies may consider reading Sandrine Müller and colleagues’ excellent tutorial (2022). ↑

http://developers.google.com/location-context/activity-recognition ↑

E.g., http://platform.openai.com/docs/guides/speech-to-text ↑

E.g., http://openai.com/research/clip ↑

E.g., http://learn.microsoft.com/en-us/azure/ai-services/computer-vision/concept-face-recognition ↑

In 2024, Prolific.com advised compensating participants at $12 per hour and charged a service fee of 30% ↑

137,000 participants x 1 hour x 130% for platform fees x $12 per hour ↑

http://trends.google.com ↑

http://w.wiki/u3m ↑

http://osf.io/th8ew ↑

http://www.icpsr.umich.edu/web/pages/ICPSR ↑

http://www.kaggle.com/datasets ↑

http://www.data.gov ↑

http://data.europa.eu ↑

http://dataforgood.fb.com ↑

http://datasetsearch.research.google.com ↑

For example, a Python command “requests.get(https://maps.googleapis.com/maps/api/geocode/json?latlng=40.714224,-73.961452)” sends a request to the Google Maps Platform API to return the address at latitude 40.714224 and longitude -73.961452. ↑

http://developer.x.com ↑

http://www.codecademy.com ↑

http://rlanders.net/scrapytutorial.html ↑

openai.com/gpt-4 ↑

www.midjourney.com ↑

Note that facial attractiveness could be more accurately and more effectively approximated by facial recognition algorithms (Saeed & Abdulazeez, 2021). ↑

Udacity offers a free course on conducting online A/B tests: http://www.udacity.com/course/ab-testing--ud257 ↑

https://www.facebook.com/business/ads ↑

In this article, the accuracy was reported using the area under receiver operating characteristic curve coefficient (AUC). AUC represents the likelihood of a classifier being correct when presented with the facial images of two randomly selected individuals—one gay and one heterosexual. The AUC of 50% indicates that the classifier is correct only half of the time, which is no better than a random draw. The AUC of 100% indicates that the classifier is always correct. AUC is an equivalent of the Wilcoxon signed-ranks test coefficient, used widely in social sciences. ↑

https://www.faceplusplus.com ↑

www.hhs.gov/ohrp ↑

www.bps.org.uk/guidelines-documents-library ↑