Citation in APA format: Morewedge, C. K., Naushan, H. F., & Mullainathan, S. (2025). Decisions with algorithms. In D. T. Gilbert, S. T. Fiske, E. J. Finkel, & W. B. Mendes (Eds.), The handbook of social psychology (6th ed.). Situational Press. https://doi.org/10.70400/PQYE5163

Decisions With Algorithms 

Carey K. Morewedge, Boston University  

Haaya F. Naushan, World Bank 

Sendhil Mullainathan, University of Chicago 

Decision making has changed in the age of information. While it was once an individual pursuit or an endeavor shared by multiple people, it is increasingly becoming a collaboration between humans and algorithms (Frey & Osborne, 2017). Algorithms are rules or procedures for combining inputs, calculating outputs, or solving problems. A computer often executes algorithms, but any person or tool following a fixed step-by-step decision-making process can use an algorithm—a few examples include statistical models, actuarial tables, and calculators. Algorithmic contributions to human decision making can range from providing information relevant to a decision, such as notifying a person of an event or communication (e.g., “Lunch at noon” or “New message”), to recommending a set of high-value options (e.g., “Chipotle, Sweetgreen, Panera Bread”), to making decisions on their behalf (e.g., “Here are directions to the nearest Sweetgreen.”).  

The stakes for collaborating well are high. Collaborations with algorithms can dramatically improve human welfare—or go terribly wrong. In healthcare, algorithmic providers perform many tasks as well as or better than human healthcare providers do (e.g., interpreting scans, diagnosing illnesses, monitoring patients), saving billions of dollars and providing services to underserved populations lacking access to human medical professionals (Topol, 2019). But algorithmic providers can also unintentionally create and perpetuate human biases and structural inequities in healthcare (Obermeyer et al., 2019). These consequences are examples of algorithmic bias, cases in which algorithms are not purely neutral transformers of information but rather make predictions that privilege or penalize categories (e.g., social groups) in ways that were not intended by its designers.      

Understanding and improving collaborative decisions made by humans and algorithms requires a new approach to psychology. Algorithms are neither person nor situation. Behavioral science cannot simply use what is known about collaborative decisions with people or decision making across contexts to understand decisions made in collaboration with algorithms. An algorithm is not a person, but it has properties much like those of the human mind. An algorithm may perform the same decision processes as a human but in a different way (Agrawal et al., 2018; Frey & Osborne, 2017). Algorithms are faster and able to consider more complex information than humans can, but they are narrower in the data and objectives they consider. Algorithms do not think or interpret rules as people do (e.g., Mullainathan & Spiess, 2017; Kleinberg et al., 2018; Dietvorst & Bartels, 2022; Logg, 2022) but instead follow rules to their logical extreme, uninfluenced by social norms such as fairness. An algorithm is not a situation, but it has some properties of one: Algorithms are constructed and changed like situations (e.g., a room, store, organization, or culture) but are not settings in which people make decisions. And algorithms can be active participants in a decision-making process.   

The successes and failures of collaborations between algorithms and humans hinge on psychological considerations in their design, testing, adoption, and regulation (Morewedge et al., 2023). Psychologists can play a critical role without becoming computer scientists, but they do need to develop a basic understanding of the role of human judgment in algorithms to consider the psychology of collaborations between humans and algorithms. All algorithms, from simple decision rules to supervised machine learning, have the same core structure (Mullainathan & Spiess, 2017), which can be modeled without going much deeper than identifying the outcome they are designed to predict and the data they use to make the prediction. Humans decide what algorithms predict and the data they use, but humans and algorithms differ substantially in the way they make predictions and use data. These differences necessitate a psychology of the design, adoption, and use of algorithms. Understanding when and how this collaboration works (and fails) is not a purely intellectual exercise. Psychology can improve algorithm design, overcome barriers to their adoption, and inform the public about the costs and benefits of their use. It can untangle what happens when humans and algorithms collaborate and suggest how best to use algorithms to improve human decision making. Consider this chapter an early roadmap of the psychology of collaborative decision making with algorithms.  

UNASSISTED HUMAN DECISION MAKING 

Before delving into the decision processes that algorithms facilitate and replace, it helps first to understand how people make decisions without assistance. Below is a three-stage theoretical and descriptive model drawn from the cognitive and managerial sciences (Figure 1). The model assumes that people make decisions they expect will maximize their utility (e.g., maximize pleasure and minimize pain; for reviews, see Kahneman, 1999; Morewedge, 2015).  

In the first stage, people search for, learn about, and recall relevant information about attributes and categories of options. Even at this early stage, people consider a subset of all options available to them (Frederick et al., 2009; Greenberg & Spiller, 2016). In the second stage, people identify a smaller consideration set of high-value options, which they compare and evaluate with respect to their present goals and the decision context (Morris et al., 2021). In the third stage, people choose one or more of the options that they believe will maximize their utility. The process may begin anew if people do not find a suitable option (e.g., Howard & Sheth, 1969; Heckhausen & Gollwitzer, 1987; Hauser & Wernerfelt, 1990; c.f., Ernst & Paulus, 2005). Decision processes may differ when certain decision-making capacities are constrained. At the end of this section, we discuss how inaccuracy occurs and explain two categories of errors: bias and noise.  

Figure 1: A Simplified Three-Stage Model of Human Decision Making Without Assistance 

Note: First, people identify important attributes and categories of options. Second, people form a consideration set of 2-7 options that they then compare and evaluate with respect to their context and goals. Third, people choose the option with the greatest expected value.  

Three Stages of Decision Making  

People typically begin making decisions by searching their memory, their environment, or both. The goal of the first stage is to identify the categories of options and attributes most relevant to the particular decision and context (e.g., March vacations; Johnson & Raab, 2003). We define an option as a mental representation of an action an agent could perform––what a person believes is a choice alternative (Kalis et al., 2013). Models proposing a hierarchical structure suggest that abstract options are first identified (e.g., option categories) before the process moves to a more concrete level (e.g., discrete options). For example, a person may first consider whether to take a beach, mountain, or city vacation. Next, they might generate concrete high-value options within the preferred category (e.g., beach) or subcategory (e.g., Caribbean, Florida, Hawaii; Kalis et al., 2013). One way to model the path people take is a decision tree—a series of forks that one sequentially explores—using statistical models such as a Markov Decision Process (e.g., Callaway, Jain, et al., 2022; Nelle & Cushman, 2022).  

In stage 2, people form a consideration set of a limited number of options they think about more carefully. Early work suggests that the size of this set typically ranges from 2-7 (e.g., Hauser & Wernerfelt, 1990), depending on the expertise of the decision maker, the importance of the decision, the number of options, and the cognitive resources available (Alba & Hutchinson, 1987; Diehl, 2005; Callaway, van Opheusden, et al., 2022). People may include options in the consideration set that were past high-value experiences even if those experiences aren’t relevant to the current context (e.g., fantastic previous vacations that they cannot presently afford). The options are then evaluated by incorporating knowledge about the unique decision context (e.g., “Three connecting flights would be too difficult for young children.”; Morris et al., 2021). People ideally evaluate each option in the consideration set in terms of its attribute values (e.g., “Which of these options can I afford?” “Which will have the best weather?” “Which destinations have direct flights?”), which are then weighted by the importance of those attributes and combined into an aggregate score (von Winterfeldt & Edwards, 1986).  

In the third stage, people compare the aggregate values of options and choose the option with the highest expected value or the highest value on important attributes (e.g., cheapest, most convenient, highest rated; Heckhausen & Gollwitzer, 1987; Todd & Gigerenzer, 2000). The expected value of an option is its utility adjusted for its probability (Von Neumann & Morgenstern, 1944). If no option has a sufficiently high expected value, people may consider more options (Callaway, van Opheusden, et al., 2022; Nelle & Cushman, 2022). Relying on this process leads people to occasionally miss out on good options, but it is efficient (Simon, 1955). 

Bounded Rationality  

People do not always have the necessary information, cognitive resources, or motivation to engage in the deliberative decision processes described above, known as System 2 (Kahneman, 2002). If resources are constrained or a decision is unimportant, people may exhibit bounded rationality and rely on a more intuitive System 1 decision process (for reviews, see Todd & Gigerenzer, 2000; Kahneman, 2002; Shah & Oppenheimer, 2008; Milkman, Chugh, & Bazerman, 2009; Pearson et al., 2022). People may satisfice, choosing an option that is good enough, (i.e., the best option they can find given their resources and motivation; Schwartz et al., 2002; Simon, 1990).  

System 1 and System 2 are typically presented as metaphors for ways people think, arranged in a corrective process. Evidence suggests that people form an intuitive System 1 answer to every problem, which is monitored by a more deliberative System 2 that may accept, correct, or reject the answer generated by System 1 (for reviews, see Gilbert et al., 1988; Sloman, 1996; Stanovich & West, 2000; Kahneman, 2002; Evans, 2003; Morewedge & Kahneman, 2010; for critiques see, Keren & Schul, 2009; Kruglanski & Gigerenzer, 2011; Mercier & Sperber, 2011; Oppenheimer & Kelso, 2015; Melnikoff & Bargh, 2018). 

When decision processes are constrained, people may rely on heuristics, attentional strategies, and/or intuitive judgments to reduce the effort associated with deliberative decision making. Heuristics are simple decision rules that reduce the effort required for decision making (Todd & Gigerenzer, 2000). Heuristics may reduce effort by (1) reducing the number of attributes of each option considered, (2) reducing the difficulty associated with retrieving and storing the values of each attribute, (3) simplifying how attributes are weighted, (4) requiring less information integration, or (5) by reducing the number of options considered (Shah & Oppenheimer, 2008).  

For instance, people using a lexicographic heuristic (Fishburn, 1967) first identify the most important attribute that differentiates their choice options and then pick the option with the highest value on that attribute. If there is a tie, they would move to the second most important attribute and pick the option that is highest on that attribute, and so on, until an option is chosen. This heuristic reduces the number of attributes considered (i.e., possibly to one attribute), reduces the difficulty of storing attribute values (i.e., only one needs to be considered at a time), simplifies weighting (i.e., puts 100% of the weight on a single attribute), only requires comparison between options on a single attribute, and reduces consideration of all attributes with low values on the first attribute considered. 

Constrained decision makers may strategically attend to option attributes that are easiest to process, most important, or have been most rewarding in the past (for a review, see Pearson et al., 2022). When given a choice of either certain or risky options (e.g., gambles), people attend more to the certain option when they are presented as a choice between gains (e.g., keep $15 for sure or a 25% chance to keep $60), and attend more to the gamble when they are presented as a choice between losses (e.g., lose $45 for sure or a 75% chance to lose $60). This difference in attention to options increases with time pressure, which in turn increases framing effects: a tendency to prefer certain options to gambles when choices are described in terms of gains and a tendency to prefer gambles to certain options when the same choices are described in terms of losses (Kahneman & Tversky, 1984; Roberts et al., 2022).  

Broadly, intuitive judgments rely on processes in associative memory, such as associative coherence, processing fluency, and attribute substitution (Morewedge & Kahneman, 2010). Related concepts are linked in memory via associative links. Activating a concept by thoughts of or exposure to a stimulus spreads that activation to concepts, actions, states, and emotions with which it is associated. Thinking of or seeing a SNAKE, for instance, may evoke a constellation of related associations including thoughts of other snakes, fear, a flight response, other dangers, and influence judgments (e.g., snakes may seem more prevalent after than before the stimulus was presented). These linkages are important for predicting near future states of the world (e.g., what happens next), choice, problem-solving and reasoning, and the recognition of objects and scenes (Rowe, 1991; Bar, 2004; Ratcliff et al., 2016). Associative reasoning may lead people to favor intuitive but wrong answers, even for relatively simple decisions. For example, when people are told a bat and ball cost $1.10, the bat costs $1 more than the ball, and asked “How much does the ball cost?”, they are more likely to give an intuitive incorrect answer (10¢) than the non-intuitive correct answer (5¢; Frederick, 2005).       

When information is chronically or temporarily more accessible in memory, people may favor that information or use its processing fluency as an attribute on which to make their decision (Schwarz et al., 1991; Shah & Oppenheimer, 2007). Most people find it easier to call to mind information about Tokyo than Osaka and hence correctly decide that Tokyo is more populous. Most people also find it easier to call to mind information about Dallas than San Antonio and hence incorrectly decide that Dallas is more populous. It is pleasant to process stimuli easily, and the positive affect accompanying stimuli that are high in processing fluency increases their perceived value. People also prefer stimuli that are chronically more familiar or whose recognition has been facilitated via a prime (Zajonc, 1986; Reber et al., 1998; Montoya et al., 2017).  

Constrained decision makers sometimes solve difficult decisions by solving an entirely different problem. This attribute substitution occurs when difficult decisions (y) are replaced with easier decisions to which they are semantically related (y’), and the decision maker fails to catch the substitution (Kahneman & Frederick, 2002). When people are uncertain about how dangerous a thing is (e.g., nuclear power plants vs. coal power plants), for instance, they may rely on an affect heuristic. They may substitute their fear of snakes (y’) for their dangerousness (y). This is not always wrong, but it can create errors when more dangerous stimuli evoke less fear than less dangerous stimuli. Most people believe snakes are more dangerous than bathtubs, for instance, despite the fact that accidents in bathtubs cause more deaths per year (e.g., Slovic et al., 2007). 

Accuracy, Bias, and Noise  

Good judgment and decision making are high in correspondence or coherence or both (Hammond, 2000). Correspondence refers to the accuracy of real-world predictions and requires empirical observations to test (e.g., “It will rain tomorrow.”). Coherence means consistency with the principles of logic and probability or the axioms of Expected Utility Theory (e.g., “It is more likely to rain tomorrow in Las Vegas than in Las Vegas and Seattle.”; preferring a sure $1 to a 1% chance to win $99 and a 99% chance to win $0; e.g., Oppenheimer & Kelso, 2015). Errors in decision making tend to originate from two sources, bias and noise. Bias refers to errors that are systematic. Noise refers to errors that are random. If a bathroom scale always gives readings that are too high or too low, it is biased. If the bathroom scale gives different readings depending on where you stand, it is noisy (Kahneman et al., 2016). Bias and noise may independently contribute to errors in decision making, or noise may determine the amount of bias that can occur (e.g., Dunning et al., 1989; Lee & Morewedge, 2022).  

Biases can arise from applying the wrong strategies and rules to a decision (e.g., Lerner & Tetlock, 1999). They can occur when using the right rules but insufficiently correcting initial decisions one knows to be wrong (e.g., anchoring bias, superstition; Tversky & Kahneman, 1974; Epley & Gilovich, 2001; Risen, 2016; Lee & Morewedge, 2022). Or they can arise from anomalies in the psychophysical mapping of external stimuli to their mental representation (e.g., reference dependence, loss aversion, diminishing marginal utility, decision weights; Lichtenstein & Slovic, 1971; Kahneman & Tversky, 1979). Bias influences decisions when people have unrepresentative associations, memories, or explicit beliefs about the world or themselves (e.g., Stewart et al., 2006); when people neglect information that contradicts those beliefs (Nickerson, 1999); and when people fail to correct for internal and external differences in decision contexts and the contexts in which options are experienced (e.g., differences in decision utility and experienced utility; for reviews, see Arkes, 1991; Payne et al., 1993; Gilovich et al., 2002; Hsee & Hastie, 2006; Gilbert & Wilson, 2007; Weber & Johnson, 2009; Morewedge, 2015). 

Common sources of noise in decision making include unknowable and knowable uncertainty as well as errors in mental processes. Whether a gambler will roll a 1 or a 6, a couple will remain married or divorce, or checked luggage will arrive at its destination—each have a chance element that makes them unknowable (i.e., aleatory uncertainty). There are other unknowns that people could but do not know (i.e., epistemic uncertainty): People could know that the die is rigged, that the husband is planning to move by himself to Montana, or that a connecting airport often loses bags (Ülkümen et al., 2009). In addition, the mental processes involved in decision making are often noisy (Hilbert, 2012; Kahneman et al., 2021). Attention and memory and mental representations are imperfect, so not all stimuli in the environment are noticed and remembered (Miller, 1956; Treisman, 1969), and people are sometimes inconsistent in their answers and choices (Dehaene, 2003; Lee & Morewedge, 2022).     

COLLABORATIVE DECISION MAKING BETWEEN HUMANS AND ALGORITHMS 

Although people have relied on other people and technology to help them make decisions for millennia, decision making in the 21st century is typically done in collaboration with an algorithm. These collaborations take three forms: Algorithms can inform, make recommendations to, and decide on behalf of human decision makers. We assume that each form usually encompasses the forms preceding it. (Recommendation systems also inform; Algorithms that decide also inform and make a recommendation.) Google Scholar is an algorithm that performs tasks in all three ways. It provides basic bibliographic information (e.g., find all works authored by “Daniel Wegner”), and it sends notifications of new works by authors people follow (i.e., informing). Google Scholar auto-completes searches and ranks results, listing more relevant articles first and less relevant articles later (i.e., recommending). It collates duplicates of the same paper published on the Internet and tracks its influence (i.e., it decides which are versions of the same paper and how often the paper has been cited). In Table 1, we provide more examples where people collaborate with algorithms in personal and professional decisions.  

Table 1: Three Kinds of Decisions with Algorithms  

Informing  

At the most basic level, algorithms are facilitating and replacing human decision processes by virtue of their suprahuman ability to find, store, and analyze information (Ward, 2013; Rai, 2020). The speed and ease of using algorithms to find information often makes it faster and less effortful to retrieve information by querying a search engine than by retrieving it from one’s own memory (Ward, 2021). This encourages reliance on algorithms such as search engines as sources of information (Storm et al., 2017), particularly for people who are less inclined toward effortful thinking (Barr et al., 2015).  

People are cognitively offloading attention, knowledge, information, and memories to algorithms. Notification algorithms alert people to new information and communications and remind them of events and commitments. Algorithms are used to track behavior (e.g., steps taken, screen time, social media activity) by people for the personal use and at work by their employers (Raveendhran & Fast, 2021). People use algorithms as external memory sources in which to retain information (Marsh & Rajaram, 2019). Fewer than half of the adults in the United States have memorized more than five telephone numbers, and 21% report using search engines to find their own telephone number (Long, ​​​​2022, April 25). This offloading has led algorithms to become partners in human transactive memory systems (Ward, 2013). When people believe they will be able to access information on a computer or the Internet later, they are less likely to encode and be able to recall that information (Sparrow, Liu, & Wegner, 2011), creating problems when the ease of accessing information through algorithms leads people to conflate knowledge they can access with the knowledge they possess (Ward, 2021).  

Algorithms also create new information. Algorithms discover new rules and test large numbers of variables and interactions that would be too difficult for human experts to discover or test on their own. Algorithms can detect early-stage breast cancer in scans earlier than humans can (Yala et al., 2022), for instance, and provide novel input to physicians’ diagnoses and recommendations. Algorithms analyze and organize large volumes of data in ways that would be nearly impossible for humans to do. Take, for example, the Panama Papers: Algorithms performed preliminary scanning, compiling, and analyzing of the 11.5 million leaked financial documents (2.6 terabytes). Without this assistance, journalists would not have been able to conduct the hundreds of investigative reports on international corruption that were based on these data (Diakopoulos, 2019).  

Recommendations  

Algorithms are improving the efficiency and quality of human decision making through recommendation systems such as auto-complete, predictive text, and algorithmic curation. Recommendation systems tend to rely on content-based, collaborative, or hybrid filtering models. Content-based filtering uses the similarity between choice options to make recommendations, suggesting stimuli similar to those that were evaluated positively or were chosen in the past. Because content-based filtering is stimulus-focused, it is limited in its ability to recommend new categories of options that have not yet been rated or previously consumed. Collaborative filtering is the most popular type of recommendation system (Yang et al., 2014). It generates recommendations by leveraging inter-user or inter-item relationships and is person-focused. User-based collaborative filtering, for instance, forms groups of similar users and makes recommendations based on the behavior of the users in that group. It is limited when data are limited (e.g., when the algorithm is first launched, a person begins using it, or a new stimulus is introduced), or when there is little similarity between a user and other users (Da’u & Salim, 2020; Ko et al., 2022).  

Auto-complete and predictive text algorithms help humans write their queries, communications, and documents by recommending words and sentences while people are typing them. Algorithms recommend how to complete queries in search engines (Tahery & Farzi, 2020) and how to complete words, phrases, and sentences in word-processing documents, emails, and text messages based on what has already been typed by users or by people with whom they are communicating (Weiss et al., 2015). Less common applications now allow algorithms to recommend longer spans of text, such as consumer reviews or the opening paragraphs of business cases (e.g., ChatGPT; Dale, 2021).  

Algorithmic curation determines which advertisements, media, information, jobs, goods, and services people see based on the demographic, geographic, psychological, and behavioral segments into which they are categorized (e.g., Matz & Netzer, 2017; Martin & Palmatier, 2020). Algorithmic curation organizes content so that what is ranked most relevant and personalized is prioritized earlier in a feed (Berman & Katona, 2020). This benefits most users, who see only a fraction of the content available on a platform. On social media platforms such as Instagram, for instance, users rarely see more than 30% of the videos and photographs in their feeds. Of course, algorithms also benefit the firms that own these platforms by delivering profitable content and advertising to users. For cases where firm incentives can harm users, see the discussion of the principal-agent problem and agency bias in Section 6.    

Algorithmic curation in recommender systems generally improves the quality of information and choice options that people consider in their decision making (Haubl & Triffs, 2000). Having more options leads to better choices when people can find options that match their preferences (attribute weightings). Attributes that make choice options unambiguously better or worse are vertical attributes—for example, price and quality (e.g., the square footage and condition of a house). Attributes that reflect personal preferences such as taste and social identity are horizontal attributes (e.g., whether a house is painted beige or yellow). When options are not in a particular order, however, people may not benefit from having additional options in their consideration set (Hauser & Wernerfelt, 1990). They may rationally pick lower-value options that appear early in their search process—or even defer a decision—because search costs (e.g., inspecting additional options; Stigler, 1961) outweigh the utility of finding better options (e.g., Iyengar & Lepper, 2000; Johnson et al., 2004). Algorithmic curation lowers search costs (e.g., time, effort, money) by ranking options in order of their value or only presenting high-value options. This allows people to easily find high-value options even when choice sets are extremely large. In this capacity, algorithmic curation replaces the role memory plays in the generation of alternatives (Alba et al., 1997).  

Algorithmic curation also reduces noise in decision making by helping humans better differentiate between options (Alba et al., 1997; West et al., 1999; Diehl et al., 2003). Carefully comparing options and their attributes (e.g., price, quality, size) requires attention and effort, which suggests that comparisons may be more thorough when there are fewer options and attributes to compare (Hauser & Wernerfelt, 1990; Kassam et al., 2011; Morewedge et al., 2019). It is easier to compare two job offers than 10, for instance, and easier to compare jobs that only differ on salary and benefits than jobs that differ on salary, benefits, industry, and location. Moreover, people are more sensitive to differences when the values of options and attributes are narrowly rather than widely dispersed (e.g., Stewart et al., 2006). If the options in a consideration set have similar values on an attribute, then any two options with different values should be further apart in relative rank and seem more subjectively different. A television that costs $400 will seem much more expensive than a television that costs $300 when the other televisions being considered are $375, $350, and $325 versus when the other options are $600, $500, and $200.  

Algorithmic curation may increase the diversity of information that people receive (Flaxman et al., 2016; Villermet et al., 2021). In a panel of 200,000 U.S. adults whose news consumption was tracked for 4 years, for instance, use of Facebook and Reddit increased information source diversity (non-redundancy in the sources from which they receive information; Kitchens et al., 2020; cf., Eady et al., 2019). Algorithmic curation seems to reduce online polarization in social media and increase the diversity of content by encouraging the passing of high-quality content from more distant social sources and encouraging connections between more distant individuals in networks (Barberá et al. 2015; Berman & Katona, 2020). Ties on Facebook often include other people with different ideological views (e.g., family members, colleagues, hobby-related associations; Goel et al., 2010), for instance, which can increase exposure to different information.  

These findings stand in relief to early concerns that algorithmic curation would create filter bubbles, in which people would only receive information that matches their existing attitudes and beliefs (Pariser, 2011).[1] Algorithmic curation is less likely to reduce the diversity of content to which a person is exposed than their personal decisions to consume content that reflects their attributes and beliefs (e.g., partisan affiliation; Bakshy et al., 2015) and homophily, the tendency to affiliate with others who have similar attitudes, beliefs, and preferences. Using Facebook, where curation is influenced more by the homophily of social networks, tends to steer users toward more partisan information sources (Levy, 2021). By contrast, using Reddit, where curation is less influenced by the homophily of social networks, tends to steer users toward less partisan information (Kitchens et al., 2020). Although algorithmic curation may not be the primary cause, it is important to note that there is still considerable debate about the existence and drivers of polarization in social media (for a collection of articles, see Haidt & Bail, ongoing).     

Decisions  

Algorithms augment and replace human decision making in at least two ways: monitoring and outsourcing. Algorithms can provide a second opinion that helps people monitor the quality of human judgment and decision making (e.g., algorithmic management; Jarrahi et al., 2021). When algorithmic and human decisions converge, it may suggest the judgment or decision was good. When algorithmic and human decisions diverge, it may indicate that the human judge made a mistake, is biased, or that the judgment or decision is an unusual or difficult case warranting additional consideration. If an algorithm and a human radiologist arrive at the same medical diagnosis, for instance, it may suggest no additional scrutiny of the patient’s symptoms is needed before recommending a treatment. If an algorithm and a radiologist differ in their diagnoses, however, additional scrutiny by the radiologist or other physicians may be warranted before recommending a treatment for the patient (Sutton et al., 2020). If police are stopping and frisking Black men more often than an algorithm predicts they should, the algorithm may reveal racial bias in policing (Goel et al., 2017).  

Many routine decisions are now entirely outsourced to algorithms. Outsourcing is more common on the supply side than on the demand side (e.g., institutions and experts rather than individual consumers), but technologies such as autonomous vehicles are likely to increase the diffusion of outsourcing decisions to algorithms throughout everyday life (Sharif et al., 2017; Rahwan et al., 2019; Yurtsever et al., 2020). In the United States alone, for instance, there are more than 900 million GPS devices in use, and 60% of Americans use GPS each week. From Inuit hunters to taxi drivers in Tokyo, people around the world rely on wayfinding algorithms that use GPS to provide them with directions on land and sea (Aporta & Higgs, 2005; Wakabayashi et al., 2011; Furchtgott-Roth, 2022). People use facial recognition algorithms instead of passcodes to unlock their phones and homes. They are beginning to use generative artificial intelligence such as ChatGPT and DALL-E for purposes good and bad, from creating new art, prose, and code to cheating on written exams.  

Experts in many fields—such as finance, medicine, services, and journalism—outsource decisions (with varying supervision) to algorithms so that they can redirect their time and labor to the tasks that are most important or too challenging to be automated. Algorithms perform high-frequency trading across financial markets for institutional investors (Chaboud et al., 2014). Credit score algorithms make loan approval decisions (Huang et al., 2007). Dating services use algorithms to decide which applicants to admit to their platforms and how to match members (Abolfathi & Santamaria, 2020; Yalcin et al., 2022). Marketers use algorithms to determine how much to charge consumers and how much to pay for online and mobile advertisements to individual consumers (Chen et al., 2016; Rafieian & Yoganarasimhan, 2020). News outlets use algorithms to generate financial reports about companies, a generally formulaic task from which journalists have been mostly freed. Journalists now have time to do more advanced investigative reporting when companies have an unusual earnings quarter (e.g., getting quotes, doing research, and putting unexpected outcomes into context). Algorithms are also used in tandem with journalists to sort, code, and catalog raw data (Diakopoulos, 2019). These are but a few examples of the many ways in which algorithms increase the quality and efficiency of professional and expert decision making.  

A SIMPLE MODEL OF ALGORITHMS 

A simple model of algorithmic judgment and decision making is a computational process that predicts an outcome from data (Kleinberg et al., 2018). Algorithms can range from simple decision rules such as “pick the cheapest/most popular/most highly rated” to complicated supervised learning models (for an overview of the critical elements, similarities, and differences of modern algorithms, see Appendix). Modern algorithms perform well, statistically. Their performance can vary with data quality and suitability, modeling choices, and optimization decisions. However, the most critical decision for algorithm design is one that psychologists are well suited to make. In all cases, humans determine problem specification. Humans decide the data the algorithm will use and the outcome it will predict from that data.  

There are three key parts to problem specification. You can use a simple linear regression model as a guide (see equation below).  

y = α + βx + ε 

When building an algorithm, you (1) first choose what it will do. This is the outcome it is designed to predict, the y in the regression. You next choose which data (2) variables and (3) cases it will use to make this prediction––think of the variables as the x in the above equation and the cases as the sample (N) on which the solution to the regression is based (e.g., participants). Another way to think about this is as a spreadsheet, with the data variables in the columns (the x used to predict y) and the cases in the rows (the N whose data is used to predict y). In this equation, the data on which the algorithm is trained will determine the parameter weights and error in this regression (i.e., α, β, and ε). Algorithms may perform a variety of complicated operations on this data to predict the outcome you requested. An algorithm may perform feature engineering––transforming data (e.g., apply a log function to normalize the distribution of a variable) or creating new combinations of variables (e.g., creating interaction terms). The algorithm may perform feature selection––choosing which predictors to include from the potential variables or “features.” In all cases, the first three decisions—the (1) outcome predicted, (2) variables, and (3) cases––remain its instruction manual. Whatever transformations and operations an algorithm may perform, human decisions dictate what the algorithm predicts and what data it uses, which have profound effects on its validity and accuracy and biases (Table A1, appendix). 

Imagine you were building an algorithm to track sentiment on a social media platform such as Facebook or Twitter. You first would need to decide what sentiment to track (i.e., the outcome). Your algorithm could predict subjective outcomes like valence and intensity of language, the amount of moral outrage expressed in posts, or the expression of one or more basic emotions (happiness, anger, sadness, disgust, fear). Your algorithm could predict objective outcomes like the frequency of negative and positive emotion words. Your second decision would be how to code the training data and who will code it. What scales will human coders use? Will emojis be counted and if so, how? To what cultural backgrounds and social groups will human coders belong (e.g., WEIRD college students, Amazon Mechanical Turk workers, interns)? A third decision is which sentences would be coded––what corpus of text would you use for the training data? What part will be analyzed (e.g., abstract or full paper)? What unit of text will be analyzed (e.g., paragraphs, sentences, words)? In practice, most algorithms are trained on a convenience sample of (undisclosed) text containing biases and irregularities that may not be discovered until long after the algorithm has been put into use.     

If you were designing a more complicated algorithm to help you hire an assistant professor, you would still need to make the same three decisions. You would first need to decide what it will predict about the candidates (y). It could predict who is most likely to get tenure, most similar or different from previous hires, likely to win the most awards, publish the most papers, be cited the most, or a variety of other outcomes. Second, you would need to decide which variables to use to train the algorithm. Will you include unpublished papers and conference talks or only published papers? What demographic information should you include or exclude? Would you train the algorithm on data that is likely to be relevant and important to a human hiring committee that is likely to be biased (e.g., interview performance)? Finally, you would need to decide on what cases to train the algorithm. It is unlikely that there would be enough data if you only trained it on previous hires in that department, and it would incorporate any biases that tipped the scales towards those candidates in the past. You could train it on other hires at your university or in other institutions’ psychology departments, but they might have different preferences and values. You could include applicants who did not secure tenure track positions in the past, but then you’d have to curtail the information it used to make predictions.   

Even building a generative artificial intelligence algorithm such as ChatGPT, which can create text that at times passes for human writing, would require that you make the same three problem specification decisions. First, you would need to decide what the algorithm predicts. ChatGPT is a large language model that predicts how a human would respond to a text prompt. It is essentially a very powerful auto-completion algorithm. Second, you would need to decide which variables to include in its training data. Third, on which cases would you train it? ChatGPT scrapes text from the Internet, and it is trained on the prompts and responses generated by a team of human artificial intelligence trainers, but we know little about what text it uses or who its trainers are.   

PSYCHOLOGICAL CONSIDERATIONS IN THE DESIGN, ADOPTION, AND USE OF ALGORITHMS 

The next three sections identify major psychological frictions (a) in choices made in the process of algorithmic design, (b) in decisions to trust and adopt algorithms in decision processes, and (c) in collaborations between algorithms and humans. We borrow the term friction from economics, which describes monetary or non-monetary costs incurred when making a transaction (e.g., brokerage fees or time). Each friction marks a point where ignoring human psychology is costly and where psychology can inform and improve collaborations between humans and algorithms. A roadmap of the three sections is provided in Table 2. 

Table 2: Psychological Frictions in Algorithm Design, Adoption, and Collaborations with Algorithms 

Psychological Frictions in the Design of Algorithms 

Building an algorithm requires human designers to make at least three choices with psychological frictions: specifying the data used to train the algorithm, the (1) variables and (2) cases (i.e., sample), and (3) specifying the outcome it will predict (i.e., its objective). Modern algorithms are statistically accurate when the cases they are used to predict are similar to the cases on which they were trained. Algorithmic biases that are of deep concern to psychologists arise when these criteria are not met. The accuracy, morality, and legality of algorithms may be impaired when training data is unrepresentative of the new cases to which the algorithm will be applied and when training data includes legally protected cases or variables (such as race, gender, age, or disabilities) that should have been excluded.  

Specifying the outcome predicted by an algorithm is perhaps the most important decision for psychology to inform. Choosing the wrong outcome to predict creates an alignment problem, where an algorithm optimizes on an outcome that is misaligned with its users’ values (Christian, 2020). When algorithm designers know which outcome an algorithm should predict, an alignment problem can arise from an imperfect connection between the observable human behaviors on which algorithms are trained to optimize (e.g., people’s choices), and the underlying human preferences, values, and utility for which that human behavior serves as a proxy. This gives rise to a host of statistical and regulatory problems (see Frictions From Predictions below).  

A second problem that algorithm designers have done less to unravel, and where psychology can make immediate contributions, is how to pick outcomes for algorithms in cases where people do not behave in accordance with their preferences and values (Morewedge et al., 2023). One source of misalignment arises in the many cases where people cannot explain their behavior, when they do not know what preferences or values make them behave as they do. Another source of misalignment stems from the cases in which people make choices that don’t believe they should. As an example, insert your favorite self-control dilemma here. In these cases, algorithm designers must decide whether to pursue prescriptive or predictive objectives: 1) to train algorithms to make decisions as humans should or 2) to train algorithms to mimic how humans do make decisions. The strengths and weaknesses of each approach are deeply interwoven with a variety of conflicts in motivation and preferences that have long been studied by psychologists through the lens of fairness, self-control, intertemporal choice, and want-should conflicts. 

Frictions from Training Data  

Training on unrepresentative cases

Algorithmic predictions are more accurate when applied to cases that are like those on which they are trained. However, algorithmic bias may arise when an algorithm makes predictions for a different group of cases, even if that group is simply a more heterogenous sample from the same broad population (Danks & London, 2017). Voice recognition software that is trained primarily on White Americans’ speech, for example, is more accurate at recognizing White Americans’ speech than Black Americans’ speech (Koenecke et. al., 2020). Almost all facial recognition algorithms exhibit performance differences by the race of faces tested. Many show an interaction between the origin of the algorithm and its ability to detect faces by race akin to “other-race” effects in human face perception (Buolamwini & Gebru, 2018; Cavazos et al., 2020). Algorithms trained on Asian or Caucasian faces exhibit greater accuracy in identifying faces of the race on which they were trained (Asian or Caucasian, respectively). A tractable solution to this issue is to train algorithms with data cases that are representative of the cases for which they will make predictions, but eliminating bias may require using training data that is costly to acquire (Raji & Buolamwini, 2019).  

It is worth noting why these examples are cases of algorithmic bias. Algorithmic bias hinges on whether differences in an algorithm’s predictions across categories is intended or not. Often algorithms are intentionally designed to make different predictions for social groups. If an ad delivery algorithm delivers ads for infant diapers to parents 100% of the time and never delivers ads for infant diapers to people without children, its designers and the public would think it accurate, not biased. In the examples above, however, the algorithms were designed to recognize spoken English and human faces. If an English voice recognition algorithm better recognized spoken English than French, or a human facial recognition algorithm better recognized human than feline faces, these outcomes would be intended design features, not algorithmic bias. The examples above exhibit bias because the different performance of voice recognition software for White and Black Americans and the different performance of facial recognition software for Asian and Caucasian faces were unintentional features of the algorithms. The algorithms’ performance unintentionally varied across social groups with potentially important consequences. While driving, for instance, White Americans may find it easier to use voice commands for smartphone applications such as Google Maps than Black Americans do, making driving safer for White Americans (e.g., they are less likely to need to use a touchscreen interface). Conversely, White Americans may be more easily surveilled by government and private organizations with facial recognition software in the United States than Asian Americans.  

Training on protected cases

Legal issues arise when algorithms are trained on cases and samples that their designers did not have the right to use. Many modern privacy laws, for instance, grant users the right to be forgotten. This gives users the right to request that their data be deleted (e.g., credit history, purchases, browsing history). Even when deleted, however, these data can still cast an “algorithmic shadow.” Once data are used to train an algorithm, the algorithm will not change if the data are later deleted (Li, 2022). Consider a person with a rare genetic condition who submits material for DNA testing to learn vital health information, then requests their data be deleted after receiving that information. If an algorithm is trained on their data and links their genetic condition to a chronic medical condition, they or their descendants might later be charged higher medical insurance rates. This issue is of sufficient concern to legal scholars and policy makers that algorithmic disgorgement is recommended for algorithms that were trained on protected data. Destroying an algorithm (i.e., disgorgement), even when the data on which it was trained have already been deleted, is sometimes the only way to maintain the privacy of people in the sample on which it was trained (Li, 2022).  

Training on protected variables

Algorithmic bias and discrimination can occur when algorithms incorporate variables in their training data that they do not have permission to use or indirectly incorporate those variables through their correlates. If protected social categories (e.g., race, gender, age, sexuality, disabilities) are explicitly excluded from training data, algorithms may still include those variables in their predictions because membership in social categories is often correlated with other variables. Personality tests that predict a variety of desirable and undesirable workplace behaviors, for instance, also predict psychiatric diagnoses (e.g., measures of neuroticism that are predictive of anxiety and depressive disorders; Melson-Silimon et al., 2019). As an example of the consequences of this kind of algorithmic bias, algorithms trained to predict whether criminal defendants can safely be released on bail while awaiting trial do not use race as an input. Yet, they are twice as likely to classify Black than White defendants who ultimately do not re-offend as high risk (Corbett-Davies et al., 2017).  

Without access to and control over an algorithm, algorithmic bias is often difficult to mitigate and its cause difficult to identify. Algorithmic audits are one way to detect these kinds of biases. New content is submitted to an algorithm to test if the algorithm exhibits bias in the way that it handles that content. Even when advertisers specify ad delivery to the same target audience (ignoring social category memberships), for instance, a predicted “suitability score” appears to direct Facebook ads to different social groups based on the context of the ad and imagery used. In one algorithmic audit study, despite explicitly specifying that an ad should be delivered similarly across race and gender, ads for employment in the lumber industry were delivered in 90% of instances to men and in 70% of instances to White users in the aggregate, whereas ads for janitors were delivered in 65% of instances to women and in 75% of instances to Black users in the aggregate. These results hold when the images used in ads are made invisible to the human eye but detectable to computers (Ali et al., 2019). One Facebook advertising audit study revealed algorithmic bias may paradoxically be related to profitability of advertising to different social groups. Women were more likely than men to click on an ad promoting job opportunities and training in STEM fields (science, technology, engineering, and math), but algorithmic curation probably led the STEM ad to be less likely to be displayed to women than men because men are cheaper to advertise to (they tend to make fewer household purchases; Lambrecht & Tucker, 2019). 

Frictions from Predictions  

Goodhart’s law

Algorithms are designed to optimize on the outcome they have been designed to predict, and that outcome needs to be observable and measurable. But performance on desired outcomes (e.g., readiness for college, teaching quality) is often difficult or impossible to directly measure, and only indirect measures may be used to assess performance toward desired outcomes. Goodhart’s law refers to cases in which the attempt to optimize a measure of a desired outcome erodes the statistical relationship between the measure and the desired outcome (Goodhart, 1984). It encompasses a variety of statistical problems that arise from using indirect measures, ranging from noise due to imperfect measurement to problems that arise when the causal path between the desired outcome and indirect measure is uncertain (for a review in the computer science literature, see Manheim & Garrabrant, 2018).  

Psychologists have wrestled with some of these measurement problems, particularly in their development of measures of formative constructs, which are caused by measurable variables (e.g., health, socioeconomic status). Socioeconomic status, for instance, is a product of three inputs: income, education, and the social status associated with an occupation. A singular measure often does not capture a formative construct as well as it could a reflective construct, which influences or has a causal effect on its measures. Intelligence, for instance, is assumed to be the cause of performance on measures of intelligence (e.g., IQ; Edwards & Bagozzi, 2000). This distinction is important because if only one measure of a formative construct such as health care spending is used as a proxy for the broader construct (e.g., health), disparities in the measured construct can be amplified. If health care spending on patients is used as the sole proxy for patient health, for instance, systematic differences in the amount spent on Black and White patients create racial biases in more general decisions about how health care is delivered (Obermeyer et al., 2019). As a field that has spent more than a century indirectly measuring latent constructs (e.g., happiness, personality), psychology could do much to inform how to indirectly discern and measure the human preferences and values that algorithms are designed to follow.  

An important variant of Goodhart’s law for psychology is more than a statistical problem: Using an indirect measure of a desired outcome to reward performance often creates an incentive to manipulate it (adversarial Goodhart; Manheim & Garrabrant, 2018). Humans are highly sensitive to reward structures and will adapt their behavior to perform better when desired outcomes are rewarded (Sutton & Barto, 2018). Standardized exams like the Scholastic Aptitude Test (SAT), for instance, are widely used as an indirect measure of the literacy, writing skills, and numeracy of students applying for college. The widespread use of the SAT to determine college admissions, however, has incentivized firms such as Kaplan and Princeton Review to create courses and materials aimed only at improving students’ test scores. Goodhart’s law also applies to university faculty. Universities often seek to reward faculty for effective teaching with salary raises and use instructor teaching ratings to determine how to distribute these raises. However, faculty are then incentivized to make changes to increase their instructor ratings that have no or a negative effect on the quality of their courses (e.g., handing out candy during course evaluations, reducing the difficulty of assignments, inflating grades).  

In the context of algorithm design, this issue is critical. If people learn which outcome an algorithm is predicting and rewarding (e.g., what metric the algorithm is using to prioritize content or recommend options), they are likely to change their behavior to optimize their score on that predicted outcome. For instance, many websites engage in search engine optimization, using tactics to increase the likelihood that they will be ranked higher in search engine recommendations in a way that does not benefit their readers. Social media platforms such as Facebook tend to prioritize content according to how likely users will engage with that content (e.g., news, photographs, videos, microblogs), and engagement may be as good an indirect proxy for quality as any other usable measure. Yet, if social media content is prioritized by the likelihood that users engage with it, and users learn this measure determines which content is seen first, then they may change the way they write posts to increase the likelihood that others will engage with their posts (e.g., use posts to solicit opinions on published research rather than summarize its findings). Indeed, professional journalists report adapting the news content they publish on Facebook to increase user engagement and prioritization in its feed. Social media editors accentuate surprising and emotional elements of news stories to increase the probability that they will be shared (Berger & Milkman, 2012). To optimize attention and exposure, they publish more stories about entertainment and fewer about foreign politics and economic policy on social media relative to the stories published in their own news outlets (e.g., websites; Lischka, 2021). Eventually, attempts to “game” the algorithm may change what content is produced and shared in ways that do not maximize user utility. Algorithm designers at YouTube found that videos loaded with speculation about popular events were most likely to sustain user engagement, for instance, which may result in users being exposed to increasingly extreme political content over time (Brady et al., 2020).  

Psychological research is needed to address both statistical and adversarial issues related to Goodhart’s law in algorithm design. Much methodological and statistical work has been done in psychology to understand how to measure attitudes, values, and preferences, and more work is needed to adapt this work to inform algorithm design. Dealing with the adversarial variant of Goodhart’s law is another place where psychology could make valuable contributions. The simplest current solutions are to (i) directly measure and reward performance on desired outcomes, which is often impossible, or (ii) keep secret any indirect measures of performance.   

Do as i do, or do as i should?

A central friction in designing algorithms to collaborate with people is determining whether to build a prescriptive or predictive algorithm: if the algorithm should decide as humans should or as humans do. Platform designers generally ignore this conundrum and design algorithms to predict what people do (Khambatta et al., 2023). A consequence is that many social media users spend considerable time on platforms that provide them with dissatisfying experiences that decrease their happiness (Kleinberg et al., 2022). This is a critical design choice, one that should take into consideration want-should conflicts in human decision making. 

To make an algorithm do as humans should, designers need to specify the data the algorithm should use and the objective it should predict (Acemoglu & Autor, 2011). The ability to train an algorithm to make decisions the way humans should is hindered by (1) the inaccessibility of many psychological processes involved in human decision making, (2) the frequency with which normative human decisions are illegal, and (3) cases where accurate decisions conflict with other criteria such as fairness.   

It is difficult to build algorithms that perform tasks that do as people should, such as tasks relying on creative and social intelligence, because it is difficult to stipulate the values and processes underlying intuitive judgments and decisions, creative tasks, social interactions, and moral judgments (e.g., Frey & Osborne, 2017). In many cases, intuitive and moral decisions resemble perceptual decisions (Kahneman & Frederick, 2002). People have access to the output of intuitive decision processes (e.g., “I prefer the socks on the right.” or “Consensual sex between adult siblings is wrong.”) but often lack insight into the processes by which they make intuitive decisions (Nisbett & Wilson, 1977; Haidt, 2001; Wilson & Dunn, 2004). Generative artificial intelligence algorithms such as DALL-E and ChatGPT can copy and recombine images and writing resembling what humans have created and written in the past, but the ability to create algorithms that produce novel artistic works and understand the complexities of values and morals and social interaction is challenged by our own difficulty articulating and specifying how humans make decisions in these spheres.  

A second challenge in building an algorithm that decides as humans should is that in many cases, the normative judgments and decisions made by humans violate the law, and building a law-abiding algorithm may be harmful to humans. Imagine an autonomous vehicle (e.g., a self-driving car) that always drives at or below speed limits. If most humans are driving faster than the posted speed limit, the autonomous vehicle may be more likely to cause an accident than if it is not bound by traffic laws. In the early days of autonomous vehicle testing, Google programmed its self-driving cars never to drive faster than 25 miles per hour to avoid scaring other drivers or pedestrians. In 2015, a Google self-driving car following this rule was pulled over by Mountain View, California, police for driving too cautiously (24 mph in a 35-mph zone) and causing a traffic jam. Another Google self-driving car stopped at a crosswalk out of an abundance of caution and was rear-ended by the surprised human driver behind it (“Google car pulled over for being too slow,” 2015, November 13). 

A third challenge occurs when the potential standards for how algorithms should make predictions conflict. In most cases, it is impossible for algorithms to optimize raw accuracy for predictions about members of different social groups (the rate of false positive and false negatives across social groups; see Appendix) and at the same time maintain statistical parity across social groups (all groups having similar average predictions scores; Kleinberg et al., 2016). Consequently, algorithmic designers and regulators face difficult trade-offs with substantive consequences (e.g., bail decisions, advertising, medical tests and diagnoses). Criminal bail decisions should ideally maximize public safety (e.g., prevent violent crime) and should fairly impose the social and economic costs of incarceration (e.g., exhibit no racial bias). White and Black defendants with similar criminal charges and histories, for instance, should be equally likely to receive bail.  

Current solutions require a trade-off between these two goals. Algorithms that ignore fairness requirements and purely maximize public safety (e.g., COMPAS) exhibit racial bias in their recommendations for whom to grant bail and whom to detain. Even when race is not included as a variable in their decisions, they are more likely to recommend bail for White defendants than for similar Black defendants. All current algorithms that impose fairness constraints increase the risk of violent crime, including algorithms that detain an equal proportion of defendants in each race group (i.e., statistical parity), algorithms that detain an equal proportion of defendants in each race group after controlling for a limited set of “legitimate” risk factors (i.e., conditional statistical parity), and algorithms that make decisions across all race groups that are equally accurate (i.e., predictive equality; Corbett-Davies et al., 2017).   

Training an algorithm to imitate human decision making—do as I do—circumvents the need for humans to articulate what they are doing. Training autonomous vehicles is an example of how such a process works (Le Mero et al., 2022). An autonomous vehicle learns how to drive by observing the choices and actions of an expert human driver in a range of scenarios. Like an auto-completion program in a text string, these imitation learning algorithms learn how, given a particular context, a human expert will respond to a near-human standard. These algorithms learn to predict what the human expert will do next.  

One weakness of imitative algorithms is an inability to generalize. They are trained by imitation, not rules, and have difficulty predicting what a human expert will do in edge cases and in novel, unseen, and unrepresented scenarios. Imitative algorithms are highly context dependent. They struggle to generalize to novel contexts that vary on important dimensions (e.g., different norms and laws; ratio of drivers, pedestrians, and cyclists; weather; Danks & London, 2017). An autonomous vehicle trained by observing Boston drivers may, for instance, decide to “squeeze the tomato” (i.e., run a traffic light that just turned red) in other cities where that behavior is unusual. These shortcomings of imitative algorithms reveal strengths of the human mind. An American vacationing in England can land at Heathrow, rent a car, and drive (relatively) safely on a different side of the road. The human ability to map inarticulable skills and rules onto new contexts far surpasses what any algorithm can do today.  

Imitative algorithms also reveal cracks in human reasoning. Because they learn by mimicking human decisions, imitative algorithms—like humans—sometimes make decisions that are normative and inequitable or illegal, potentially harming society and exposing their designers and users to legal damages (Logg, 2019). A canonical example is a hiring algorithm developed by Amazon to find job candidates and rate them using a five-star system much like the one it uses for consumer reviews. Because the algorithm was trained on past hiring decisions and because a disproportionately larger number of men than women were hired at Amazon, the algorithm inferred that variables correlated with gender predicted hiring. It then favored resumes that included words and affiliations suggesting the candidates were male while penalizing resumes that included words and affiliations suggesting the candidates were female (e.g., “women’s chess club captain” and graduates of all-women’s colleges). Moreover, because variables such as gender influence so many factors that are difficult to enumerate or articulate, the designers were unable to debias the algorithm and ultimately abandoned its use (Dastin, 2018, October 10). 

A third issue with imitative algorithms arises when their development outpaces their scientific validation. Algorithms can make accurate predictions even when designers and scientists have no idea how or why they work. Affective computing algorithms, for instance, are designed to detect signs of deception in facial micro-expressions (i.e., subtle movements in musculature). In direct comparisons, affective computing algorithms exhibit deception-detection capabilities that are superior to those of humans (Pérez-Rosas et al., 2015; Su & Levine, 2016; Monaro et al., 2022). Although these algorithms are in use by law enforcement organizations such as the Department of Homeland Security (Bratton, 2019, July 17), their designers cannot tell how the algorithms are differentiating truth and deception. This is problematic because these algorithms rely heavily on limited training data: Many are trained on the same set of 121 court statements (Monaro et al., 2022), and all are based on controversial psychological assumptions (for a review, see Zloteanu, 2020). The validity of detecting deception from facial micro-expressions is questionable for several reasons. People can consciously manipulate micro-expressions such as an authentic “Duchenne smile,” the algorithms may generate many false negatives, people experiencing a genuine emotion often do not exhibit associated facial expressions, and many emotions are difficult to detect from facial expressions without contextual cues (Krumhuber & Manstead, 2009; Gosselin et al. 2010; Gunnery & Hall, 2014; Namba et al. 2016; Barrett, 2017). If affective computing algorithms proliferate before they are well-validated, as the polygraph did (Saxe et al., 1985), they are likely to be used inappropriately in ways that impinge on the privacy and rights of private citizens (Richardson, 2020). 

When designers can choose how to design algorithms, they usually make them decide as people do. Social media platforms, for instance, tend to prioritize content that maximizes immediate attention and engagement (e.g., time spent on the platform and clicks) rather than long-term consumer welfare and satisfaction with the platform (Kleinberg et al., 2022). Two psychological phenomena explain why people continue to use these algorithms (e.g., Facebook) while finding their experience to be so unsatisfying: 1) People often want algorithms to mimic their reflective, not intuitive decisions, and 2) people are often conflicted between choice options that they want and choice options that they believe they should choose (e.g., want-should conflicts; Morewedge et al., 2023).  

First, human decision making is often intuitive. Cases where more reflective and deliberative processes correct or reject intuition appear to be the exception, not the norm (e.g., Todd & Gigerenzer, 2000; Kahneman, 2003; Shah & Oppenheimer, 2008; Morewedge & Kahneman, 2010). Intuitive decision making typically does well but can fall short of its ideal when scrutinized by deliberation (e.g., Frederick, 2005; Duckworth et al., 2018). When pressed for time, people eat the chocolate cake, but given time to reflect, they prefer the fruit salad (Shiv & Fedorikhin, 1999). Tentative evidence suggests people expect algorithms to emulate their reflective decisions, not their intuition. When people decide how autonomous vehicles should prioritize human lives, for instance, people direct autonomous vehicles to save the lives of children and sacrifice the elderly (Awad et al., 2018). Yet, people are more morally outraged by autonomous vehicles that explicitly incorporate these intuitive social preferences (De Freitas & Cikara, 2021). Training algorithms to decide as people (typically) do, therefore, will create algorithms that fail to live up to the higher standard of reflective decision making to which they are held.  

Even when people have the resources to deliberate about their choices, people are often conflicted about which goal to pursue. The things people believe they should do are not the same as they things they want to (e.g., smoke, drink, gamble, take drugs, procrastinate, fail to exercise, and engage in needlessly risky sexual behavior; Schelling, 1984). Want-should conflicts arise when some choice options bring greater pleasure in the moment and others bring more net value over the long run (Milkman et al., 2009), and when preferences reverse between actual and ideal selves (Higgins et al., 1994). Netflix users tend to populate their watch list with highbrow films (shoulds), for instance, but tend to churn more quickly through lowbrow ones (wants; Milkman et al., 2009).  

Although we typically think that people are better off choosing options that provide greater benefits in the long term, algorithms that choose what people want may not be so bad. Wants can convey preferences and emotional needs that are more difficult to articulate than shoulds (Bazerman et al., 1998), and algorithms that always prioritize shoulds over wants might never be used. A streaming service that only recommends educational movies and shows might struggle to find paying users, as would a meal delivery service that only recommended raw and steamed vegetables. Designing algorithms that collaborate well with human decision makers requires a delicate balance between wants and shoulds rather than always defaulting to shoulds.  

Psychological Frictions in the Adoption of Algorithms 

Algorithms can only benefit human decision making when people consent to use them. Algorithms are tools, and their adoption is guided by practical considerations that govern the acceptance of any new technology, like their performance, risk, convenience, familiarity, and cost. A few psychological considerations, however, differentiate decisions to trust and use algorithms from doing so with other technologies. First, the incredible opacity and complexity of algorithms—the perception that their decisions are a black box—can inhibit the propensity to rely on and trust algorithms. Second, because algorithms often mimic or replace human decision processes, algorithms can threaten people’s identity and evoke biased evaluations that disparage algorithms or overestimate their and other humans’ decision making. At an interpersonal level, the designers of algorithms often have incentives that conflict with the incentives of their users. Using an algorithm requires people to decide whether the cost of divulging personal information is worth the benefits its use would provide. Because most algorithms are provided by for-profit firms, using algorithms typically entails a principal-agent problem, where people rely on advisers who are incented to recommend decisions that are not to their users’ benefit. For a discussion of additional barriers to adoption, see Gray and colleague (Gray et al., 2023)  

Transparency, Explainability, and Understanding  

The complexity of algorithms is both a strength and a weakness. It allows algorithms to make predictions with accuracy that often far surpasses human decision makers (Mullainathan & Speiss, 2017; Gunning et al., 2019). It also creates opacity inhibiting their adoption. How algorithms convert data to predictions is typically hidden from users. Designers may better understand how a specific kind of algorithm works, but even designers cannot fully understand how and why deep-learning models—such as those used for speech recognition, image recognition, and natural language processing—work (Gunning et al., 2019; Rai, 2020).  

People typically exhibit an illusion of explanatory depth for complex systems. They generally overestimate how well they understand how systems such as helicopters, light switches, and fiscal cliffs work (Rozenbuilt & Keil, 2002). Algorithms appear to be an exception. Whether algorithms are designed to predict which jokes people will find funny or diagnose skin lesions as cancerous or benign, people rightfully perceive algorithms as a black box. People report having a poor subjective understanding of how algorithms make decisions, and they perform at chance when their objective understanding is tested (Cadario et al., 2021). Because people are more willing to adopt and accept new ideas and technologies that they can understand (e.g., Fernbach, Rogers et al., 2013; Fernbach, Sloman, et al., 2013), this lack of understanding inhibits the use of algorithms.  

As a result, they also generate inaccurate lay theories about how algorithms work that further impair their adoption. People believe that algorithms that undergird medical artificial intelligence will not consider their unique health history as a human provider would, for instance, unless they are explicitly told that algorithms will provide them with personalized health care (Longoni et al., 2019). More generally, people tend to overestimate their understanding of human decision making. Although people perform at chance when explaining how humans and algorithms make the same medical decision, they believe they better understand how humans make those decisions, which is one reason why people prefer to rely on humans than on comparable algorithmic medical providers (Cadario et al., 2021).  

Fortunately, explaining how algorithms work increases how willing people are to rely on algorithms (Yeomans et al., 2019; Cadario et al., 2021). Understanding the reasons for algorithmic predictions can also increase their acceptance and perceived legitimacy (Holzinger et al., 2019; Shin, 2021; Demirdag, 2022). A home buyer may want to know why an algorithm approved or declined their loan, for instance, or offered them a different rate than other customers. In the explainable artificial intelligence literature (XAI), causability means providing such an explanation for predictions that humans can understand (Chou, et al., 2022).   

Governments, legal systems, and organizations also are hesitant to rely on decision processes they do not understand, whose legitimacy is difficult to verify, and whose decision quality is difficult to assess. Opacity can obscure prejudicial predictions (Rai, 2020). It can be difficult to discern whether and why algorithms generate different predictions for individual users, for instance, and even members of different social groups (e.g., Lambrecht & Tucker, 2019). Explainability may be a prerequisite for algorithms with applications in fields such as defense, medicine, and law (Gunning et al., 2019) or decisions related to moral or social preferences (e.g., Bigman & Gray, 2018; Kleinberg et al., 2018; Dietvorst & Bartels, 2022). An example is the Right of Explanation in AI in the European Union’s General Data Protection Regulation, which gives people the right to demand that a company explain how an algorithm made a decision based on their data (e.g., why it approved or denied their loan application).  

As the highest-performing algorithms are often the least explainable (Gunning et al., 2019), exploring how to balance performance, transparency, and explainability is an area where psychology can make important contributions to algorithm development, adoption, and regulation. Early work suggests it may help to add interpretability constraints (e.g., fewer inputs, monotonic relationships between features and predictions), explain the inputs used to make decisions (Cadario et al., 2021; Shin, 2021; Demirdag, 2022), or provide an approximation of the model such as an interpretable decision. ​​​Alternatively, models can be explained by perturbing interpretable features (Rai, 2020) or adapting game-theoretic Shapley values to understand how features contribute to predictions ​​(Lundberg, Erion & Lee, 2018). ​More work is needed to see how each of these approaches increases objective and subjective understanding of algorithms in users and experts.  

Positively Biased Evaluations of Self and Other Humans  

Algorithms are more accurate and efficient than humans in a multitude of judgments and decisions (for an early review, see Dawes et al., 1989), but people still prefer to rely on humans in a variety of domains (e.g., health care, tasks requiring social intelligence, painting, poetry, providing romantic advice). Initially perceived as evidence of domain-general algorithm aversion, a recent reconciliation suggests these phenomena reflect a narrower domain-specific preference for human judges and decision makers, driven by biased self–evaluations in domains where negative comparisons may threaten identity (Morewedge, 2022).  

A history of comparing actuarial or statistical prediction and clinical judgment has shown that human experts are often unwilling to outsource their work to superior algorithms (e.g., linear models; Dawes et al., 1989). This preference for humans over algorithms was initially characterized as a domain-general prejudice, an algorithm aversion (Dietvorst et al., 2015; Burton et al., 2020). Although the initial evidence did not suggest people were prejudiced across all domains toward algorithms, it did show that people are more critical of mistakes made by algorithms and are less likely to trust them after seeing them err (Dietvorst et al., 2015). People believe that algorithmic decision processes are rigid and less personalized than human decision processes (Longoni et al., 2019; Newman et al., 2020); cannot learn as well as humans (Berger et al., 2021; Reich et al., 2022); and are incapable of assessing qualitative, subjective, and moral attributes (Bigman & Gray, 2018; Castelo et al., 2019; Longoni & Cian, 2022; Yalcin et al. ​​2022).  

This literature, however, struggled to parsimoniously explain why in many domains people simultaneously exhibit algorithm appreciation (e.g., for data analysis, directions, financial advice, piloting airplanes). In many cases, people prefer to rely on algorithms than on decisions made by themselves and other humans (Logg et al., 2019). Many people prefer to rely on algorithms, for instance, to predict the weather. The Weather Company is currently the fifth most prevalent smartphone application in the United States after Google, Facebook, Amazon, and Apple (Comscore, 2022, June). Also, when perceptions of humans and algorithms are compared to objective criteria, people exhibit less biased evaluations of algorithms than humans (e.g., Cadario et al., 2021) and believe algorithms are less biased than humans (Demirdag & Shu, 2021). News articles are seen as less biased, for instance, if people believe they were written by algorithms rather than by human journalists (Nam et al., 2022).   

A theoretical integration suggests that cases where people prefer humans to algorithms reflect positively biased evaluations of self and other humans in a narrower set of domains (Morewedge, 2022). Like other self-serving evaluations (for a recent review, see Sedikides et al., 2021), this approach proposes that these biased evaluations tend to occur for tasks central to one’s identity in which negative comparisons can threaten the self (Campbell & Sedikidies, 1999), particularly where evaluative criteria are ambiguous enough to allow biased distortions to influence judgment (see Figure 2, adapted from Morewedge, 2022).  

Figure 2: Preferences for Humans Relative to Algorithms by Identity-Relevance and Ambiguity of Evaluative Criteria  

Note: If the preference for humans relative to algorithms is driven by biased self-evaluations, it should increase with the (1) identity-relevance of the judgment, decision, skill, or task (x-axis), and (2) ambiguity of its evaluative criteria (e.g., data and process it required; y-axis). In those domains, negative comparisons are most likely to threaten the self and elicit self-serving evaluations. Quadrants are populated with examples from references in Morewedge (2022).

This theoretical framework proposes that two factors can predict when people will prefer to rely on humans than on algorithms: 1) the identity relevance of tasks and 2) the ambiguity of their evaluative criteria. As the identity relevance of tasks increases, threatening comparisons are more likely to evoke self-enhancing and self-protective biases that influence evaluations of humans and algorithms (e.g., Agarwal et al., 2024; Campbell & Sedikides, 1999; Leung et al., 2018). As the ambiguity of evaluative criteria increases, so too does the opportunity for these biases to influence evaluations (e.g., Dunning et al., 1989). Broadly, people tend to identify with symbolic, hedonic, and subjective preferences; tasks in which they possess expertise; and preferences and abilities that are distinctive associations with their desirable social group memberships (Ellemers et al., 2002; Leung et al., 2018; Castelo et al., 2019; Logg et al., 2019; Sedikides et al., 2021; Kang & Kim, 2022; Longoni & Cian, 2022). When evaluative criteria are sufficiently ambiguous, these are the domains in which they are more likely to prefer to rely on themselves or other humans than on algorithms. 

This theoretical framework also proposes paths to increase the adoption of algorithms. Interventions reducing identity threat could make people like experts more likely to adopt algorithms. One path toward this end is to make what algorithms do less identity-relevant. Most decisions require multiple steps and vary in ease or typicality, and the components that algorithms perform could be described as supportive, easy, or routine. Psychologists used to compute their own statistics by hand, for instance, but have readily adopted software that performs these computations for them, as these algorithms are performing work that psychologists view as supporting their job rather than replacing it. Another path toward this end is to bolster the self against threats, such as through self-affirmation. Finding ways to reduce the identity threat posed by algorithms become increasingly prevalent as algorithms diffuse through the workplace and replace much of the work with which people identify (e.g., Puntoni et al., 2021; Selenko et al., 2022). A third path toward reducing self-threat is to give people control or perceived control over the decision process (Dietvorst et al., 2018; Longoni et al., 2019). Interventions reducing the opportunities for bias to influence judgment could increase the perceived or real specificity and objectivity of evaluative criteria (e.g., Castelo et al., 2019). Explaining the criteria an algorithm uses to make a decision can increase subjective comprehension of how it works and willingness to use it (Cadario et al., 2021). 

Conflicts of Interest  

Privacy versus personalization

Privacy is one price everyone must pay for the benefits of algorithmic curation (Tucker, 2012). People must tell an algorithm who they are before it can provide them with personalized choices curated to their tastes, but people value and desire safeguarding their privacy. There are real dangers in sharing data, and people also feel psychological ownership of their personal data and want to limit its sharing (Morewedge et al., 2021). Sharing personal data can be costly (Acquisti et al., 2015). Using age, birthday, and place of birth, algorithms can predict the Social Security numbers of many adults in the United States (Gross & Acquisti, 2005). Algorithms can track, compile, and reveal professionally or social undesirable behavior people engage in at home or in the workplace (Raveendhran & Fast, 2021). A variety of theories (e.g., social contract theory, justice theory, control theory, social exchange theory, reactance theory, behavioral decision theory; for a review, see Martin & Murphy, 2017) explore whether and when people explicitly consider this trade-off (i.e., the “privacy calculus”) and seek to explain the psychological processes involved when it is explicitly considered.  

Important conversations center on the degree to which people implicitly or explicitly consider their privacy when disclosing personal data, the extent to which they value their privacy, how they decide with whom it is safe to share personal data, and their concerns and awareness of the extent to which it is may be used and shared (intentionally or not) with third parties (e.g., firms, governments, criminals; Goldfarb & Tucker, 2011; John et al., 2011; Kim et al., 2019; Martin & Palmatier, 2020). Most of this literature is concerned with the perceived risks and benefits of personal data disclosed through online behavior (e.g., Raveendran & Fast, 2021), but algorithms are increasingly being used to collect data about people’s offline behaviors. Technologies such as video surveillance and facial recognition software are in widespread use by governments and both private and public organizations, and physical location tracking through technologies such as GPS can easily link devices (e.g., smartphones) with personal data. The psychology that underlies how people understand and make trade-offs between the costs and benefits of divulging their personal information in these contexts is an area where much more research is needed.   

Principal-agent problems

Principals tend to delegate tasks to agents when doing so is more efficient. Most homeowners (principals in this scenario) could repaint their home, for instance, but it is usually more efficient to hire a professional painter (agents in this scenario). As agents, algorithms have preterhuman abilities. Algorithms can sort through thousands of television shows or flights or potential romantic partners in seconds to find the few that are most likely to match what the principal is looking for. Financial and state incentives for designing and maintaining algorithms (e.g., profit, control, surveillance), however, can create conflicts of interest in their collaborations with humans. Sometimes referred to as the agency problem (Shapiro, 2005; Dalton et al., 2007), the principal-agent relationship is concerned with such conflicts of interest (Michaely & Womack, 1999; Ljungqvist et al., 2007), incentives, and costs that arise when decision-making power is delegated. Search engines are typically supported by advertising revenue, for instance, creating a conflict between incentives to provide users with the best information and capture revenue through sponsored search advertising.  

Traditional analyses of the principal-agent problem are predicated on determining the optimal degree of risk-sharing between principals and agents (Grossman & Hart, 1992). Today, we search for Pareto optimality in collaborations between humans and algorithms––the balance point where we cannot give benefits to one group without harming the other. Beyond risk sharing, we need a better understanding of principal-agent relationships between humans and algorithms. Research is needed to explore how to design incentive and monitoring schemes that enable the advantage of delegation to be realized, despite principals having different incentives and information than agents (Hamman et al., 2010). The opacity of deep learning algorithms requires new solutions to the principal-agent problem. A lack of interpretability means that conventional solutions based on independence, equity, and the market for corporate control (Dalton et al. 2007) are insufficient measures.  

Modern collaboration with algorithms can be viewed as an algo-social complex, where an entity can act as principal and agent simultaneously and the relationship can be framed as a multi-principal problem with i) corporations, ii) individual algorithm users, and iii) algorithms themselves as principals. In this hierarchy, the principal corporate entity delegates to an algorithmic agent, which in turn acts as principal and delegates interaction to human users, who (in a self-reinforcing cycle) are both principal and agent. Human principals rely on algorithmic agents to improve their judgment through measures such as cognitive offloading, algorithmic curation, and optimizing choice architecture. Conversely, algorithmic principals delegate the task of engagement to human agents for stimulation that generates data from which they learn. The informational asymmetry is evident as is the possibility of malfeasance. The algorithms we interact with have far more information about our behavior than we ourselves are capable of recording (e.g., Kim et al., 2019).  

The first tension occurs when the interests of multiple principals are not in congruence (Dalton et al., 2007; Gailmard, 2009). This increases informational asymmetries and makes monitoring more difficult (Shapiro, 2005). This tension is also evident in how algorithms as agents can be used to take self-interested or immoral actions that the principal would be reluctant to do more directly (Hamman et al., 2010). Delegation by corporate entities or individual users allows them to indirectly behave unethically through algorithmic agents, diffusing responsibility and avoiding blame. An advertising recommendation algorithm driven by reinforcement learning that optimizes for profit and disregards potential social harms may advertise alcohol, unhealthy food, or cigarettes to children. There is growing concern over the conflicts of interest that occur with the use of algorithms to provide entertainment and literature, such as music curation algorithms (Bonini & Gandini, 2019), and their role in shaping popular culture (Morris, 2015).  

Actions taken by agents are more self-interested and less prosocial than actions taken directly (Hamman et al., 2010). Applying the principal-agent framing to influential tech companies such as Google (Rieder & Sire, 2014) and Amazon (Farronato et al., 2023) shows how incentives orient the user experience in self-serving ways. For Google, whose revenue is primarily through advertising, these incentives lead to a fundamental conflicts of interest magnified by the company’s dominant market position. Similarly, self-preferencing at Amazon, whereby products sold by Amazon are favored in search results over products offered by other sellers, disrupts the free market by reducing competition, which makes consumers worse off—the requisite argument for anti-trust action. An algorithm’s incentives are determined by the corporate entities that control them, placing individual users at the disadvantaged end of a power differential. Regulations ensuring that the monitoring and incentive schemes in the user-algorithm relationship are prioritized over the corporate-algorithm relationship may be needed to protect consumer welfare.  

A second tension is driven by the reciprocal pattern of human behavior driving algorithmic behavior and vice versa. Human users can give decision-making authority to algorithmic agents with different interests under an informational asymmetry where users often cannot interpret the decisions made on their behalf. Human users also may exhibit malfeasance by including variables and cases in training data that gives rise to algorithmic bias or unethically misusing algorithms (Kim, 2020). Algorithms do not bear the costs of failure, so they may perform malfeasant actions or take excessive risks (i.e., moral hazard; Borch, 2022). When a biased algorithm unfairly denies bail to a marginalized person, the effect is casual—a manifestation of algorithmic malfeasance. This tension is relevant for other domains of law, governance and management, medicine, finance, marketing, and manufacturing (Borch, 2022). Financial decision-making is outsourced by many professional investors to high-frequency trading algorithms. The risks of such outsourcing became clear in 2010, when the U.S. financial markets experienced what was called a major “flash crash.” In the span of a few minutes, the interactions of automated trading algorithms produced huge losses (Kirilenko et al., 2017). Such events are likely to occur elsewhere because the delegation of decision-making power generates informational asymmetries and moral hazard. Moral hazards are commonly discussed in banking (Hellmann et al., 2000), but discussions have yet to fully incorporate the role of algorithms and the principal-agent problem. Although the theory of agency has been incorporated into an understanding of organizational behavior (Jensen & Meckling, 1976), this popular theory does not account for non-human agents. It is necessary to update previously established theories to adjust for algorithms, with the goal of better understanding how to mitigate risks and maximize utility.  

Frictions in Collaborations with Algorithms 

Four psychological frictions in collaborative decision making with algorithms create intentional and unintentional changes in knowledge, skills, and behavior. Cognitive offloading (1) to algorithms can increase access to skills and information while decreasing the skills and information that people themselves possess. Collaborations with algorithms may change social norms (2) by exposing people to beliefs and behavior that are unrepresentative of the social groups to which they belong. In extreme cases, the structure of algorithms may create digital addiction (3). It is worth exploring whether algorithms can create digital addictions by making natural rewards addictive through changes in the reinforcement structures with which those stimuli are delivered (e.g., social rewards; Lindström et al., 2021). These and other psychological frictions inherent in collaborative decision making with algorithms may also (4) increase unintentional or intentional unethical behavior and raise questions about how responsibility is attributed for the unethical and erroneous decisions made by algorithms.   

Consequences of Cognitive Offloading 

People have long enhanced their cognitive abilities through cognitive offloading, depending on external sources (e.g., abacuses, books, notepads, computers, phones, calendars, other people) for information or skills.  People tend to think, know, and remember collaboratively (Sloman & Fernbach, 2018) and engage in cognitive offloading with available technologies (Risko & Gilbert, 2016). In collaborative groups, people retain direct access to some memories stored internally (within themselves) and store other memories externally within other members of the group (i.e., transactive memory; Wegner, 1987). Through such a distributed cognitive system, people do not need to know everything. They can leverage their own knowledge as well as the knowledge others possess (Sloman & Fernback, 2018). Similarly, collaborations with algorithms are changing what people know, how people interact, and where knowledge is (and is perceived to be) stored.  

People are now outsourcing knowledge, skills, and memories to algorithms. People do not encode information as well when they have stored it on a computer compared with when they cannot store it (Sparrow et al., 2011). Drivers who rely on GPS wayfinding algorithms for directions similarly do not encode information about their location and route. Those drivers navigate that route a second time as poorly as drivers who are navigating it for the first time (Fenech et al., 2010). Access to these algorithms is also changing the knowledge people believe others possess. People may assume others have access to the same algorithms and information that they do (with some exceptions, such as older generations). When friends plan to meet at a new restaurant, for instance, people may share the name of the restaurant but not its address or directions because they assume their friends will also use an algorithm to find it.  

Offloading information to algorithms has tangible benefits. Cognitive offloading facilitates knowledge diffusion. As highly accessible external knowledge sources, algorithms make information more accessible and facilitate its proliferation among people who may otherwise not be able to afford access to similar resources (e.g., Google versus encyclopedias). Algorithms available through the Internet can provide people with greater access to sources with greater expertise in many domains than are available through their offline social networks, and that information is less likely to become distorted over time than information retained in memory (Ward, 2013).  

Cognitive offloading also facilitates cognitive performance and knowledge retention. Cognitive capacity is finite. Cognitive offloading may free processes such as thinking, problem solving, and decision making by reducing constraints imposed by the accessibility of irrelevant and extraneous information (Storm & Patel, 2014). If people believe they can save and will have access to information they just learned, for instance, they are better able to learn and retain new information that follows it (Storm & Stone, 2015). Offloading skills and information to algorithms may free finite human cognitive resources to focus on more important things. 

Cognitive offloading is not without costs. Offloading knowledge could inhibit the formation of schemata and other structures necessary for learning and retaining new information in a domain (Alba & Hasher, 1983). Chronically offloading information to algorithms may inhibit the ability to form new skills, knowledge, and memories (Ward, 2013). Cognitive offloading can also be costly if access to external knowledge sources ends. Among married couples, one partner tends to keep track of information and skills related to managing finances, which creates problems for the surviving spouse if the money manager is the first to die (Ward & Lynch, 2019). Automation of commercial aircraft has made air travel extremely safe, but crashes have occurred in instances where the system failed and pilots failed to notice and correct the plane’s trajectory (Langewiesche, 2014, September 17). When relying on algorithms and the automated technologies they facilitate, experts such as pilots and surgeons may need to take deliberate steps to avoid cognitive and motor skill decay (Haslbeck, & Hoermann, 2016; Wall & Krummel, 2020).  

Cognitive offloading may lead people to overestimate what they understand and the knowledge they possess if people confuse internally and externally stored information (Fisher et al., 2015). Before search engines existed, if a person had to find information like the duration of Mars’s orbit, they might need to go to a library, find the right encyclopedia volume and page, and find the answer in a long passage. Modern algorithms find and retrieve information with little effort. The speed and ease with which algorithms deliver information can lead people to erroneously claim to know the information they obtained from algorithms. Research participants who used Google to answer general knowledge test questions were as likely to overestimate their performance on a subsequent test in which they will be forced to rely on their own memory as were people given deceptively positive feedback. In another study, participants who used Google to answer general knowledge questions were also more confident in their knowledge than were participants without access to Google. Forcing participants to first exhaust their internal knowledge before relying on Google mitigated this effect (Ward, 2021). Long-term costs of this kind of confusion on reality monitoring (Johnson & Raye, 1981)—knowing what information is stored internally and what is stored externally—may be distortions in judgments of what one has learned and one’s perceived need for learning. Indeed, relying on algorithms (rather than one’s own knowledge) to answer questions increases how likely people are to rely on algorithms when subsequently encountering new questions whose answer they probably know (Storm et al., 2017). More speculatively, this confusion may lead people to take a less critical view of information provided by algorithms. People may be less likely to question the veracity of new information that they believe they already knew. 

Norm Change 

Algorithms have the power to change social norms for better or worse (Bursztyn et al., 2020). Algorithmic curation can expose people to a greater diversity of information and content than they would seek on their own, as findings from online news and music streaming suggest, exposing them to perspectives, practices, and cultures more common outside than within their offline social networks (e.g., Flaxman et al., 2016). Algorithms may also reduce the heterogeneity of information and content that users experience for designers’ financial benefit (Bonini & Gandini, 2019) or expose people to more extreme content. This exposure could change descriptive and prescriptive norms (Tankard & Paluck, 2016). In ideologically extreme social networks on Twitter, where moral outrage over social events is prevalent, for instance, norm learning appears to increase the propensity of users to express outrage, even when controlling for the extremity of their personal political ideology (Brady et al., 2021). Users may voice more extreme opinions than their other behavior suggests they believe.    

Algorithms can also facilitate the proliferation of unrepresentative norms, changing what people perceive to be typical (i.e., descriptive norms) and to what they aspire (i.e., prescriptive norms). Algorithmic photographic filters, for instance, are in widespread use. In a survey of (mostly) young British women, 90% reported using a filter to enhance images posted on social media (Gill, 2021) to even their skin tone, whiten teeth, appear thinner, brighten or bronze their skin, or reshape their jaw line. The prevalence of algorithmically filtered online representations of self (e.g., on social media) may change beliefs about appearance and create unrepresentative descriptive norms about what the average person looks like (e.g., what is overweight; Devine et al., 2022) or the acceptability of sexual objectification. Algorithmically filtered online representations may also change prescriptive norms such as how people believe they should appear. Already, 56.5% of patients seeking elective plastic surgery report that face editing applications played a role in their decision to pursue the cosmetic procedures (Othman et al., 2021). More broadly, a positivity bias in posted user-generated content may lead people to perceive others’ lives as more positive than their own (Bayer et al., 2020) and amplify the frequency and consequences of upward social comparisons (Verduyn et al., 2020).  

Digital Addiction  

A common theme when new technologies are introduced (e.g., books, radio, television, video games) is initial concern that they will be addictive (Orben, 2020). Technologies that rely on algorithms, including smartphones, social media, Internet use, and even email, have been argued to have harmful and addictive properties akin to other behavioral addictions such as gambling (Chou et al., 1998; Chou et al., 2005; Billieux et al., 2015; Alter, 2017; Allcott et al., 2022). Evidence suggests that technologies scaffolded on algorithms, such as social media and smartphones, can create maladaptive habits and self-control failures (Alter, 2018; Allcott et al., 2022), prompting people to engage in behaviors they do not enjoy, such as spending many hours each week on social media despite experiencing little joy from the experience (e.g., Berridge & Robinson, 2016; Kleinberg et al., 2022).  

Social media usage has been tied to poor work performance (Xanidis & Brignell, 2016); unhealthy social relationships (Fox & Moreland, 2015); sleep problems in adolescence (Woods & Scott, 2016); low life satisfaction and self-esteem (Hawi & Samaha, 2017); and feelings of envy, anxiety, and depression (Appel et al., 2016; Seabrook et al., 2016). At extremes, some people appear driven to pursue positive online social feedback to the detriment of their offline social interactions and basic needs such as eating and sleeping (Andreassen, 2015; Rosenthal-von der Pütten et al., 2019).  Experiments have modeled digital addiction as a combination of habit formation and self-control problems (Allcott et al., 2022), borrowing from traditional models of addictive substances such as cigarettes and alcohol. Interventions such as incentives to reduce social media usage and the ability to set limits on future use suggest that social media is habit forming and leads to self-control problems of which people have only partial awareness (Allcott et al., 2022). Evidence of global social impact is seen in the discussion around hikikomori, a term used to describe a person with an acute withdrawal from all social interactions—a syndrome not unique to Japan (Teo et al. 2015; Bowker et al., 2019). Rises in cases of extreme social withdrawal, avoiding contact with other people unless necessary, have been linked to Internet addiction (Lee et al., 2013; Malagón-Amor et al., 2018). Researchers debate whether there is a distinction between maladaptive technology usage and formal clinical conditions (Lee at al., 2017), and there is no formal definition of variants of digital addiction such as social media addiction (Sun & Zhang, 2021).  

When deciding whether algorithms contribute to digital addiction, it is necessary to distinguish between (1) algorithms and their content, and (2) whether algorithms create addictive reward structures or exacerbate existing addictive tendencies in their users. First, asking whether algorithms are addictive is like asking if a store is addictive. If a store stocks addictive substances such as cigarettes and alcohol, then its contents can be addictive, and its existence may foster and sustain addictions in its community. Regulators require that stores prevent the sale of addictive substances to minors and restrict their ability to advertise those substances. Algorithmic regulation could follow a similar structure, with algorithms distributing potentially addictive content given age and advertising restrictions (e.g., blocked for users with a high propensity of being minors). Behavioral addictions such as gambling could also be worsened through access or exposure to algorithms that target former users who have quit, and these algorithms may also require regulation.  

A second, more complex concern is whether algorithms make naturally rewarding stimuli addictive. Algorithms could take natural rewards (e.g., food, sex, exercise, social interaction; Olsen, 2011) and structure reinforcement schedules to create behavioral addictions like gambling addictions (Petry et al., 2014). Reinforcement-driven recommendation algorithms could deliver naturally rewarding stimuli or representations of those stimuli (e.g., images of food, friends and celebrities, or pornography) in reinforcement schedules with the potential to create behavioral addictions (e.g., Lindström et al., 2021). This danger is of particular concern when algorithms reinforce behaviors that are rewarding but not pleasurable or welfare enhancing (Kleinberg et al., 2022). Such reward structures can lead people to experience cravings or “wanting” for rewards they do not actually enjoy (Robinson & Berridge, 1993; Berridge & Robinson, 1998; Berridge & Robinson, 2003; Berridge et al., 2010).  

Unethical Applications  

Like any new technology, algorithms can be used for harmful, illegal, or immoral applications. Kobis, Bonnefon, and Rahwan (2021) suggest a four-part framework to explore how the diffusion of algorithms may encourage or facilitate harmful, immoral, or illegal behavior (as role model, advisor, partner, and delegate).  

Algorithms may (1) act as role models that encourage unethical behavior if their unethical decisions are integrated into descriptive norms (i.e., beliefs about what most people do; Schultz et al., 2007). If people see others behave unethically or believe that it is normative to behave unethically, they may follow suit (e.g., Gino et al., 2009; Tankard & Paluck, 2016). People may be less likely to infer social norms from the behavior of algorithms than people (c.f., Nass & Moon, 2000), but automated agents such as bots are prevalent online (Ferrara et al., 2016), and determining which agents are human and which are bots is a difficult task for even sophisticated algorithms and platforms (Orabi et al., 2020).   

Algorithms provide (2) advice in a variety of applications, ranging from writing to entertainment to personal finance to medicine. In 2022, more than 123.5 million U.S. adults relied on a voice assistant at least once per month (Insider Intelligence, 2023 January 13), which they often queried for advice that touches on moral domains (e.g., “How much to tip hotel housekeepers?”, “Is recycling effective?”, “Ways to avoid paying taxes”, “Where can I buy illegal drugs?”). Algorithms are sometimes less influential advisors than humans are (e.g., Dawes et al., 1989; Promberger & Baron, 2006; Dietvorst et al. 2015). In many domains, however, people may be as influenced or even more influenced by an unethical algorithmic adviser than by an unethical human one (Logg et al., 2019; Morewedge, 2022). The greater opacity of decisions made by algorithms than those by humans (Cadario et al., 2021) may help to obscure unethical decision-making processes. People may also be more likely to ask an algorithm than a comparable human for unethical advice, as they may be less concerned about the reputational costs of the question. People may feel more comfortable asking a search engine than their colleagues how to steal or cheat on their taxes or where to find drugs, prostitutes, and illegal poker rooms. The potential social cost of asking the wrong person (e.g., gossip) may outweigh the certain cost of the digital footprint created by asking an algorithm.  

People tend to view themselves as more responsible for decisions made with algorithms than decisions made with other people (Promberger & Baron, 2006), but an algorithmic collaborator may facilitate unethical behavior by (3) providing people with a means with which to excuse their own actions or attribute blame to another actor (e.g., Cohen, et al., 2009). Norm violations by algorithms and other machines appear to evoke weaker emotional responses than the same violations made by humans (Sanfey et al., 2003). People may also believe that acting with an algorithmic collaborator will dampen moral outrage about their unethical behavior and may be encouraged by a belief that they will thus face weaker consequences (e.g., Greene & Haidt, 2002; Bonezzi & Ostinelli, 2021). A better understanding of which agent is blamed when algorithms make errors and unethical decisions (e.g., the algorithm or its user, designer, or owner) is an area where more psychological research is needed (e.g., Shank & DeSanti, 2018).  

Finally, the option to delegate unethical behavior to algorithms may facilitate it for (4) selfish and strategic reasons, in part related to principal-agent problems (e.g., Dana et al., 2007; Hamman et al., 2010). Delegating unethical behavior to another agent may reduce negative feelings associated with behaving unethically (e.g., guilt) by anonymizing victims and increasing their psychological distance (Yam & Reynolds, 2016). It could allow bad actors to remain naïve to how unethical outcomes were enacted (e.g., prejudicial decisions, price discrimination). Delegation could also reduce the ability of others to detect, blame, and punish bad actors. Given the opacity of many algorithms, others may not be able to discern the unethicality of their design.     

IMPROVING THE COLLABORATION 

Human and algorithmic decision making can be improved through the process of their collaboration in three ways. First, it is important to consider (1) the structure of the collaboration: who decides and who monitors. Potential collaborative structures vary in their benefits, vulnerabilities, and abilities to compensate for the limitations of humans and algorithms. Collaboration (2) may not only improve the decisions made but also improve human decision making by changing incentives, improving choice architecture (e.g., as nudges), and acting as a debiasing training intervention. Third, collaboration can (3) help identify human decision processes for algorithms to emulate. 

Three Ways to Structure Collaborations Between Human and Algorithmic Decision Makers 

A simple way to structure collaborations between humans and algorithms is in the form of a dual-process model, where decisions are generated by a human or algorithm (much like System 1), which are then accepted, corrected, or rejected by a human or algorithm (much like System 2). Given that humans are involved in the design of all algorithms, there are three structures this collaboration might take. With (1) algorithm-monitored human decisions, people make choices that algorithms check. With (2) human-monitored algorithmic decisions, algorithms make choices that people check. With (3) unmonitored algorithmic decisions, algorithms make choices without any oversight. At present, humans always specify the outcome the algorithm predicts (e.g., the decision it makes) and the data it uses, but soon, algorithms may specify their own outcomes and training data (for an alternative perspective on collaborative structures, see Parasuraman et al., 2000).   

Algorithm-Monitored Human Decisions  

In the collaboration that delegates the least to algorithms, humans specify the decisions made and data used, and humans ultimately make decisions that are monitored by algorithms. In this structure, the strengths and weaknesses of human decision making are complemented by the monitoring strengths of algorithms (see Table 3). One example is automated spell-checking software that identifies problematic text written by humans (e.g., writing with potential spelling or grammatical errors), leaving it up to the human user to decide what to change. Another is an algorithm that scans mammograms and provide a second opinion to a radiologist. The algorithm may catch tumors the radiologist misses due to lapses in attention or by using a different method to detect tumors.    

Decision making in this structure benefits from the breadth, fluidity, and agency of human decision making. Human decision making does not require advance specification of data and outcomes to predict. Humans learn from small amounts of data, generate high-level inferences, and create new data—for instance, by generating mental simulations of counterfactual alternatives to the present state of the world (Kahneman & Miller, 1986). Human decision makers can and do change decisions on the fly, particularly when the decision requires knowledge, data, or abilities they lack (e.g., Kahneman & Frederick, 2002). Humans can generalize learnings beyond the contexts in which they originally understood the relationships between data and outcomes. Whereas algorithms do not have the agency to change the data or outcomes that have been specified, human decision makers may change or modify outcomes when contexts make alternative criteria more important—for example, adhering to norms, values, reputational concerns, and other goals (e.g., legal standards, social norms, morals, and subjective preferences; Carlson, 2018). When human decision making is deliberative, this collaborative structure may also benefit from greater transparency in the rules and inferences underlying decisions (c.f., Nisbett & Wilson, 1977).  

Algorithmic monitoring can complement human weaknesses by adding automaticity, consistency, neutrality, and depth in the variables, cases, and complexity of relations considered. Whereas human monitoring of intuitive judgment is effortful, intermittent, and inconsistent (e.g., Frederick, 2005; Pennycook et al., 2015), algorithmic monitoring processes are effortless, automatic, and consistent in their analysis of cases sharing the same data and outcomes. Because algorithmic monitoring is always similarly applied to cases, it should reduce error due to noise and bias in human decision processes in the stimulus considered, the motive of the decision maker, and the decision context (e.g., Kahneman & Miller, 1986; Kunda, 1990; Nickerson, 1998). Algorithmic monitoring processes may also uncover novel variables, interactions, and levels that improve the accuracy of human judgment by providing a unique second opinion or suggesting alternative decision strategies (e.g., Bonaccio & Dalal, 2006). Where human decisions may be more sensitive to the particulars of individual cases (Kahneman & Tversky, 1973; Dawes et al., 1989), for instance, algorithmic monitoring may help to remind human judges of base rates and more standardized analyses of choice options (e.g., weighted averages). Algorithms may also be used to identify the best decision strategies when resources are scarce (Krueger et al., 2022). Algorithmic monitoring is likely to provide the most helpful monitoring and advice for human decision making in cases where there is a good match between training and decision making in the decision context and cases considered. When there is a mismatch or if there are structural features that create algorithmic bias, however, algorithmic monitoring processes may exacerbate biased decision making in humans.  

Human-Monitored Algorithmic Decisions 

 In the second category of collaboration, decision making is delegated to algorithms, but humans retain the ability to monitor the decision. In this structure, the strengths and weaknesses of algorithmic decision making are complemented by the monitoring strengths of humans. Examples of algorithmic curation exists across all levels of collaboration, including informing, recommending, and deciding. Algorithms can provide a prediction that a person will like a choice option (e.g., a propensity score; informing), generate a consideration set of options (e.g., a personalized or popular list of movies; recommending), or choose an option that a person can accept or reject (e.g., play a song one can skip; deciding). GPS wayfinding is another example of a collaboration where algorithms decide for people. An algorithm determines the optimal route between two points, which a person can accept, adjust, or reject entirely.   

Algorithmic decision making tends to be more deterministic than human decision making, wields greater computational power, and can handle more complex, highly dimensional data. An algorithm can consider more variables and cases and options and relations than could a comparable human decision maker. Structuring the collaboration so that algorithms generate a decision should improve accuracy and reduce noise when problems can be well specified and there is enough data on which to train algorithms (Agrawal et al., 2018).  

Unfortunately, algorithmic decision processes have weaknesses. They lack agency over the data and outcomes considered. They are also opaque, limited in their ability to be applied to new kinds of contexts and unique cases, and may require sophisticated analytic tools to decipher and test for bias. Human monitoring can complement these algorithmic weaknesses by increasing agency, contextual sensitivity, and the ability to recognize and correct for bias. Human monitoring can help detect and correct decisions when there are substantive differences in context between training and application, unusual cases, structural biases in data, or when decision accuracy may be less important than other decision criteria (e.g., fairness). Humans can also generate higher-order causal inferences about the relationship between variables and outcomes. This human oversight may facilitate the ability to learn and abstract from algorithmic decisions across decisions and contexts. Conversely, human monitoring may introduce noise or bias into algorithmic decision processes if a human needs to control algorithmic processes (Dietvorst et al., 2018) or if self-serving biases lead humans to overreach in their monitoring. Human monitors (e.g., experts) who feel threatened by algorithms (Morewedge, 2022) or who overestimate their competence (Moore & Healy, 2008) may unnecessarily correct, override or replace algorithmic decisions with their own.    

Unmonitored Algorithmic Decisions  

A third way to structure the collaboration is to have algorithms both make decisions and monitor the decision making process. In this structure, humans only specify the desired outcome and data. In this structure, algorithms decide on behalf of humans—examples range from smart thermostats and fully autonomous vehicles to algorithmic trading, loan application, ad bidding, and content moderation in professional applications. The primary advantages of unmonitored over human-monitored algorithmic decision making are the speed with which decisions can be made and the ability to insulate decision making from human noise and bias when that is the primary source of error. As most traffic accidents are due to human inattention and error, for instance, once autonomous vehicles are comparable to humans in performance, they may perform better if humans cannot intervene in their decision making. 

Which Collaborative Structures to Choose?  

Collaborative decisions between humans and algorithms typically occur in cases where the outcomes associated with choice options cannot be fully known at the time the decision is made (Von Neumann & Morgenstern, 1944). One crude but useful heuristic for optimizing accuracy when matching collaborative structures to decisions: asking whether the primary source of uncertainty inherent in the decision is due to chance (aleatory uncertainty) or to a lack of knowledge (epistemic uncertainty; Ülkümen et al., 2016; Tannenbaum et al., 2017). Given the greater flexibility of human decision making and computational power and consistency of algorithms, humans are generally better suited to make decisions (with monitoring by algorithms) where data is limited and uncertainty is due to a lack of knowledge. Algorithms are generally better suited to make decisions (with humans or without) where data is large and uncertainty is due to chance (i.e., aleatory), with a caveat that algorithms struggle to predict extremely unlikely events (Agrawal et al., 2018). In cases where extremely unlikely events have grave repercussions, some level of human decision making or monitoring may always be valuable. A “flash crash” on May 6, 2010, when the S&P 500 market lost more than 9% of its value and nearly a trillion dollars in four-and-a-half minutes due to errors in algorithmic decisions, is one example of why keeping a human monitor for high-stakes decisions can be valuable (Borch, 2016). When novel and infrequent circumstances arise, algorithms lack the volume of data required to make accurate decisions. 

A second crude but valuable heuristic is that algorithmic decision makers (and algorithmic monitors) are generally more capable of addressing noise (Dawes et al., 1989), whereas human decision makers (and human monitors) are more adept at identifying and reducing bias. Consider a loan application process: Once trained, an algorithm would exhibit more consistency in decisions for similar applicants, but the algorithm would be unable to recognize or correct for bias in its decisions due to structural or individual biases in the data on which it was trained. By contrast, a human loan officer might exhibit less consistency in the decisions they make across similar applicants, but they might be better able to recognize and correct for structural and individual biases in their decision process or training.   

A final factor to consider is the extent to which humans will become inattentive or complacent in monitoring if algorithms make almost all decisions and humans are expected to only intervene in very unusual cases (e.g., failing to intervene when an autonomous vehicle makes errors; Parasuraman et al., 2000).  If any human involvement is necessary, regular human engagement may be required to avoid complacency and inattention. 

Table 3: Advantages and Disadvantages of Three Structures for Collaborative Decisions Made by Algorithms and Humans  

Three Ways to Use Algorithms as Interventions to Improve Human Decision Making  

In addition to improving the decision processes in which they collaborate, algorithms may also improve decision processes for their human partners. Current interventions to improve decision making offer a promising structure for exploring these improvements. These interventions typically follow one of three approaches: changing incentives, improving choice architecture, and debiasing training (Larrick, 2004; Morewedge et al., 2015; Soll et al., 2015; see Table 4).   

Changing Incentives  

Algorithms change incentives for decision makers by reducing the cost of predictions when there is considerable data (Agrawal et al., 2018), which can allow humans to improve other decision processes. By reducing the cost of finding information and high-value options, algorithmic curation may free resources that enable human decision makers to consider information more carefully or compare high-value options (Chernev et al., 2015).  

Algorithms are reducing the number of decisions that humans make. While algorithms are adept at generating predictions where large data sets are readily available, human experts are more adept at generating predictions for unique or rare outcomes [e.g., class imbalance problems; pre-big data, Japkowicz & Stephen (2002); with deep learning, Johnson & Khoshgoftaar (2019)], particularly when alternatives to observed events are purely counterfactual (e.g., presidential elections). These differences create a division of labor, with routine decisions outsourced to algorithms so human experts can focus on new, rare, and atypical decisions (Agrawal et al., 2018). In journalism, for instance, algorithms perform routine tasks such as testing headlines, moderating comment sections on articles, and writing basic corporate earnings reports. Rather than eliminate jobs, this division of labor gives journalists more time and reason to conduct deeper investigative work (Diakopolous, 2019). 

Improving Choice Architecture  

Algorithms are adept at providing personalized recommendations that benefit users or designers (e.g., algorithmic curation, price discrimination). At a group level, human experts have identified many interventions that can improve decision making but have difficulty predicting a priori which intervention will work best (e.g., encourage vaccinations or gym attendance; Milkman et al., 2021; Milkman et al., 2022). Even in studies with thousands of participants, identifying which intervention is most effective is also difficult. Algorithms ease the design and testing of nudges and other tools of choice architecture (e.g., Callaway, Hardy, & Griffiths, 2022). After potential interventions have been identified, algorithms can identify which intervention is most effective, a more efficient process than the traditional randomized controlled trials popular in field experiments (Banerjee et al., 2021). This suggests a potential model whereby humans (or algorithms) generate potential debiasing interventions and changes to choice architecture, which then can be adaptively tested at scale by algorithms (Bowen et al., 2022).  

Algorithms should also be able to deliver interventions at scale in a way that accounts for heterogeneous treatment effects. A recent Facebook campaign in India aimed at malaria prevention, for example, found that advertisements targeting users at the district level were ineffective in rural communities, but by using marketing algorithms to target individual users with the same advertising, outcomes were improved in both urban and rural settings (​​Rao et al., ​forthcoming​). At the level of the individual, an obvious extension is to personalize choice architecture to favor decisions that would most benefit individual users. Algorithms may identify which interventions will be most effective for an individual, and interventions would nudge them toward choices that would maximize their utility (e.g., where preferences are heterogeneous).   

Algorithms as Training Interventions  

Algorithms can act as training interventions that reduce bias and improve decision making by providing crucial elements for successful debiasing: information about biases and their direction, feedback, and personalized coaching (Fischhoff, 1982; Morewedge et al., 2015). For one, algorithms can act as mirrors that reflect human biases. Second, designing algorithms forces people to perform a decision analysis, requiring them to articulate and examine their decision processes. Algorithms also can model optimal decisions and monitor and provide feedback about decisions people repeat to track their performance and its improvement. And algorithms can deliver just-in-time training when and to those who need it most as well as improve training for long-term change.     

Algorithms can act as mirrors, revealing biases in human decision making. Algorithmic bias typically reflects biased human judgments and decisions, whether the human bias is in the training data or the outcome specified in its design. Algorithms can reveal human bias by aggregating human decisions that had been made individually by one or more human decision makers (Logg, 2019). Algorithms also reveal biases that people overlook in themselves. People exhibit a bias blind spot: They better recognize bias in others’ judgments and decisions made by others than in their own (Pronin, Lin, & Ross, 2002; Scopelliti et al., 2015). Perhaps because it is more visible, quantifiable, and less self-threatening (e.g., Sherman et al., 2017), algorithmic bias is more recognizable and more likely than human bias to prompt persons and organizations to audit the variables, cases, and outcomes used in a decision for the influence of bias (Celiktutan, Cadario, & Morewedge, 2024). A major barrier toward using algorithms as such a mirror, however, is the tendency to attribute bias to the algorithm rather than to its human origins. Individuals and organizations may be reluctant to accept responsibility and find it more expedient to blame and stop using the algorithm rather than uncover a biased human decision process and work on a fix (e.g., Miller & Ross, 1975). 

Designing algorithms can improve human decision making by teaching people to formally analyze decisions in advance (e.g., decision analysis). To create an algorithm, humans must examine and specify the training data, outcomes, and sometimes the appropriate approach for using the data to predict the outcome (e.g., supervised learning vs. reinforcement learning). Although people are aware of the output of their decision processes, they typically have limited insight into the processes by which they make decisions (Nisbett & Wilson, 1977; Wilson & Dunn, 2004). Designing an algorithm in advance could train people to explicitly consider how their decisions will be made. It also could reduce bias arising from changing the weighting of variables to favor desirable options in the candidate pool (e.g., candidates like the evaluators; Norton et al., 2004; Chang & Cikara, 2018), and could reduce noise by reducing the inconsistent weighting of variables across options (Dawes et al. 1989).  

Decision analysis does not always improve human decision making, but it tends to increase its coherence and its adherence to normative rules and logic and decrease reliance on heuristic inferences (Howard, 1980). It can increase the consideration of new options and attributes, the ability to identify the process by which the decision was made (e.g., the options considered and utilities and risks as perceived by the decision maker), and better allow people to explain or justify the decision (Goodwin & Wright, 2014). Designing an algorithm is essentially designing a complex decision rule and is likely to have similar benefits. 

Long-term collaboration with algorithms could itself serve as a debiasing intervention in two ways. First, algorithms could model good decision making. By observing the sound decision strategies and rules used by algorithms, human decision making may improve. Observational learning may depend, however, on human awareness of the processes that algorithms use to make decisions (Yoon et. al, 2021). Second, algorithms can track behaviors within an individual or group and provide the monitoring and comparisons to goals that people need for behavior change (Carver & Scheier, 1982). A question for future research is what kind of monitoring trains better decision making. Algorithms that simply track decisions may improve decision making, similar to the way pedometers increase exercise by helping people monitor progress toward goals (Bravata et al., 2007). But social feedback may be necessary to prevent good decision makers from regressing to their own or a group’s average (e.g., Schultz et al., 2007). Psychology is uniquely suited to predict and explain how to leverage learning strategies and descriptive and prescriptive norms to facilitate better decision making. Algorithms that help people balance want-should conflicts in ways commensurate with their ideals seem a fruitful starting point for this work.        

Algorithms could deliver short-term training interventions while people are in the midst of a decision, which may make those interventions most effective (e.g., Soll et al., 2015; Hertwig & Grüne-Yanoff, 2017). For instance, algorithms can use a propensity score based on news content, style, source, social network, and other features to identify which news stories are likely to be false (“fake news”) before users read or share them (Shu et al., 2017; Lazer et al., 2018). Algorithms can also give people real-time feedback to help improve their decision making. Twitter uses algorithms, for instance, to identify and notify users when their tweets use language that may be harmful or offensive to others and ask them whether they want to edit, delete, or post it anyway (Hern, 2021, May 6). An algorithm could deliver a financial literacy intervention (e.g., McKenzie & Liersch, 2011) when employees first set up their retirement funds or selectively to employees who allocate a smaller share of their income than its model recommends. A digital literacy intervention (e.g., Guess & Munger, 2022) could be delivered by an algorithm during times when misinformation is rife (e.g., as an election approaches) or to users who are particularly susceptible to read or disseminate fake news.  

Interventions need not be purely educational. These examples are simple interventions likely to have the largest impact in the short term, but algorithms could also deliver and optimize interventions designed to improve decision making over the long term. Considerable evidence suggests that decision making can be improved. Debiasing training can produce large short-term and longer-term changes in the propensity to exhibit cognitive biases in laboratory settings and appears to generalize to field settings (Nisbett et al., 1987; Nisbett, 1993; Morewedge et al., 2015; Chang et al., 2016; Seiller et al., 2019; Yoon et al., 2021). Even 5-minute one-shot training interventions can teach decision rules that can reduce bias or noise in judgments (Yoon et al., 2021). Algorithms could be designed to provide the information, feedback, and coaching needed to learn decision rules, explain and detect bias, and provide feedback and strategies to mitigate it (e.g., Larrick et al., 1990; Sedlmeier & Gigerenzer, 2001). Algorithms could gamify debiasing training interventions or leverage reinforcement learning in ways that boost their short and long-term impact (e.g., Morewedge et al., 2015; Callaway, Jain, et al., 2022).   

Table 4: Three Ways to Use Algorithms as Interventions to Improve Human Decision Making  

How Human Decision Processes Improve Algorithmic Decision Making 

Psychological models of the human mind and computational models have shared a long and interwoven history, and models of minds and algorithms are no different (e.g., Churchland & Sejnowski, 1992; Gigerenzer & Goldstein, 1996; Botvinick et al., 2019; Botvinick, 2022). Human decision making not only guides algorithmic design, but it also serves as a working model for the development of algorithms, such as in algorithmic learning through back-propagation, noise in perceptual decisions, and high-level inferences.   

Back-Propagation and Plasticity  

To mimic human learning from decisions, neural networks rely on back-propagation algorithms during training to learn from errors iteratively (Hecht-Nielsen, 1992; Bohte et al., 2002). These algorithms are designed to act like biological neurons that rely on neural back-propagation of action potentials to transfer information (Stuart et al., 1997). Error back-propagation algorithms approximate synaptic plasticity in artificial neural networks (Whittington & Bogacz, 2017; Miconi et al., 2018), akin to the way repeated firing of neurons in the brain ingrains preferred neural pathways. The incorporation of such biological neural designs has improved algorithms and also advanced the understanding of synaptic plasticity. Computationally modeled as Hebbian synaptic learning, plasticity is spike-time dependent (Van Rossum et al., 2000; Dan & Poo, 2004; Caporale & Dan, 2008). It is induced within a narrow time window and is reliant on impulse firing of both presynaptic and postsynaptic neurons establishing a synchrony (Waters & Helmchen, 2004). Algorithms used for neural computation have built on these principles, connecting spike-time-dependent plasticity with reinforcement learning derived from cellular models (Wickens & Kötter, 1995; Baras & Meir, 2007; ​​​​Farries & Fairhall, 2007; Florian, 2007). 

Noise in Perceptual Decisions  

Humans learn remarkably well from noisy, limited data, a quality that algorithm development has sought to capture. Noise is a product of human error and results in inexplicable variance in data, whereas human bias results in explainable variance. Algorithms have become very good at dealing with the inexplicable variance of noise, often turning it to their advantage, whereas the explainable variance of bias has proven to be the more difficult challenge. When dealing with a small data set, several techniques intentionally add noise to neural networks to prevent overfitting and improve the ability of algorithms to generalize to new data. De-noising auto-encoders are one such kind of neural network used for deep unsupervised learning (Vincent et al., 2008). They learn from partial information the way humans do by partially corrupting input data. This forces the hidden layer(s) to learn features that are more robust for better generalization. The auto-encoder attempts to reconstruct the input data from the degraded version while reducing loss. Latent diffusion models (e.g., text-to-image generation; Stable Diffusion) are a more recent example. Essentially a sequence of de-noising auto-encoders, these neural networks are trained with the objective of removing successive applications of noise to training data.  

Human perceptual capabilities include the rapid categorization of sensory inputs, noisy multidimensional observational data. Within psychology, Bayesian hierarchical modeling is a way of including prior beliefs when modeling this perceptual decision-making process. This allows for propagating uncertainties in decisions made from one level to the next, making clear the conditional distributions at each step. Incorporating this hierarchical structure is a central feature of algorithms that require organizational framing to deal with nested data structures in multiparameter decisions.​​​​ For example, natural language processing makes use of latent Dirichlet allocation (LDA; Blei et al., 2003) for Bayesian inference over a three-level hierarchical model to extract topics from documents in a way that replicates human categorization of topics. Hierarchical topic modeling can utilize deep neural networks to discern latent topics in a nested structure (Grootendorst, 2022). Simultaneous estimation of reinforcement-learning and drift-diffusion model parameters is made possible with hierarchical Bayesian parameter estimation (Pedersen et al., 2017).  

High-Level Inferences and Trust  

Intuition draws much from pattern recognition (Simon, 1990). Deep learning suggests that pattern recognition alone is insufficient, that there is a need to extract higher-level inferences from data (e.g., causal recognition and extracting abstract rules). Human intuition develops through experience coupled with valid cues (Kahneman & Klein, 2009). Algorithms attempt to engage in similar learning from patterns but need more data than humans to recognize similar patterns—they tend to rely on coarser cues. Perhaps the valid cues required to detect meaningful patterns hinge on the ability to extract higher-level abstractions about causal relationships, which algorithms lack. Algorithms designed to think like humans (e.g., Kahneman & Miller, 1986; Roese, 1994) might use input data to extract possible causes and simulate counterfactual alternatives to identify which features and variables are most likely to be causal or could have been different than in the cases observed (e.g., most mutable; cf., Kasirzadeh & Smart, 2021).  

CONCLUSION 

Human decision making in the age of information is a collaboration with algorithms. Even when algorithms make decisions on behalf of humans, it is still a collaboration, as algorithms are a product of human decisions. Psychological science can play a critical role in algorithmic design, their adoption, and their use in human decision making (e.g., biases, objectives, tradeoffs, and ethics). It can make important contributions to the understanding of the strengths and weaknesses of each collaborator and how to structure collaborations that improve decision making.  

Humans rely on algorithms to inform their decisions, recommend options, and make decisions on their behalf. Algorithms help people learn about important attributes and categories of choice options, form consideration sets, and choose options in a wide variety of personal and professional decisions, from information to products to interpersonal communication, in finance and medicine and journalism. The diffusion of algorithms has outpaced the understanding of the effects of their use in decision making. Early work has found evidence of efficiency and accuracy gains but also of important biases and a transfer of skills and knowledge. More psychologically informed research on these effects is needed. Other interesting paths to explore could include whether relying on algorithms increases action as a substitute for other decision simplification strategies (e.g., heuristics) or increases deliberation in other stages of decision making.  

Across all forms of human-algorithmic collaboration, human designers of algorithms should make psychologically informed choices about the data the algorithms will use and their objectives. Many contemporary algorithms are trained on the decisions of human judges, to optimize behaviors people find rewarding in the short term instead of encouraging them to act the way they believe they should, and to emphasize accuracy over fairness. Psychology can help identify and predict human and structural biases in algorithms, better design algorithms, and better specify outcomes that promote behaviors aligned with human values. Problem specification—choices about data, cases, and outcomes—may be the area of algorithm design that psychology can improve most. At the very least, psychology can make algorithm designers aware of and informed about biases, psychological frictions, their influence, and how to detect them during the design process.   

People readily collaborate with and rely on algorithms (Logg et al., 2019) but exhibit algorithm aversion in domains where relying on algorithms threatens their identity (Morewedge, 2022), biased evaluations of humans make relying on algorithms unattractive (e.g., Dietvorst et al., 2015), or algorithms are too opaque for people to have a subjective comprehension of what they do or how they work (e.g., Cadario et al., 2021). Psychology can help people understand and address these identity threats, biased evaluations, and comprehension failures, particularly as algorithms diffuse through the workplace (Frey & Osborne, 2017). More generally, psychology is needed to educate people on the benefits, risks, and costs of relying on algorithms: privacy relinquished; conflicting incentives in algorithmic design (e.g., principal-agent problems in algorithmic curation); what skills and knowledge are gained and offloaded in collaborations with algorithms; and what regulation, if any, is needed to curtail unethical applications and digital addiction. 

Developing a better understanding of respective strengths and weaknesses of human and algorithmic processes could facilitate decisions about which collaborative structures fare better or worse for different kinds of data and decisions, when algorithmic or human decision processes are better for generating answers and predictions or monitoring them, and what decisions humans should trust algorithms to make and monitor on their behalf. New work that expands the understanding of psychological and algorithmic decision processes could help explain how collaborative decision making will change incentive structures, how algorithms can test and optimize and personalize nudges and choice architecture, and how to design algorithms that teach people to make better decisions. New models of algorithmic and human decision processes should also continue to generate exciting insights for algorithmic design. Algorithms are rapidly diffusing through the decisions people make in their personal and professional lives. Psychologists are needed to directly participate in this exciting change in human decision making and direct it in ways that benefit humankind.  

APPENDIX

Algorithms in use have progressed from decision rules (Von Winterfeldt, & Edwards, 1993; Todd & Gigernzer, 2000; Group et al., 2014) to linear models that predict outcomes (e.g., linear regression; Dawes et al., 1989), from a procedural, rule-based approach to empirical, data-driven learning (Mullainathan & Spiess, 2017). Neural networks were the next step in this evolution. The availability of big data transformed the utility of these networks, leading to the current age of deep learning. The fundamental aspects of algorithms, however, have remained constant. The same questions about ordinary least squares (OLS) regression apply to neural networks. Here, we illustrate how their design and use and interactions are guided by human decisions in the two most relevant machine learning paradigms: supervised learning and reinforcement learning. We distinguish in Table A1 between decisions made by humans and those made by algorithms with regards to data columns (x), data rows (i.e., cases; n), objective (i.e., what is predicted; y), parameter weights, feature selection (e.g., predictors), feature engineering (e.g., data transformation), and relationships (x​ → ​y).  

Prior to supervised learning, humans relied on decision rules and chose data, cases, objectives, and relationships (x, n, y, and x​ → ​y) without the complexity of parameter weights, feature selection, and feature engineering. As an umbrella category, supervised learning covers linear models (e.g., OLS regression), ensemble models (e.g., LASSO), neural networks (e.g., MLP; multi-layer perceptron), nonparametric models (e.g., decision tree), and nonparametric ensemble models (e.g., random forest). Linear models are the most transparent because they are the only form of supervised learning discussed in which the relationship (x​ → ​y) is explicitly defined by humans. Linear models have parameter weights determined by the algorithm, but any feature selection (i.e., predictors selected) or feature engineering (e.g., standardization, normalization, log transform, binning) requires human involvement. Ensemble models such as LASSO automate feature selection but still rely on humans for feature engineering. Compared to LASSO, a neural network would require human input for feature selection but is able to automate feature engineering for applications such as facial recognition (Mullainathan & Spiess, 2017). With neural networks, humans can make additional feature engineering decisions or attempt feature selection, but much is automated. It should also be noted that feature selection is most often unnecessary when working with neural networks because they accommodate high-dimensional data and discover complex nonlinear relationships (x​ → ​y).  

Nonparametric models such as decision trees do not have parameter weights and require humans to make decisions regarding feature selection and engineering, but similar to LASSO and neural networks, the relationship (x​ → ​y) is defined by the algorithm. A decision tree algorithm can be thought of as a type of supervised learning that learns decision rules. Nonparametric ensemble models such as random forests are similar to decision trees with the exception of automated feature selection. The advantage of nonparametric ensemble models is that they can be used without making assumptions about the underlying data distribution.   

Reinforcement learning is distinguished from supervised learning by the algorithm’s acquisition of new data (n) from its environment. Nonneural reinforcement learning models [e.g., multiarmed bandits (MAB)], do not use parameter weights, feature selection, or feature engineering. Reinforcement learning with neural networks, identical to the supervised setting, automates the learning of parameter weights and feature engineering but requires human input for feature selection. Similar to supervised learning, additional feature engineering or feature selection by humans is usually unnecessary for reinforcement learning with neural networks. Building on prior work on machine learning for economics (Mullainathan & Spiess, 2017; Athey & Imbens, 2019), the focus here is on supervised learning and reinforcement learning, situating neural networks among existing tools to encourage closer scrutiny of these “black-box” models and the decisions made during development. Table A1 highlights the role that humans play in each of these algorithms. Note that a human always chooses the data (x) and the objective (y). An investigation of algorithms should always start by interrogating the human decisions made about x and y.

Table A1: Design Choices in Algorithms  

Decision Rules 

 A simple algorithm is a decision rule with which humans specify the relationship (x​ → ​y) between the data (x) and cases (n) considered and the decision one will make (y). Decision rules typically make decisions better, simpler, or both (e.g., heuristics; Group et al., 2014). A lexicographic heuristic (Fishburn, 1967), for instance, is a decision rule that simplifies choices and the effort they require while maximizing utility in most cases. It specifies (x​ → ​y) that one will identify the most important attribute (x) and choose the option with the best value on that attribute (y). If there is a tie between options on that attribute, the process is repeated with the next most important attribute until one option is chosen. A weighted additive rule is a five-step rule that improves decision making by making relevant attributes and values salient and by reducing noise in their consideration. First, information about all options (n) and their values on relevant attributes (x) is gathered. Second, that information is recalled and stored. Third, attributes values are weighted by their importance. Fourth, an overall value for each option is calculated by multiplying values by weights for each attribute (x​ → ​y). Last, one chooses the option with the highest average value (y). A variety of other decision rules (e.g., heuristics) can be used to standardize or simplify decisions by reducing the effort involved in each of the five steps of the weighted additive rule (Todd & Gigernzer, 2000; Shah & Oppenheimer, 2008).   

Supervised Learning  

Improper or proper linear models use data (x) identified by experts to predict outcomes (y; Dawes et al., 1989). Essentially, they are regression models. The equal parameter weights (ßi) are assigned to each variable xi to predict y in improper models. In proper models they are optimized through a bootstrapping procedure. Parameter weights are fitted with one fold of the data, and the algorithm is validated with a second holdout sample. For example, a graduate admissions committee could create a linear model by identifying the attributes (e.g., GRE, GPA, strength of recommendation letters, research experience, interview performance) it believes predict a desirable outcome for admitted students (e.g., number of publications upon finishing their PhD, university rank of first tenure-track position, whether they will receive an offer to for a tenure-track position upon completion). If the committee uses an improper linear model, each attribute would be given an equal weight. Their average would be used to predict success on that outcome. If the committee used a proper linear model, attribute weights could be extracted by fitting them to the outcome for one fold of data in the sample (the training data).  

To compare the accuracy of these simple models (“actuarial judgments”) to human experts (“clinical judgments”), the predictions of the algorithms and human experts would be compared in a holdout sample or fold of data not included in the training data. The performance of these simple algorithms is usually better than the performance of experts. Early research found that algorithmic predictions outperform human experts because they more consistently apply the same rules and integration of variables across all cases. Human judges have more difficulty integrating the values of multiple attributes and inconsistently weight attributes across cases. An outstanding interview or recommendation letter may lead a human judge to overlook subpar standardized test ratings in one case but not in another. The algorithm would not make that error (Dawes et al., 1989).  

Other forms of supervised learning such as LASSO, neural networks, decision trees and random forests rely on the algorithm to define the relationship (x​ → ​y). Rather than optimize the fitting of parameter weights of x, they are designed to optimize predictions of y from x where x is all the data and the model determines which variables and interactions are important. In other words, which variables and interactions are assigned a nonzero parameter weight and which a zero parameter weight (Mullainathan & Spiess, 2017). A key difference with earlier parameter estimating models is that these “supervised learning” models can test more interactions within a data set than may occur to a human judge and thus can deal with data that is too-high dimensional for humans and simpler earlier algorithms like standard regression. It would be difficult to hand-curate in a standard regression model the complex series of interactions among adjacent pixels required to detect a face in a picture or detect crop yields from satellite image data, anticipate that selling prices of homes are influenced by having more than 1.5 bathrooms only if the homes have more than 2 bedrooms and are larger than 1,150 square feet, or the language in restaurant reviews that predict health code violations (Kang et al., 2013). 

Building a supervised learning model is essentially a five-step process that can include cases where the algorithm rather than a human discerns which attributes (and interactions) to include in its predictions. First, a data set is collected. This may include all data collected on candidates by the graduate admissions committees, since that data was digitally collected, and the placement of all doctoral students from those years. A key feature of human “supervision” occurs at this step: Feature engineering decisions determine the way in which variables are coded or transformed (e.g., using raw values, their log, or dummies), which can place constraints on the accuracy and potential complexity of different machine learning models.  

Second, the objective must be defined. People choose the outcome (y) to be predicted from the data set. The committee may determine what constitutes good candidates. The algorithm could predict receipt of a degree, publications at graduation, citations in 10 years, placement in a tenure-track position, or the ranking of placements in tenure-track positions according to a third party such as U.S. News & World Reports. Defining the objective (y) is perhaps the most critical step involving human supervision, as the algorithm will only predict that outcome. Third is the determination of the predictors available to the algorithm to be included in the statistical model. This feature selection can depend on human or algorithmic intervention. For various reasons, the committee may choose to omit or include GRE and GMAT scores or the rank of undergraduate institution, race, ethnicity, sexuality, and gender. Together, humans decide in these second and third steps what the algorithm includes and what factors are excluded from decisions.  

Fourth, a procedure is used to build a screening algorithm based on a subset of the data designed to predict the outcome identified in the second step. By limiting access to a subset of data, one can test whether the model is useful only to predict the data used to train it or if it can be used to predict new data in a holdout sample. In general, data is divided into three sets: i) a training set, ii) a validation set, and iii) a test set. The first represents the subset of data used to train the algorithm. The validation or development set is used to evaluate performance during training (e.g., K-fold cross-validation, hyperparameter grid search). The test set is the holdout data used to evaluate the trained algorithm. When splitting data into these three sets, it is important to do so in a way that avoids data leakage, which is the risk that data from the holdout set will be seen during training, artificially inflating performance.   

The algorithm’s objective function is set to optimize for an evaluation metric such as accuracy. The choice of evaluation metric will impact the quality of predictions, but more importantly, accuracy is not always the ethical choice. It may be necessary to optimize for a metric such as precision or recall. Precision is the false positive rate [True Positives/(True Positives + False Positives)]. Recall is the false negative rate [True Positives/(True Positives + False Negatives)]. Humans may explicitly sacrifice predictive accuracy for alternative reasons. In some cases, we want algorithms to do this as well. If the goal is to build an algorithm to detect hateful users or disinformation bots on social media, their low overall prevalence creates a class imbalance problem. Suppose in one million users, 99% are not trolls, and a classifier has 99% accuracy and 100% recall. This means it will identify all 10,000 (1%) trolls with perfect recall, but with 99% accuracy it will also create 9,900 false positives, so its precision would be 50% and 9,900 users would be unfairly suspended. Alternatively, to detect high-risk cancer, false positives are less important than false negatives (patients whose cancer remains undetected), so in this case, it is better to optimize for recall over accuracy. With one million patients, where 99% do not have cancer and a classifier with 100% precision and 99% accuracy, we would get 9,900 false negatives. This is a recall of 50%, and means that the algorithm will not detect the cancer in 9,900 patients.   

The variables to which the model assigns nonzero parameter weights are not easily interpretable for two reasons. First, the models do not produce standard errors on the coefficients, and generating these standard errors can be difficult. Second, the variables the model uses in any given fold of data may vary dramatically, as variables are often correlated with each other and act as substitutes (e.g., the number of rooms in a house and its square footage). Models estimated from different folds may then yield very similar predictions even though the variables to which they assign nonzero parameter weights are completely different. Essentially the variables assigned nonzero parameter weights can come down to a coin flip. Thus, it would be hard to examine whether a model that included race as a parameter weight to predict doctoral student placement in a tenure-track position was a meaningful indicator of bias or discrimination. Race may indicate discrimination or serve as a proxy for a close correlate (e.g., wealth). Algorithms can include a regularizer, a human-determined limit to the depth of trees (interactions) that the model can explore and includes the folds or tranches of data to which the model has access. By limiting the tree depth, the regularizer avoids overfitting the data. It is important to note that this also intentionally introduces bias into the prediction to increase its accuracy when making out-of-sample predictions. Because the regularizer limits the complexity of the model, a variable correlated with race that underlies its influence on the focal outcome may be omitted from the screening algorithm.       

In the fifth step, the algorithm is validated with the holdout test data set that was excluded from its training. Empirical tuning or hyperparameter tuning is the process of adjusting parameters to optimize the model’s performance in this holdout sample. The use of a holdout sample creates a firewall principle: The data used to build the predictive model should be kept separate from the data used to evaluate its predictions.  

These five steps are followed in the development of all the supervised learning algorithms herein discussed, including deep-learning models. Modern deep supervised learning is driven by transformer models (Vaswani et al., 2017). Google’s BERT (Devlin et al., 2018), for example, can be fine-tuned for classification or regression tasks. Importantly, the exact same dimensions that are relevant for scrutinizing non-deep neural networks apply to deep supervised learning models.      

Reinforcement Learning  

As humans, we learn through interacting with our environment, psychologically we are driven by a reward system that allows us to learn from trial and error in a way that improves our decision making over time (for reviews, see Sutton & Barto, 1981; Gershman & Daw, 2017). Algorithmic decision making with reinforcement learning (RL) requires an agent to interact with its environment, observe the consequences of its actions, and, in response to receiving rewards, it learns to alter its own behavior. As a learning paradigm, reinforcement learning is rooted in behavioral psychology (Sutton & Barto, 2018; Botvinick et al., 2019). Learning through trial and error requires mimicking human behavior and optimizing through interactions. Reinforcement learning agents learn from a reward system while balancing the exploration-exploitation tradeoff to find an optimal policy. They learn optimal behaviors.  

This type of trial-and-error learning is achieved with a perception-action learning loop (Arulkumaran et al., 2017) which can be modeled as a Markov decision process (MDP). The MDP model has: i) a set of states S, ii) a set of agents A, iii) transition dynamics that map a state-action pair at time t onto a distribution of states at time t + 1, iv) an instantaneous reward function and v) a discount factor that emphasizes immediate rewards. The MDP model relies on the Markov property, that only the current state affects the next state, meaning the future is conditionally independent of the past given the present state. This requires the states to be fully observable, which is an unrealistic assumption. Hence, we can choose to generalize the MDP to a partially observed MDP (POMDP). We distinguish between model-based and model-free methods based on whether we can specify a priori the full MDP. Model-based “planning” methods require complete knowledge of the MDP model in terms of transition and reward functions. Model-free methods learn an optimal policy simply based on observations and rewards received.  

Reinforcement learning is useful for making decisions in stochastic settings, especially in time-dependent scenarios where it is necessary to balance the trade-off between long-term and short-term rewards. Experimental design has benefited from interpretable reinforcement learning in the form of multiarmed bandits and contextual bandits (Athey & Imbens, 2019). The advantage of reinforcement learning in experiments is the ability of the algorithms to model bounded rationality. Statistics and econometrics have long been interested in the exploration-exploitation tradeoff of perfect rationality; for instance, the upper confidence bound (UCB) algorithm is an exploration strategy based on the principle of “optimism in the face of uncertainty” (Lai & Robbins, 1985). When dealing with high-dimensional problems, deep reinforcement learning (DRL) makes use of deep learning to solve previously intractable decision-making problems. Deep neural networks are capable of learning from low-dimensional feature representations and function approximations, allowing for reinforcement learning with high-dimensional states and action spaces. 

Reinforcement learning is best suited for systems with inherent time delays and evaluation via long-term future rewards that require decisions to be made and carried out without immediate knowledge of effectiveness (Yu et al., 2021). It includes automated driving algorithms (Kiran et al., 2021), health care treatment regimes and resource allocation (Yu et al., 2021), personalized user recommendations in news (Zheng et al., 2018), music (Wang et al., 2014), and retail (Rohde et al., 2018), automated bidding algorithms in marketing for display advertising (Jin et al., 2018) or for use in electricity markets (Zhang et al., 2019), and competitive video games (Berner et al., 2019). In each of these domains, it is necessary to learn optimal behaviors. Reinforcement learning allows for sequential decision making that simultaneously accounts for sampled, evaluative, and delayed feedback. A breakthrough moment was AlphaGo (Silver et al., 2017), a partially supervised reinforcement learning algorithm composed of neural networks trained using both supervised and RL in combination with a search algorithm. A more recent breakthrough is the incorporation of the transformer architecture into reinforcement learning models, producing AlphaFold 2 (Jumper et al., 2021), an algorithm capable of accurately predicting 3D protein structure from a chain of amino acids, a formerly intractable task. Notably, non-deep generative causal models (interpretable algorithms) performed better than standard deep reinforcement learning algorithms on some benchmarks by using causal reasoning (Kansky et al., 2017).

AUTHOR NOTE

We thank Sharad Goel, George Loewenstein, Barbara Mellers, and Katherine Milkman for their helpful comments and suggestions.  

REFERENCES 

Abolfathi, N., & Santamaria, S. (2020). Dating disruption-how tinder gamified an industry. MIT Sloan Management Review, 61(3), 7-11. https://sloanreview.mit.edu/article/dating-disruption-how-tinder-gamified-an-industry/

Acemoglu, D., & Autor, D. (2011). Skills, tasks and technologies: Implications for employment and earnings. In Handbook of Labor Economics (Vol. 4, pp. 1043-1171). Elsevier. https://doi.org/10.3386/w16082

Acquisti, A., Brandimarte, L., & Loewenstein, G. (2015). Privacy and human behavior in the age of information. Science, 347(6221), 509-514. https://doi.org/10.1126/science.aaa1465

Agarwal, S., De Freitas, J., Ragnhildstveit, A., & Morewedge, C. K. (2024). Acceptance of automated vehicles is lower for self than others. Journal of the Association for Consumer Research, 9(3), 269-281. https://doi.org/10.1086/729900

Agrawal, A., Gans, J., & Goldfarb, A. (2018). Prediction machines: The simple economics of artificial intelligence. Harvard Business Press. 

Alba, J. W., & Hasher, L. (1983). Is memory schematic? Psychological Bulletin, 93(2), 203–231. https://doi.org/10.1037/0033-2909.93.2.203

Alba, J. W., & Hutchinson, J. W. (1987). Dimensions of consumer expertise. Journal of Consumer Research, 13(4), 411-454. https://doi.org/10.1086/209080

Alba, J., Lynch, J., Weitz, B., Janiszewski, C., Lutz, R., Sawyer, A., & Wood, S. (1997). Interactive home shopping: consumer, retailer, and manufacturer incentives to participate in electronic marketplaces. Journal of Marketing, 61(3), 38-53. https://doi.org/10.2307/1251788

Ali, M., Sapiezynski, P., Bogen, M., Korolova, A., Mislove, A., & Rieke, A. (2019). Discrimination through optimization: How Facebook's Ad delivery can lead to biased outcomes. Proceedings of the ACM on human-computer interaction, 3(CSCW), 1-30. https://doi.org/10.1145/3359301

Allcott, H., Gentzkow, M., & Song, L. (2022). Digital addiction. American Economic Review, 112(7), 2424-63. https://doi.org/10.1257/aer.20210867

Alter, A. (2017). Irresistible: The rise of addictive technology and the business of keeping us hooked. Penguin Press. 

Amato, P. R. (2010). Research on divorce: Continuing trends and new developments. Journal of Marriage and Family, 72(3), 650-666. https://doi.org/10.1111/j.1741-3737.2010.00723.x

Andreassen, C. S. (2015). Online social network site addiction: A comprehensive review. Current Addiction Reports, 2(2), 175-184. https://doi.org/10.1007/s40429-015-0056-9

Aporta, C., & Higgs, E. (2005). Satellite culture: global positioning systems, Inuit wayfinding, and the need for a new account of technology. Current Anthropology, 46(5), 729-753. https://doi.org/10.1086/432651

Appel, H., Gerlach, A. L., & Crusius, J. (2016). The interplay between Facebook use, social comparison, envy, and depression. Current Opinion in Psychology, 9, 44-49. https://doi.org/10.1016/j.copsyc.2015.10.006

Arkes, H. R. (1991). Costs and benefits of judgment errors: Implications for debiasing. Psychological Bulletin, 110(3), 486-498. https://doi.org/10.1037/0033-2909.110.3.486

Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017). A brief survey of deep reinforcement learning. arXiv preprint arXiv:1708.05866. https://doi.org/10.48550/arXiv.1708.05866

Athey, S., & Imbens, G. W. (2019). Machine learning methods that economists should know about. Annual Review of Economics, 11, 685-725. https://doi.org/10.1146/annurev-economics-080217-053433

Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., ... & Rahwan, I. (2018). The moral machine experiment. Nature, 563(7729), 59-64. https://doi.org/10.1038/s41586-018-0637-6

Bakshy, E., Messing, S., & Adamic, L. A. (2015). Exposure to ideologically diverse news and opinion on Facebook. Science, 348(6239), 1130-1132. https://doi.org/10.1126/science.aaa1160

Banerjee, A., Chandrasekhar, A. G., Dalpath, S., Duflo, E., Floretta, J., Jackson, M. O., ... & Shrestha, M. (2021). Selecting the most effective nudge: Evidence from a large-scale experiment on immunization (No. w28726). National Bureau of Economic Research. https://doi.org/10.3386/w28726

Bar, M. (2004) Visual objects in context. Nature Reviews Neuroscience, (5) 617. https://doi.org/10.1038/nrn1476

Baras, D., & Meir, R. (2007). Reinforcement learning, spike-time-dependent plasticity, and the BCM rule. Neural Computation, 19(8), 2245-2279. https://doi.org/0.1162/neco.2007.19.8.2245

Barrett, L. F. (2017) How Emotions Are Made: The Secret Life of the Brain. Pan Books. 

Barberá, P., Jost, J. T., Nagler, J., Tucker, J. A., & Bonneau, R. (2015). Tweeting from left to right: Is online political communication more than an echo chamber?. Psychological Science, 26(10), 1531-1542. https://doi.org/10.1177/0956797615594620

Barr, N., Pennycook, G., Stolz, J. A., & Fugelsang, J. A. (2015). The brain in your pocket: Evidence that Smartphones are used to supplant thinking. Computers in Human Behavior, 48, 473-480. https://doi.org/10.1016/j.chb.2015.02.029

Bayer, J. B., Triệu, P., & Ellison, N. B. (2020). Social media elements, ecologies, and effects. Annual Review of Psychology, 71, 471-497. 

Bazerman, M. H., Tenbrunsel, A. E., & Wade-Benzoni, K. (1998). Negotiating with yourself and losing: Making decisions with competing internal preferences. Academy of Management Review, 23(2), 225-241. https://doi.org/10.5465/amr.1998.533224

Berger, J., & Milkman, K. L. (2012). What makes online content viral?. Journal of Marketing Research, 49(2), 192-205. 

Berger, B., Adam, M., Rühr, A., & Benlian, A. (2021). Watch me improve—Algorithm aversion and demonstrating the ability to learn. Business & Information Systems Engineering, 63(1), 55–68. 

Berman, R., & Katona, Z. (2020). Curation algorithms and filter bubbles in social networks. Marketing Science, 39(2), 296-316. https://doi.org/10.1287/mksc.2019.1208

Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison, C., ... & Zhang, S. (2019). Dota 2 with large scale deep reinforcement learning. arXiv preprint arXiv:1912.06680. https://doi.org/10.48550/arXiv.1912.06680

Berridge, K. C., Ho, C. Y., Richard, J. M., & DiFeliceantonio, A. G. (2010). The tempted brain eats: pleasure and desire circuits in obesity and eating disorders. Brain research, 1350, 43-64. https://doi.org/10.1016/j.brainres.2010.04.003

Berridge, K. C., & Robinson, T. E. (2003). Parsing reward. Trends in neurosciences, 26(9), 507-513. https://doi.org/10.1016/S0166-2236(03)00233-9

Berridge, K. C., & Robinson, T. E. (2016). Liking, wanting, and the incentive-sensitization theory of addiction. American Psychologist, 71(8), 670. https://doi.org/10.1037/amp0000059

Berridge, K. C., & Robinson, T. E. (1998). What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?. Brain research reviews, 28(3), 309-369. https://doi.org/10.1016/s0165-0173(98)00019-8

Bigman, Y. E., & Gray, K. (2018). People are averse to machines making moral decisions. Cognition, 181, 21-34. https://doi.org/10.1016/j.cognition.2018.08.003

Bigman, Y. E., Wilson, D., Arnestad, M. N., Waytz, A., & Gray, K. (2022). Algorithmic discrimination causes less moral outrage than human discrimination. Journal of Experimental Psychology: General. https://doi.org/10.1037/xge0001250

Billieux, J., Maurage, P., Lopez-Fernandez, O., Kuss, D. J., & Griffiths, M. D. (2015). Can disordered mobile phone use be considered a behavioral addiction? An update on current evidence and a comprehensive model for future research. Current Addiction Reports, 2(2), 156-162. https://doi.org/10.1007/s40429-015-0054-y

Bitzer, S., Park, H., Blankenburg, F., & Kiebel, S. J. (2014). Perceptual decision making: drift-diffusion model is equivalent to a Bayesian model. Frontiers in human neuroscience, 8, 102. https://doi.org/10.3389/fnhum.2014.00102

Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of Machine Learning Research, 3(Jan), 993-1022. 

Bohte, S. M., Kok, J. N., & La Poutre, H. (2002). Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing, 48(1-4), 17-37. https://doi.org/10.1016/S0925-2312(01)00658-0

Bonaccio, S., & Dalal, R. S. (2006). Advice taking and decision making: An integrative literature review, and implications for the organizational sciences. Organizational behavior and human decision processes, 101(2), 127-151. https://doi.org/10.1016/j.obhdp.2006.07.001

Bond, C. F., & DePaulo, B. M. (2006). Accuracy of deception judgments. Personality and Social Psychology Review, 10(3), 214–234. https://doi.org/10.1207/s15327957pspr1003_2

Bonezzi, A., & Ostinelli, M. (2021). Can algorithms legitimize discrimination?. Journal of Experimental Psychology: Applied, 27(2), 447-459. https://doi.org/10.1037/xap0000294

Bonini, T., & Gandini, A. (2019). “First week is editorial, second week is algorithmic”: Platform gatekeepers and the platformization of music curation. Social Media+ Society, 5(4). https://doi.org/10.1177/2056305119880006

Bonnefon, J. F., & Rahwan, I. (2020). Machine thinking, fast and slow. Trends in Cognitive Sciences, 24(12), 1019-1027. https://doi.org/10.1016/j.tics.2020.09.007

Borch, C. (2016). High-frequency trading, algorithmic finance and the Flash Crash: reflections on eventalization. Economy and Society, 45(3-4), 350-378. https://doi.org/10.1080/03085147.2016.1263034

Borch, C. (2022). Machine learning, knowledge risk, and principal-agent problems in automated trading. Technology in Society, 68, 101852. https://doi.org/10.1016/j.techsoc.2021.101852

Botvinick, M. M. (2022). Realizing the promise of AI: a new calling for cognitive science. Trends in Cognitive Sciences, 26(12), 1013-1014. https://doi.org/10.1016/j.tics.2022.08.004

Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., & Hassabis, D. (2019). Reinforcement learning, fast and slow. Trends in Cognitive Sciences, 23(5), 408-422. https://doi.org/10.1016/j.tics.2019.02.006

Bowen, D., Green, E., & Simmons, J. (2022). Making megastudies more effective. Technical report, 2022b. URL https://dsbowen.github.io/download/megastudies.  

Bowker, J. C., Bowker, M. H., Santo, J. B., Ojo, A. A., Etkin, R. G., & Raja, R. (2019). Severe social withdrawal: Cultural variation in past hikikomori experiences of university students in Nigeria, Singapore, and the United States. The Journal of Genetic Psychology, 180(4-5), 217-230. https://doi.org/10.1080/00221325.2019.1633618

Brady, W. J., Crockett, M. J., & Van Bavel, J. J. (2020). The MAD model of moral contagion: The role of motivation, attention, and design in the spread of moralized content online. Perspectives on Psychological Science, 15(4), 978-1010.  
https://doi.org/10.1177/1745691620917336

Brady, W. J., McLoughlin, K., Doan, T. N., & Crockett, M. J. (2021). How social learning amplifies moral outrage expression in online social networks. Science Advances, 7(33). https://doi.org/10.1126/sciadv.abe5641

Bratton, W. J. (2019, July 17). For police, is AI deception technology a “truth meter” yet? The Hill. https://thehill.com/opinion/criminal-justice/453395-for-police-is-ai-deception-technology-a-truth-meter-yet/ 

Bravata, D. M., Smith-Spangler, C., Sundaram, V., Gienger, A. L., Lin, N., Lewis, R., ... & Sirard, J. R. (2007). Using pedometers to increase physical activity and improve health: a systematic review. Jama, 298(19), 2296-2304. https://doi.org/10.1001/jama.298.19.2296

Buolamwini, J., & Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of the 1st Conference on Fairness, Accountability and Transparency, 81, 77-91. 

Bursztyn, L., Egorov, G., & Fiorin, S. (2020). From extreme to mainstream: The erosion of social norms. American Economic Review, 110(11), 3522-48. https://doi.org/10.1257/aer.20171175

Burton, J. W., Stein, M. K., & Jensen, T. B. (2020). A systematic review of algorithm aversion in augmented decision making. Journal of Behavioral Decision Making, 33(2), 220-239. https://doi.org/10.1002/bdm.2155

Cadario, R., Longoni, C., & Morewedge, C. K. (2021). Understanding, explaining, and utilizing medical artificial intelligence. Nature Human Behaviour, 5(12), 1636-1642. https://doi.org/10.1038/s41562-021-01146-0

Callaway, F., Hardy, M., & Griffiths, T. (2022, January 29). Optimal nudging for cognitively bounded agents: A framework for modeling, predicting, and controlling the effects of choice architectures. https://doi.org/10.31234/osf.io/7ahdc

Callaway, F., Jain, Y. R., van Opheusden, B., Das, P., Iwama, G., Gul, S., ... & Lieder, F. (2022). Leveraging artificial intelligence to improve people’s planning strategies. Proceedings of the National Academy of Sciences, 119(12). https://doi.org/10.1073/pnas.2117432119

Callaway, F., van Opheusden, B., Gul, S., Das, P., Krueger, P. M., Griffiths, T. L., & Lieder, F. (2022). Rational use of cognitive resources in human planning. Nature Human Behaviour, 1-14. https://doi.org/10.1038/s41562-022-01332-8

Campbell, W. K., & Sedikides, C. (1999). Self-threat magnifies the self-serving bias: A meta-analytic integration. Review of General Psychology, 3(1), 23-43. https://doi.org/10.1037/1089-2680.3.1.23

Caporale, N., & Dan, Y. (2008). Spike timing-dependent plasticity: a Hebbian learning rule. Annual Review of Neuroscience, 31(1), 25-46. https://doi.org/10.1146/annurev.neuro.31.060407.125639

Carlson, M. (2018). Automating judgment? Algorithmic judgment, news knowledge, and journalistic professionalism. New Media & Society, 20(5), 1755-1772. https://doi.org/10.1177/1461444817706684

Carver, C. S., & Scheier, M. F. (1982). Control theory: A useful conceptual framework for personality–social, clinical, and health psychology. Psychological Bulletin, 92(1), 111-135. https://doi.org/10.1037/0033-2909.92.1.111

Castelo, N., Bos, M. W., & Lehmann, D. R. (2019). Task-dependent algorithm aversion. Journal of Marketing Research, 56(5), 809-825. https://doi.org/10.1177/0022243719851788

Cavazos, J. G., Phillips, P. J., Castillo, C. D., & O’Toole, A. J. (2020). Accuracy comparison across face recognition algorithms: Where are we on measuring race bias?. IEEE Transactions on Biometrics, Behavior, and Identity Science, 3(1), 101-111. https://doi.org/10.1109/TBIOM.2020.3027269

Celiktutan, B., Cadario, R., & Morewedge, C. K. (2024). People see more of their biases in algorithms. Proceedings of the National Academy of Sciences, 121(16), e2317602121. https://doi.org/10.1073/pnas.2317602121

Chaboud, A. P., Chiquoine, B., Hjalmarsson, E., & Vega, C. (2014). Rise of the machines: Algorithmic trading in the foreign exchange market. The Journal of Finance, 69(5), 2045-2084. https://doi.org/10.1111/jofi.12186

Chang, L. W., & Cikara, M. (2018). Social decoys: Leveraging choice architecture to alter social preferences. Journal of Personality and Social Psychology, 115(2), 206. https://doi.org/10.1037/pspa0000117

Chang, W., Chen, E., Mellers, B., & Tetlock, P. (2016). Developing expert political judgment: The impact of training and practice on judgmental accuracy in geopolitical forecasting tournaments. Judgment and Decision Making, 11(5), 509-526. https://doi.org/10.1017/S1930297500004599

Chen, H. L., Jegadeesh, N., & Wermers, R. (2000). The value of active mutual fund management: An examination of the stockholdings and trades of fund managers. Journal of Financial and Quantitative Analysis, 35(3), 343-368. https://doi.org/10.2307/2676208

Chen, L., Mislove, A., & Wilson, C. (2016, April). An empirical analysis of algorithmic pricing on amazon marketplace. In Proceedings of the 25th International Conference on World Wide Web (pp. 1339-1349). https://doi.org/10.1145/2872427.2883089

Chernev, A., Böckenholt, U., & Goodman, J. (2015). Choice overload: A conceptual review and meta-analysis. Journal of Consumer Psychology, 25(2), 333-358. https://doi.org/10.1016/j.jcps.2014.08.002

Chou, C., Chou, J., & Tyan, N. C. N. (1998). An Exploratory Study of Internet Addiction, Usage and Communication Pleasure. 

Chou, C., Condron, L., & Belland, J. C. (2005). A review of the research on Internet addiction. Educational Psychology Review, 17(4), 363-388. https://doi.org/10.1007/s10648-005-8138-1

Chou, Y. L., Moreira, C., Bruza, P., Ouyang, C., & Jorge, J. (2022). Counterfactuals and causability in explainable artificial intelligence: Theory, algorithms, and applications. Information Fusion, 81, 59-83. https://doi.org/10.1016/j.inffus.2021.11.003

Christian, B. (2020). The alignment problem: Machine learning and human values. WW Norton & Company. 

Churchland, P. S., & Sejnowski, T. J. (1992). The computational brain. MIT press. 

Clark, A., & Chalmers, D. (1998). The extended mind. Analysis, 58(1), 7-19. https://doi.org/10.1093/analys/58.1.7

Cohen, T. R., Gunia, B. C., Kim-Jun, S. Y., & Murnighan, J. K. (2009). Do groups lie more than individuals? Honesty and deception as a function of strategic self-interest. Journal of Experimental Social Psychology, 45(6), 1321-1324. https://doi.org/10.1016/j.jesp.2009.08.007

Comscore. (2022, June). Latest rankings. Retrieved from https://www.comscore.com/Insights/Rankings#tab_mobile_apps 

Connolly, T., & Zeelenberg, M. (2002). Regret in decision making. Current Directions in Psychological Science, 11(6), 212-216. https://doi.org/10.1111/1467-8721.002

Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017, August). Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining (pp. 797-806). https://doi.org/10.1145/3097983.3098095

Curci, A., Lanciano, T., Battista, F., Guaragno, S., & Ribatti, R. M. (2019). Accuracy, confidence, and experiential criteria for lie detection through a videotaped interview. Frontiers in Psychiatry, 9(748). https://doi.org/10.3389/fpsyt.2018.00748

Dale, R. (2021). GPT-3: What’s it good for?. Natural Language Engineering, 27(1), 113-118. https://doi.org/10.1017/S1351324920000601

Dalton, D. R., Hitt, M. A., Certo, S. T., & Dalton, C. M. (2007). The fundamental agency problem and its mitigation. Academy of Management Annals, 1(1), 1-64. https://doi.org/10.5465/078559806

Dan, Y., & Poo, M. M. (2004). Spike timing-dependent plasticity of neural circuits. Neuron, 44(1), 23-30. https://doi.org/10.1016/j.neuron.2004.09.007

Dana, J., Weber, R. A., & Kuang, J. X. (2007). Exploiting moral wiggle room: experiments demonstrating an illusory preference for fairness. Economic Theory, 33(1), 67-80. https://doi.org/10.1007/s00199-006-0153-z

Danks, D., & London, A. J. (2017, August). Algorithmic Bias in Autonomous Systems. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (Vol. 17, pp. 4691-4697). https://doi.org/10.24963/ijcai.2017/654

Dastin, J. (2018, October 10). Amazon scraps secret AI recruiting tool that showed bias against women. Reuters. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G 

Da’u, A., & Salim, N. (2020). Recommendation system based on deep learning methods: a systematic review and new directions. Artificial Intelligence Review, 53(4), 2709-2748. https://doi.org/10.1007/s10462-019-09744-1

Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243(4899), 1668-1674. https://doi.org/10.1126/science.2648573

De Freitas, J., & Cikara, M. (2021). Deliberately prejudiced self-driving vehicles elicit the most outrage. Cognition, 208, 104555. https://doi.org/10.1016/j.cognition.2020.104555

Dehaene, S. (2003). The neural basis of the Weber–Fechner law: a logarithmic mental number line. Trends in cognitive sciences, 7(4), 145-147. https://doi.org/10.1016/s1364-6613(03)00055-x

Demirdag, B. I. (2022). Consumer responses to algorithmic decisions. University of Calfornia at Los Angeles, https://escholarship.org/content/qt2kz01678/qt2kz01678.pdf.  

Demirdag, I., & Shu, S. (2021). Bias neglect: When human bias, but not algorithmic bias, is disregarded. ACR North American Advances. 

DePaulo, B. M., Lindsay, J. J., Malone, B. E., Muhlenbruck, L., Charlton, K., & Cooper, H. (2003). Cues to deception. Psychological Bulletin, 129(1), 74. https://doi.org/10.1037/0033-2909.129.1.74

Devine, S., Germain, N., Ehrlich, S., & Eppinger, B. (2022). Changes in the prevalence of thin bodies bias young women’s judgments about body size. Psychological Science, 33(8), 1212-1225. https://doi.org/10.1177/09567976221082941

Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. https://doi.org/10.48550/arXiv.1810.04805

Diakopoulos, N. (2019). Automating the news. Harvard University Press. 

Dickson, P. R., & Ginter, J. L. (1987). Market segmentation, product differentiation, and marketing strategy. Journal of Marketing, 51(2), 1-10. https://doi.org/10.2307/1251125

Diehl, K. (2005). When two rights make a wrong: Searching too much in ordered environments. Journal of Marketing Research, 42(3), 313-322. https://doi.org/10.1509/jmkr.2005.42.3.313

Diehl, K., Kornish, L. J., & Lynch Jr, J. G. (2003). Smart agents: When lower search costs for quality information increase price sensitivity. Journal of Consumer Research, 30(1), 56-71. https://doi.org/10.1086/374698

Dietvorst, B. J., & Bartels, D. M. (2022). Consumers object to algorithms making morally relevant tradeoffs because of algorithms’ consequentialist decision strategies. Journal of Consumer Psychology, 32(3), 406-424. https://doi.org/10.1002/jcpy.1266

Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: people erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114-126. https://doi.org/10.1037/xge0000033

Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018). Overcoming algorithm aversion: People will use imperfect algorithms if they can (even slightly) modify them. Management Science, 64(3), 1155-1170. https://doi.org/10.1287/mnsc.2016.2643

Duckworth, A. L., Milkman, K. L., & Laibson, D. (2018). Beyond willpower: Strategies for reducing failures of self-control. Psychological Science in the Public Interest, 19(3), 102-129. https://doi.org/10.1177/1529100618821893

Dunning, D., Meyerowitz, J. A., & Holzberg, A. D. (1989). Ambiguity and self-evaluation: The role of idiosyncratic trait definitions in self-serving assessments of ability. Journal of Personality and Social Psychology, 57(6), 1082. https://doi.org/10.1037/0022-3514.57.6.1082

Eady, G., Nagler, J., Guess, A., Zilinsky, J., & Tucker, J. A. (2019). How many people live in political bubbles on social media? Evidence from linked survey and Twitter data. Sage Open, 9(1). https://doi.org/10.1177/2158244019832705

Edwards, J. R., & Bagozzi, R. P. (2000). On the nature and direction of relationships between constructs and measures. Psychological Methods, 5(2), 155–174. https://doi.org/10.1037/1082-989X.5.2.155

Ekman, P. (2006). Darwin, deception, and facial expression. Annals of the New York Academy of Sciences, 1000(1), 205–221.  

https://doi.org/10.1196/annals.1280.010

Ekman, P. (2009). Telling lies: Clues to deceit in the marketplace, politics, and marriage (revised edition). WW Norton & Company. 

Ellemers, N., Spears, R., & Doosje, B. (2002). Self and social identity. Annual Review of Psychology, 53(1), 161-186. https://doi.org/10.1146/annurev.psych.53.100901.135228

Epley, N., & Gilovich, T. (2001). Putting adjustment back in the anchoring and adjustment heuristic: Differential processing of self-generated and experimenter-provided anchors. Psychological science, 12(5), 391-396. https://doi.org/10.1111/1467-9280.0037

Ernst, M., & Paulus, M. P. (2005). Neurobiology of decision making: a selective review from a neurocognitive and clinical perspective. Biological psychiatry, 58(8), 597-604. https://doi.org/10.1016/j.biopsych.2005.06.004

Evans, J. S. B. T. (2003). In two minds: dual-process accounts of reasoning. Trends in cognitive sciences, 7(10), 454-459. https://doi.org/10.1016/j.tics.2003.08.012

Farries, M. A., & Fairhall, A. L. (2007). Reinforcement learning with modulated spike timing–dependent synaptic plasticity. Journal of neurophysiology, 98(6), 3648-3665. https://doi.org/10.1162/neco.2007.19.6.1468

Farronato, C., Fradkin, A., & MacKay, A. (2023). Self-Preferencing at Amazon: Evidence from search rankings (No. w30894). National Bureau of Economic Research. https://doi.org/10.3386/w30894

Fenech, E. P., Drews, F. A., & Bakdash, J. Z. (2010, September). The effects of acoustic turn-by-turn navigation on wayfinding. In Proceedings of the human factors and ergonomics society annual meeting (Vol. 54, No. 23, pp. 1926-1930). Sage CA: Los Angeles, CA: SAGE Publications. https://doi.org/10.1518/107118110X12829370263287

Ferguson, T. S. (1989). Who solved the secretary problem?. Statistical science, 4(3), 282-289. https://doi.org/10.1214/ss/1177012493

Fernbach, P. M., Rogers, T., Fox, C. R., & Sloman, S. A. (2013). Political extremism is supported by an illusion of understanding. Psychological Science, 24(6), 939-946. https://doi.org/10.1177/0956797612464058

Fernbach, P. M., Sloman, S. A., Louis, R. S., & Shube, J. N. (2013). Explanation fiends and foes: How mechanistic detail determines understanding and preference. Journal of Consumer Research, 39(5), 1115-1131. https://doi.org/10.1086/667782

Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016). The rise of social bots. Communications of the ACM, 59(7), 96-104. https://doi.org/10.1145/2818717

Fischhoff, B. (1982). Debiasing. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 422-444). Cambridge, UK: Cambridge University Press. 

Fishburn, P. C. (1967). Methods of estimating additive utilities. Management Science, 13, 435–453. http://doi.org/10.1287/mnsc.13.7.435

Fisher, M., Goddu, M. K., & Keil, F. C. (2015). Searching for explanations: How the Internet inflates estimates of internal knowledge. Journal of Experimental Psychology: General, 144(3), 674. https://doi.org/10.1037/xge0000070

Flaxman, S., Goel, S., & Rao, J. M. (2016). Filter bubbles, echo chambers, and online news consumption. Public Opinion Quarterly, 80(S1), 298-320. https://doi.org/10.1093/poq/nfw006

Florian, R. V. (2007). Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Computation, 19(6), 1468-1502. https://doi.org/10.1162/neco.2007.19.6.1468

Fox, J., & Moreland, J. J. (2015). The dark side of social networking sites: An exploration of the relational and psychological stressors associated with Facebook use and affordances. Computers in Human Behavior, 45, 168–176. https://doi.org/10.1016/j.chb.2014.11.083

Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19(4), 25-42. https://doi.org/10.1257/089533005775196732

Frederick, S., Novemsky, N., Wang, J., Dhar, R., & Nowlis, S. (2009). Opportunity cost neglect. Journal of Consumer Research, 36(4), 553-561. https://doi.org/10.1086/599764 

Frey, C. B., & Osborne, M. A. (2017). The future of employment: How susceptible are jobs to computerisation?. Technological forecasting and social change, 114, 254-280. https://doi.org/10.1016/j.techfore.2016.08.019

Furchtgott-Roth, D. (2022, March 10). America needs GPS backup. Forbes. https://www.forbes.com/sites/dianafurchtgott-roth/2022/03/10/america-needs-gps-backup/?sh=2ba3084d5785 

Gailmard, S. (2009). Multiple principals and oversight of bureaucratic policy-making. Journal of Theoretical Politics, 21(2), 161-186. https://doi.org/10.1177/0951629808100762

Gershman, S. J., & Daw, N. D. (2017). Reinforcement learning and episodic memory in humans and animals: an integrative framework. Annual review of psychology, 68, 101. https://doi.org/10.1146/annurev-psych-122414-033625

Gigerenzer, G., & Goldstein, D. G. (1996). Mind as computer: Birth of a metaphor. Creativity Research Journal, 9(2-3), 131-144. https://doi.org/10.1080/10400419.1996.9651168

Gilbert, D. T. (1999). What the mind’s not. In Chaiken, S. & Trope, Y. (Eds)., Dual-process theories in social psychology, 3-11. 

Gilbert, D. T., Pelham, B. W., & Krull, D. S. (1988). On cognitive busyness: When person perceivers meet persons perceived. Journal of personality and social psychology, 54(5), 733-740. https://doi.org/10.1037/0022-3514.54.5.733

Gilbert, D. T., & Wilson, T. D. (2007). Prospection: Experiencing the future. Science, 317(5843), 1351-1354. https://doi.org/10.1126/science.1144161

Gill, R. (2021). Changing the perfect picture: Smartphones, social media and appearance pressures. City, University of London. 

Gilovich, T., Griffin, D., & Kahneman, D. (Eds.). (2002). Heuristics and biases: The psychology of intuitive judgment. Cambridge university press. 

Gino, F., Ayal, S., & Ariely, D. (2009). Contagion and differentiation in unethical behavior: The effect of one bad apple on the barrel. Psychological Science, 20(3), 393-398. https://doi.org/10.1111/j.1467-9280.2009.02306.x

Goel, S., Mason, W., and Watts, D. J. 2010. Real and perceived attitude agreement in social networks. Journal of Personality and Social Psychology, 99(4), 611-621. https://doi.org/10.1037/a0020697

Goel, S., Perelman, M., Shroff, R., & Sklansky, D. A. (2017). Combatting police discrimination in the age of big data. New Criminal Law Review, 20(2), 181-232. https://doi.org/10.1525/nclr.2017.20.2.181

Goldfarb, A., & Tucker, C. (2011a). Online display advertising: targeting and obtrusiveness. Marketing Science, 30, 389–404. https://doi.org/10.1287/mksc.1100.0583

Goodhart, C.A.E. (1984). Problems of monetary management: The UK experience. In Monetary theory and practice. Palgrave, London. https://doi.org/10.1007/978-1-349-17295-5_4

Goodwin, P., & Wright, G. (2014). Decision analysis for management judgment. John Wiley & Sons. 

Google car pulled over for being too slow. (2015, November 13). BBC. https://www.bbc.com/news/technology-34808105

Gosselin, P., Perron, M., & Beaupré, M. (2010). The voluntary control of facial action units in adults. Emotion, 10(2), 266–271. https://doi.org/10.1037/a0017748

Gray, K., Yam, K. C., Zhen’an, A. E., Wilbanks, D., & Waytz, A. (2023). The psychology of robots and artificial intelligence. In D. T. Gilbert, S. T. Fiske, E. Finkel, and W. Mendes (eds.), The handbook of social psychology, 6th edition.

Greenberg, A. E., & Spiller, S. A. (2016). Opportunity cost neglect attenuates the effect of choices on preferences. Psychological Science, 27(1), 103-113. https://doi.org/10.1177/0956797615608267

Greene, J., & Haidt, J. (2002). How (and where) does moral judgment work?. Trends in Cognitive Sciences, 6(12), 517-523. https://doi.org/10.1016/S1364-6613(02)02011-9

Grootendorst, M. (2022). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794. https://doi.org/10.48550/arXiv.2203.05794

Gross, R., & Acquisti, A. (2005, November). Information revelation and privacy in online social networks. In Proceedings of the 2005 ACM workshop on Privacy in the electronic society (pp. 71-80). https://doi.org/10.1145/1102199.1102214

Grossman, S. J., & Hart, O. D. (1992). An analysis of the principal-agent problem. In Dionne, G. & Harrington, S. E. (Eds)., Foundations of insurance economics (pp. 302-340). Springer, Dordrecht. 

Guess, A. M., & Munger, K. (2023). Digital literacy and online political behavior. Political Science Research and Methods, 11(1), 110-128. https://doi.org/10.1017/psrm.2022.17

Gunnery, S. D., & Hall, J. A. (2014). The Duchenne smile and persuasion. Journal of Nonverbal Behavior, 38(2), 181–194. https://doi.org/10.1007/s10919-014-0177-1

Gunning, D., Stefik, M., Choi, J., Miller, T., Stumpf, S., & Yang, G. Z. (2019). XAI—Explainable artificial intelligence. Science Robotics, 4(37), eaay7120. https://doi.org/10.1126/scirobotics.aay7120

Haidt, J. (2001). The emotional dog and its rational tail: a social intuitionist approach to moral judgment. Psychological Review, 108(4), 814. https://doi.org/10.1037/0033-295x.108.4.814

Haidt, J., & Bail, C. (ongoing). Social media and political dysfunction: A collaborative review. Unpublished manuscript, New York University 

Hamman, J. R., Loewenstein, G., & Weber, R. A. (2010). Self-interest through delegation: An additional rationale for the principal-agent relationship. American Economic Review, 100(4), 1826-46. https://doi.org/10.1257/aer.100.4.1826

Hammond, K. R. (2000). Coherence and correspondence theories in judgment and decision making. In T. Connoly, H. R. Arkes, & K. R. Hammond (eds.), Judgment and decision making: An interdisciplinary reader, 53-65. Cambridge: Cambridge University Press. 

Hartwig, M., & Bond, C. F. (2014). Lie Detection from Multiple Cues: A Meta-analysis: Lie detection from multiple cues. Applied Cognitive Psychology, 28(5), 661–676.  https://doi.org/10.1002/acp.3052

Haslbeck, A., & Hoermann, H. J. (2016). Flying the needles: flight deck automation erodes fine-motor flying skills among airline pilots. Human Factors, 58(4), 533-545. https://doi.org/10.1177/0018720816640394

Häubl, G., & Trifts, V. (2000). Consumer decision making in online shopping environments: The effects of interactive decision aids. Marketing Science, 19(1), 4-21. https://doi.org/10.1287/mksc.19.1.4.15178

Hauser, J. R., & Wernerfelt, B. (1990). An evaluation cost model of consideration sets. Journal of Consumer Research, 16(4), 393-408. https://doi.org/10.1086/209225

Hawi, N. S., & Samaha, M. (2017). The relations among social media addiction, self-esteem, and life satisfaction in university students. Social Science Computer Review, 35(5), 576-586. https://doi.org/10.1177/0894439316660340

Hecht-Nielsen, R. (1992). Theory of the backpropagation neural network. In Wechsler, H. (Ed)., Neural networks for perception (pp. 65-93). Academic Press. https://doi.org/10.1016/B978-0-12-741252-8.50010-8

Heckhausen, H., & Gollwitzer, P. M. (1987). Thought contents and cognitive functioning in motivational versus volitional states of mind. Motivation and emotion, 11(2), 101-120. https://doi.org/10.1007/BF00992338

Hellmann, T. F., Murdock, K. C., & Stiglitz, J. E. (2000). Liberalization, moral hazard in banking, and prudential regulation: Are capital requirements enough?. American economic review, 90(1), 147-165. https://doi.org/10.1257/aer.90.1.147

Henkel, L. A. (2014). Point-and-shoot memories: The influence of taking photos on memory for a museum tour. Psychological Science, 25, 396–402. https://doi.org/10.1177/0956797613504438

Hern, A. (2021, May 6). Twitter launches prompt asking users to rethink abusive tweets. The Guardian. https://www.theguardian.com/technology/2021/may/06/twitter-launches-prompt-in-bid-to-reduce-abusive-language 

Hertwig, R., & Grüne-Yanoff, T. (2017). Nudging and boosting: Steering or empowering good decisions. Perspectives on Psychological Science, 12(6), 973-986. https://doi.org/10.1177/1745691617702496

Higgins, E. T., Roney, C. J. R., Crowe, E., & Hymes, C. (1994). Ideal versus ought predilections for approach and avoidance distinct self-regulatory systems. Journal of Personality and Social Psychology, 66(2), 276–286. https://doi.org/10.1037/0022-3514.66.2.276

Hilbert, M. (2012). Toward a synthesis of cognitive biases: how noisy information processing can bias human decision making. Psychological Bulletin, 138(2), 211. https://doi.org/10.1037/a0025940

Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019). Causability and explainability of artificial intelligence in medicine. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 9(4), e1312. https://doi.org/10.1002/widm.1312

Howard, J. A., & Sheth, J. N. (1969). The theory of buyer behavior. John Wiley & Sons: New York. 

Howard, R. A. (1980). An assessment of decision analysis. Operations Research, 28(1), 4-27. https://doi.org/10.1287/opre.28.1.4

Hsee, C. K., & Hastie, R. (2006). Decision and experience: why don't we choose what makes us happy?. Trends in Cognitive Sciences, 10(1), 31-37. https://doi.org/10.1016/j.tics.2005.11.007

Hsee, C. K., Loewenstein, G. F., Blount, S., & Bazerman, M. H. (1999). Preference reversals between joint and separate evaluations of options: a review and theoretical analysis. Psychological Bulletin, 125(5), 576. https://doi.org/10.1037/0033-2909.125.5.576

Huang, C. L., Chen, M. C., & Wang, C. J. (2007). Credit scoring with a data mining approach based on support vector machines. Expert Systems with Applications, 33(4), 847-856. https://doi.org/10.1016/j.eswa.2006.07.007

Insider Intelligence. (2023, January 13). Voice assistants in 2023: Usage, growth, and future of the AI voice assistant market. https://www.insiderintelligence.com/insights/voice-assistants/#:~:text=This%20year%2C%20123.5%20million%20US,over%20the%20next%20few%20years. 

Iyengar, S. S., & Lepper, M. R. (2000). When choice is demotivating: Can one desire too much of a good thing? Journal of Personality and Social Psychology, 79(6), 995–1006. https://doi.org/10.1037/0022-3514.79.6.995

Japkowicz, N., & Stephen, S. (2002). The class imbalance problem: A systematic study. Intelligent Data Analysis, 6(5), 429-449. https://doi.org/10.1177/20539517211020332

Jarrahi, M. H., Newlands, G., Lee, M. K., Wolf, C. T., Kinder, E., & Sutherland, W. (2021). Algorithmic management in a work context. Big Data & Society, 8(2), 20539517211020332. https://doi.org/10.1177/20539517211020332

Jensen, M. C., & Meckling, W. H. (1976). Theory of the firm: Managerial behavior, agency costs and ownership structure. In Corporate Governance (pp. 77-132). Gower. https://doi.org/10.1016/0304-405X(76)90026-X

Jin, J., Song, C., Li, H., Gai, K., Wang, J., & Zhang, W. (2018, October). Real-time bidding with multi-agent reinforcement learning in display advertising. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (pp. 2193-2201). https://doi.org/10.1145/3269206.3272021

John, L. K., Acquisti, A., & Loewenstein, G. (2011). Strangers on a plane: Context-dependent willingness to divulge sensitive information. Journal of Consumer Research, 37, 858–873. https://doi.org/10.1086/656423

Johnson, E. J., Moe, W. W., Fader, P. S., Bellman, S., & Lohse, G. L. (2004). On the depth and dynamics of online search behavior. Management Science, 50(3), 299-308. https://doi.org/10.1287/mnsc.1040.0194

Johnson, J. G., & Raab, M. (2003). Take the first: Option-generation and resulting choices. Organizational behavior and human decision processes, 91(2), 215-229. https://doi.org/10.1016/S0749-5978(03)00027-X

Johnson, J. M., & Khoshgoftaar, T. M. (2019). Survey on deep learning with class imbalance. Journal of Big Data, 6(1), 1-54. https://doi.org/10.1186/s40537-019-0192-5

Johnson, M. K., & Raye, C. L. (1981). Reality monitoring. Psychological Review, 88(1), 67–85. https://doi.org/10.1037/0033-295X.88.1.67

Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., ... & Hassabis, D. (2021). Highly accurate protein structure prediction with AlphaFold. Nature, 596(7873), 583-589. https://doi.org/10.1038/s41586-021-03819-2

Kahneman, D. (1999). Objective happiness. In D. Kahneman, E. Diener, & N. Schwarz (Eds.), Well-being: The foundations of hedonic psychology (pp. 3–25). Russell Sage Foundation. 

Kahneman, D. (2003). A perspective on judgment and choice: Mapping bounded rationality. American Psychologist, 58(9), 697–720. https://doi.org/10.1037/0003-066X.58.9.697

Kahneman, D., & Frederick, S. (2002). Representativeness revisited: Attribute substitution in intuitive judgment. In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristics and biases: The psychology of intuitive judgment (pp. 49–81). Cambridge University Press. https://doi.org/10.1017/CBO9780511808098.004

Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise: A failure to disagree. American Psychologist, 64(6), 515. https://doi.org/10.1037/a0016755

Kahneman, D., & Miller, D. T. (1986). Norm theory: Comparing reality to its alternatives. Psychological Review, 93(2), 136. https://doi.org/10.1037/0033-295X.93.2.136

Kahneman, D., Rosenfield, A. M., Gandhi, L., & Blaser, T. (2016). Noise: How to overcome the high hidden cost of inconsistent decision making. Harvard Business Review. https://hbr.org/2016/10/noise

Kahneman, D., Sibony, O., & Sunstein, C. R. (2021). Noise: a flaw in human judgment. Hachette UK. 

Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80(4), 237. https://doi.org/10.1037/h0034747

Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47(2), 263-292. https://doi.org/10.2307/1914185

Kahneman, D., & Tversky, A. (1984). Choices, values, and frames. American Psychologist, 39(4), 341. https://doi.org/10.1037/0003-066X.39.4.341

Kalis, A., Kaiser, S., & Mojzisch, A. (2013). Why we should talk about option generation in decision making research. Frontiers in Psychology, 4, 555. https://doi.org/10.3389/fpsyg.2013.00555

Kang, S., & Kim, J. W. (2022). The fragility of experts: A moderated-mediation model of expertise, expert identity threat, and overprecision. Academy of Management Journal, 65(2), 577-605. https://doi.org/10.5465/amj.2019.0899

Kang, J. S., Kuznetsova, P., Luca, M., & Choi, Y. (2013, October). Where not to eat? Improving public policy by predicting hygiene inspections using online reviews. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (pp. 1443-1448). https://doi.org/10.5465/amj.2019.0899

Kansky, K., Silver, T., Mély, D. A., Eldawy, M., Lázaro-Gredilla, M., Lou, X., ... & George, D. (2017, July). Schema networks: Zero-shot transfer with a generative causal model of intuitive physics. In International conference on machine learning (pp. 1809-1818). PMLR. https://doi.org/10.48550/arXiv.1706.04317

Kasirzadeh, A., & Smart, A. (2021, March). The use and misuse of counterfactuals in ethical machine learning. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 228-236). https://doi.org/10.1145/3442188.3445886

Kassam, K. S., Morewedge, C. K., Gilbert, D. T., & Wilson, T.z D. (2011). Winners love winning and losers love money. Psychological science, 22(5), 602-606. https://doi.org/10.1177/0956797611405681

Keren, G., & Schul, Y. (2009). Two is not always better than one: A critical evaluation of two-system theories. Perspectives on psychological science, 4(6), 533-550. https://doi.org/10.1111/j.1745-6924.2009.01164.x

Khambatta, P., Mariadassou, S., Morris, J., & Wheeler, S. C. (2023). Tailoring recommendation algorithms to ideal preferences makes users better off. Scientific Reports, 13(1), 9325. https://doi.org/10.1038/s41598-023-34192-x

Kim, E. S. (2020). Deep learning and principal–agent problems of algorithmic governance: The new materialism perspective. Technology in Society, 63, 101378. https://doi.org/10.1016/j.techsoc.2020.101378

Kim, T., Barasz, K., & John, L. K. (2019). Why am I seeing this ad? The effect of ad transparency on ad effectiveness. Journal of Consumer Research, 45(5), 906-932. https://doi.org/10.1093/jcr/ucy039

Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Al Sallab, A. A., Yogamani, S., & Pérez, P. (2021). Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems. https://doi.org/10.48550/arXiv.2002.00444

Kirilenko, A., Kyle, A. S., Samadi, M., & Tuzun, T. (2017). The flash crash: High‐frequency trading in an electronic market. The Journal of Finance, 72(3), 967-998. https://doi.org/10.1111/jofi.12498

Kitchens, B., Johnson, S. L., & Gray, P. (2020). Understanding Echo Chambers and Filter Bubbles: The Impact of Social Media on Diversification and Partisan Shifts in News Consumption. MIS Quarterly, 44(4), 1619-1649. https://doi.org/10.25300/MISQ/2020/16371

Kleinberg, J., Ludwig, J., Mullainathan, S., & Sunstein, C. R. (2018). Discrimination in the Age of Algorithms. Journal of Legal Analysis, 10, 113-174. https://doi.org/10.1093/jla/laz001

Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016). Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807. https://doi.org/10.48550/arXiv.1609.05807

Kleinberg, J., Mullainathan, S., & Raghavan, M. (2022). The Challenge of Understanding What Users Want: Inconsistent Preferences and Engagement Optimization. arXiv preprint arXiv:2202.11776. https:/doi.org/10.48550/arXiv.2202.11776

Ko, H., Lee, S., Park, Y., & Choi, A. (2022). A survey of recommendation systems: Recommendation models, techniques, and application fields. Electronics, 11(1), 141. https://doi.org/10.3390/electronics11010141

Köbis, N., Bonnefon, J. F., & Rahwan, I. (2021). Bad machines corrupt good morals. Nature Human Behaviour, 5(6), 679-685. https://doi.org/10.1038/s41562-021-01128-2

Koenecke, A., Nam, A., Lake, E., Nudell, J., Quartey, M., Mengesha, Z., ... & Goel, S. (2020). Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences, 117(14), 7684-7689. https://doi.org/10.1073/pnas.1915768117

Krueger, P., Callaway, F., Gul, S., Griffiths, T., & Lieder, F. (2022, January 7). Identifying Resource-Rational Heuristics for Risky Choice. https://doi.org/10.31234/osf.io/mg7dn

Kruglanski, A. W., &  Gigerenzer, G. (2011). Intuitive and deliberate judgments are based on common principles. Psychological Review, 118(1), 97 -109.  https://doi.org/10.1037/a0020762

Krumhuber, E. G., & Manstead, A. S. R. (2009). Can Duchenne smiles be feigned? New evidence on felt and false smiles. Emotion, 9(6), 807–820. https://doi.org/10.1037/a0017844

Kunda, Z. (1990). The case for motivated reasoning. Psychological Bulletin, 108(3), 480–498. https://doi.org/10.1037/0033-2909.108.3.480

Lai, T. L., & Robbins, H. (1985). Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6(1), 4-22. https://doi.org/10.1016/0196-8858(85)90002-8

Lambrecht, A., & Tucker, C. (2019). Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads. Management Science, 65(7), 2966-2981. https://doi.org/10.1287/mnsc.2018.3093

Langewiesche, W. (2014, September 17). The human factor. Vanity Fair. https://www.vanityfair.com/news/business/2014/10/air-france-flight-447-crash 

Larrick, R. P. (2004). Debiasing. In D. J. Koehler & N. Harvey (Eds.), Blackwell handbook of judgment and decision making (316-338). Blackwell Publishing. https://doi.org/10.1002/9780470752937.ch16

Larrick, R. P., Morgan, J. N., & Nisbett, R. E. (1990). Teaching the use of cost-benefit reasoning in everyday life. Psychological Science, 1(6), 362-370. https://doi.org/10.1111/j.1467-9280.1990.tb00243.x

Lazer, D. M., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., ... & Zittrain, J. L. (2018). The science of fake news. Science, 359(6380), 1094-1096. https://doi.org/10.1126/science.aao2998

Le Mero, L., Yi, D., Dianati, M., & Mouzakitis, A. (2022). A Survey on Imitation Learning Techniques for End-to-End Autonomous Vehicles. IEEE Transactions on Intelligent Transportation Systems. https://doi.org/10.1109/TITS.2022.3144867

Lee, E. W., Ho, S. S., & Lwin, M. O. (2017). Explicating problematic social network sites use: A review of concepts, theoretical frameworks, and future directions for communication theorizing. New Media & Society, 19(2), 308-326. https://doi.org/10.1177/1461444816671891

Lee, Y. S., Lee, J. Y., Choi, T. Y., & Choi, J. T. (2013). Home visitation program for detecting, evaluating and treating socially withdrawn youth in Korea. Psychiatry and Clinical Neurosciences, 67(4), 193-202. https://doi.org/10.1111/pcn.12043

Lee, C. Y., & Morewedge, C. K. (2022). Noise increases anchoring effects. Psychological Science, 33(1), 60-75. https://doi.org/10.1177/09567976211024254

Lerner, J. S., & Tetlock, P. E. (1999). Accounting for the effects of accountability. Psychological Bulletin, 125(2), 255–275. https://doi.org/10.1037/0033-2909.125.2.255

Leung, E., Paolacci, G., & Puntoni, S. (2018). Man versus machine: Resisting automation in identity-based consumer behavior. Journal of Marketing Research, 55(6), 818-831. https://doi.org/10.1177/0022243718818423

Levy, R. E. (2021). Social media, news consumption, and polarization: Evidence from a field experiment. American Economic Review, 111(3), 831-870. https://doi.org/10.1257/aer.20191777

Li, T. C. (2023). Algorithmic Destruction. SMU Law Review, Forthcoming.

Lichtenstein, S., & Slovic, P. (1971). Reversals of preference between bids and choices in gambling decisions. Journal of Experimental Psychology, 89(1), 46. https://doi.org/10.1037/h0031207

Lindström, B., Bellander, M., Schultner, D. T., Chang, A., Tobler, P. N., & Amodio, D. M. (2021). A computational reward learning account of social media engagement. Nature Communications, 12(1), 1-10. https://doi.org/10.1038/s41467-020-19607-x

Lischka, J. A. (2021). Logics in social media news making: How social media editors marry the Facebook logic with journalistic standards. Journalism, 22(2), 430-447. https://doi.org/10.1177/1464884918788472

Ljungqvist, A., Marston, F., Starks, L. T., Wei, K. D., & Yan, H. (2007). Conflicts of interest in sell-side research and the moderating role of institutional investors. Journal of Financial Economics, 85(2), 420-456. https://doi.org/10.1016/j.jfineco.2005.12.004

Logg, J. M. (2019). Using Algorithms to Understand the Biases in Your Organization. Harvard Business Review. https://hbr.org/2019/08/using-algorithms-to-understand-the-biases-in-your-organization

Logg, J. M. (2022). The psychology of Big Data: Developing a “theory of machine” to examine perceptions of algorithms. American Psychological Association. https://doi.org/10.1037/0000290-011 

Logg, J. M., Minson, J. A., & Moore, D. A. (2019). Algorithm appreciation: People prefer algorithmic to human judgment. Organizational Behavior and Human Decision Processes, 151, 90-103. https://doi.org/10.1016/j.obhdp.2018.12.005

Long, M. (2022, April 25). What is my phone number? WhistleOut. https://www.whistleout.com/CellPhones/Guides/what-is-my-phone-number 

Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019). Resistance to medical artificial intelligence. Journal of Consumer Research, 46(4), 629-650. https://doi.org/10.1093/jcr/ucz013

Longoni, C., & Cian, L. (2022). Artificial intelligence in utilitarian vs. hedonic contexts: The “word-of-machine” effect. Journal of Marketing, 86(1), 91-108. https://doi.org/10.1177/0022242920957347

Lundberg, S. M., Erion, G. G., & Lee, S. I. (2018). Consistent individualized feature attribution for tree ensembles. arXiv preprint arXiv:1802.03888. https://doi.org/10.48550/arXiv.1802.03888

Malagón-Amor, Á., Martín-López, L. M., Córcoles, D., González, A., Bellsolà, M., Teo, A. R., ... & Bergé, D. (2018). A 12-month study of the hikikomori syndrome of social withdrawal: Clinical characterization and different subtypes proposal. Psychiatry Research, 270, 1039-1046. https://doi.org/10.1016/j.psychres.2018.03.060

Manheim, D., & Garrabrant, S. (2018). Categorizing variants of Goodhart's Law. arXiv preprint arXiv:1803.04585. https://doi.org/10.48550/arXiv.1803.04585

Marsh, E. J., & Rajaram, S. (2019). The digital expansion of the mind: Implications of internet usage for memory and cognition. Journal of Applied Research in Memory and Cognition, 8(1), 1-14. https://doi.org/10.1016/j.jarmac.2018.11.001

Martin, K. D., & Murphy, P. E. (2017). The role of data privacy in marketing. Journal of the Academy of Marketing Science, 45(2), 135-155. https://doi.org/10.1007/s11747-016-0495-4

Martin, K. D., & Palmatier, R. W. (2020). Data privacy in retail: Navigating tensions and directing future research. Journal of Retailing, 96(4), 449-457. https://doi.org/10.1016/j.jretai.2020.10.002

Matz, S. C., & Netzer, O. (2017). Using Big Data as a window into consumers’ psychology. Current Opinion in Behavioral Sciences, 18, 7-12. https://doi.org/10.1016/j.cobeha.2017.05.009

McKenzie, C. R., & Liersch, M. J. (2011). Misunderstanding savings growth: Implications for retirement savings behavior. Journal of Marketing Research, 48(SPL), S1-S13. https://doi.org/10.1509/jmkr.48.SPL.S1

Melnikoff, D. E., & Bargh, J. A. (2018). The mythical number two. Trends in cognitive sciences, 22(4), 280-293. https://doi.org/10.1016/j.tics.2018.02.001

Melson-Silimon, A., Harris, A. M., Shoenfelt, E. L., Miller, J. D., & Carter, N. T. (2019). Personality testing and the Americans with Disabilities Act: Cause for concern as normal and abnormal personality models are integrated. Industrial and Organizational Psychology, 12(2), 119-132. https://doi.org/10.1017/iop.2018.156

Melumad, S., Hadi, R., Hildebrand, C., & Ward, A. F. (2020). Technology-augmented choice: How digital innovations are transforming consumer decision processes. Customer Needs and Solutions, 7(3), 90-101. https://doi.org/10.1007/s40547-020-00107-4

Mercier, H., & Sperber, D. (2011). Why do humans reason? Arguments for an argumentative theory. Behavioral and Brain Sciences, 34(2), 57-74. https://doi.org/10.1017/S0140525X10000968

Michaely, R., & Womack, K. L. (1999). Conflict of interest and the credibility of underwriter analyst recommendations. The Review of Financial Studies, 12(4), 653-686. https://doi.org/10.1093/rfs/12.4.653

Miconi, T., Stanley, K., & Clune, J. (2018, July). Differentiable plasticity: Training plastic neural networks with backpropagation. In International Conference on Machine Learning (pp. 3559-3568). PMLR. 

Milkman, K. L., Chugh, D., & Bazerman, M. H. (2009). How can decision making be improved?. Perspectives on Psychological Science, 4(4), 379-383. http://doi.org/10.1111/ j.1745-6924.2009.01142.x

Milkman, K. L., Gandhi, L., Patel, M. S., Graci, H. N., Gromet, D. M., Ho, H., ... & Duckworth, A. L. (2022). A 680,000-person megastudy of nudges to encourage vaccination in pharmacies. Proceedings of the National Academy of Sciences, 119(6), e2115126119. https://doi.org/10.1073/pnas.2115126119

Milkman, K. L., Gromet, D., Ho, H., Kay, J. S., Lee, T. W., Pandiloski, P., ... & Duckworth, A. L. (2021). Megastudies improve the impact of applied behavioural science. Nature, 600(7889), 478-483. https://doi.org/10.1038/s41586-021-04128-4

Milkman, K. L., Rogers, T., & Bazerman, M. H. (2009). Highbrow films gather dust: Time-inconsistent preferences and online DVD rentals. Management Science, 55(6), 1047-1059. http://doi.org/10.1287/mnsc.1080.0994

Miller, D. T., & Ross, M. (1975). Self-serving biases in the attribution of causality: Fact or fiction?. Psychological Bulletin, 82(2), 213-225. https://doi.org/10.1037/h0076486

Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2), 81. https://doi.org/10.1037/h0043158 

Modelling Animal Decisions Group, Fawcett, T. W., Fallenstein, B., Higginson, A. D., Houston, A. I., Mallpress, D. E., ... & McNamara, J. M. (2014). The evolution of decision rules in complex environments. Trends in Cognitive Sciences, 18(3), 153-161. https://doi.org/10.1016/j.tics.2013.12.012

Monaro, M., Maldera, S., Scarpazza, C., Sartori, G., & Navarin, N. (2022). Detecting deception through facial expressions in a dataset of videotaped interviews: A comparison between human judges and machine learning models. Computers in Human Behavior, 127, 107063. https://doi.org/10.1016/j.chb.2021.107063

Montoya, R. M., Horton, R. S., Vevea, J. L., Citkowicz, M., & Lauber, E. A. (2017). A re-examination of the mere exposure effect: The influence of repeated exposure on recognition, familiarity, and liking. Psychological Bulletin, 143(5), 459-498. https://doi.org/10.1037/bul0000085

Moore, D. A., & Healy, P. J. (2008). The trouble with overconfidence. Psychological Review, 115(2), 502-517. https://doi.org/10.1037/0033-295X.115.2.502

Morewedge, C. K. (2015). Utility: Anticipated, experienced, and remembered. In D. J. Koehler & N. Harvey (Eds.), Blackwell handbook of judgment and decision making 2 (295-330). Blackwell Publishing. https://doi.org/10.1002/9781118468333.ch10

Morewedge, C. K. (2022). Preference for human, not algorithm aversion. Trends in Cognitive Sciences, 26(10), 824-826. https://doi.org/10.1016/j.tics.2022.07.007

Morewedge, C. K., & Kahneman, D. (2010). Associative processes in intuitive judgment. Trends in Cognitive Sciences, 14(10), 435-440. https://doi.org/10.1016/j.tics.2010.07.004

Morewedge, C. K., Monga, A., Palmatier, R. W., Shu, S. B., & Small, D. A. (2021). Evolution of consumption: A psychological ownership framework. Journal of Marketing, 85(1), 196-218. https://doi.org/10.1177/0022242920957007

Morewedge, C. K., Mullainathan, S., Naushan, H. F., Sunstein, C. R., Kleinberg, J., Raghavan, M., & Ludwig, J. O. (2023). Human bias in algorithm design. Nature Human Behaviour, 7(11), 1822-1824. https://doi.org/10.1038/s41562-023-01724-4

Morewedge, C. K., Yoon, H., Scopelliti, I., Symborski, C. W., Korris, J. H., & Kassam, K. S. (2015). Debiasing decisions: Improved decision making with a single training intervention. Policy Insights from the Behavioral and Brain Sciences, 2(1), 129-140. https://doi.org/10.1177/2372732215600886

Morewedge, C. K., Zhu, M., & Buechel, E. C. (2019). Hedonic contrast effects are larger when comparisons are social. Journal of Consumer Research, 46(2), 286-306. https://doi.org/10.1093/jcr/ucy070

Morris, A., Phillips, J., Huang, K., & Cushman, F. (2021). Generating options and choosing between them depend on distinct forms of value representation. Psychological Science, 32(11), 1731-1746. https://doi.org/10.1177/09567976211005702

Morris, J. W. (2015). Curation by code: Infomediaries and the data mining of taste. European Journal of Cultural Studies, 18(4-5), 446-463. https://doi.org/10.1177/1367549415577387

Mullainathan, S., & Spiess, J. (2017). Machine learning: An applied econometric approach. Journal of Economic Perspectives, 31(2), 87-106. https://doi.org/10.1257/jep.31.2.87

Nam, J., Norton, M., & Waytz, A. (2021). Robo-journalism: Less biased, and more likely to be shared. Advances in Consumer Research, 49, 917. 

Namba, S., Makihara, S., Kabir, R. S., Miyatani, M., & Nakao, T. (2016). Spontaneous facial expressions are different from posed facial expressions: Morphological properties and dynamic sequences. Current Psychology, 1–13. https://doi.org/10.1007/s12144-016-9448-9

Nass, C., & Moon, Y. (2000). Machines and mindlessness: Social responses to computers. Journal of Social Issues, 56(1), 81-103. https://doi.org/10.1111/0022-4537.00153

Nelle, J., & Cushman, F. (2022). Evidence for Dynamic Consideration Set Construction in Open-Ended Problems. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 44, No. 44). 

Newman, D. T., Fast, N. J., & Harmon, D. J. (2020). When eliminating bias isn’t fair: Algorithmic reductionism and procedural justice in human resource decisions. Organizational Behavior and Human Decision Processes, 160, 149-167. https://doi.org/10.1016/j.obhdp.2020.03.008

Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175-220. https://doi.org/10.1037/1089-2680.2.2.175

Nisbett, R. E. (1993). Rules for Reasoning. Psychology Press. 

Nisbett, R. E., Fong, G. T., Lehman, D. R., & Cheng, P. W. (1987). Teaching reasoning. Science, 238(4827), 625-631. https://doi.org/10.1126/science.3672116

Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3), 231. https://doi.org/10.1037/0033-295X.84.3.231

Norton, M. I., Vandello, J. A., & Darley, J. M. (2004). Casuistry and social category bias. Journal of Personality and Social Psychology, 87(6), 817. https://doi.org/10.1037/0022-3514.87.6.817

Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447-453. https://doi.org/10.1126/science.aax2342

​​ Olsen, C. M. (2011). Natural rewards, neuroplasticity, and non-drug addictions. Neuropharmacology, 61(7), 1109-1122. https://doi.org/10.1016/j.neuropharm.2011.03.010

Oppenheimer, D. M., & Kelso, E. (2015). Information processing as a paradigm for decision making. Annual Review of Psychology, 66(1), 277-294. https://doi.org/10.1146/annurev-psych-010814-015148

Orabi, M., Mouheb, D., Al Aghbari, Z., & Kamel, I. (2020). Detection of bots in social media: A systematic review. Information Processing & Management, 57(4), 102250. https://doi.org/10.1016/j.ipm.2020.102250

Orben, A. (2020). Teenagers, screens and social media: A narrative review of reviews and key studies. Social Psychiatry and Psychiatric Epidemiology, 55(4), 407-414. https://doi.org/10.1007/s00127-019-01825-4

Othman, S., Lyons, T., Cohn, J. E., Shokri, T., & Bloom, J. D. (2021). The influence of photo editing applications on patients seeking facial plastic surgery services. Aesthetic Surgery Journal, 41(3), NP101-NP110. https://doi.org/10.1093/asj/sjaa065

Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 30(3), 286-297. https://doi.org/10.1109/3468.844354

Pariser, E. (2011). The filter bubble: What the Internet is hiding from you. Penguin. 

Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993). The adaptive decision maker. Cambridge University Press. 

Pearson, D., Watson, P., Albertella, L., & Le Pelley, M. E. (2022). Attentional economics links value-modulated attentional capture and decision making. Nature Reviews Psychology, 1-14. https://doi.org/10.1038/s44159-022-00053-z

Pedersen, M. L., Frank, M. J., & Biele, G. (2017). The drift diffusion model as the choice rule in reinforcement learning. Psychonomic Bulletin & Review, 24(4), 1234-1251. https://doi.org/10.3758/s13423-016-1199-y

Pennycook, G., Fugelsang, J. A., & Koehler, D. J. (2015). What makes us think? A three-stage dual-process model of analytic engagement. Cognitive Psychology, 80, 34-72. https://doi.org/10.1016/j.cogpsych.2015.05.001

Pérez-Rosas, V., Abouelenien, M., Mihalcea, R., & Burzo, M. (2015). Deception detection using real-life trial data. Proceedings of the 2015 ACM on International Conference on Multimodal Interaction – ICMI, 15, 59–66. https://doi.org/10.1145/2818346.2820758

Petry, N. M., Blanco, C., Auriacombe, M., Borges, G., Bucholz, K., Crowley, T. J., Grant, B. F., Hasin, D. S., & O’Brien, C. (2014). An overview of and rationale for changes proposed for pathological gambling in DSM-5. Journal of Gambling Studies, 30(2), 493-502. https://doi.org/10.1007/s10899-013-9370-0

Porter, S., & Brinke, L. T. (2008). Reading between the lies: Identifying concealed and falsified emotions in universal facial expressions. Psychological Science, 19(5), 508-514. https://doi.org/10.1111/j.1467-9280.2008.02116.x

Promberger, M., & Baron, J. (2006). Do patients trust computers?. Journal of Behavioral Decision Making, 19(5), 455-468. https://doi.org/10.1002/bdm.542

Pronin, E., Lin, D. Y., & Ross, L. (2002). The bias blind spot: Perceptions of bias in self versus others. Personality and Social Psychology Bulletin, 28(3), 369-381. https://doi.org/10.1177/0146167202286008

Puntoni, S., Reczek, R. W., Giesler, M., & Botti, S. (2021). Consumers and artificial intelligence: An experiential perspective. Journal of Marketing, 85(1), 131-151. https://doi.org/10.1177/0022242920953847

Rafieian, O., & Yoganarasimhan, H. (2021). Targeting and privacy in mobile advertising. Marketing Science, 40(2), 193-218. https://doi.org/10.1287/mksc.2020.1235

Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J. F., Breazeal, C., ... & Wellman, M. (2019). Machine behaviour. Nature, 568(7753), 477-486. https://doi.org/10.1038/s41586-019-1138-y

Rai, A. (2020). Explainable AI: From black box to glass box. Journal of the Academy of Marketing Science, 48(1), 137-141. https://doi.org/10.1007/s11747-019-00710-5

Raji, I. D., & Buolamwini, J. (2019, January). Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial AI products. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (pp. 429-435). https://doi.org/10.1145/3306618.3314244

Rao N., Donati. D, Orozco-Olvera V. & Muñoz-Boudet, A. M. (forthcoming) Facebook Ads vs. Malaria: A Cluster-Randomized Trial in India. 

Ratcliff, R., Smith, P. L., Brown, S. D., & McKoon, G. (2016). Diffusion decision model: Current issues and history. Trends in Cognitive Sciences, 20(4), 260-281. https://doi.org/10.1016/j.tics.2016.01.007

Raveendhran, R., & Fast, N. J., (2021). Humans judge, algorithms nudge: The psychology of behavior tracking acceptance. Organizational Behavior and Human Decision Processes, 164, 11-26. https://doi.org/10.1016/j.obhdp.2021.01.001

Reber, R., Winkielman, P., & Schwarz, N. (1998). Effects of perceptual fluency on affective judgments. Psychological Science, 9(1), 45-48. https://doi.org/10.1111/1467-9280.00008

Reich, T., Kaju, A., & Maglio, S. J. (2022). How to overcome algorithm aversion: Learning from mistakes. Journal of Consumer Psychology.  https://doi.org/10.1002/jcpy.1313

Richardson, S. (2020). Affective computing in the modern workplace. Business Information Review, 37(2), 78-85. https://doi.org/10.1177/0266382120930866

Rieder, B., & Sire, G. (2014). Conflicts of interest and incentives to bias: A microeconomic critique of Google’s tangled position on the Web. New Media & Society, 16(2), 195-211. https://doi.org/10.1177/1461444813481195

Risen, J. L. (2016). Believing what we do not believe: Acquiescence to superstitious beliefs and other powerful intuitions. Psychological Review, 123(2), 182. https://doi.org/10.1037/rev0000017

Risko, E. F., & Gilbert, S. J. (2016). Cognitive offloading. Trends in Cognitive Sciences, 20(9), 676-688. https://doi.org/10.1016/j.tics.2016.07.002

Roberts, I. D., Teoh, Y. Y., & Hutcherson, C. A. (2022). Time to pay attention? Information search explains amplified framing effects under time pressure. Psychological Science, 33(1), 90-104. https://doi.org/10.1177/09567976211026983

Robinson, T. E., & Berridge, K. C. (1993). The neural basis of drug craving: An incentive-sensitization theory of addiction. Brain Research Reviews, 18(3), 247-291. https://doi.org/10.1016/0165-0173(93)90013-p

Roese, N. J. (1994). The functional basis of counterfactual thinking. Journal of Personality and Social Psychology, 66(5), 805. https://doi.org/10.1037/0022-3514.66.5.805

Rohde, D., Bonner, S., Dunlop, T., Vasile, F., & Karatzoglou, A. (2018). Recogym: A reinforcement learning environment for the problem of product recommendation in online advertising. arXiv preprint arXiv:1808.00720.  
https://doi.org/10.48550/arXiv.1808.00720

Rosenthal-von der Pütten, A. M., Hastall, M. R., Köcher, S., Meske, C., Heinrich, T., Labrenz, F., & Ocklenburg, S. (2019). “Likes” as social rewards: Their role in online social comparison and decisions to like other people's selfies. Computers in Human Behavior, 92, 76-86. https://doi.org/10.1016/j.chb.2018.10.017

Rowe, P.G. (1991). Design thinking. MIT press. 

Rozenblit, L., & Keil, F. (2002). The misunderstood limits of folk science: An illusion of explanatory depth. Cognitive Science, 26(5), 521-562. https://doi.org/10.1207/s15516709cog2605_1

Sanfey, A. G., Rilling, J. K., Aronson, J. A., Nystrom, L. E., & Cohen, J. D. (2003). The neural basis of economic decision making in the ultimatum game. Science, 300(5626), 1755-1758. https://doi.org/10.1126/science.1082976

Saxe, L., Dougherty, D., & Cross, T. (1985). The validity of polygraph testing: Scientific analysis and public controversy. American Psychologist, 40(3), 355–366. https://doi.org/10.1037/0003-066X.40.3.355

Schelling, T. C. (1984). Choice and consequence. Harvard University Press. 

Schultz, P. W., Nolan, J. M., Cialdini, R. B., Goldstein, N. J., & Griskevicius, V. (2007). The constructive, destructive, and reconstructive power of social norms. Psychological Science, 18(5), 429-434. https://doi.org/10.1111/j.1467-9280.2007.01917.x

Schwarz, N., Bless, H., Strack, F., Klumpp, G., Rittenauer-Schatka, H., & Simons, A. (1991). Ease of retrieval as information: Another look at the availability heuristic. Journal of Personality and Social Psychology, 61(2), 195. https://doi.org/10.1037/0022-3514.61.2.195

Schwartz, B., Ward, A., Monterosso, J., Lyubomirsky, S., White, K., & Lehman, D. R. (2002). Maximizing versus satisficing: happiness is a matter of choice. Journal of Personality and Social Psychology, 83(5), 1178-1197. https://doi.org/10.1037/0022-3514.83.5.1178

Scopelliti, I., Morewedge, C. K., McCormick, E., Min, H. L., Lebrecht, S., & Kassam, K. S. (2015). Bias blind spot: Structure, measurement, and consequences. Management Science, 61(10), 2468-2486. https://doi.org/10.1287/mnsc.2014.2096

Seabrook, E. M., Kern, M. L., & Rickard, N. S. (2016). Social networking sites, depression, and anxiety: A systematic review. JMIR Mental Health, 3(4), e5842. https://doi.org/10.2196/mental.5842

Sedikides, C., Alicke, M. D., & Skowronski, J. J. (2021). On the utility of the self in social perception: An egocentric tactician model. In Advances in Experimental Social Psychology (Vol. 63, pp. 247-298). Academic Press. https://doi.org/10.1016/bs.aesp.2020.11.005

Sedlmeier, P., & Gigerenzer, G. (2001). Teaching Bayesian reasoning in less than two hours. Journal of Experimental Psychology: General, 130(3), 380. https://doi.org/10.1037//0096-3445.130.3.380

Selenko, E., Bankins, S., Shoss, M., Warburton, J., & Restubog, S. L. D. (2022). Artificial Intelligence and the Future of Work: A Functional-Identity Perspective. Current Directions in Psychological Science, 31(3), 272-279. https://doi.org/10.1177/09637214221091823

Sellier, A. L., Scopelliti, I., & Morewedge, C. K. (2019). Debiasing training improves decision making in the field. Psychological Science, 30(9), 1371-1379. https://doi.org/10.1177/0956797619861429

Shah, A. K., & Oppenheimer, D. M. (2007). Easy does it: The role of fluency in cue weighting. Judgment and Decision Making, 2(6), 371-379. 

Shah, A. K., & Oppenheimer, D. M. (2008). Heuristics made easy: An effort-reduction framework. Psychological bulletin, 134(2), 207. https://doi.org/10.1037/0033-2909.134.2.207

Shank, D. B., & DeSanti, A. (2018). Attributions of morality and mind to artificial intelligence after real-world moral violations. Computers in Human Behavior, 86, 401-411. https://doi.org/10.1016/j.chb.2018.05.014

Shapiro, S. P. (2005). Agency theory. Annual Review of Sociology, 263-284. https://doi.org/10.1146/annurev.soc.31.041304.122159

Shariff, A., Bonnefon, J. F., & Rahwan, I. (2017). Psychological roadblocks to the adoption of self-driving vehicles. Nature Human Behaviour, 1(10), 694-696. https://doi.org/10.1111/spc3.12364

Sherman, D. K., Brookfield, J., & Ortosky, L. (2017). Intergroup conflict and barriers to common ground: A self-affirmation perspective. Social and Personality Psychology Compass, 11(12), e12364. 

Shin, D. (2021). The effects of explainability and causability on perception, trust, and acceptance: Implications for explainable AI. International Journal of Human-Computer Studies, 146, 102551. https://doi.org/10.1016/j.ijhcs.2020.102551

Shiv, B., & Fedorikhin, A. (1999). Heart and mind in conflict: The interplay of affect and cognition in consumer decision making. Journal of Consumer Research, 26(3), 278-292. https://doi.org/10.1086/209563

Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017). Fake news detection on social media: A data mining perspective. ACM SIGKDD Explorations Newsletter, 19(1), 22-36. https://doi.org/10.1145/3137597.3137600

Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., ... & Hassabis, D. (2017). Mastering the game of go without human knowledge. Nature, 550(7676), 354-359. https://doi.org/10.1038/nature24270

Simon, H. A. (1990). Bounded rationality. In Utility and probability (pp. 15-18). Palgrave Macmillan, London. 

Sloman, S. A. (1996). The empirical case for two systems of reasoning. Psychological Bulletin, 119(1), 3. https://doi.org/10.1037/0033-2909.119.1.3

Sloman, S., & Fernbach, P. (2018). The knowledge illusion: Why we never think alone. Penguin. 

Slovic, P., Finucane, M. L., Peters, E., & MacGregor, D. G. (2007). The affect heuristic. European Journal of Operational Research, 177(3), 1333-1352. https://doi.org/10.1016/j.ejor.2005.04.006

Soll, J. B., Milkman, K. L., & Payne, J. W. (2015). A user's guide to debiasing. In D. J. Koehler & N. Harvey (Eds.), Blackwell handbook of judgment and decision making 2 (924-951). Blackwell Publishing. https://doi.org/10.1002/9781118468333.ch33

Sparrow, B., Liu, J., & Wegner, D. M. (2011). Google effects on memory: Cognitive consequences of having information at our fingertips. Science, 333(6043), 776-778. https://doi.org/10.1126/science.1207745 

Stanovich, K. E., & West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate?. Behavioral and Brain Sciences, 23(5), 645-665. https://doi.org/10.1017/S0140525X00003435

Stewart, N., Chater, N., & Brown, G. D. (2006). Decision by sampling. Cognitive Psychology, 53(1), 1-26. https://doi.org/10.1016/j.cogpsych.2005.10.003

Stigler, G. J. (1961). The economics of information. Journal of Political Economy, 69(3), 213-225. https://doi.org/10.1086/258464

Storm, B. C., & Patel, T. N. (2014). Forgetting as a consequence and enabler of creative thinking. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40, 1594–1609. https://doi.org/10.1037/xlm0000006

Storm, B. C., & Stone, S. M. (2015). Saving-enhanced memory: The benefits of saving on the learning and remembering of new information. Psychological Science, 26(2), 182–188. https://doi.org/10.1177/0956797614559285

Storm, B. C., Stone, S. M., & Benjamin, A. S. (2017). Using the Internet to access information inflates future use of the Internet to access other information. Memory, 25(6), 717-723. https://doi.org/10.1080/09658211.2016.1210171

Stuart, G., Spruston, N., Sakmann, B., & Häusser, M. (1997). Action potential initiation and backpropagation in neurons of the mammalian CNS. Trends in Neurosciences, 20(3), 125-131. https://doi.org/10.1016/s0166-2236(96)10075-8

Study reveals where drivers are most reliant on their GPS. (n.d.). UnitedTires Library. https://www.utires.com/articles/where-drivers-need-gps-the-most/ 

Su, L., & Levine, M. (2016). Does “lie to me” lie to you? An evaluation of facial clues to high-stakes deception. Computer Vision and Image Understanding, 147, 52–68. https://doi.org/10.1016/j.cviu.2016.01.009

Sun, Y., & Zhang, Y. (2021). A review of theories and models applied in studies of social media addiction and implications for future research. Addictive Behaviors, 114, 106699. https://doi.org/10.1016/j.addbeh.2020.106699

Sutton, R. S., & Barto, A. G. (1981). Toward a modern theory of adaptive networks: Expectation and prediction. Psychological Review, 88(2), 135. https://doi.org/10.1037/0033-295X.88.2.135

Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. MIT Press. 

Sutton, R. T., Pincock, D., Baumgart, D. C., Sadowski, D. C., Fedorak, R. N., & Kroeker, K. I. (2020). An overview of clinical decision support systems: Benefits, risks, and strategies for success. NPJ Digital Medicine, 3(1), 1-10. https://doi.org/10.1038/s41746-020-0221-y

Sweeney, L. (2013). Discrimination in online ad delivery. Communications of the ACM, 56(5), 44-54. https://doi.org/10.1145/2460276.2460278

Tahery, S., & Farzi, S. (2020). Customized query auto-completion and suggestion—A review. Information Systems, 87, 101415. https://doi.org/10.1287/mnsc.2015.2344

Tankard, M. E., & Paluck, E. L. (2016). Norm perception as a vehicle for social change. Social Issues and Policy Review, 10(1), 181-211. https://doi.org/10.1111/sipr.12022

Tannenbaum, D., Fox, C. R., & Ülkümen, G. (2017). Judgment extremity and accuracy under epistemic vs. aleatory uncertainty. Management Science, 63(2), 497-518. https://doi.org/10.1287/mnsc.2015.2344

Teo, A. R., Fetters, M. D., Stufflebam, K., Tateno, M., Balhara, Y., Choi, T. Y., ... & Kato, T. A. (2015). Identification of the hikikomori syndrome of social withdrawal: Psychosocial features and treatment preferences in four countries. International Journal of Social Psychiatry, 61(1), 64-72. https://doi.org/10.1177/0020764014535758

Todd, P. M., & Gigerenzer, G. (2000). Précis of simple heuristics that make us smart. Behavioral and Brain Sciences, 23(5), 727-741. https://doi.org/10.1017/S0140525X00003447

Topol, E. J. (2019). High-performance medicine: The convergence of human and artificial intelligence. Nature Medicine, 25(1), 44-56. https://doi.org/10.1038/s41591-018-0300-7

Treisman, A. M. (1969). Strategies and models of selective attention. Psychological Review, 76(3), 282-299. https://doi.org/10.1037/h0027242

Tucker, C. E. (2012). The economics of advertising and privacy. International Journal of Industrial Organization, 30(3), 326-329. https://doi.org/10.1016/j.ijindorg.2011.11.004

Tversky, A. (1972). Elimination by aspects: A theory of choice. Psychological Review, 79(4), 281. https://doi.org/10.1037/h0032955

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124-1131. https://doi.org/10.1126/science.185.4157.1124

Ülkümen, G., Fox, C. R., & Malle, B. F. (2016). Two dimensions of subjective uncertainty: Clues from natural language. Journal of Experimental Psychology: General, 145(10), 1280. https://doi.org/10.1037/xge0000202

Van Rossum, M. C., Bi, G. Q., & Turrigiano, G. G. (2000). Stable Hebbian learning from spike timing-dependent plasticity. Journal of Neuroscience, 20(23), 8812-8821. https://doi.org/10.1523/jneurosci.20-23-08812.2000

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30. 

Verduyn, P., Gugushvili, N., Massar, K., Täht, K., & Kross, E. (2020). Social comparison on social networking sites. Current Opinion in Psychology, 36, 32-37. 

Villermet, Q., Poiroux, J., Moussallam, M., Louail, T., & Roth, C. (2021, September). Follow the guides: disentangling human and algorithmic curation in online music consumption. In Fifteenth ACM Conference on Recommender Systems (pp. 380-389). https://doi.org/10.1145/3460231.3474269

Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P. A. (2008, July). Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning (pp. 1096-1103). https://doi.org/10.1145/1390156.1390294

Von Neumann, J., & Morgenstern, O. (1944). Theory of games and economic behavior. Princeton University Press. 

Von Winterfeldt, D. and Edwards, W. (1986) Decision analysis and behavioral research. Cambridge University Press. 

Wakabayashi, Y., Itoh, S., & Nagami, Y. (2011). The use of geospatial information and spatial cognition of taxi drivers in Tokyo. Procedia-Social and Behavioral Sciences, 21, 353-361. https://doi.org/10.1016/j.sbspro.2011.07.030

Wall, J., & Krummel, T. (2020). The digital surgeon: How big data, automation, and artificial intelligence will change surgical practice. Journal of Pediatric Surgery, 55, 47-50. https://doi.org/10.1016/j.jpedsurg.2019.09.008

Wang, X., Wang, Y., Hsu, D., & Wang, Y. (2014). Exploration in interactive personalized music recommendation: A reinforcement learning approach. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 11(1), 1-22. https://doi.org/10.1145/2623372

Ward, A. F. (2013). Supernormal: How the Internet is changing our memories and our minds. Psychological Inquiry, 24(4), 341-348. https://doi.org/10.1080/1047840X.2013.850148

Ward, A. F. (2021). People mistake the Internet’s knowledge for their own. Proceedings of the National Academy of Sciences, 118(43), e2105061118. https://doi.org/10.1073/pnas.2105061118

Ward, A. F., & Lynch Jr, J. G. (2019). On a need-to-know basis: How the distribution of responsibility between couples shapes financial literacy and financial outcomes. Journal of Consumer Research, 45(5), 1013-1036. https://doi.org/10.1093/jcr/ucy037

Waters, J., & Helmchen, F. (2004). Boosting of action potential backpropagation by neocortical network activity in vivo. Journal of Neuroscience, 24(49), 11127-11136. https://doi.org/10.1523/JNEUROSCI.2933-04.2004

Weber, E. U., & Johnson, E. J. (2009). Mindful judgment and decision making. Annual Review of Psychology, 60, 53. https://doi.org/10.1146/annurev.psych.60.110707.163633  

Wegner, D. M. (1987). Transactive memory: A contemporary analysis of the group mind. In Theories of group behavior (pp. 185-208). Springer, New York, NY. 

Weiss, S. M., Indurkhya, N., & Zhang, T. (2015). Fundamentals of predictive text mining. Springer. 

West, P.M., Ariely, D., Bellman, S. et al. (1999). Agents to the rescue?. Marketing Letters 10, 285–300. https://doi.org/10.1023/A:1008127022539

Whittington, J. C., & Bogacz, R. (2017). An approximation of the error backpropagation algorithm in a predictive coding network with local Hebbian synaptic plasticity. Neural Computation, 29(5), 1229-1262. https://doi.org/10.1162/NECO_a_00949

Wickens, J., & Kötter, R. (1994). 10 Cellular models of reinforcement. Models of Information Processing in the Basal Ganglia, 187–214. 

Wilson, T. D., & Dunn, E. W. (2004). Self-knowledge: Its limits, value, and potential for improvement. Annual Review of Psychology, 55(1), 493-518. https://doi.org/ 10.1146/annurev.psych.55.090902.141954

Woods, H. C., & Scott, H. (2016). # Sleepyteens: Social media use in adolescence is associated with poor sleep quality, anxiety, depression and low self-esteem. Journal of Adolescence, 51, 41-49. https://doi.org10.1016/j.adolescence.2016.05.008

Xanidis, N., & Brignell, C. M. (2016). The association between the use of social network sites, sleep quality and cognitive function during the day. Computers in Human Behavior, 55(Part A), 121-126. https://doi.org/10.1016/j.chb.2015.09.004

Yala, A., Mikhael, P. G., Lehman, C., Lin, G., Strand, F., Wan, Y. L., ...  & Barzilay, R. (2022). Optimizing risk-based breast cancer screening policies with reinforcement learning. Nature Medicine, 28(1), 136-143. https://doi.org/10.1038/s41591-021-01599-w

Yalcin, G., Lim, S., Puntoni, S., & van Osselaer, S. M. (2022). Thumbs up or down: Consumer reactions to decisions by algorithms versus humans. Journal of Marketing Research, 59(4), 696-717. https://doi.org/10.1177/00222437211070016

Yalcin, G., Themeli, E., Stamhuis, E., Philipsen, S., & Puntoni, S. (2022). Perceptions of justice by algorithms. Artificial Intelligence and Law, 1-24. https://doi.org/10.1007/s10506-022-09312-z

Yam, K. C., & Reynolds, S. J. (2016). The effects of victim anonymity on unethical behavior. Journal of Business Ethics, 136(1), 13-22. https://doi.org/10.1007/s10551-014-2367-5

Yang, X., Guo, Y., Liu, Y., & Steck, H. (2014). A survey of collaborative filtering based social recommender systems. Computer Communications, 41, 1-10. https://doi.org/10.1016/j.comcom.2013.06.009

Yeomans, M., Shah, A., Mullainathan, S., & Kleinberg, J. (2019). Making sense of recommendations. Journal of Behavioral Decision Making, 32(4), 403-414. https://doi.org/10.1002/bdm.2118

Yoon, H., Scopelliti, I., & Morewedge, C. K. (2021). Decision making can be improved through observational learning. Organizational Behavior and Human Decision Processes, 162, 155-188. https://doi.org/10.1016/j.obhdp.2020.10.011

Yu, C., Liu, J., Nemati, S., & Yin, G. (2021). Reinforcement learning in healthcare: A survey. ACM Computing Surveys (CSUR), 55(1), 1-36. https://doi.org/10.1145/3477600

Yurtsever, E., Lambert, J., Carballo, A., & Takeda, K. (2020). A survey of autonomous driving: Common practices and emerging technologies. IEEE Access, Access, IEEE, 8, 58443-58469. https://doi.org /10.1109/ACCESS.2020.2983149 

Zajonc, R. B. (1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology, 9(2, Pt.2), 1–27. https://doi.org/10.1037/h0025848

Zhang, Z., Zhang, D., & Qiu, R. C. (2019). Deep reinforcement learning for power system applications: An overview. CSEE Journal of Power and Energy Systems, 6(1), 213-225. https://doi.org/10.17775/CSEEJPES.2019.00920

Zheng, G., Zhang, F., Zheng, Z., Xiang, Y., Yuan, N. J., Xie, X., & Li, Z. (2018, April). DRN: A deep reinforcement learning framework for news recommendation. In Proceedings of the 2018 world wide web conference (pp. 167-176). https://doi.org/10.1145/3178876.3185994

Zloteanu, M. (2020). Reconsidering facial expressions and deception detection. Handbook of facial expression of emotion, 3, 238-284. 

ENDNOTES

The Handbook of Social Psychology, 6th edition © 2025 by Situational Press is licensed under Creative Commons- Attribution-NonCommercial-NoDerivatives 4.0 International. This work may be copied and distributed only in unmodified form, for noncommercial purposes, and with attribution.

Increased content homogeneity across all users may also occur if algorithmic curation favors popular content. ↑