# ind_items

authors_index: oshotse, abraham berda, yael goldberg, amir
secondary_title_index: american sociological review
title_index: cultural tariffing: appropriation and the right to cross cultural boundaries
abstract_index:
       why are some acts of cultural boundary-crossing considered permissible whereas others are repudiated as cultural appropriation? we argue that perceptions of cultural appropriation have formed in response to the emergence of cultural omnivorousness as a dominant form of high-status consumption, making boundary-crossing a source of cultural capital. consequently, the right to adopt a practice from a culture that is not one’s own is determined on the basis of the costs and benefits one is presumed to accrue. people express disapproval at boundary-crossing if they believe that it devalues or extracts value at the expense of the target culture. we call this process cultural tariffing. we test our theory in a between-subject experimental design, demonstrating that those who enjoy a privileged social position, as inferred from their social identity or socioeconomic status, have less normative latitude to cross cultural boundaries. this is explained by perceptions that these actors are either devaluing or exploiting the target
  culture. while symbolic boundaries and cultural distinction theories are inconsistent with our results, we find that americans who are disenchanted about group-based social mobility are the ones most likely to be outraged by cultural boundary-crossing. cultural tariffing, we therefore posit, is a form of symbolic redistribution.     

full_text_index:
       cultural tariffing: appropriation and the right to cross cultural boundaries abraham oshotse , yael berda† , amir goldberg‡   emory university, † the hebrew university of jerusalem, ‡ stanford university  november 13, 2023  abstract why are some acts of cultural boundary-crossing considered permissible whereas others are repudiated as cultural appropriation? we argue that perceptions of cultural appropriation have formed in response to the emergence of cultural omnivorousness as a dominant form of high-status consumption, making boundary-crossing a source of cultural capital. consequently, the right to adopt a practice from a culture that is not one’s own is determined on the basis of the costs and benefits one is presumed to accrue. people express disapproval at boundary-crossing if they believe that it devalues or extracts value at the expense of the target culture. we call this process cultural tariffing. we test our theory in a between-subject experimental design, demonstrating that those who enjoy a privileged social position, as inferred from their social identity or socioeconomic status, have less normative latitude to cross cultural boundaries. this is explained by perceptions that these actors are either devaluing or exploiting the target culture. while symbolic boundaries and cultural distinction theories are inconsistent with our results, we find that americans who are disenchanted about group-based social mobility are the ones most likely to be outraged by cultural boundary-crossing. cultural tariffing, we therefore posit, is a form of symbolic redistribution.   nofar gueta provided invaluable research support, and was central to developing some of the early ideas informing this research. we thank ashley martin, dale miller, adina sterling, brian lowery, members of the computational culture lab, and participants of the scancor, cornell sociology colloquium, and hbs negotiations, organizations & markets seminars for their helpful comments.  
  introduction cultural boundaries are becoming increasingly fluid (bauman 2000; brubaker 2016). from names to dress codes, practices that once uniquely identified specific social groups are being widely adopted by outsiders. not all boundaries are equally traversable, however. caryn elaine johnson, for example, an academy award winner african-american actress who is professionally known as whoopi goldberg, describes herself as jewish despite being the daughter of a baptist clergyman and having no jewish ancestry. “goldberg is my name,” she professed in a 2011 interview, “it’s part of my family, part of my heritage. just like being black.” 1 the public’s general indifference to goldberg’s identity fluidity stands in contrast to the allegations directed at hilaria baldwin, an entrepreneur who is also the spouse of celebrated actor alec baldwin, for disingenuously claiming spanish heritage. born hilary hayward-thomas in boston to parents of english, french and german descent, baldwin also gave her five children spanish first names. “their names are after people who were important to me,” she said after being accused of cultural appropriation, “they’re not names that we pulled out of a hat.” 2 recent years have seen an influx of scandals in which cultural boundary-crossing—from non-latino white proprietors selling burritos,3 to italian fashion house gucci’s introduction of an indian turban as a haute couture accessory4 —is repudiated as cultural appropriation. why is it that some forms of cultural borrowing are seen as unproblematic while others are rejected as appropriation? cultures have been borrowing, imitating and adapting each other’s practices—from language through cuisine to religious traditions—for millennia (bhabha 2012; henrich 2015). the idea that such cross-cultural adoption is a form of unauthorized taking is a fairly recent historical phenomenon. as figure 1 illustrates, “cultural appropriation” was only introduced into english discourse three decades ago, and has dramatically increased in prevalence since.5 [insert figure 1 about here]  as the term suggests, an act of boundary-crossing is deemed appropriation when the actor adopting the cultural element is seen to be using something she does not rightfully own. the different reactions to goldberg’s and baldwin’s identity traversals suggest, however, that people are not simply primordially averse to all forms of boundary-crossing. contra 1  
  conventional sociological wisdom, moreover, the license to cross cultural boundaries is not the prerogative of the privileged. dominant sociological theories, from symbolic boundaries to cultural distinction, cannot explain this pattern. instead, we propose a historically contextualized theory of the conditions under which cultural boundary-crossing is perceived as transgressive. recent decades have seen a shift towards identity fluidity and cultural omnivorousness, making cultural boundary-crossing a source of cosmopolitan capital (peterson and kern 1996; goldberg, hannan, and kovacs 2016; friedman and reeves 2020). people, we contend, consequently understand cultural practices as valuable currency (schneider 2003; buyukokutan 2011). the right to adopt a practice from a culture that is not one’s own is therefore determined on the basis of the cost one is presumed to have paid, and the benefits one is presumed to have accumulated. we further argue that people find an act of cultural boundary-crossing transgressive if they believe that it devalues that target culture, or extracts more value than would have accrued to a member of the target culture for performing the same practice. we refer to this mental accounting process as cultural tariffing. we test our theoretical predictions in a (preregistered) between-subject set of experiments, demonstrating that acts of cultural boundary-crossing beget disapproval when the actor is a member of a higher status group, relative to the group whose culture is being adopted, or has relatively high status within her group. testing the underlying mechanisms producing these reactions, we further demonstrate that people disapprove of such boundarycrossing because they believe that it devalues the target culture or that it extracts value at the expense of the target group. consistent with our theory, we find that americans who are disenchanted about group-based social mobility are the ones most likely to respond with dismay to cultural boundary-crossing scenarios. we argue that this relates to their beliefs about redistributive justice. they grant members of disadvantaged groups more license to cross cultural boundaries as a form of symbolic redistribution.  theory what is cultural appropriation? to appropriate something is to take it without permission or authority. in the cultural realm, such permission is conventionally determined on the basis of group membership. 2  
  cultural appropriation is therefore defined as an act of taking from a culture that is not one’s own (ziff and rao 1997). this seemingly straightforward definition is difficult to apply in practice, however, as group membership, cultural uniqueness or even the act of taking itself are ambiguous concepts (scafidi 2005). is a north-african immigrant to france who uses african-american lingo and fashion (beaman 2017) partaking in cultural appropriation or in permissible cultural borrowing? what about a jewish israeli serving arab hummus to her guests (hirsch 2011)? scholars have consequently elaborated on this definition, often conditioning appropriation on the power relations between the groups involved (ziff and rao 1997), the harm generated by cultural taking (matthes 2016; lalonde 2021; tuvel 2021) or the social and historical context in which this taking occurs (schneider 2003). our intention is not to add to this discussion. rather, we are concerned with why certain acts of cultural exchange are conventionally perceived as transgressive, whereas others are not. we therefore use the sociological imagery of cultural boundary-crossing (lamont and molnar 2002). we refer to the actor doing the boundary-crossing as the adopter, and to members of the group whose culture is being adopted as the target. a cultural boundary is crossed when two conditions are met. the first requires the adopter to be an outsider to the target culture. this implies that there exists a boundary delineating the target group, and that the adopter is conventionally understood as being outside that boundary. this is not always a trivial determination. the inclusion of italian musical elements in mozart’s work, for example, can be seen as appropriative or not depending on whether one sees him as germanic, or as a member of a broader european musical movement (young 2010, p. 14). we therefore focus on instances in which the adopter and target are separated by an unambiguous social boundary. second, the cultural practice being adopted needs to be conventionally associated with the target culture, but not with the adopter’s. hamburgers, for example, may have originated in germany, but have since become strongly associated with american culture. an american chef serving a hamburger is not crossing a cultural boundary. if, however, the chef were to garnish this burger with kimchi—fermented cabbage that is a staple of korean cuisine—a cultural boundary would be crossed. in what follows below, we focus on practices that are strongly and unambiguously associated with a target group. two additional definitions are in order. by “group” we mean a collection of people who share a set of unique cultural practices. group boundaries are often coextensive with other social and cultural dimensions of difference such as race, religion or ethnicity. in fact, some 3  
  groups intersect multiple such dimensions.6 but a group need not necessarily be delineated along these lines. gay and lesbian identities, for example, are associated with consistent and distinct cultural practices (ghaziani 2015). for our purposes, any identity which is conventionally perceived as having distinctive practices qualifies as a target group. of course, certain dimensions play particularly important roles in organizing social difference. in the u.s., race is an especially potent source of social cleavage. we will return to this important issue when we discuss our results. for the time being, however, we develop our argument in a manner that is abstracted from the specific dimensions of difference that undergird american society. second, by “practice” we mean a symbolic expression of group identity, such as cuisine, language or apparel. this definition specifically narrows our focus to expressions that are visible and transmittable, and excludes characteristics—such as skin tone—that are genetically inheritable. it also leaves out what young (2010) terms object appropriation, namely the taking of physical objects (such as egyptian mummies displayed in the british museum). our attention, instead, is directed at symbols that are freely reproduced. young (2010) differentiates between two types of such symbolic boundary-crossing. content boundarycrossing occurs when the adopter incorporates specific symbols, styles or motifs from the target culture into her own behavior. subject boundary-crossing, in contrast, happens when the adopter represents the target group, for example, when a painter documents members of a culture she does not belong to. although this distinction is potentially consequential, we develop our argument in general terms, encompassing both content and subject boundary-crossing. with this conceptual terminology clarified, we can now offer a definition of cultural appropriation: cultural boundary-crossing that is perceived as transgressive. note that our definition departs from conventional discussions by philosophers and cultural theorists in that we do not aspire to delineate the objective conditions that make an act of cultural boundary-crossing appropriative. rather, taking a sociological approach (schoon 2022), our definition hinges on the perception of an observer : an act of boundary-crossing is appropriative if observers generally perceive it as such (even if they do not use the language of appropriation to describe it). as various scholars have pointed out (e.g., buyukokutan 2011; tuvel 2021), discussions of cultural appropriation often conflate analytical and normative definitions, with the term sometimes used to neutrally connote boundary-crossing in general and other times to connote wrongful instances of cultural exchange. our nomenclature 4  
  obviates this confusion. we use boundary-crossing to denote acts of intergroup cultural exchange in general, and appropriation to denote those that are conventionally perceived as transgressive. as scholars of cultural appropriation have pointed out, boundary-crossing acts are often considered transgressive if they are seen as harmful to the target group (e.g., matthes 2016; lalonde 2021). some acts are inherently more harmful than others. a non-native speaking a foreign language with a purposefully exaggerated accent, for example, is unmistakably ridiculing the target culture. a child mispronouncing a foreign word while learning at school, in contrast, is not. such differences in the attributes of cultural boundary-crossing acts, and the contexts in which they are enacted, likely explain a significant proportion of the variance in people’s reactions. our focus, however, is the attributes of the adopter, not the act. people find others’ behaviors legitimate if they believe that they comply with the “rules of the game” (johnson, dowd, and ridgeway 2006). as we saw earlier, these rules appear to be applied differently to different actors. holding the act’s attributes constant, we therefore ask: when are people normatively licensed to cross cultural boundaries and when are they not?  sociological theories of cultural boundary-crossing sociological theory generally suggests that high-status actors have more freedom to cross cultural boundaries. two literatures are especially relevant. the first argues that people are innately averse to all forms of boundary-crossing. a large body of research demonstrates that behaviors that defy categorical codes breed confusion and dismay (zuckerman 1999; goldberg et al. 2016). in particular, acts that breach distinctions between categories of people are often perceived as threats to social order (zerubavel 1999; martin 2000; smangs 2016). this line of work implies that people are primordialists who see cultural categories as natural entities and who therefore universally resist cultural boundary-crossing. the vast sociological literature on this topic finds that some acts of boundary-crossing are seen as more transgressive than others (e.g., silver, childress, lee, slez, and dias 2022). this work demonstrates that variation in people’s reactions to categorical mixing often stems from their evaluations of actors’ identities. they grant high-status individuals more normative latitude to defy categorical codes than they give those lower on the status distribution (rao, monin, and durand 2005; sgourev and althuizen 2014). by extension, this would suggest  5  
  that primordial objections to cultural boundary-crossing would be relaxed the more the adopter enjoys high status. a second sociological argument, drawing on bourdieu’s (1986) theory of cultural distinction, maintains that members of groups who benefit from the social status quo are the ones most motivated to protect the symbolic boundaries that sustain it. from culinary consumption (johnston and baumann 2007) to child naming (lieberson 2000), research consistently finds that those enjoying privileged access to resources employ various tactics to symbolically distinguish themselves from members of less advantaged groups. distinction theory would therefore predict that resistance to cultural boundary-crossing should originate from high-status actors protecting existing social order, and that outrage will ensue when members of low-status groups adopt higher status culture. members of lower status groups, in contrast, should, by this rationale, have the strongest incentive to partake in cultural mixing that undercuts symbolic boundaries that sustain their disadvantage. indeed, those exhibiting refined taste are least accepting of practices that defy conventional categorical boundaries (goldberg et al. 2016). there are, of course, limits to how fiercely high-status actors can protect their cultural privilege. actively denying members of lower status groups access to prestigious cultural resources bears the risk of accusations of exclusivity and irrelevance (lena 2019). yet, to protect their cultural distinctiveness, elites often find ways to redraw cultural boundaries when pressured to be inclusive (accominotti, khan, and storer 2018). the reality of contemporary discourse on cultural appropriation, however, appears to be inconsistent both with high-status privilege and high-status distinction. acts of cultural boundary-crossing are ubiquitous; they are seldom explicitly rejected as appropriation. when such accusations are invoked, they tend to be most commonly directed at high-status individuals. in fact, in a qualitative analysis of recent cultural appropriation scandals reported in the media, we found that the overwhelming majority of cases were focused on prominent individuals, typically borrowing cultural elements associated with minority, especially socioeconomically disadvantaged, groups. the various scandals involving the kardashian-jenner family, whose members include celebrity business-people and socialites, are instructive. in 2018, for example, the reality tv show star kim kardashian was widely repudiated in the media for wearing cornrows, typically associated with the fulani people of africa, to the mtv awards ceremony. her sister, kendall jenner, has repeatedly been criticized on social media for doing the same. more recently, jenner was 6  
  widely chastised for culturally appropriating mexican culture after launching a tequila brand and starring in a commercial promoting it in 2021. this was not always the case. when elvis presley thrust rock ’n’ roll into mainstream american consciousness in the mid 1950s, for example, he and other white artists like him were mostly not seen as taking something they do not own.7 many, including some black rock ’n’ roll singers, saw this racial boundary-crossing as an opportunity to elevate the status of african-american culture (cook 2004). opposition to white artists’ adoption of this african-american musical genre came predominantly from segregationists who saw it as a threat to their vision of a racially divided america (altschuler 2003). incidentally, the primary racialized controversies presley endured in his time were accusations of racism that an investigation by the popular black publication jet magazine failed to find support for (guralnick 1995). it was only decades later that elvis, and the rise of rock ’n’ roll as the dominant popular musical genre of its time, were critically reassessed as a form of cultural appropriation (e.g., hall 1997).  omnivorousness and cultural appropriation the tendency to see certain types of boundary-crossing as appropriative, we argue, can only be fully understood in the context of broader historical processes. the last half century has seen tectonic shifts in americans’ (and westerners’, more broadly) attitudes about the value of ethnic diversity and the desirability of multicultural exchange (jimenez 2010; alba and nee 2005; joyner and kao 2005). this period also gave rise to cultural omnivorousness as a dominant form of high-status cultural consumption (peterson and kern 1996; lena 2019). privileged individuals in contemporary western and industrialized societies exhibit broad and varied cultural tastes. rather than adhering to categorical boundaries, these affluent, educated and predominantly (though, not exclusively) white consumers actively seek out cultural forms that are associated with other, especially lower status groups (grazian 2005; johnston and baumann 2007; ocejo 2017). the rise of cultural omnivorousness had two important consequences. first, it made cultural practices, hitherto seen as valuable only within their respective cultural traditions, desirable objects of consumption (grazian 2005; peterson 1999; hahl, zuckerman, and kim 2017). second, it rendered cultural boundary-crossing a form of cultural capital in and of itself, which can often be directly transferable into economic advantage (koppman 2016).  7  
  although it is rooted in a multicultural and inclusive zeitgeist (fishman and lizardo 2013), recent work demonstrates that omnivorousness embodies a logic of distinction that sees cultural breadth as a strategy to maintain high social status (ollivier 2008; khan 2012; friedman and reeves 2020). thus, individuals who borrow cultural elements from other social groups are not necessarily doing so to blur the symbolic boundaries separating them. rather, by projecting openness and carrying a cachet of cosmopolitan sophistication, they are reaffirming their exclusive high status (goldberg et al. 2016).  cultural tariffing it is against this historical backdrop, we contend, that cultural boundary-crossing came to be popularly understood through the prism of ownership and value. consequently, people assess its legitimacy by weighing the costs and benefits accruing to the parties involved. they approve of boundary-crossing as long as they believe that the adopter does not capture more value than the target.8 the right to cross a cultural boundary is contingent on having paid a price. we refer to this mental accounting as cultural tariffing. social position and tariffing observers perform these value calculations on the basis of salient information about the adopter’s and target’s social identities. in particular, they rely on information pertaining to the adopter’s relative position of social advantage to assess whether the costs and benefits accruing to the adopter are commensurate with the target’s. this, we argue, occurs in two ways. the first happens when the adopter’s social group has higher status than the target’s. this explains why baldwin’s adoption of spanish names is repudiated as appropriation whereas goldberg’s adoption of a jewish name is not. observers, we contend, interpret baldwin’s white identity and goldberg’s african-american identity as positions of relative social advantage and disadvantage, respectively. they use this information to make inferences about these different adopters’ benefits and costs, relative to typical members of their respective target groups. in the american multiethnic and multiracial context, the status ordering of various social groups is not always evident, or consensual. we therefore expect this kind of intergroup accounting to result in disapproval especially when the status difference between the adopter and target groups is unambiguous.9 8  
  hypothesis 1 (h1): acts of cultural boundary-crossing will generate disapproval if there exists an unambiguous status difference between the adopter and target groups that favors the adopter. a second way by which observers make inferences about actors’ positional advantages, or disadvantages, relates to their socioeconomic position within their group. consider, for example, the musical careers of eminem and macklemore, two white american and commercially successful rappers. while macklemore was raised in an upper-middle class family in seattle, eminem grew up in poverty to a single-parenting mother. the latter’s life ordeals are commonly featured in his lyrics. though both singers have expressed unequivocal support for racial equality and identification with the african-american community, eminem is often seen as having more legitimate right to this mode of expression whereas macklemore has more frequently been disparaged as a cultural appropriator.10 this dynamic, we contend, embodies a more general principle. adopters belonging to a high-status group but who occupy a position of low socioeconomic status within that group have more normative latitude to cross boundaries than those occupying a high-status position. the former are seen as having paid a price—in eminem’s case, analogous to the social and economic burden carried by african-american males (tate 2003)—and as paying authentic homage to the target culture. the latter, on the other hand, are seen as deriving benefits without having suffered a cost. their advantaged social position confers benefits that their low-status counterparts do not have. hypothesis 2 (h2): acts of cultural boundary-crossing will generate disapproval the higher the adopter’s socioeconomic status within her social group.  devaluation and extraction people, we argue, do not disapprove of high-status individuals’ boundary-crossing merely because they are averse to those who occupy advantaged social positions. nor are they simply reacting with outrage to the inequality between the adopter and target. rather, they see the act of cultural boundary-crossing itself as reproducing this inequality (lena 2019). this occurs through two complementary mechanisms. we refer to the first as devaluation. devaluation occurs when boundary-crossing undermines the prestige or reinforces the low status of the target identity. this can happen when 9  
  actors intentionally parody the target identity, for example, when they mockingly wear a costume for a halloween party. yet, devaluation most commonly occurs without explicit derogatory intentions. when white upper-middle class new yorkers, for example, display tribal african or native-american art in their otherwise contemporarily designed homes, these artifacts’ juxtaposition to western aesthetics underscores their “primitivity” (halle 1996). even when outsiders are explicitly well-intentioned their actions can still devalue the target identity. white participants in brown-saracino’s (2010) study of gentrification, for example, highly value the original multicultural character of the neighborhoods they move into. however, their attempts to protect the neighborhood’s authentic nature exoticizes these target identities as nobly savage (johnston and baumann 2007). devaluation occurs because the act of boundary-crossing portrays the target as inferior. this occurs when a boundary-crossing act is seen as disrespectful toward the target culture (tuvel 2021). as lamont et al (2016) demonstrate, acts of disrespect are perceived as symbolic assaults on the target’s worth. they are acts of devaluation. most boundarycrossing acts, however, are not unequivocally respectful or disrespectful. people therefore rely on their inference about the actor’s intent in determining whether the act is respectful or not. consider macklemore again. though he is clearly aware of his advantaged social position— in fact, one of his earliest songs is titled “white privilege”—his detractors express doubt about the authenticity of his intentions. they see his homage to black culture a form of cheap talk that ultimately devalues hip-hop by exoticizing it. we argue that observers make inferences about the adopter’s intentions of respect, or lack thereof, on the basis of the adopter’s connection to the target culture. adopters who have a history of connection to the target culture have invested a cost in acquainting themselves with that culture and are therefore seen as acting out of intentions of respect. those who do not have such a history, in contrast, have not paid a cost and are therefore perceived to be devaluing the target culture. hypothesis 3 (h3): acts of cultural boundary-crossing will generate disapproval the more superficial the adopter’s connection to the target culture. second, people see boundary-crossing as reinforcing inequality when members of the adopting group derive value that does not accrue to members of the target group. we refer to this mechanism as extraction. in its most simple form, extraction happens when the 10  
  adopter’s social position provides direct economic advantages that members of the target group do not have, such as when a white gallerist selling indigenous art has exclusive access to affluent buyers. often, the adopter’s benefits are more subtly produced. extraction regularly occurs because the meaning of a borrowed cultural practice becomes reinterpreted by virtue of its enactment by a member of the adopting group. consider, for example, the white youth in rodriquez’s (2006) study of an new england hip-hop scene. black men participating in hip-hop culture are often associated with the genre’s violent connotations. white men, in contrast, consume hip-hop through a colorblind interpretative frame that neutralizes these racially coded and mostly negative meanings. they can be “cool” without having to carry the burden of being black (tate 2003). extraction similarly occurs when the act of boundary-crossing confers benefits only to the adopting party, because of the cultural connotations associated with that identity. the educated middle-class bartenders and butchers in ocejo’s (2017) ethnography, for example, emphasize their handmade production processes. while they can charge a premium for that authenticity, the products of working-class craftspeople following similar procedures are discounted as unsophisticated. as many have pointed out (e.g. young 2010), cultural appropriation is often experienced by its targets as a form of theft. this subjective sense is especially exacerbated when the adopter is able to derive significant value from the act of cultural boundary-crossing. for example, macklemore’s success with white audiences traditionally outside the scope of hiphop fans is seen by his critics as extractive. they attribute his ability to access resources mostly unavailable to african-american musicians to his advantaged social position (tate 2003). thus, members of the adopting group capture the value that was created, through hard work, by members of the target group. we therefore hypothesize: hypothesis 4 (h4): acts of cultural boundary-crossing will generate disapproval the more value derived by the adopter relative to members of the target group.  research design we employed a between-subject experimental design to test the hypotheses stated above. respondents were presented with stimuli describing a hypothetical cultural boundary-crossing scenario, and were asked to provide their reactions. we created 16 scenarios overall, four  11  
  scenarios for each of the four hypotheses. each scenario includes two stimuli: one testing the treatment condition and another the control condition. each respondent was randomly presented with four scenarios, one per hypothesis, and was randomly assigned to condition within scenario. scenarios were presented in random order. our hypotheses and research design were pre-registered with osf.11 for each scenario, respondents were randomly assigned to one of two experimental conditions in which we manipulate a parameter relating to the adopter, the target, or the context in which boundary-crossing is occurring. this manipulation is designed to affect respondents’ perceptions of the extent to which the boundary-crossing act is transgressive (per its corresponding hypothesis). we refer to the condition designed to generate heightened perceptions of transgression, and consequently disapproval, as the transgressive condition, and to the condition designed to elicit muted perceptions of transgression as the permissible condition.12 we expect the transgressive conditions to generate more disapproval. respondents’ reactions are measured after each stimulus. figure 2 provides a schematic overview of the experimental procedure. [insert figure 2 about here]  for example, for the scenarios testing the socioeconomic status hypothesis (h2), the adopters in the permissible and the transgressive conditions are, respectively, white individuals of low and high socioeconomic status. an example of such stimuli is illustrated in figure 3. this scenario depicts a white female who adopts traditional indian fashion. this female is of high socioeconomic status in the transgressive condition, and of low socioeconomic status in the permissible condition. [insert figure 3 about here]  this research design provides two important advantages, above and beyond the ability to identify causal effects with a randomized controlled trial. first, because each respondent evaluates four different scenarios, we can account for fixed between-person differences in the tendency to express disapproval. indeed, we find that between-person variability alone accounts for 45% to 48% in variance in respondent reactions, suggesting that people vary significantly in the extent to which they disapprove boundary-crossing in general. second, 12  
  because each hypothesis is tested using four different scenarios, we can include scenario fixed effects in our models. this enables us to ensure that our results are robust to specific operationalization decisions, as we discuss below.  stimuli the various stimuli we use directly manipulate attributes related to the hypothesis being tested, while keeping other attributes of the cultural boundary-crossing act constant across conditions. we used four scenarios for each hypothesis (numbered with a digit referencing the hypothesis and a letter referencing the scenario’s sequential order). these scenarios are summarized in table 1 and are presented in full in appendix f. [insert table 1 about here]  stimuli related to the group status hypothesis (h1) manipulate the identity of the target group, the adopter group, or both. in all of these scenarios the high status group is white and the low status group is black. to test the socioeconomic status hypothesis (h2), we manipulated the relative socioeconomic status of a white person adopting a minority practice. we manipulate the perceived authentic connection of a white adopter to a minority group in the stimuli used to test the devaluation hypothesis (h3). finally, in the stimuli used to test the extraction hypothesis (h4) we manipulate the value derived to a white adopter. we discuss these different scenarios in more detail when reviewing the results. our objective is to test the extent to which some people have more latitude to cross cultural boundaries than others, holding the type of boundary-crossing act constant. nevertheless, we recognize that different types of boundary-crossing acts might generate more or less disapproval. though this source of variation is outside the scope of our theory, we vary the types of stimuli that we use within each scenario. our purpose is not to identify the effects of that variance on observers’ disapproval but rather to account for this variance in testing our hypotheses. we explain our approach and provide full descriptions of the stimuli in appendix f.  13  
  variables respondents were asked to provide four types of reactions, each scaled on a 5-point likert scale, ranging from “not at all” to “extremely” (see appendix a for details). these reactions either gauge respondents’ disapproval of the behavior depicted in the scenario, or tap their characterization of the behavior, as we explain below. respondent disapproval is our dependent variable. the characterization of the behavior measures the underlying mechanisms, devaluation (h3) and extraction (h4), which we hypothesize are the channels through which boundary-crossing is interpreted as transgressive and therefore generates disapproval. we therefore model them as mediators. dependent variables. we used two different variables to measure respondents’ disapproval of the adopter’s boundary-crossing. people approve others’ behaviors as long as these behaviors are compliant with the “rules of the game” (johnson et al. 2006). we therefore first asked respondents whether the behavior described in a scenario is appropriate. for ease of interpretation, we reverse-coded this variable (and labeled it inappropriate). perceptions of inappropriateness are related to assumptions about what is normatively acceptable, but do not necessarily reflect one’s own beliefs about what is wrong or undesirable. we therefore also asked respondents about whether the scenario made them feel upset. these two reactions are related, but distinct. a person can be upset even if she recognizes that a behavior is generally considered permissible, or recognize that it is seen by most as inappropriate even if she does not find it upsetting. asking respondents for their emotional response also has the advantage of gauging their subjective reactions, without requiring them to consider the expectations of others or the feasibility of denying the adopter the opportunity to cross boundaries. because feeling upset motivates people to act (carver and harmon-jones 2009; hutcherson and gross 2011), it is also related to their willingness to impose costs on adopters whose boundary-crossing they deem transgressive. an alternative would have been to ask respondents directly whether they perceive an act as cultural appropriation. we did not take this approach for two reasons. first, the term “cultural appropriation” is non-colloquial and therefore unfamiliar to many participants. using it would have required providing a clear definition. second, asking directly about cultural appropriation would have primed our participants to see acts as transgressive. our approach obviates these two disadvantages. mediators. next, we asked for reactions tapping underlying mechanisms that mediate the relationship between experimental condition and disapproval. to gauge devaluation 14  
  we asked respondents whether the adopter was being respectful to the target identity. to gauge extraction we asked whether the adopter was taking advantage of the target identity. we label this variable exploitation. for ease of interpretation we reverse-coded the variable respectful (and labeled it disrespect). in addition to respondents’ reactions, we collected a host of respondent attributes, including ideological positions and sociodemographics. full descriptions of the variables used in the analyses that follow are provided in detail in appendix a.  data we recruited 2,999 respondents through the online survey platform prolific. of these, 359 respondents failed to pass post-hoc attention checks. their responses were excluded from the analysis. this means that roughly 650 respondents evaluated each scenario (with half assigned to the transgressive condition and the other half to the permissible condition). our set of respondents embodies a comprehensive cross-section of the u.s. population (table a2).  results aggregate results we begin our analysis by examining results in the aggregate. figure 4, panel a, reports respondents’ overall reactions to the experimental manipulation. the diagram plots coefficients (and their 95% confidence intervals) estimated from four linear models, one per reaction type, where the reaction is regressed on experimental condition and responses are pooled across all scenarios. these coefficients should be interpreted as the magnitude of the effect of being in the transgressive condition, relative to being in the permissible condition, on respondents’ reactions across all scenarios. all models include respondent and scenario fixed effects. these account for differences in respondents’ general tendency to express disapproval, as well as differences in the general disapproval generated by each scenario irrespective of experimental condition. standard errors are clustered by respondent to account for within-person collinearities. as the bottom two rows of the plot illustrate, the transgressive condition systematically generates more disapproval—namely, it is perceived as more upsetting and more 15  
  inappropriate—than the permissible condition (both results are significant at p 0.001). the effects are comparable in size—both correspond to roughly 0.4 of a standard deviation in the outcome. the upper two rows plot the effects of the transgressive condition on respondents’ assessments that the adopter is behaving in a disrespectful or exploitative manner. these are the theorized mechanisms mediating the relationship between the experimental condition and respondents’ disapproval. as the plot illustrates, respondents perceive significantly greater disrespect and exploitation in the transgressive condition (both results are significant at p 0.001). these effects are comparable in size to the effects on disapproval, suggesting that respondents are more offended by the transgressive condition because, as we theorize, they perceive it to be more disrespectful and exploitative. panel b explores this relationship further. it plots the portions of the effect of being in the transgressive condition on disapproval that are mediated by disrespect and exploitation.13 overall, these mediated effects account for 77% and 81% of the overall experimental effect on feeling upset or perceptions of inappropriateness, respectively. in other words, the overwhelming majority of the experimental effect is attributable to respondents’ sense that the behavior is either disrespectful or exploitative. as the plot illustrates, disrespect and exploitation correspond differently to disapproval: whereas being upset is primarily mediated by exploitation, perceptions of inappropriateness are primarily mediated by disrespect. respondents are more outraged by exploitation, while being more likely to find disrespectful boundary-crossing normatively unacceptable. [insert figure 4 about here] overall, aggregate results are consistent with the theory of cultural tariffs. being in the transgressive condition leads to disapproval. this effect is largely mediated by perceptions of disrespect and exploitation, in support of our contention that disapproval is the result of devaluation and extraction. we proceed by disaggregating these result. we begin by analyzing the first two hypotheses about social position, and proceed by examining the scenarios relating to the mechanisms of devaluation and extraction.  social position panels a and b in figure 5 plot the experimental effects on respondents’ reactions in the scenarios testing the group status (h1) and socioeconomic status (h2) hypotheses. these 16  
  effects are estimated with linear models that include scenario fixed effects. they represent the overall effect of being in the transgressive condition, relative to the permissible condition, on respondents’ reactions, accounting for fixed differences in reactions to specific scenarios. the bottom portions plot experimental effects on respondents’ disapproval (being upset or finding the act inappropriate). in support of hypotheses 1 and 2, respondents express significantly more disapproval when the adopter has a high-status social identity (h1) or has higher socioeconomic status (h2).14 although the effects are greater in magnitude for scenarios testing group status than those testing socioeconomic status, the fact that the latter are significant suggests that expressions of disapproval are not merely driven by a desire to protect disadvantaged groups. rather, consistent with cultural tariff theory, people are outraged when individuals with high socioeconomic status cross boundaries because they interpret the adopter’s status as information about the costs and benefits they are accruing. examining the various scenarios separately provides additional insights. figure 6 plots the effects of being in the transgressive condition, relative to the permissible condition, for each individual scenario. effects were estimated using simple linear models, each regressing disapproval on experimental condition (similar models estimating experimental effects on perceptions of disrespect and exploitation are reported in table b5). panels a and b relate to the scenarios testing the group status and socioeconomic status hypotheses, respectively. [insert figure 5 about here] group status hypothesis as panel a of figure 6 illustrates, effects are equally substantial and significant for the first three scenarios testing the group status hypothesis. the first scenario, “singer” (1a), in which a white singer releases a hiphop album in the transgressive condition, and a black singer releases a country album in the permissible condition, simultaneously manipulates the adopter’s identity and the target practice. to ensure that this effect is not driven by the fact that both the actor and act are manipulated in scenario 1a, in the following two scenarios—“musician” (1b) and “alter ego” (1c)—we keep the actor constant across conditions. in the “musician” scenario, a white classical musician from new england either plays country (permissible condition) or blues (transgressive condition) in clubs, whereas in the “alter-ego” scenario an asianamerican college student adopts either a preppy (permissible condition) or gangsta-rap  17  
  (transgressive condition) persona. though the actor is white in the former and asianamerican in the latter, effects for both stimuli are near identical in size and are comparable to those produced by scenario 1a. this suggests that what is driving these experimental effects is not the identity of the adopter as much as the identity of the target. consistent with our expectations, respondents express greater disapproval when the target identity is black. moreover, the effects for scenarios 1b and 1c are almost identical in magnitude, even though the latter, in which a college student adopts an accentuated alter ego identity, is more derogatory in nature. interestingly, the “alter ego” stimulus generates a strong sense of exploitation, significantly greater than its effect on perceptions of disrespect (table b5), suggesting that respondents see the adoption of a black identity as a source of social value to the adopter. consistent with our theory, an exaggerated depiction of affluent whites is seen as less outrageous than a comparably exaggerated depiction of inner-city black culture, presumably because of the former’s advantaged social position. the experimental effects for the final scenario, where the target identity (mexican) remains constant and the identity of the adopter (white or black male) is manipulated, are significant but weaker than the first three. this, we conjecture, is related to respondents’ general perceptions that latinos are not as disadvantaged as african-americans. moreover, it demonstrates that respondents are more forceful in their negative reactions to boundarycrossing acts that target low-status groups than they are in granting members of these groups more license to cross cultural boundaries. [insert figure 6 about here] socioeconomic status hypothesis as we saw above, observers impose larger tariffs when members of privileged groups borrow elements from disadvantaged cultures. if they do so to redress distributive injustice, then the adopter’s social position within her group—above and beyond advantages conferred by group membership—should be related to her latitude to cross cultural boundaries. the socioeconomic status hypothesis scenarios test this proposition. all four scenarios testing the socioeconomic status hypothesis maintain the boundarycrossing act constant and manipulate the adopter’s identity between an affluent (transgressive condition) and working-class (permissible condition) white individual. as illustrated  18  
  in panel b of figure 6, respondents’ disapproval increases in the transgressive condition for all scenarios, but effects on perceptions of appropriateness are insignificant when the target identity is not black (scenarios 2b and 2c). the two latter scenarios—one in which the adopter wears a typical indian clothing garment (sari, 2b) and the other in which the adopter wears a typical arab garment (kaffiyeh, 2c)—generate similar levels of upset, but do not affect perceptions of inappropriateness. we conjecture that these two scenarios generate less disapproval overall, relative to scenarios in which the target identity is black, because respondents associate these identities less strongly with systemic disadvantage and therefore see less value imbalance in their targeting. though a minority, indian-americans enjoy, on average, relatively high socioeconomic status in american society. arab-americans, in contrast, are more socioeconomically marginalized, but most americans are ambivalent about this disadvantage.15 consequently, respondents are less protective of boundaries separating these groups’ identities, compared to black identity. at the same time, we also find that respondents in the transgressive condition see significantly more exploitation when the adopter wears a kaffiyeh, but not when wearing a sari (table b5). we interpret this as indication that, despite their ambivalence toward arab culture, most americans associate it with systemic disadvantage. consistent with cultural tariff theory, this leads them to see cultural boundary-crossing that targets arab identity as more exploitative than one that targets indian identity.  mechanisms: devaluation and extraction respondents’ objection to boundary-crossing by high-status actors is consistent with our theory of cultural tariffing. but their disapproval may be merely driven by their general dislike for those who occupy advantaged social positions. experimental effects on respondents’ perceptions of disrespect and exploitation suggest otherwise. as panels a and b of figure 5 illustrate, the increases in people’s evaluations of transgressive conditions as disrespectful or exploitative are of similar magnitude to the increase in their disapproval. in other words, they are outraged because they believe the adopter is devaluing the target or extracting value at its expense. panels c and d illustrate how much these perceptions channel respondents’ disapproval. the red and blue dots correspond to the portions of disapproval effects that are mediated, respectively, by disrespect and exploitation. although these perceptions do not fully mediate  19  
  disapproval, they explain a substantial proportion of it. this is especially the case for the socioeconomic status hypothesis, where disapproval is practically fully mediated by disrespect and exploitation.16 two additional conclusions emerge from these mediation analyses. first, respondents’ disapproval in the group status hypothesis (h1) is equally driven by perceptions of disrespect and exploitation. namely, the adopter’s higher group status is being implicitly associated both with an intention to devalue the target culture, and with an ability to extract value that is not accessible to members of the target group. second, as panel d illustrates, exploitation is a particularly strong mediator for scenarios testing h2, suggesting that socioeconomic status is seen more as an indication about whether the adopter is taking advantage of the target group than whether she is disrespecting it. this might explain why artists like macklemore are often accused of appropriation even if they are aware of their privilege and express strong appreciation for the target culture. observers are outraged not by these boundary crossers’ derogatory or unselfconscious intentions but by the fact that they are reaping rewards without appearing to have paid significant social costs, irrespective of their intentions. the mediation analyses reported in figure 5 support our contention that devaluation and extraction are the mechanisms undergirding cultural tariffs. but they are, ultimately, indirect effects. in the remaining scenarios we test these two mechanisms directly, by manipulating the adopter’s connection to the target group (h3) or the amount of value the adopter derives from crossing (h4).17 results reported in panels a and b of figure 7 provide strong support for both hypotheses. in both cases, respondents express significantly more disapproval in the transgressive condition (p 0.001 for all outcomes). as expected, scenarios testing the devaluation hypothesis result in especially strong perceptions that the adopter is being disrespectful, whereas those testing the extraction hypothesis result in strong perceptions of exploitation. the mediation analyses reported in panels c and d provide additional support for these two mechanisms. respondent disapproval is almost fully mediated by disrespect in scenarios testing h3 and exploitation in scenarios testing h4. at the same time, disrespect and exploitation significantly mediate experimental effects for both hypotheses, suggesting that even if devaluation and extraction are analytically distinct mechanisms, they appear to blend into one another in respondents’ subjective experiences. for example, a successful white pottery artist appropriating native-american art is seen to be more disrespectful of 20  
  native-americans than a less commercially successful white artist doing exactly the same (scenario 4c). in other words, mere economic success appears to be also interpreted, to some extent, as an indication about the adopter’s intentions to devalue the target group. examining individual scenarios, as reported in panels c and d of figure 6, provides additional insight into these mechanisms. [insert figure 7 about here] devaluation hypothesis the scenarios testing the devaluation hypothesis manipulate the adopter’s perceived connection to the target identity. as illustrated in panel c of figure 6, three of the four stimuli generate greater outrage and perceptions of inappropriateness when the adopter is seen as less authentically connected to the target group. the first scenario, however—in which a white couple names their baby with a distinctively black name—does not produce experimental effects on feeling upset or inappropriateness. nevertheless, we find that when the parents of the baby are not connected to black culture respondents interpret their naming decision as more disrespectful (table b5).18 the remaining three scenarios generate comparable effects on being upset and perceptions of inappropriateness. in the second and third scenarios adopters integrate traditional cultural elements—jewish and arab in scenarios 3b and 3c respectively—into a wedding ceremony. in the fourth scenario, where a white female adopts arab dress, we manipulate the adopter’s connection to the target culture with marriage rather than personal history. this scenario generates slightly weaker effects than the two preceding scenarios. we interpret this as indication that, consistent with the devaluation hypothesis, respondents express disapproval on the basis of the actor’s perceived authentic connection to the target culture more than her formal marriage right to that culture. extraction hypothesis the second mechanism occurs when the adopter reaps greater rewards than typical members of the target group would have reaped for similar actions. the final set of scenarios directly tests this hypothesis. their experimental effects are illustrated in panel d of figure 6. in the first three scenarios testing the extraction hypothesis the actor, a white individual, remains constant while the value derived from the act of boundary-crossing is manipulated 21  
  across experimental conditions. in the ethnic passing (4a) scenario the value accrued to the adopter, who is a job candidate, is manipulated by the type of job she is applying for and the importance that employers put on diversity. this is the scenario that, irrespective of experimental condition, generates the highest level of disapproval relative to all other scenarios used in our study.19 in support of the extraction hypothesis, we find that this scenario affects respondents’ perception of exploitation (p 0.001) but not of disrespect (table b5). namely, their increased sense of dismay and inappropriateness is attributable to the fact—as the extraction hypothesis suggests—that the act of boundary-crossing generates value for the adopter. this value is seen as unfairly extracted because the actor does not suffer the societal costs associated with being latina. the two following stimuli—“store” (4b) and “white potter” (4c)—directly manipulate the amount of economic value derived by the white adopter. in the former, success is attributed to the actor’s business connections and in the latter, where it is tied to his status as the “only white potter in navajo nation,” directly to his identity. consequently, respondents report a greater increase in perceptions of exploitation in the transgressive condition for scenario 4c than scenario 4b (table b5). they see the adopter’s success as especially exploitative because he derives value, without having to bear the disadvantages associated with being native-american, that members of that target group are unable to access. in the final scenario, “interethnic potter” (4d), we rule out that the effects we have seen are simply a product of respondents’ negative reaction to winner-take-all success. here the success of the actor remains the same, but his identity varies from white (transgressive condition) to native-american (permissible condition). respondents express far greater disapproval of the success of an outsider relative to an insider to the target group, even if the latter enjoys rare and disproportionate rewards. in other words, they disapprove unequal outcomes when such inequality is rooted in group difference.  who is sensitive to cultural boundary-crossing? experimental results indicate, consistent with our theory, that people disapprove of cultural boundary-crossing when the adopter has high status, and that this occurs because they perceive such actions as devaluing or exploitative of the target culture. that they feel upset suggests that they are morally outraged by this transgression.  22  
  most americans are not fundamentally averse to the idea that some people are more successful than others. they find such inequality unpalatable, however, if they believe it is unfairly produced (mccall 2013; mijs 2021). accordingly, we posit that our respondents are offended by boundary-crossing when they think that adopters are unfairly rewarded for their social position rather than for cultural capital they deserve. if this is indeed the case, we should find that people express greater disapproval in the transgressive conditions the more they believe that inequality in the u.s. is systemic rather than meritocratic. indeed, research shows that americans vary in their perceptions about the magnitude of inequality and their beliefs about its fundamental causes (mccall, burk, laperriere, and richeson 2017). we therefore asked respondents about their belief that ethnic and racial boundaries are primary blockages to mobility and that members of minority groups are structurally disadvantaged in their ability to translate hard work and talent into success. we refer to this sentiment as group-based mobility disenchantment.20 those most disenchanted about mobility should vigorously protect disadvantaged target groups. to explore whether susceptibility to the experimental manipulation is correlated with group-based mobility disenchantment, we perform a set of moderation analyses, where the experimental effect is interacted with group-based mobility disenchantment (for a formal specification, see appendix d). the dependent variable in these analyses is a composite measure of disapproval, averaging the respondent’s reactions upset and inappropriate.21 figure 8 plots marginal effects estimated by this model. each panel plots the effect of a different moderator. these moderation effects were estimated using separate models. as panel a illustrates, the tendency to disapprove the transgressive condition increases significantly with mobility disenchantment. while respondents who are not disenchanted (namely, who strongly disagree that there exist group-based barriers to mobility) react to transgressive boundary-crossing insignificantly differently from their reactions to permissible boundary-crossing, others react with growing disapproval as their disenchantment increases.22  these results strongly suggest that people’s negative reactions to cultural  boundary-crossing are driven by their perceptions of systemic inequality. in the absence of such perceptions, they do not find the scenarios transgressive.23 [insert figure 8 about here] panels b and c of figure 8 shed further light on the mechanisms undergirding cultural tariffing. as panel b illustrates, experimental effects are significantly moderated by 23  
  political ideology. with the exception of those identifying as strongly conservative, respondents disapprove of transgressive cultural boundary-crossing. their disapproval grows as they become more liberal. panel c demonstrates that differences in reactions to cultural boundary-crossing are also generational. this is consistent with our contention that the rise of cultural appropriation as a social problem is a fairly recent historical phenomenon. respondents 40 or younger express significantly greater disapproval at transgressive cultural boundary-crossing than those born earlier. the former came of age in the late 1990s, or later, when omnivorous commodification was already in full sway. as we report in appendix d, the moderating effect of mobility disenchantment remains robust even when it is modeled together with all other moderators, demonstrating that perceptions of inequality drive disapproval of boundary-crossing above and beyond other sociodemographic attributes. the moderation analyses also rule out two alternative explanations that attribute respondents’ reactions to their motivations, as opposed to their beliefs about group-based inequality. the first, drawing on a broad sociological literature, relates cultural preferences to social status. such an approach would argue that expressions of disapproval at boundary-crossing are simply displays of high status, especially if high-status individuals are motivated to deflect accusations of their seemingly exploitative or inauthentic omnivorous behavior (hahl et al. 2017; friedman and reeves 2020). our results do not support such an argument, however. as many have noted, education is increasingly becoming a marker of status difference in the u.s. (fischer and hout 2006). as panel d of figure 8 illustrates, the tendency to react negatively to appropriation is not moderated by education. model 4 of table d3, which includes education together with other moderators, further supports the contention that education is not driving perceptions of transgression: the effects of educational attainment are weak and insignificant. this is also inconsistent with the notion that sensitivity to appropriation is a form of moral absolutism or radical identity politics inculcated through higher education (brocic and miles 2021). those with a college degree or higher disapprove of transgressive boundary-crossing as strongly as those without one.24 a different alternative hailing from psychology—identity threat theory—posits that people seek to maintain a positive image of their group identity and, specifically, are motivated to protect the group’s differentiation from other identities (tajfel 1982). perceived threat is especially heightened for minority groups, whose identity is chronically subject to scrutiny and derogation. because members of minority groups frame their identity in contrast to 24  
  the majority culture, they may be especially motivated to maintain their group’s cultural distinctiveness (mosley and biernat 2020). they might also be more sensitive to cultural appropriation in general. we find no support for any of these expectations. in fact, we find that those belonging to disadvantaged minority groups are less, rather than more, likely to express disapproval at cultural appropriation than whites (table d3, model 4). in additional analyses, which are reported in detail in appendix e, we find that respondents do not react more disapprovingly when their group is portrayed as the target of boundary-crossing, whether they are minorities or not. altogether, we find no evidence that our results are driven by identity threat.  discussion some acts of cultural boundary-crossing are met with more resistance than others. as our results demonstrate, americans—irrespective of their own cultural backgrounds—are particularly protective of cultural identities that are seen as socially disadvantaged. yet, as we showed, people’s calculus goes beyond essentialized beliefs about group-based entitlements. a socioeconomically disadvantaged individual has more latitude to cross cultural boundaries than a well-to-do counterpart with the same cultural genealogy. as we demonstrated, this inheres in people’s perceptions about the value derived and costs incurred by the adopter, relative to the target group. the more boundary-crossing is perceived to devalue the target identity or to provide the adopter access to rewards that are inaccessible to the target group’s members, the more our respondents disapprove it. they are not merely granting passage to people on the basis of their social identities. rather, they are calculating tariffs in light of their interpretations of these cultural passengers’ costs and benefits.  cultural tariffing as symbolic redistribution we do not know how exactly people perform this mental accounting. yet, as our results suggest, these calculations are grounded in people’s understandings of systemic barriers to opportunity and historical intergroup relations. their objection to the appropriation of native-american art, for example, may be related to their beliefs about the discrimination that native-americans experience in contemporary american society or to their perceptions of native-americans as victims of a historical injustice. reactions to cultural boundarycrossing, in other words, are fundamentally dependent on individuals’ shared historical 25  
  narratives about group-based power relations and inequalities. it is very likely that people translate these beliefs into value calculations in divergent ways, and that they often over- or under-weigh certain costs and benefits inconsistently (e.g., danna-lynch 2010). regardless, as we showed, the more they are disenchanted about intergroup mobility, the more they disapprove of cultural boundary-crossing. cultural tariffing, in other words, is a form of symbolic redistribution. like international trade tariffing, cultural tariffing seeks to limit flows across cultural boundaries. in many social domains, such as immigration (flores and schachter 2018) or professional certification (gieryn 1983), group members rely on legal instruments to limit such flows. in the symbolic domain, however, “goods” are freely reproduced. they cannot be materially policed. boundary work is therefore performed by sanctioning those whose behaviors are deemed transgressive. that our respondents express dismay at boundary-crossing does not necessarily mean they would be willing to translate cultural tariffing into social sanctioning. yet, feeling upset increases the likelihood that they would pursue such a course of action. anger is a motivational force that reduces people’s inhibitions about confronting perceived wrongdoers (carver and harmon-jones 2009; hutcherson and gross 2011; lerner and tiedens 2006). indeed, in a followup study, we found that respondents expressed significantly greater willingness to penalize adopters, either financially or reputationally, in the transgressive condition.25 we interpret these results as suggestive evidence that people’s disapproval is often translated into real costs—or tariffs—imposed on perceived transgressors. media portrayals of contemporary cultural debates tend to focus on heated and often exaggerated exchanges. yet, a discomfort with devaluatory or extractive boundary-crossing is not merely the purview of radical social activists. the majority of our respondents, including those identifying as conservative or denying the existence of systemic mobility barriers, respond disapprovingly to transgressive boundary-crossing. we do not know whether most, if any, of our respondents would use the language of appropriation to describe such acts of cultural borrowing. regardless, most of them subscribe, some more reservedly than others, to a worldview that finds such acts inappropriate and upsetting. moreover, the fact that mobility disenchantment strongly explains the intensity with which they react negatively to what we term cultural appropriation suggests that their behaviors are motivated by a moral resentment for distributive injustice. they seek to offset what they perceive as nonmeritocratic material inequality by granting members of the target group exclusivity to their 26  
  cultural capital.  when does cultural tariffing emerge? because race is the primary socioeconomic cleavage in american society (mcpherson, smithlovin, and cook 2001), our experimental scenarios predominantly focused on racial boundaries. indeed, we found that respondents were especially disapproving of boundary-crossing in contexts of interracial exchange. as cultural boundaries are eroding in variety of contexts (bauman 2000; brubaker 2016), however, one might wonder whether cultural tariffing is equally vigorous in domains of cultural interaction that extend beyond racial difference. our results point to two parameters that might explain when cultural boundary-crossing is subject to tariffing, and when it is not. first, we conjecture that disapproval of cultural boundary-crossing is a product of recent cultural shifts in american society. half a century ago, omnivorous consumption as a form of status display was relatively in its infancy. as boundary-crossing became a source of capital, people began evaluating it as such. we imagine that, had we conducted our experiments fifty years ago, we would have received very different results. because we cannot go back in time, we are unable to test this hypothesis. nevertheless, historical accounts demonstrate that boundary-crossing practices—such as “playing indian” (deloria 1998)—which are increasingly rejected today as inappropriate, were once widespread and perceived as legitimate. second, while many of the scenarios we tested are interracial, our findings demonstrate that people’s disapproval of cultural boundary-crossing is not exclusively related to perceptions of racial difference. in several scenarios (e.g. 2c, 3b-3d), observers resist boundary traversals when the target is either arab or jewish, two groups that are predominantly understood through ethnic or religious, but not racial, lenses.26 more generally, we find that respondents protect cultural boundaries delineating ethnic and religious groups, as long as the adopter and target occupy a shared social space.27 the effects of group-based mobility disenchantment further suggest that observers’ objection to boundary-crossing increases as they believe that there exist non-meritocratic inequities between the groups comprising the domain in question. together, these lead us to conclude that cultural tariffing is more likely to occur when (1) group-specific practices become desirable objects of consumption and when (2) observers  27  
  attribute value imbalance to historical or systemic disparities between the groups involved. this suggests that cultural tariffing dynamics may extend beyond american race and ethnic relations. as cultural omnivorousness has become a prevalent mode of consumption throughout the industrial world (fishman and lizardo 2013), for example, it stands to reason that boundary-crossing is disapproved in non-american contexts. whereas cultural fault lines are mostly racial and ethnic in the u.s., they are predominantly religious in many western european countries (drouhot 2021; bail 2008). similarly, the gentrifying classes in new york (ocejo 2017) and london (gest 2016) are “rediscovering” old crafts while displacing the working class, both economically and geographically. cultural tariffing might even extend to domains of knowledge production such as academia. an economist popularizing an old anthropological insight, for example, might be perceived as engaging in appropriation if her use of anthropological knowledge carries a cachet of intellectual sophistication while her ability to reach a broad audience is seen as a product of her discipline’s high academic status.  distinction, status and the boundary-spanning penalty our findings shed light on two broad areas of sociological inquiry, above and beyond the specific dynamics of cultural tariffing. the first is the sociological literature on categorical boundary-spanning. research in that vein consistently demonstrates that actors that defy categorical codes are penalized, but that those enjoying high status have more normative latitude to cross categorical boundaries (rao et al. 2005; phillips and zuckerman 2001). yet high-status actors are not given a carte blanche to do so. for example, they are frowned upon if their boundary-crossing implies moral impurity (phillips, turco, and zuckerman 2013). in fact, our results demonstrate that high status can be a liability. in contexts of race- and ethnicity-based intergroup relations, it limits adopters’ normative license to cross cultural boundaries. prestige is, therefore, a double-edged sword. it provides high-status actors with an aura of competence and creativity, giving them more freedom to cross cultural boundaries (durand and kremp 2015; sgourev and althuizen 2014). but, when adopters’ high status is seen as a product of systemic inequality, it also makes their boundary-crossing behavior seem devaluatory or extractive and, consequently, illegitimate. seemingly driven by an inclusive impetus, cultural omnivorousness has become a dominant mode of high-status distinction. elites are therefore required to balance openness 28  
  with subtle forms of exclusion (ollivier 2008; khan 2012). our findings suggest that the avenues for doing so are narrowing. against the backdrop of increasing inequality in western societies, economic and cultural elites are seen to be motivated by extrinsic rewards in their pursuit of lower status culture (friedman and reeves 2020). as we showed, boundarycrossing that is rooted in structural inequities is consequently met with reproach. it may be the case that knowing which forms of boundary-crossing are permissible and which are transgressive—often referred to as the difference between cultural appreciation and appropriation (scafidi 2005; lena 2019)—is becoming an important resource for navigating this duality. we leave this question for future work.  diversity and boundary-crossing second, our findings are relevant for broader discussions about the dynamics of ethnoracial relations in the u.s. recent years have seen a growing cultural emphasis on diversity in american universities, workplaces and institutions (bell and hartmann 2007; berrey 2015). conversations about race are increasingly shifting from denials of racial difference to its recognition. such recognition, however, also introduces frictions. while diversity is acknowledged and celebrated, discussions of racial and ethnic inequality are suppressed (douds 2021). cultural tariffing, especially when resulting in public repudiations of transgressive boundarycrossing, might be an avenue for such frictions to be smoothed. rejecting colorblind ideology, explicit reference to cultural appropriation acknowledges racial and ethnic difference, indeed protects it. moreover, it recognizes race- and ethnicity-based inequality. the differential license to cross cultural boundaries appears to be intended to address these inequalities. yet this remediation is, ultimately, contained to the symbolic realm, where it does not pose direct threats to the material interests of high-status individuals.  conclusion symbolic boundaries play a fundamental role in sustaining the structural barriers that separate different groups, reinforcing the material inequalities between them (lamont and molnar 2002). sociological theory would therefore predict that an ideology emphasizing these boundaries serves the maintenance of group-based inequality. though seemingly motivated  29  
  by a redistributive impetus, cultural tariffing—ironically, perhaps—appears to be doing precisely that. even though we did not ask our respondents to describe their reactions in their own words, we imagine that those most disapproving of cultural boundary-crossing would have explained that they are not seeking to undo the erosion of these symbolic boundaries. rather, given their disappointment from what they see as the failed promises of diversity and the inauthentic inclusiveness of cultural omnivores, they believe that those most negatively impacted by unfair inequities should have the greatest license to cross cultural boundaries. it remains to be seen whether cultural tariffing undermines or exacerbate group-based inequality, if it has any material impact at all.  notes 1  https://www.thejc.com/news/uk/whoopi-goldberg-i-m-jewish-and-i-talk-to-god-1.23000  2  https://www.nytimes.com/2020/12/30/style/hilaria-baldwin-interview.html  3  https://www.nydailynews.com/news/national/burrito-shop-closes-accusations-cultural-appropriation-article1.3192942 4  https://www.washingtonpost.com/nation/2019/05/16/nordstroms-indy-full-turban-gucci-draws-sikh-protests/  5  historically, the rise of the term occurred against the backdrop of a formal recognition of culture as an exercisable human right, particularly the right to cultural heritage and integrity of indigenous peoples. the international covenant on economic, social and cultural rights entered into force in 1976 and included the right to cultural heritage. only in 2005, however, unesco adopted the first legally binding international instrument on culture, the convention on the protection and promotion of the diversity of cultural expressions. while there is little evidence that international human rights treaties themselves change public perceptions, recent studies show that international treaties affect domestic policy in democracies because prior international obligations effect political climate (chilton 2014). 6  in the u.s., for example, middle-easterners are often perceived through religious, genealogical or even racial prisms (maghbouleh, schachter, and flores 2022). 7  it may very well be the case that earlier rock ’n’ roll artists who did not experience commercial success resented elvis and other white artists’ ability to reach audiences that they had no access to, or that black audiences more broadly saw elvis’ success as an unfair gain at the expense of african-american artists. however, unlike the present day, this discontent was very rarely, if ever, expressed in public. 8  in many cases, such as when the adopter sells the products of her cultural boundary-crossing, the value gained is explicitly monetary. yet, the perceived value need not necessarily be pecuniary. an act of cultural boundary-crossing can generate status by projecting sophistication without directly translating into financial rewards. we assume that observers see value in both types of acts, and treat these as fungible sources of capital (bourdieu 2018). 9  in the us this difference is most pronounced when the adopter is white and the target is black. in the analyses below we also consider several other social identities that have been historically disadvantaged  30  
  in the u.s., including latinos, native-americans and arabs. some social identities occupy ambiguous status positions. asian-americans, for example, have experienced high socioeconomic mobility but are still associated with derogatory stereotypes (lee 2015). 10  https://www.newyorker.com/magazine/2016/03/07/macklemore-the-hip-hop-villain  11  https://osf.io/tvp4q/?view_only b2e30046ab8c4d21a7cc2045fb4028d8  12  we assume that perceptions of transgression are a matter of degree. thus these conditions should be seen as eliciting stronger or weaker perceptions of transgression. 13  these effects were estimated by a structural equations model that includes both disrespect and exploitation as mediators. this model includes scenario, but not respondent, fixed effects. 14  all results are significant at p 0.001, except the effect for the socioeconomic status hypothesis on inappropriate, for which p 0.008 15  in fact, by some accounts, arabs are the most dehumanized minority in the u.s. (kteily, bruneau, waytz, and cotterill 2015). this suggests that respondents might not see them as equals who deserve fair treatment. 16  a little less than half of the experimental effects for scenarios testing the group status hypothesis are mediated by disrespect and exploitation. this suggests that respondents’ dismay is also driven by perceptions that are not directly related to their value calculations. further analyses, in which we asked respondents whether the adopter is being “culturally insensitive,” indicate that respondents’ sense of discomfort additionally relates to perceptions that members of the target group experience greater psychological harm by boundary-crossing when their group has lower status. we leave exploring this issue for future work. 17  we included respondents’ perceptions of the actor’s competence as a control for all models testing the extraction hypothesis and reported in figure 6 (see also table 1). we do so because, as we discovered when pretesting stimuli, respondents tend to interpret commercial success as an indication of competence. this attenuated their disapproval for scenario 4b (“store”). adding competence as a control did not substantively alter results in the remaining three scenarios. nevertheless, for consistency we included it as a control in all models testing the extraction hypothesis. we provide additional details in appendix c. 18  we interpret this to be mostly related to the type of act featured in this scenario. the permanence and symbolic significance of a parent’s naming decision implies appreciation of the target identity; this, we conjecture, offsets outrage or perceptions of inappropriateness generated by the experimental manipulation. indeed, we find that this scenario is seen as least offensive on average across all scenarios used in our study, irrespective of experimental condition. 19  respondents report, on average, being 1.07 points more upset in response to this scenario relative to the mean level of dismay they express at other scenarios. 20  sociologists often explore americans’ pessimism about social mobility in general (mccall 2013). instead, we ask whether respondents believe that racial minorities are less likely to be successful, compared to whites, even if they are talented and hard-working. though our theory is about cultural identity in general, not specifically race, we ask respondents about race given that it is the most dominant dimension of systemic inequality in american society. our intention is to gauge the extent to which respondents perceive american society as unequal, and how much they believe this inequality relates to group difference.  31  
  21  we do so for ease of presentation. the results we present below are mostly replicated if applied to each variable independently. 22  when limiting moderation analyses to specific scenarios, we find that respondents who are overly optimistic about mobility significantly disapprove of transgressive boundary-crossing, but only in scenarios testing the group status hypothesis (h1). this effect is overwhelmingly driven by their sense of inappropriateness, not their personal outrage. they recognize that intergroup boundary-crossing is normatively frowned upon when there is significant status imbalance between the adopter and target groups, but are not personally upset by such crossing. because they do not see the benefits accruing to boundary-crossers as unfair, they do not express greater disapproval in other scenarios that directly test value imbalance. 23  we reproduce significant moderation effects for mobility disenchantment when estimating separate models for each hypothesis, which we do not report due to space limitations. this suggests that perceptions of systemic inequality do not only motivate objections to intergroup boundary-crossing. they also undergird observers’ negative reactions to boundary-crossing when it is done by adopters with high socioeconomic status, or when it devalues or extracts value from the target group. 24  in analyses restricted to age groups, we find that this is not driven by the fact that younger respondents have not had the opportunity to complete higher education. in fact, we find fragile evidence that among those roughly one decade past college age, offense at appropriation decreases with education, mostly because those with a college education are less forgiving of permissible boundary-crossing than those without it. 25  we replicated two of the scenarios, and asked respondents to report their behavioral intentions. in scenario 1c (“alter-ego”) we asked respondents how likely they would be to “call out” the adopter. in scenario 4d (“interethnic potter”) we asked them how likely they were to buy the artwork. in both cases, respondents reported intentions to impose higher costs in the transgressive condition (p 0.001). because the ability to sanction others is mediated by contextual factors, we leave this question for future research. 26  while arabs are sometimes seen as non-white (maghbouleh et al. 2022), jews are mostly associated with whiteness (goldstein 2008) 27  in pretesting of various scenarios we found, for example, that people object when white outsiders adopt amish cultural practices—an ultra-traditionalist anabaptist christian group—but are indifferent when the adopter is a white american and the target is the maori people of new zealand.  references accominotti, fabien, shamus r. khan, and adam storer. 2018. “how cultural capital emerged in gilded age america: musical purification and cross-class inclusion at the new york philharmonic.” american journal of sociology 123:1743–1783. alba, richard and victor nee. 2005. remaking the american mainstream: assimilation and contemporary immigration. cambridge, mass.: harvard university press, f first paperback edition used edition. 32  
  altschuler, glenn c. 2003. all shook up: how rock ’n’ roll changed america. oxford university press. bail, christopher a. 2008. “the configuration of symbolic boundaries against immigrants in europe.” american sociological review 73:37–59. bauman, zygmunt. 2000. liquid modernity. cambridge, uk : malden, ma: polity, 1st edition edition. beaman, jean. 2017. citizen outsider: children of north african immigrants in france. oakland, california: university of california press, first edition. bell, joyce m. and douglas hartmann. 2007. “diversity in everyday discourse: the cultural ambiguities and consequences of “happy talk”.” american sociological review 72:895– 914. berrey, ellen. 2015. the enigma of diversity: the language of race and the limits of racial justice. university of chicago press. bhabha, homi k. 2012. the location of culture. routledge. bourdieu, pierre. 1986. distinction: a social critique of the judgement of taste. london: routledge, 1 edition edition. bourdieu, pierre. 2018. “the forms of capital.” in the sociology of economic life, pp. 78–92. routledge. brocic, milos and andrew miles. 2021. “college and the “culture war”: assessing higher education’s influence on moral attitudes.” american sociological review 86:856–895. brown-saracino, japonica. 2010. a neighborhood that never changes: gentrification, social preservation, and the search for authenticity. chicago ; london: university of chicago press, illustrated edition edition. brubaker, rogers. 2016. trans: gender and race in an age of unsettled identities. princeton: princeton university press. buyukokutan, baris. 2011. “toward a theory of cultural appropriation: buddhism, the vietnam war, and the field of u.s. poetry.” american sociological review 76:620–639. 33  
  carver, charles s. and eddie harmon-jones. 2009. “anger is an approach-related affect: evidence and implications.” psychological bulletin 135:183–204. chilton, adam s. 2014. “the influence of international human rights agreements on public opinion: an experimental study.” chicago journal of international law 15:110–137. cook, jody. 2004. “graceland national historic landmark nomination.” united states department of the interior . danna-lynch, karen. 2010. “switching roles: the process of mental weighing.” poetics 38:166–184. deloria, philip joseph. 1998. playing indian. yale university press. douds, kiara wyndham. 2021. “the diversity contract: constructing racial harmony in a diverse american suburb.” american journal of sociology 126:1347–1388. drouhot, lucas g. 2021. “cracks in the melting pot? religiosity and assimilation among the diverse muslim population in france.” american journal of sociology 126:795–851. durand, rodolphe and pierre-antoine kremp. 2015. “classical deviation: organizational and individual status as antecedents of conformity.” academy of management journal 59:65–89. fischer, claude s. and michael hout. 2006. century of difference: how america changed in the last one hundred years. russell sage foundation. fishman, robert m. and omar lizardo. 2013. “how macro-historical change shapes cultural taste: legacies of democratization in spain and portugal.” american sociological review 78:213–239. flores, rene d. and ariela schachter. 2018. “who are the “illegals”? the social construction of illegality in the united states.” american sociological review 83:839–868. friedman, sam and aaron reeves. 2020. “from aristocratic to ordinary: shifting modes of elite distinction.” american sociological review 85:323–350. gest, justin. 2016. the new minority. new york, ny: oxford university press, 1 edition edition. 34  
  ghaziani, amin. 2015. there goes the gayborhood? princeton university press, reprint edition edition. gieryn, thomas f. 1983. “boundary-work and the demarcation of science from nonscience: strains and interests in professional ideologies of scientists.” american sociological review 48:781–795. goldberg, amir, michael t. hannan, and balazs kovacs. 2016. “what does it mean to span cultural boundaries? variety and atypicality in cultural consumption.” american sociological review 81:215–241. goldstein, eric l. 2008. the price of whiteness: jews, race, and american identity. princeton: princeton university press. grazian, david. 2005. blue chicago: the search for authenticity in urban blues clubs. university of chicago press. guralnick, peter. 1995. last train to memphis: the rise of elvis presley. boston: back bay books. hahl, oliver, ezra w. zuckerman, and minjae kim. 2017. “why elites love authentic lowbrow culture: overcoming high-status denigration with outsider art.” american sociological review 82:828–856. hall, perry a. 1997. “african-american music: dynamics of appropriation and innovation.” in borrowed power: essays on cultural appropriation, edited by bruce h. ziff and pratima v. rao, pp. 31–51. rutgers university press. halle, david. 1996. inside culture: art and class in the american home. chicago; london: university of chicago press, 1st edition edition. henrich, joseph. 2015. the secret of our success: how culture is driving human evolution, domesticating our species, and making us smarter . princeton: princeton university press. hirsch, dafna. 2011. “ “hummus is best when it is fresh and made by arabs”: the gourmetization of hummus in israel and the return of the repressed arab.” american ethnologist 38:617–630. 35  
  hutcherson, cendri a. and james j. gross. 2011.  “the moral emotions:  a so-  cial–functionalist account of anger, disgust, and contempt.” journal of personality and social psychology 100:719–737. jimenez, tomas r. 2010. “affiliative ethnic identity: a more elastic link between ethnic ancestry and culture.” ethnic and racial studies 33:1756–1775. johnson, cathryn, timothy j. dowd, and cecilia l. ridgeway. 2006. “legitimacy as a social process.” annual review of sociology 32:53–78. johnston, josee and shyon baumann. 2007. “democracy versus distinction: a study of omnivorousness in gourmet food writing.” american journal of sociology 113:165–204. joyner, kara and grace kao. 2005. “interracial relationships and the transition to adulthood.” american sociological review 70:563–581. khan, shamus rahman. 2012. privilege: the making of an adolescent elite at st. paul’s school . princeton, nj: princeton university press. koppman, sharon. 2016. “different like me: why cultural omnivores get creative jobs.” administrative science quarterly 61:291–331. kteily, nour, emile bruneau, adam waytz, and sarah cotterill. 2015. “the ascent of man: theoretical and empirical evidence for blatant dehumanization.” journal of personality and social psychology 109:901. lalonde, dianne. 2021. “does cultural appropriation cause harm?” politics, groups, and identities 9:329–346. lamont, michele and and virag molnar. 2002. “the study of boundaries in the social sciences.” annual review of sociology 28:167–195. lamont, michele, graziella moraes silva, jessica welburn, joshua guetzkow, nissim mizrachi, hanna herzog, and elisa reis. 2016. getting respect: responding to stigma and discrimination in the united states, brazil, and israel . princeton university press. lee, jennifer. 2015. “from undesirable to marriageable: hyper-selectivity and the racial mobility of asian americans.” the annals of the american academy of political and social science 662:79–93. 36  
  lena, jennifer c. 2019. entitled: discriminating tastes and the expansion of the arts. princeton: princeton university press. lerner, jennifer s. and larissa z. tiedens. 2006. “portrait of the angry decision maker: how appraisal tendencies shape anger’s influence on cognition.” journal of behavioral decision making 19:115–137. lieberson, stanley. 2000. a matter of taste: how names, fashions, and culture change. new haven, ct: yale university press. maghbouleh, neda, ariela schachter, and rene d. flores. 2022. “middle eastern and north african americans may not be perceived, nor perceive themselves, to be white.” proceedings of the national academy of sciences 119:e2117940119. martin, john levi. 2000. “what do animals do all day?: the division of labor, class bodies, and totemic thinking in the popular imagination.” poetics 27:195–231. matthes, erich hatala. 2016. “cultural appropriation without cultural essentialism?” social theory and practice 42:343–366. mccall, leslie. 2013. the undeserving rich: american beliefs about inequality, opportunity, and redistribution. cambridge university press. mccall, leslie, derek burk, marie laperriere, and jennifer a. richeson. 2017. “exposure to rising inequality shapes americans’ opportunity beliefs and policy support.” proceedings of the national academy of sciences 114:9593–9598. mcpherson, miller, lynn smith-lovin, and james m cook. 2001. “birds of a feather: homophily in social networks.” annual review of sociology 27:415–444. mijs, jonathan j b. 2021. “the paradox of inequality: income inequality and belief in meritocracy go hand in hand.” socio-economic review 19:7–35. mosley, ariel j. and monica biernat. 2020. “the new identity theft: perceptions of cultural appropriation in intergroup contexts.” journal of personality and social psychology pp. no pagination specified–no pagination specified. ocejo, richard e. 2017. masters of craft: old jobs in the new urban economy. princeton university press. 37  
  ollivier, michele. 2008. “modes of openness to cultural diversity: humanist, populist, practical, and indifferent.” poetics 36:120–147. peterson, richard a. 1999. creating country music: fabricating authenticity. university of chicago press. peterson, richard a. and roger m. kern. 1996. “changing highbrow taste: from snob to omnivore.” american sociological review 61:900–907. phillips, damon j., catherine j. turco, and ezra w. zuckerman. 2013. “betrayal as market barrier: identity-based limits to diversification among high-status corporate law firms.” american journal of sociology 118:1023–1054. phillips, damon j. and ezra w. zuckerman. 2001. “middle-status conformity: theoretical restatement and empirical demonstration in two markets.” american journal of sociology 107:379–429. rao, hayagreeva, philippe monin, and rodolphe durand. 2005. “border crossing: bricolage and the erosion of categorical boundaries in french gastronomy.” american sociological review 70:968–991. rodriquez, jason. 2006. “color-blind ideology and the cultural appropriation of hip-hop.” journal of contemporary ethnography 35:645–668. scafidi, susan. 2005. who owns culture?: appropriation and authenticity in american law . new brunswick, n.j: rutgers university press, none ed. edition edition. schneider, arnd. 2003. “on ‘appropriation’. a critical reappraisal of the concept and its application in global art practices.” social anthropology 11:215–229. schoon, eric w. 2022. “operationalizing legitimacy.” american sociological review 87:478– 503. sgourev, stoyan v. and niek althuizen. 2014. “ “notable” or “not able”: when are acts of inconsistency rewarded?” american sociological review 79:282–302. silver, daniel, clayton childress, monica lee, adam slez, and fabio dias. 2022. “balancing categorical conventionality in music.” american journal of sociology 128:224–286. 38  
  smangs, mattias. 2016. “doing violence, making race: southern lynching and white racial group formation.” american journal of sociology 121:1329–1374. tajfel, h. 1982. “social psychology of intergroup relations.” annual review of psychology 33:1–39. tate, greg. 2003. everything but the burden: what white people are taking from black culture. crown. tuvel, rebecca. 2021. “putting the appropriator back in cultural appropriation.” the british journal of aesthetics 61:353–372. young, james o. 2010. cultural appropriation and the arts. john wiley & sons. zerubavel, eviatar. 1999. social mindscapes: an invitation to cognitive sociology. cambridge, ma: harvard university press, reprint edition edition. ziff, bruce h. and pratima v. rao. 1997. introduction to cultural appropriation: a framework for analysis, pp. 1–27. rutgers university press. zuckerman, ezra w. 1999. “the categorical imperative: securities analysts and the illegitimacy discount.” american journal of sociology 104:1398–1438.  39  
  tables  40  
  41  black male adopting country music as a country musician as an affluent new england preppy black male  writer is working class female is working class male is working class  white male adopting hiphop  as a gangsta rapper  writer is affluent female is affluent male is affluent  m/c/e  female married to arab man  m / c / sp  male is commercially unsuccessful male is local native-american  male is white from the east coast  m / c / sp  r/c/e  commercially successful due to business ties male is commercially successful  m/c/p  r/c/e  analytical thinking is valued by employers commercially unsuccessful  diversity is valued by employers  r/c/e  couple grew up in black neighborhood couple immersed in jewish culture couple immersed in arab culture  couple has no connection to black culture couple recently learned about jewish culture couple recently learned about arab culture female recently visited egypt  r/c/p  male is working class  m / c / sp  r/c/e  r/c/e  m / s / sp  m/s/e  m/s/e  m / c / sp  m / c / sp  type*  male is affluent  white male  as a blues musician  permissible condition  transgressive condition  * m mimetic/r recombinant, s subject/c content, e ephemeral/s semi-permanent/p permanent (see appendix f)  label description h1: intergroup 1a singer singer releasing an album in an ethnic musical genre 1b musician white new england classical male musician performing in clubs 1c alter ego asian-american male student adopting an alter ego persona 1d costume male organizing a mexican-themed “calavera party” h2: intragroup 2a author white male writes a book on the travails of black georgians in the 1940s 2b sari white female wears an indian sari as a fashion accessory 2c kaffiyeh white male wears an arab kaffiyeh as a fashion accessory 2d rapper white male releases a hiphop album h3: devaluation 3a baby white couple names their baby with a distinctively black name 3b jewish white couple includes jewish traditions wedding in their wedding 3c arab white couple includes arab traditions wedding in their wedding 3d dress female dressed in traditional arab garb h4: extraction 4a passing white female passes as latina during job application 4b store white male proprietor of latino clothing store 4c white white male hopi potter potter 4d interethnic male is a commercially successful hopi potter potter  table 1: description of scenarios  
  figures  1.6  10 -7  cultural appropriation in written english  1.4  probability  1.2 1 0.8 0.6 0.4 0.2 0 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010  year  figure 1: the probability, by year, of the bigram “cultural appropriation” as indexed by the google ngram viewer project. the probability is calculated as the likelihood of the term appearing in a document during that year. the google ngram viewer is the most comprehensive collection of texts published in the english language, containing upwards of 8 million books.  42  
  1  select one of 4 scenarios  2  assign to experimental condition permissible  transgressive  3  get reactions  is this scenario: 1. 2. 3. 4.  making you upset inappropriate disrespectful exploitative  repeat for each hypothesis  figure 2: a schematic overview of the experimental procedure.  permissible lauren anderson, 28, works as a cashier at a car wash facility in los angeles. she rents a modest apartment with her three housemates. lauren has recently discovered indian fashion, and has begun incorporating traditional indian saris into her regular outfits. she especially likes to wear these at work, where she feels they make her “stand out.”  transgressive lauren anderson, 28, works in her family's boutique real estate agency that caters to some of the wealthiest residents of los angeles. she lives in a beachfront house owned by her parents. lauren has recently discovered indian fashion, and has begun incorporating traditional indian saris into her regular outfits. she especially likes to wear these at work, where she feels they make her “stand out.”  figure 3: example of a scenario (2c) testing the intragroup hypothesis (h2). subjects in both conditions are presented with the same two images and slightly modified accompanying texts. the differences between the texts presented in each condition are highlighted.  43  
  experimental effects  a  effect mediation  b  mechanisms  exploitation  inappropriate  disrespect  exploitation  inappropriate  upset disrespect  upset disapproval 0  0.1 0.2 0.3 0.4 0.5 0.6  0  0.1  0.2  0.3  0.4  figure 4: (a) the effect of being in the transgressive condition (relative to the permissible condition) on respondents’ reactions. coefficients were estimated using linear models, pooled across all scenarios, with respondent and scenario fixed effects (standard errors clustered by respondent). (b) indirect effects estimated from a mediation model with disapproval (either upset or inappropriate) as the dependent variable and disrespect and exploitation as mediators (with scenario fixed effects).  44  
  a  b  group status (h1)  socioeconomic status (h2)  mechanisms exploitation  exploitation  disrespect  disrespect  inappropriate  inappropriate  upset  upset disapproval 0  c  0.2  0.4  0.6  0.8  0  d  mediation (h1)  0.1 0.2 0.3 0.4 0.5  mediation (h2)  mediation by exploitation  inappropriate  inappropriate  upset  upset mediation by disrespect  0  0.1  0.2  0.3  0.4  0  0.1  0.2  figure 5: social position hypotheses. experimental effects on respondents’ reactions for scenarios testing the group status (a and c) and socioeconomic status (b and d) hypotheses. upper panels plot coefficients estimated by a linear model with scenario fixed effects (with 95% confidence intervals). lower panels plot the indirect effects of the transgressive condition on being upset or finding the scenario inappropriate, which are mediated by perceptions of disrespect (red) or exploitation (blue).  45  
  group status (h1)  socioeconomic status (h2)  costume  rapper  alter ego  kaffiyeh  musician  sari  singer upset 0  author  inappropriate  0.2 0.4 0.6 0.8  1  1.2 1.4  -0.4  devaluation (h3) interethnic potter  arab wedding  white potter  jewish wedding  store  baby  passing 0  0.2  0.4  0  0.2  0.4  0.6  extraction (h4)  dress  -0.2  -0.2  0.6  0 0.2 0.4 0.6 0.8 1 1.2 1.4  figure 6: disapproval by stimulus. the effect of being in the transgressive condition, relative to the permissible condition, on disapproval (upset in red and inappropriate in blue), by individual scenario (with 95% confidence intervals). grayed coefficients are insignificant.  46  
  a  b  devaluation (h3)  extraction (h4)  mechanisms exploitation  exploitation  disrespect  disrespect  inappropriate  inappropriate  upset  upset disapproval 0  c  0.2  0.4  0.6  0  0.2  d  mediation (h3)  0.4  0.6  0.8  1  mediation (h4)  mediation by exploitation  inappropriate  inappropriate  upset  upset  6  5  0.  0.  4  3  0.  0.  0.4  2  0.3  0.  0.2  0.  0.1  0  0  1  mediation by disrespect  figure 7: mechanisms. experimental effects on respondents’ reactions for scenarios testing the devaluation (a and c) and extraction (b and d) hypotheses. upper panels plots coefficients estimated by a linear model with scenario fixed effects (with 95% confidence intervals). lower panels plot the indirect effects of the transgressive condition on being upset or finding the scenario inappropriate, which are mediated by perceptions of disrespect (red) or exploitation (blue).  47  
  a 3.5  offense  2.5 2  ly  st  g ron  1.5 e gre isa  d  re ag  e  dis  er  n  e  e  re ag  gl  on  gre ya  3  2.5  2 er ral ral ral ive ative ative e b ibe libe neith rvat i l l e erv serv k g s a n n ns n o we co co g co str k a on r t we s  d 3.5  age  offense  offense  h eit  str  c 3 2.5  2  30  ideology  permissible transgressive  3  offense  b 3.5  mobility disenchantment  education  3  2.5  2 31-40  41-50  51-60   60  ol  o ch hs  no  hig  c hs hig  ol ho  e eg  so  ll co me  rs  elo ch ba  te ua  ad gr  figure 8: moderation models. each panel plots marginal effects (with 95% confidence intervals) of an individual trait’s moderation of the experimental effect on disapproval. moderators are estimated independently. all models include scenario fixed effects. (a) group-based mobility disenchantment. (b) ideological identification. (c) age. (d) educational attainment.  48  
  a  variables and descriptive statistics  experiment variables respondents were presented four vignettes, each depicting one scenario. after each vignette they were asked to provide their reactions to the following questions: upset: “does [adopter’s] behavior make you upset?” inappropriate: “is [adopter’s] behavior appropriate?” (reverse coded) disrespect: “does [adopter] seem respectful of [target] culture to you?” (reverse coded) exploitation: “do you think [adopter] is taking advantage of [target] culture?” all reactions are scaled on a 5-point likert scale which included the options “not at all”, “a little”, “moderately”, “quite a bit”, and “extremely”. reactions were randomly ordered for each respondent. descriptive statistics and rank correlations are reported in table a1. for each vignette we altered the specific wordings of the questions in order to appropriately match the scenario contained within the vignette. for instance, for the alter ego stimulus, which features an adopter named nick who light-heartedly dresses up as stereotypes of different cultural/ethnic groups, the question prompting upset is phrased as “does nick’s behavior make you feel upset?”. in the passing scenario, where white adopter mary ethnically “passes” as a latina job applicant “maria,” we phrase the disrespect and exploitation questions as “does mary seem respectful of hispanic culture to you?” and “is mary taking advantage of hispanic culture?”. competence: in addition to these reactions we also asked respondents about their perception of the adopter’s competence for stimuli testing the extraction hypothesis (h4). these were worded as follows: 4a: “does mary seem like a capable worker?” 4b: “does david seem like a smart businessman to you?” 4c and 4d: “do you think [daniel/nolan] is a talented craftsman?” the variable disapproval is constructed by averaging upset and inappropriate.  49  
  table a1: descriptive statistics and rank correlations (reactions) variables (1) upset (2) inappropriate (3) disrespect (4) exploitation  (1) 1.00 0.58 0.52 0.67  (2)  (3)  1.00 0.74 0.58  1.00 0.54  n 10,471 10,266 10,236 10,354  min 1 1 1 1  max 5 5 5 5  mean 1.737 3.286 3.265 2.259  sd 1.078 1.329 1.313 1.321  respondent demographics we collected a variety of demographic variables. their descriptive statistics and correlations are presented in table a2. these variables were collected and constructed as follows: age: age in years. gender : male, female, other (please specify). race: “what is your race?”. options include: white/caucasian, black/african descent, american indian or alaskan native, native hawaiian or pacific islander, east asian, south asian, other (please specify). multiple selection permitted. latino: “are you of hispanic, latino, or spanish origin?” yes/no. identity : “do you self identify as” arab or middle-eastern, jewish. multiple selection permitted. education: some high-school or less, high-school, some college or associate’s degree, bachelor’s degree, graduate degree. ideology : 7-point likert scale of ideological identification, including “extremely liberal”, “liberal”, “slightly liberal”, “moderate or middle of the road”, “slightly conservative”, “conservative”, “extremely conservative”, and “something else”. disenchantment: respondent were asked how much they agree with the statement “it is less likely that an american youth who is a racial minority will go on to become a successful individual compared to the typical white youth, even if he or she is talented and hard-working.” agreement was measured using a 5-point likert scale which includes the options “strongly disagree”, “disagree”, “neither agree nor disagree”, “agree”, and “strongly agree”. the race, latino and identity variables were transformed into dummy variables (given that options are not mutually exclusive). when respondents selected “other” and specified a racial identity, it was coded as one of those dummies if applicable (for example, “mexican” 50  
  was coded as latino). respondents identifying as black, native-american, latino or arab were coded as disadvantaged minority. all other minorities were coded as high-status minority.  51  
  table a2: descriptive statistics and rank correlations (demographics) variables (1) male (2) female (3) other gender (4) education (5) age (6) conservative (7) disenchantment (8) white (9) black (10) latino (11) native am. (12) pacific islander (13) east asian (14) south asian (15) arab (16) jewish observations  (1) 1.00 -0.97 -0.12 0.00 -0.02 0.12 -0.06 0.00 -0.06 0.02 -0.03 -0.01 0.03 0.02 -0.02 0.02 2,640  (2)  (3)  (4)  (5)  (6)  (7)  (8)  (9)  (10)  (11)  (12)  (13)  (14)  (15)  1.00 -0.12 0.01 0.04 -0.10 0.03 -0.01 0.06 -0.02 0.02 -0.01 -0.02 -0.01 0.02 -0.03  1.00 -0.05 -0.06 -0.08 0.10 0.03 -0.01 0.00 0.05 0.05 -0.03 -0.02 0.01 0.03  1.00 0.29 -0.03 0.03 0.07 -0.10 -0.08 -0.06 0.02 0.03 -0.02 -0.01 0.10  1.00 0.14 -0.14 0.27 -0.05 -0.10 0.01 -0.02 -0.23 -0.17 -0.06 0.00  1.00 -0.46 0.03 -0.00 -0.04 -0.00 -0.02 -0.04 -0.04 -0.01 -0.04  1.00 -0.07 0.03 0.02 0.01 -0.03 0.07 0.03 -0.02 0.04  1.00 -0.45 -0.01 -0.06 -0.02 -0.57 -0.28 -0.05 0.12  1.00 -0.03 0.00 -0.02 -0.11 -0.05 0.04 -0.05  1.00 0.11 0.03 -0.08 -0.03 -0.03 -0.04  1.00 -0.01 -0.04 -0.01 0.00 -0.00  1.00 0.04 -0.01 -0.01 -0.01  1.00 -0.04 -0.02 -0.10  1.00 0.10 -0.05  1.00 0.00  min 0 0 0 1 18 1 1 0 0 0 0 0 0 0 0 0  max 1 1 1 5 81 7 5 1 1 1 1 1 1 1 1 1  mean .490 .495 .015 3.56 33.6 3.01 3.39 .746 .088 .115 .019 .002 .138 .033 .018 .064  sd .499 .500 .122 .969 12.8 1.64 1.18 .435 .284 .319 .138 .051 .345 .178 .132 .244  52  
  b  analyses  in this section we provide the full regression tables of results reported in graphical form in the main text, as well as additional results referred to in the main text but not presented in graphical form. table b1: pooled experimental effect with scenario and respondent fixed effects  transgressive  constant scenario fe respondent fe observations r2  (1) upset 0.409 (0.000)  (2) inappropriate 0.455 (0.000)  (3) disrespect 0.425 (0.000)  (4) exploitation 0.541 (0.000)  1.532 (0.000) yes yes 10471 0.581  3.054 (0.000) yes yes 10264 0.578  3.049 (0.000) yes yes 10231 0.581  1.989 (0.000) yes yes 10344 0.559  p-values in parentheses, p 0.05, p 0.01, p 0.001 standard errors clustered by respondent, results reported in figure 4  53  
  table b2: experimental effect on disapproval by hypothesis with scenario fixed effects h1  transgressive  constant scenario fe observations r2  h2  h3  h4  (1) upset 0.560 (0.000)  (2) inapp. 0.822 (0.000)  (3) upset 0.289 (0.000)  (4) inapp. 0.127 (0.007)  (5) upset 0.184 (0.000)  (6) inapp. 0.317 (0.000)  (7) upset 0.543 (0.000)  (8) inapp. 0.435 (0.000)  1.318 (0.000) yes 2593 0.111  2.813 (0.000) yes 2593 0.230  1.487 (0.000) yes 2633 0.0349  3.155 (0.000) yes 2564 0.0610  1.530 (0.000) yes 2613 0.0450  3.153 (0.000) yes 2496 0.127  1.823 (0.000) yes 2632 0.154  3.159 (0.000) yes 2613 0.266  p-values in parentheses, p 0.05, p 0.01, p 0.001, standard errors clustered by respondent results reported in figures 5 and 7  table b3: experimental effect on mediators by hypothesis with scenario fixed effects h1  transgressive  constant scenario fe observations r2  h2  h3  h4  (1) dis. 0.532 (0.000)  (2) exp. 0.632 (0.000)  (3) dis. 0.123 (0.008)  (4) exp. 0.340 (0.000)  (5) dis. 0.502 (0.000)  (6) exp. 0.224 (0.000)  (7) dis. 0.421 (0.000)  (8) exp. 0.859 (0.000)  3.025 (0.000) yes 2602 0.212  1.859 (0.000) yes 2543 0.119  3.216 (0.000) yes 2581 0.106  2.029 (0.000) yes 2622 0.0401  2.950 (0.000) yes 2453 0.0633  1.872 (0.000) yes 2554 0.0947  3.063 (0.000) yes 2600 0.271  2.238 (0.000) yes 2626 0.248  p-values in parentheses, p 0.05, p 0.01, p 0.001, standard errors clustered by respondent dis. disrespect exp. exploitation. results reported in figures 5 and 7  54  
  table b4: experimental effect on disapproval, by scenario  transgressive constant observations r2  transgressive constant observations r2  transgressive constant observations r2  transgressive competence constant observations r2  singer upset inapp.  0.616 1.129 (0.000) (0.000)  1.139 2.161 (0.000) (0.000) 665 665 0.123 0.187  intergroup hypothesis (h1) musician alter ego upset inapp. upset inapp.    0.744 0.853 0.752 1.035 (0.000) (0.000) (0.000) (0.000)    1.469 3.349 1.320 3.121 (0.000) (0.000) (0.000) (0.000) 602 602 664 664 0.127 0.161 0.131 0.191  costume upset inapp.  0.145 0.273 (0.034) (0.007)  1.361 2.673 (0.000) (0.000) 662 662 0.007 0.011  author upset inapp.  0.261 0.275 (0.000) (0.005) 1.363 2.584 (0.000) (0.000) 647 647 0.021 0.012  intragroup hypothesis (h2) sari kaffiyeh upset inapp. upset inapp.   0.220 -0.108 0.222 0.0196 (0.012) (0.293) (0.003) (0.830) 1.714 3.413 1.497 3.500 (0.000) (0.000) (0.000) (0.000) 650 650 649 617 0.010 0.002 0.014 0.000  rapper upset inapp.  0.444 0.316 (0.000) (0.000) 1.378 3.132 (0.000) (0.000) 687 650 0.051 0.020  baby upset inapp. 0.0325 0.0818 (0.589) (0.449) 1.341 2.563 (0.000) (0.000) 650 650 0.000 0.001  devaluation hypothesis (h3) jewish wedding arab wedding upset inapp. upset inapp. 0.272 0.377 0.256 0.475 (0.002) (0.000) (0.001) (0.000) 1.713 3.439 1.589 3.420 (0.000) (0.000) (0.000) (0.000) 667 667 649 602 0.014 0.022 0.017 0.046  dress upset inapp. 0.172 0.349 (0.012) (0.000) 1.472 3.204 (0.000) (0.000) 647 577 0.010 0.027  passing upset inapp.  0.411 0.150 (0.000) (0.018) yes yes  3.396 4.837 (0.000) (0.000) 686 686 0.234 0.395  extraction hypothesis (h4) store white potter upset inapp. upset inapp.    0.493 0.593 0.607 0.371 (0.000) (0.000) (0.000) (0.000) yes yes yes yes    2.026 4.312 3.117 4.360 (0.000) (0.000) (0.000) (0.000) 627 610 650 650 0.096 0.347 0.157 0.253  interethnic potter upset inapp.  0.953 1.345 (0.000) (0.000) yes yes  1.741 3.231 (0.000) (0.000) 660 660 0.338 0.521  p-values in parentheses, p 0.05, p 0.01, p 0.001, results reported in figure 6  55  
  table b5: experimental effect on mediators, by scenario  exp. 0.954 (0.000) 1.488 (0.000) 665 0.153  group status hypothesis (h1) musician alter ego dis. exp. dis. exp.    0.451 0.371 0.639 1.091 (0.000) (0.000) (0.000) (0.000)    3.593 2.329 3.553 1.903 (0.000) (0.000) (0.000) (0.000) 611 552 664 664 0.046 0.027 0.074 0.165  costume dis. exp. 0.101 0.0675 (0.313) (0.445) 2.673 1.791 (0.000) (0.000) 662 662 0.002 0.001  author dis. exp.  0.270 0.330 (0.004) (0.000) 2.469 1.863 (0.000) (0.000) 647 647 0.013 0.021  socioeconomic status hypothesis (h2) sari kaffiyeh dis. exp. dis. exp. -0.101 0.120 0.0567 0.331 (0.331) (0.262) (0.520) (0.000) 3.453 2.447 3.605 1.971 (0.000) (0.000) (0.000) (0.000) 650 650 617 641 0.001 0.002 0.001 0.020  rapper dis. exp.  0.261 0.566 (0.002) (0.000) 3.348 1.839 (0.000) (0.000) 667 684 0.014 0.055  singer transgressive constant observations r2  transgressive constant observations r2  dis. 0.926 (0.000) 2.327 (0.000) 665 0.134  exp. 0.00236 (0.975) 1.557 (0.000) 650 0.000  devaluation hypothesis (h3) jewish wedding arab wedding dis. exp. dis. exp. 0.446 0.250 0.676 0.416 (0.000) (0.018) (0.000) (0.000) 3.039 2.361 3.037 1.833 (0.000) (0.000) (0.000) (0.000) 667 667 579 605 0.029 0.008 0.092 0.035  dis. 0.372 (0.000) 3.084 (0.000) 557 0.032  passing dis. exp. 0.0789 0.433 (0.234) (0.000) yes yes  4.844 4.197 (0.000) (0.000) 686 686 0.390 0.259  extraction hypothesis (h4) store white potter dis. exp. dis. exp.    0.526 0.725 0.530 1.116 (0.000) (0.000) (0.000) (0.000) yes yes yes yes    4.205 2.026 4.486 3.213 (0.000) (0.000) (0.000) (0.000) 601 621 650 650 0.314 0.092 0.297 0.208  interethnic potter dis. exp.  1.199 1.389 (0.000) (0.000) yes yes  3.315 2.063 (0.000) (0.000) 660 660 0.511 0.341  baby transgressive constant observations r2  transgressive competence constant observations r2  dis. 0.515 (0.000) 2.663 (0.000) 650 0.039  dress  p-values in parentheses, p 0.05, p 0.01, p 0.001, dis. disrespect exp. exploitation  56  exp. 0.240 (0.002) 1.722 (0.000) 632 0.014  
  c  competence  in this section we provide further justification for our decision to include the adopter’s perceived competence as a control in models testing the extraction hypothesis (h4). as tables c1 and c2 demonstrate, the inclusion of this control in the model only affects results for stimulus 4b (“store”). once the perceived business competence of the store owner is accounted for, experimental effects are in the expected direction. other stimuli are not affected. table c1: experimental effect on upset by scenario, extraction hypothesis (h4) passing  (4) 0.493 (0.000)  white potter (5) (6) 0.529 0.607 (0.000) (0.000)  interethnic potter (7) (8) 1.116 0.953 (0.000) (0.000)  -0.669 (0.000)  -0.0988 (0.430)  -0.821 (0.013)  0.354 (0.340)  moderately  -1.015 (0.000)  -0.373 (0.004)  -0.899 (0.004)  -0.107 (0.759)  quite a bit  -1.751 (0.000)  -0.810 (0.000)  -1.484 (0.000)  -0.584 (0.088)  extremely  -1.951 (0.000)  -1.272 (0.000)  -1.823 (0.000)  -0.796 (0.020)  transgressive  (1) 0.503 (0.000)  competent a little  constant observations adjusted r2  2.491 (0.000) 686 0.035  (2) 0.411 (0.000)  3.396 (0.000) 686 0.229  store (3) 0.00483 (0.955)  1.864 (0.000) 636 -0.002  p-values in parentheses   p 0.05, p 0.01, p 0.001  57  2.026 (0.000) 627 0.089  1.818 (0.000) 650 0.044  3.117 (0.000) 650 0.150  1.097 (0.000) 660 0.261  1.741 (0.000) 660 0.333  
  table c2: experimental effect on inappropriate by scenario, extraction hypothesis (h4) passing  (4) 0.526 (0.000)  white potter (5) (6) 0.379 0.530 (0.000) (0.000)  interethnic potter (7) (8) 1.554 1.199 (0.000) (0.000)  -0.209 (0.030)  -0.702 (0.000)  -0.750 (0.020)  -0.291 (0.468)  moderately  -0.488 (0.000)  -1.047 (0.000)  -1.259 (0.000)  -0.611 (0.108)  quite a bit  -1.453 (0.000)  -1.668 (0.000)  -1.947 (0.000)  -1.313 (0.000)  extremely  -2.799 (0.000)  -2.669 (0.000)  -2.813 (0.000)  -2.128 (0.000)  transgressive  (1) 0.131 (0.119)  competent a little  constant observations adjusted r2  4.254 (0.000) 686 0.002  (2) 0.0789 (0.234)  4.844 (0.000) 686 0.386  store (3) -0.446 (0.000)  3.687 (0.000) 604 0.032  p-values in parentheses   p 0.05, p 0.01, p 0.001  58  4.205 (0.000) 601 0.309  2.722 (0.000) 650 0.019  4.486 (0.000) 650 0.292  1.616 (0.000) 660 0.319  3.315 (0.000) 660 0.507  
  d  moderation analyses  to explore whether susceptibility to the experimental manipulation varies by individual, we perform a set of moderation analyses, where the experimental effect is interacted with a moderator. formally, moderation is estimated with the following model: dis β0 β1 ais β2 mi αmi ais β3 ss is  (d1)  where i indexes individual respondents and s indexes scenario, dis is the respondent’s level of disapproval, mi is the moderator in question (a fixed individual trait, e.g. mobility disenchantment), ais is an indicator of whether the respondent was assigned to the transgressive condition for a given scenario, and ss is a scenario fixed effect. standard errors are clustered by respondent. dis is a composite measure of disapproval, averaging the respondent’s reactions upset and inappropriate. we include scenario fixed effects to account for variance in general disapproval generated by each scenario. the interaction term α estimates moderation, namely, differences in susceptibility to the experimental manipulation. where moderators include multiple levels we estimate individual coefficients for each level rather than treating the variable as continuous. table d1 reports the moderation effects of mobility disenchantment, and table d2 reports moderation for political ideology, age and education. marginal effects from these models are illustrated in figure 8  59  
  table d1: moderation of experimental effect on disapproval by mobility disenchantment  transgressive  (1) all 0.209  (0.004)  disenchantment disagree neither agree strongly agree  0.135 0.238 0.373 0.460  (0.037) (0.000) (0.000) (0.000)  disenchantment transgressive disagree transgressive neither transgressive agree transgressive strongly agree transgressive constant stimulus fe observations r2  -0.000270 0.131 0.242 0.467 2.013 yes 10185 0.218  (0.997) (0.120) (0.002) (0.000) (0.000)  p-values in parentheses standard errors clustered by respondent   p 0.05, p 0.01, p 0.001  results reported in figure 8a  60  
  table d2: moderation of experimental effect on disapproval by sociodemographics  transgressive  (1) 0.669 (0.000)  political ideology liberal weak liberal neither weak conservative conservative strong conservative liberal transgressive weak liberal transgressive neither transgressive weak conservative transgressive conservative transgressive strong conservative transgressive  -0.118 -0.189 -0.251 -0.343 -0.341 -0.286 -0.203 -0.296 -0.392 -0.448 -0.426 -0.624  -0.0573 -0.129 -0.145 -0.209 -0.127 -0.195 -0.158 -0.215  2.478 yes 10185 0.212  (3) 0.533 (0.001)  (0.009) (0.000) (0.000) (0.000) (0.000) (0.017) (0.000) (0.000) (0.000) (0.000) (0.000) (0.000)  age 31 to 40 41 to 50 51 to 60 above 60 31 to 40 transgressive 41 to 50 transgressive 51 to 60 transgressive above 60 transgressive education high school some college bachelors graduate high school transgressive some college transgressive bachelors transgressive graduate transgressive constant stimulus fe observations r2  (2) 0.481 (0.000)  (0.000)  2.359 yes 10185 0.184  (0.117) (0.007) (0.007) (0.000) (0.008) (0.002) (0.030) (0.004)  (0.000)  0.0268 0.0636 0.0776 0.0699 -0.162 -0.123 -0.0907 -0.198 2.245 yes 10185 0.175  p-values in parentheses, p 0.05, p 0.01, p 0.001, standard errors clustered by respondent results reported in figure 8b-d  61  (0.863) (0.677) (0.610) (0.651) (0.330) (0.445) (0.571) (0.225) (0.000)  
  table d3 explores the relationships between these moderators when their effects are modelled together. it reports the results of ols models with multiple moderators, pooled across all respondents and scenarios and including both scenario and respondent fixed effects. these models are estimated as follows:  dis β0 β1 ais α1 m 1i ais ... αn m ni ais β3 ss β4 ri is  (d2)  where ri are respondent fixed effects and standard errors are clustered by respondent. the α coefficients estimate moderation effects. model 1 reports the standalone moderation effect of mobility disenchantment. when political ideology is included in model 2, this effect is attenuated but remains significant. sensitivity to boundary-crossing, in other words, increases with mobility disenchantment above and beyond one’s self-professed strength of political liberalism. though ideological identification and mobility disenchantment are related (0.456 rank correlation in our sample) they are not one and the same. disapproval of cultural boundary-crossing does not appear to be a mere display of political identity. model 3, which also includes age as a moderator, suggests that younger respondents’ tendency to disapprove cultural appropriation is rooted in their pessimism about mobility. once their beliefs about mobility are accounted for, age is no longer a significant moderator of the experimental effect.  62  
  table d3: ols models of disapproval with experimental effect moderators  transgressive  (1) 0.247 (0.000)  (2) 0.0665  (3) (0.683)  (4)  0.107  (0.514)  0.0837  (0.714)  moderators (interactions with transgressive condition) disenchantment: disagree neither agree strongly agree  -0.0591 0.100 0.242 0.466  (0.466) (0.219) (0.001) (0.000)  -0.0828 0.0525 0.145 0.300  (0.291) (0.513) (0.050) (0.001)  -0.0831 0.0487 0.141 0.293  (0.290) (0.545) (0.058) (0.001)  -0.0881 0.0369 0.125 0.275  (0.262) (0.647) (0.094) (0.002)  0.146 0.189 0.159 0.215 0.291 0.466  (0.358) (0.236) (0.289) (0.156) (0.051) (0.002)  0.139 0.186 0.145 0.206 0.279 0.455  (0.379) (0.244) (0.338) (0.176) (0.061) (0.003)  0.135 0.178 0.145 0.207 0.253 0.424  (0.383) (0.259) (0.328) (0.167) (0.085) (0.005)  -0.0654 -0.0636 0.0078 -0.0874  (0.137) (0.294) (0.911) (0.265)  -0.0662 -0.0792 0.0007 -0.114  (0.143) (0.195) (0.992) (0.162)  education: high school some college bachelors graduate  -0.0873 -0.0302 0.0113 -0.0266  (0.592) (0.848) (0.943) (0.869)  minority: high status disadvantaged  0.0671 -0.122  (0.146) (0.009)  gender: female other  0.163 0.406  (0.000) (0.008)  2.296 10183 0.464  (0.000)  ideology: conservative weak conservative neither weak liberal liberal strong liberal age: 31 to 40 41 to 50 51 to 60  60  constant observations adjusted r2  2.296 10183 0.459  (0.000)  2.297 10183 0.461  (0.000)  2.297 10183 0.461  (0.000)  p-values in parentheses. p 0.10, p 0.05, p 0.01, p 0.001 all models include scenario and respondent fixed effects. standard errors clustered by respondent.  63  
  e  identity threat  identity threat theory argues that individuals are motivated to protect their group’s differentiation from other identities. in its most basic form, this theory predicts that individuals are particularly offended by boundary-crossing when it is targeted at their own group, as it undermines their self-image (tajfel 1982). we find little evidence in support of this prediction. to test it, we estimate the extent to which respondents are likely to express more disapproval when the scenario portrays their group as the target of cultural boundarycrossing, relative to outsiders to that group. we regress our measure of disapproval on an indicator of whether or not the respondent is a member of the target group, and include both respondent and scenario-condition fixed effects. we include scenario-condition fixed effects because our purpose is not to estimate susceptibility to experimental condition. rather, we seek to evaluate the extent to which, observing the exact same scenario-condition combination, members of the target group express greater disapproval. as panel a (blue coefficients) of figure e1 illustrates, the effects of being in the target group are statistically insignificant. contra the expectations of identity threat theory, respondents are not more sensitive to boundary-crossing when their group is the target. because our experiment was not designed to directly test identity threat theory, these results should not be seen as a rejection of that theory. nevertheless, these results are quite robust. in fact, when estimating a scenario fixed effects model that controls for experimental condition, we find that being a member of the target group significantly reduces disapproval. this effect is driven by the fact that white respondents are generally less sensitive than other people to boundary-crossing targeting white identity. a variant of identity threat theory predicts that perceived threat will be especially heightened for minority groups. minority group members are especially motivated to maintain their group’s cultural distinctiveness because their identity is defined in contrast to the hegemonic white identity (mosley and biernat 2020). this implies that members of minority groups should express greater disapproval when their group is the target of cultural boundary-crossing. we find no evidence for that either, as illustrated in panel a (red coefficients). namely, models identical to those reported in blue, but in which the indicator for being the target identity is only applied to members of minority groups (defined as not choosing any identity other than white), reproduce similarly insignificant effects. in that respect, our findings are inconsistent with mosley and biernat’s (2020), who find 64  
  that black respondents are more likely to see boundary-crossing as transgressive than white respondents. they attribute this finding to minority respondents’ tendency to set a lower threshold for what constitutes racism. we speculate that these discrepant findings relate to the fact that, in their experimental paradigm, mosley and biernat explicitly ask respondents whether an actor portrayed in a hypothetical scenario is appropriating the target culture. framing the exercise in the language of appropriation, we suspect, primes respondents to think about cultural boundary-crossing as harmful. alternatively, minorities’ experiences of identity threat might sensitize them more to cultural appropriation in general, irrespective of whether theirs is the target group. to test for that possibility, we interact ethnic/racial identity with experimental condition in a scenario fixed effects moderation model that predicts disapproval (eq. d1 with multiple moderators). the estimates produced by this model are reported in panel b of figure e1 (and in table e1). these indicate that certain minorities are indeed slightly more sensitive to cultural transgressive boundary-crossing than others. counter-intuitively, however, these are not members of the most systematically disadvantaged groups in american society. rather, members of socioeconomically successful ethnic minorities—east-asians, southasians and jews—express more disapproval in reaction to transgressive, relative to permissible, boundary-crossing. disadvantaged and marginalized minorities—blacks, nativeamericans and arabs—are not more likely to express disapproval in light of appropriation than white americans are. latinos, quite surprisingly, are even slightly less likely to express disapproval.  65  
  reaction if target identity  disapproval moderation jewish south asian east asian arab pacific islander native american latino black non-minority  minorities exploitation all ethnicities disrespect inappropriate upset  -0.2  -0.1  0  0.1  0.2  0  0.5  1  1.5  figure e1: sensitivity to cultural boundary-crossing by race/ethnicity. (a) the effect of having the target identity on reactions for all respondents (blue) or minority respondents only (red), as estimated by a model with respondent and scenario-condition fixed effects. (b) the moderation effect of being in the transgressive condition on disapproval, moderated by identity, estimated by a model with scenario fixed effects. the gray line corresponds to the main moderation effect of the omitted category (self-reported race is white or none).  66  
  table e1: effect of having the target identity on disapproval  target  (1) upset 0.0373 (0.346)  (2) inapp. 0.0165 (0.729)  (3) dis. -0.0124 (0.783)  (4) exp. 0.0136 (0.774)  minority target  constant scen.-condition fe respondent fe observations r2  1.732 (0.000) yes yes 10471 0.593  3.284 (0.000) yes yes 10264 0.601  3.267 (0.000) yes yes 10231 0.597  2.257 (0.000) yes yes 10344 0.576  (5) upset  (6) inapp.  (7) dis.  (8) exp.  0.00512 (0.906)  -0.0366 (0.484)  -0.0587 (0.242)  -0.0219 (0.678)  1.737 (0.000) yes yes 10471 0.593  3.289 (0.000) yes yes 10264 0.601  3.269 (0.000) yes yes 10231 0.597  2.261 (0.000) yes yes 10344 0.576  p-values in parentheses, p 0.05, p 0.01, p 0.001, standard errors clustered by respondent results reported in figure e1a  67  
  table e2: moderation of experimental effect on disapproval by racial/ethnic identity  transgressive black transgressive black latino transgressive latino native-american transgressive native-american east asian transgressive east asian south asian transgressive south asian pacific islander transgressive pacific islander jewish transgressive jewish arab transgressive arab constant scenario fe observations r2  (1) offense 0.380 0.0202 -0.00422 0.108 -0.154 0.155 -0.122 0.0745 0.162 0.0952 0.282 -0.148 0.776 0.0196 0.220 0.0892 -0.114 2.276 yes 10185 0.181  (0.000) (0.700) (0.954) (0.026) (0.014) (0.177) (0.346) (0.069) (0.004) (0.239) (0.013) (0.470) (0.040) (0.777) (0.020) (0.455) (0.539) (0.000)  p-values in parentheses, p 0.05, p 0.01, p 0.001, standard errors clustered by respondent results reported in figure e1b  68  
  f  stimuli  we address four dimensions of variability in acts of boundary-crossing. first, we distinguish between recombinant and mimetic acts. recombinant acts, such as fusion cuisine, mix elements from multiple cultural identities, whereas mimetic acts, such as a non-latino proprietor operating a taco truck, copy a cultural practice in its entirety. recombinant acts cross boundaries through the act’s atypicality while mimetic acts do so through the variety of practices adopted by the actor. as goldberg et al. (2016) demonstrate, the former subvert boundaries, and tend to be resisted especially by high-status individuals. they are also important vehichles of innovation. second, drawing on young’s (2010) nomenclature, we distinguish between subject boundarycrossing in which the adopter represents the target, and content boundary-crossing in which the adopter adopts elements of the target culture. subject boundary-crossing maintains a distance between the adopter and target. an adopter engaging in content boundarycrossing, in contrast, often implicitly makes a claim to that identity as (at least partially) her own. both types of boundary-crossing can result in devaluation or extraction. we therefore see no a-priori reason why one should inherently generate greater disapproval than the other. third, we distinguish between ephemeral and permanent acts of boundary-crossing. wearing a party costume, for example, is ephemeral, while naming a child or bodily modification with a tattoo are permanent (even if reversible). we assume that the latter indicate, ceteris paribus, greater respect for the target culture and should therefore, on average, generate less disapproval. finally, it is likely that observers differentiate between instances in which the adopter engages in boundary-crossing for commercial reasons versus instances in which the rewards to boundary-crossing are social. we assume that profit-oriented scenarios are more likely to generate disapproval as they are unequivocally instrumental. nevertheless, we expect the dynamics of cultural tariffing to operate whether the adopter is explicitly engaged in commercial activity or in behavior that generates social currency. the latter can also vary in instrumentality. a public display of boundary-crossing musical preferences, for example, is more likely to project status than private musical consumption. we do not hypothesize about or directly test the effects of any of these dimensions on observers’ disapproval. rather, we introduce variation along these dimensions in our scenarios (non-systematically), as detailed in table 1. this allows us to rule out the possibility 69  
  that the effects reported below are exclusively driven by attributes related to the act rather than the actor. the following tables provide full descriptions of all the stimuli used in the experiment.  70  
  71  1a  1a  singer  singer  label h1: intergroup  permissible  transgressive  condition  jordan anderson, known by his stage name jordiredd, is an up-and-coming rapper based in detroit. he is a lifelong musician who received training in a number of musical instruments. recently he’s decided to try his hand at hip-hop and has begun recording music. jordan isn’t troubled by the fact that he is white and the genre is predominantly black. “adopting the proper lingo and style was important for me to succeed”, he admits, “it wasn’t natural, so it took a lot of work to sound this way”. he is due to release an ep entitled “motown outlaw”, which early listeners have described as a “skillful yet bittersweet ode to the inner-city detroit lifestyle”. jordan anderson, known by his stage name carolina jordan, is an up-and-coming country singer based in north carolina. he is a lifelong musician who received musical training in a number of musical instruments. recently he’s decided to try his hand at country music and has begun recording. jordan isn’t troubled by the fact that he is black and the genre is predominantly white. “adopting the proper lingo and style was important for me to succeed”, he admits, “it wasn’t natural, so it took a lot of work to sound this way”. he is due to release an ep entitled “outlaw holler”, which early listeners have described as a “skillful yet bittersweet ode to the rural appalachia lifestyle”.  description  table f1: detailed descriptions of stimuli (h1 a) image  
  72  1b  1b  condition  musician permissible  musician transgressive  label h1: intergroup frankie alan is an up-and-coming blues musician, which is a primarily black musical genre. trained as a classical guitarist, he recently decided to try his hand in blues and has begun playing the club scene in chicago. because he is white, frankie had to work hard on sounding and looking the part. “it was really important that i speak and move like a black musician,” says frankie. when on stage he adopts strong african-american lingo and behaves in a stereotypically “inner city” manner. “it wasn’t natural”, he admits “but otherwise i would have looked like a fish out of water” frankie alan is an up-and-coming country musician, which is a southern musical genre. trained as a classical guitarist, he recently decided to try his hand at country and has begun playing the club scene in nashville. because he is from new england, frankie had to work hard on sounding and looking the part. “it was really important that i speak and move like a southerner”, says frankie. when on stage he adopts a strong twang and behaves in a stereotypically “cowboy” manner. “it wasn’t natural”, he admits “but otherwise i would have looked like a fish out of water”.  description  table f2: detailed descriptions of stimuli (h1 b) image  
  73  1c  1c  alter ego  alter ego  label h1: intergroup  permissible  transgressive  condition  nick is a college student who grew up in a middle class suburban household. an eccentric and flamboyant personality, he is known among his peers for putting on a character. he especially likes to behave as if he were a gangsta rapper, speaking with strong urban affectations, wearing exaggerated bling jewelry and using a lot of explicit slang as he constantly refers to his “ghetto life”. “everybody knows that i’m just putting on a show,” he says. “i do this because i like to entertain, and god knows college life needs a bit of spicing up”. nick is a college student who grew up in a middle class suburban household. an eccentric and flamboyant personality, he is known among his peers for putting on a character. he especially likes to behave as if he were a preppy, speaking with exaggerated new england affectations, wearing polo shirts and cardigans tied around his neck, and constantly referring to his yacht or his love for lacrosse. “everybody knows that itas just a show,” he says. “i do this because i like to entertain, and god knows college life needs a bit of spicing up”.  description  table f3: detailed descriptions of stimuli (h1 c) image  
  74  1d  1d  condition  costume permissible  costume transgressive  label h1: intergroup adam is excited about his upcoming annual fall dinner party where there’s typically an elaborate theme which he usually goes all out for. this year the theme is a “calavera dinner”, in reference to the mexican dia de los muertos (day of the dead) celebration. the day of the dead is one of the biggest and oldest events in mexico, and has been recognized by the un as an “intangible cultural heritage”. keeping with the theme of the calavera dinner, adam is planning to wear traditional mexican attire including a colorfully embroidered jalisco suit, accessorize with cempasuchil flowers and ornamentation, and wear the customary “sugar skull” white face-paint with multi-colored decoration. adam is excited about his upcoming annual fall dinner party where there’s typically an elaborate theme which he usually goes all out for. this year the theme is a “calavera dinner”, in reference to the mexican dia de los muertos (day of the dead) celebration. the day of the dead is one of the biggest and oldest events in mexico, and has been recognized by the un as an “intangible cultural heritage”. keeping with the theme of the calavera dinner, adam is planning to wear traditional mexican attire including a colorfully embroidered jalisco suit, accessorize with cempasuchil flowers and ornamentation, and wear the customary “sugar skull” white face-paint with multi-colored decoration.  description  table f4: detailed descriptions of stimuli (h1 d) image  
  75  2a  2a  condition  author permissible  author transgressive  label h2: intragroup jonathan huntsman is a first-time author. he grew up in a wealthy nyc neighborhood. he writes during the day in his brooklyn apartment that his parents have bought for him. the novel, red and white, is about an africanamerican family from north carolina who struggle as daylaborers on a winery in segregated north georgia during the 1940’s. the book is based on a combination of true stories of black families working on farms in georgia and north carolina during jim crow and attempts to portray an authentic account of poverty. jonathan huntsman is a first-time author. he grew up in a working class nyc neighborhood. he works at the local walmart by day, and crafts his novel by night while moonlighting as a parking lot attendant. the novel, red and white, is about an african-american family from north carolina who struggle as day-laborers on a winery in segregated north georgia during the 1940’s. the book is based on a combination of true stories of black families working on farms in georgia and north carolina during jim crow and attempts to portray an authentic account of poverty.  description  table f5: detailed descriptions of stimuli (h2 a) image  
  76  2b  2b  sari  sari  label h2: intragroup  permissible  transgressive  condition  lauren anderson, 28, works as a cashier at a car wash facility in los angeles. she rents a modest apartment with her three housemates. lauren has recently discovered indian fashion, and has begun incorporating traditional indian saris into her regular outfits. she especially likes to wear these at work, where she feels they make her “stand out.”  lauren anderson, 28, works in her family’s boutique real estate agency that caters to some of the wealthiest residents of los angeles. she lives in a beachfront house owned by her parents. lauren has recently discovered indian fashion, and has begun incorporating traditional indian saris into her regular outfits. she especially likes to wear these at work, where she feels they make her “stand out.”  description  table f6: detailed descriptions of stimuli (h2 b) image  
  77  2c  2c  condition  kaffiyeh permissible  kaffiyeh transgressive  label h2: intragroup nathan anderson, 28, works in his family’s boutique real estate agency that caters to some of the wealthiest residents of los angeles. he lives in a beachfront house owned by his parents. nathan has recently discovered arab fashion, and has begun incorporating arab elements into his outfits, wearing a traditional taqiyeh hat or a kaffiyeh scarf around his neck. he especially likes to wear these at work, where he feels they make him “feel special.” nathan anderson, 28, works as a cashier at a car wash facility in los angeles. he rents a modest apartment with his three housemates while supporting his disabled single mother. nathan has recently discovered arab fashion, and has begun incorporating arab elements into his outfits, wearing a traditional taqiyeh hat or a kaffiyeh scarf around his neck. he especially likes to wear these at work, where he feels they make him “feel special.”  description  table f7: detailed descriptions of stimuli (h2 c) image  
  78  2d  2d  condition  rapper permissible  rapper transgressive  label h2: intragroup jordan anderson, known by his stage name jordiredd, is an up-and-coming rapper based in detroit. he grew up in an affluent neighborhood, and displayed an early talent for music. recognizing his interest in music, his parents invested heavily in his musical education. recently he’s decided to try his hand at hip-hop and has begun recording music. according to his parents, jordan went through a profound transformation in becoming jordiredd. “when i see him perform he speaks and acts so differently from the jordi i know”, his mother says. jordan anderson, known by his stage name jordiredd, is an up-and-coming rapper based in detroit. he grew up in an impoverished household, and displayed an early talent for music. a self-taught musician, his single mother could not afford investing in his musical education. recently he’s decided to try his hand at hip-hop and has begun recording music. according to his mother, jordan went through a profound transformation in becoming jordiredd. “when i see him perform he speaks and acts so differently from the jordi i know”, she says.  description  table f8: detailed descriptions of stimuli (h2 d) image  
  79  3a  3a  baby permissible  baby transgressive  label condition h3: devaluation  the murphys, are a young newlywed couple. they have just had a baby boy, whom they’ve decided to name tyrone. though they are both white, they both grew up in a predominantly black neighborhood in atlanta, where a majority of their friends were africanamerican.  the murphys, are a young newlywed couple. they have just had a baby boy, whom they’ve decided to name tyrone. they are not black, have had little contact with black people, and do not personally know anyone with the name tyrone.  description  image  table f9: detailed descriptions of stimuli (h3 a)  
  80  3b  3b  jewish wedding  jewish wedding  label h3: devaluation  permissible  transgressive  condition  bradley and kristen have been engaged for 2 months and are currently planning their wedding. they are fascinated by jewish culture. neither of them is jewish and neither has had any exposure to judaism, but they recently became fascinated with the culture after becoming acquainted via a netflix documentary on the orthodox jewish community in brooklyn. they plan to include several elements of a traditional jewish wedding in their ceremony, such as signing a ketubah marriage contract written in hebrew, the groom breaking a glass after taking their vows, and a traditional hassidic horah dance. bradley and kristen have been engaged for 2 months and are currently planning their wedding. they are fascinated by jewish culture. neither of them is jewish but both grew up in a majority jewish neighborhood, have many observant jewish friends and have attended multiple jewish weddings. they plan to include several elements of a traditional jewish wedding in their ceremony, such as signing a ketubah marriage contract written in hebrew, the groom breaking a glass after taking their vows, and a traditional hassidic horah dance.  description  table f10: detailed descriptions of stimuli (h3 b) image  
  81  3c  3c  arab wedding  arab wedding  label h3: devaluation  permissible  transgressive  condition  bradley and kristen have been engaged for 2 months and are currently planning their wedding. they are fascinated by arab culture. neither of them is arab and neither has had any exposure to arab culture, but they recently became fascinated with the culture after becoming acquainted via a netflix documentary on the arab-american community in dearborn, michigan. they plan to include several elements of a traditional arab wedding in their ceremony, such as a henna tattoo party, a zaffe wedding procession, and a traditional debka dance. bradley and kristen have been engaged for 2 months and are currently planning their wedding. they are fascinated by arab culture. neither of them is arab but both grew up in a majority arab neighborhood, have many arab friends, and have attended multiple arab weddings. they plan to include several elements of a traditional arab wedding in their ceremony, such as a henna tattoo party, a zaffe wedding procession, and a traditional debka dance.  description  table f11: detailed descriptions of stimuli (h3 c) image  
  82  3d  3d  dress  dress  label h3: devaluation  permissible  transgressive  condition  lisa has recently married her boyfriend of 3 years, and they spent their honeymoon in cairo, egypt for two weeks. upon returning, lisa’s friends have noticed that she has begun dressing in arabic styles (for example, lots of kaftan dresses and abayas), and has begun using arabic terminology when she speaks. when they asked her about it, lisa shrugged it off, “well what’s the big deal? since the trip i’ve fallen in love with arab culture” she said, “am i not allowed to wear this? i find it beautiful.” lisa has recently married her boyfriend of 3 years, and they spent their honeymoon in cairo, egypt with his family for two weeks. upon returning, lisa’s friends have noticed that she has begun dressing in arabic styles (for example, lots of kaftan dresses and abayas), and has begun using arabic terminology when she speaks. when they asked her about it, lisa shrugged it off, “well usman is arabic, right?” she said, referring to her husband, “and i’ve been sharing a lifestyle with him for a few years. we’re family now, which makes me a part of that world.”  description  table f12: detailed descriptions of stimuli (h3 d) image  
  83  4a  4a  passing  passing  label h4: extraction  permissible  transgressive  condition  mary is a recent college graduate. though she is not hispanic, she has recently begun calling herself “maria,” often includes words in spanish when she speaks to others and tends to refer in passing to her “mexican” heritage. she is currently looking for a job as an analyst in the business consulting industry, where employers value data analytics and seek to hire quantitative people.  mary is a recent college graduate. though she is not hispanic, she has recently begun calling herself “maria,” often includes words in spanish when she speaks to others and tends to refer in passing to her “mexican” heritage. she is currently looking for a job as a creative writer in the advertising industry, where employers value cultural diversity and seek to hire underrepresented minorities.  description  table f13: detailed descriptions of stimuli (h4 a) image  
  84  4b  4b  store permissible  store transgressive  label condition h4: extraction david is a creative writer in a high-end ad agency based in downtown los angeles. though he is not hispanic, he is fascinated by latino-american culture. to get inspiration, he often takes long walks along the streets of east l.a., a run-down, low income and predominantly hispanic neighborhood. drawing on his experiences in the neighborhood he has recently launched an independent men’s clothing line called “forasteros” (spanish for “outsiders”) which sells latino-inspired urban wear for young professionals. items include shirts with latin american iconography, sneakers with mexican prints and guatemalan bandannas. luckily for david, one of his business connections was willing to feature his clothing line on a leading fashion website. success immediately followed, bringing in significant profits and drawing the attention of social media influencers. david is a creative writer in a high-end ad agency based in downtown los angeles. though he is not hispanic, he is fascinated by latino-american culture. to get inspiration, he often takes long walks along the streets of east l.a., a run-down, low income and predominantly hispanic neighborhood. drawing on his experiences in the neighborhood he has recently launched an independent men’s clothing line called “forasteros” (spanish for “outsiders”) which sells latino-inspired urban wear for young professionals. items include shirts with latin american iconography, sneakers with mexican prints and guatemalan bandannas. despite receiving positive reviews, davidas clothing line received very little media attention. his online clothing store has been a failure, leading to significant financial losses.  description  image  table f14: detailed descriptions of stimuli (h4 b)  
  85  4c  4c  white potter  white potter  label h4: extraction  description  daniel wilson is a pottery artist located in chinle, arizona. growing up in a white suburban household on the east coast, wilson moved west after discovering native american art. inspired by traditional hopi craftsmanship, he paints his designs with bee-weed and then fires them in a handmade oven to create blushes on the clay surface. although he does not have native american roots, wilson is unbothtransered: “in order for me to succeed, adopting native gressive methods is key”, he confesses, “it took a lot of work to unlearn the contemporary techniques i acquired in art school”. unlike the vast majority of native american pottery artists who struggle to sell their artwork, wilson has been exceptionally successful. initially picked up by the press as the “the only white potter in navajo nation,” his work has been presented at leading galleries and has been sold to wealthy collectors. daniel wilson is a pottery artist located in chinle, arizona. growing up in a white suburban household on the east coast, wilson moved west after discovering native american art. inspired by traditional hopi craftsmanship, he paints his designs with bee-weed and then fires them in a handmade oven to create blushes on the clay surface. although he does not have native american roots, wilson is permissible unbothered: “in order for me to succeed, adopting native methods is key”, he confesses, “it took a lot of work to unlearn the contemporary techniques i acquired in art school”. like the vast majority of native american pottery artists, wilson struggles to make ends meet. despite attending dozens of art fairs and pitching his work to art galleries and magazines, he has so far failed to sell any of his artworks.  condition  image  table f15: detailed descriptions of stimuli (h4 c)  
  86  4d  4d  condition  interethnic permisspotter ible  interethnic tranpotter gressive  label h4: extraction daniel wilson is a pottery artist located in chinle, arizona. growing up in a white suburban household on the east coast, wilson moved west after discovering native american art. inspired by traditional hopi craftsmanship, he paints his designs with bee-weed and then fires them in a handmade oven to create blushes on the clay surface. although he does not have native american roots, wilson is unbothered: “in order for me to succeed, adopting native methods is key”, he confesses, “it took a lot of work to unlearn the contemporary techniques i acquired in art school”. unlike the vast majority of native american pottery artists who struggle to sell their artwork, wilson has been exceptionally successful. initially picked up by the press as the “the only white potter in navajo nation,” his work has been presented at leading galleries and has been sold to wealthy collectors. nolan “apple blossom” nampeyo is a native american pottery artist located in chinle, arizona. born and raised in the heart of navajo nation, nampeyo has been immersed in native american culture throughout his life. inspired by traditional hopi craftsmanship, he paints his designs with bee-weed and then fires them in a handmade oven to create blushes on the clay surface. proud of his native american roots, nampeyo confesses that ain order for me to succeed, adopting native methods is keya. “it wasn’t easy”, he adds, “and took a lot of work to unlearn the contemporary techniques i acquired in art school”. unlike the vast majority of native american pottery artists who struggle to sell their artwork, nampeyo has been exceptionally successful. initially picked up by the press as the epitome of a “new generation of native american artists”, his work has been presented at leading galleries and has been sold to wealthy collectors.  description  image  table f16: detailed descriptions of stimuli (h4 d)  
       


authors_index: toscano, marion e.
secondary_title_index: journal of lgbtq issues in counseling
abstract_index:
       whether in research or clinical setting, the use of standardized measures allows for more accurate comparisons and interpretations. to this end and to better understand the factors of self-acceptance - sexual and gender identities (fsa-sgi) scale, data from 1237 individuals who identify as lgbqt, regardless of gender identity, was analyzed in order to create a norms group whose descriptives will be utilized to create standardized scores for future research and clinical work. additionally, the fsa-sgi evidenced incremental validity over personality and internalized homophobia; thus, indicating that for
  lgbtqtindividuals, psychological well-being is comprised of both internal and external factors.     

title_index:
       the continued exploration of self-acceptance: creating a normative sample and examining the incremental validity of the fsa-sgi     

full_text_index:
       journal of lgbtq issues in counseling  issn: (print) (online) journal homepage: www.tandfonline.com/journals/wlco21  the continued exploration of self-acceptance: creating a normative sample and examining the incremental validity of the fsa-sgi marion e. toscano to cite this article: marion e. toscano (2024) the continued exploration of self-acceptance: creating a normative sample and examining the incremental validity of the fsa-sgi, journal of lgbtq issues in counseling, 18:1, 39-57, doi: 10.1080/26924951.2023.2293294 to link to this article: https://doi.org/10.1080/26924951.2023.2293294  view supplementary material  published online: 31 jan 2024.  submit your article to this journal  article views: 148  view related articles  citing articles: 1 view citing articles  full terms & conditions of access and use can be found at https://www.tandfonline.com/action/journalinformation?journalcode wlco21  
  journal of lgbtq issues in counseling 2024, vol. 18, no. 1, 39–57 https://doi.org/10.1080/26924951.2023.2293294  the continued exploration of self-acceptance: creating a normative sample and examining the incremental validity of the fsa-sgi marion e. toscano counseling department, mount mary university, milwaukee, wisconsin, usa  abstract  keywords  whether in research or clinical setting, the use of standardized measures allows for more accurate comparisons and interpre tations. to this end and to better understand the factors of self-acceptance - sexual and gender identities (fsa-sgi) scale, data from 1237 individuals who identify as lgbqt, regardless of gender identity, was analyzed in order to create a norms group whose descriptives will be utilized to create standar dized scores for future research and clinical work. additionally, the fsa-sgi evidenced incremental validity over personality and internalized homophobia; thus, indicating that for lgbtq t individuals, psychological well-being is comprised of both internal and external factors.  self-acceptance; lgbtqt; fsa-sgi; normative sample; norms group; incremental validity; predictive validity  creating and validating a new measure is not a one-step procedure; it is a process that takes several steps including defining latent dimensions, creating items, testing the suitability for the items to be analyzed, exploring a factor structure, confirming a factor structure, and examining reliability and validity (boateng et al., 2018; worthington & whittaker, 2006). in addition to the above-mentioned process, additional steps remain, including the creation of a normative sample or norms group, as well as the examination of incremental validity or the ability of the measure to explain more about an outcome than what can already be known. this paper will briefly review the factors of selfacceptance – sexual and gender identities (fsg-sgi) scale as well as explore the creation of a norms group and examine the fsa-sgi ability to predict psychological outcomes over and above already existing measures. fsa-sgi  the fsa-sgi was designed by toscano (2022) as a way to quantitatively measure three of the most common areas that lgbtq t individuals report contact marion e toscano toscanom@mtmary.edu counseling department, mount mary university, milwaukee, wi, usa. supplemental data for this article can be accessed online at https://doi.org/10.1080/26924951.2023.2293294.  2024 taylor & francis group, llc  
  40  m. e. toscano  as affecting their self-acceptance; self-acceptance, safety, and connection. according to toscano (2022): self-acceptance (sa) is defined as the acknowledgement and acceptance of one’s sexual and gender identities ... safety is defined as the acknowledgement and acceptance of one’s sexual and gender identities in absence of or despite concerns of being harassed, bullied, etc., ... [and] connection is defined as the acknowledgement and acceptance of one’s sexual and gender identities in absence of or despite concerns of being rejected by family and friends, disowned, shunned from religious communities, or otherwise socially isolated. (p. 291)  in that article, in addition to defining the latent dimensions, toscano (2022) evidenced that fsa-sgi was suitable for factor analysis, conducted both exploratory and confirmatory factor analyses, the latter of which was found to be acceptable across all fit indices; as well as indicated good internal consistency and convergent validity (see toscano, 2022 for more detailed information). beyond the psychometrics, the value of the fsa-sgi is as multifaceted as the scale itself. the fsa-sgi is the first multi-faceted measure created spe cifically for use in the lgbtq t communities that examines both the intra(self-acceptance) and interpersonal (safety and connection) aspects of self-acceptance. in addition to the research implications of not needing to utilize and balance multiple scales, which may have different definitions, focuses, and/or may not have been validated for use with this population, there are theoretical and clinical benefits as well. in terms of theoretical implications, the fsa-sgi presents data that can be used to argue against the notion that acceptance of self is all that is needed for psychological wellbeing in lgbtq t individuals. the fact that the two social factors cor relate to the personal factor, indicates that understanding of self is, at least in part, related to the acceptance others have for the individual. clinically, the fsa-sgi offers counselors and counselors-in-training a simple, 18-item measure that can assess a client’s overall sense of selfacceptance, as well as three distinct areas. this ability to look at the whole picture as well as its parts provides the clinician with a more personalized approach when working with lgbtq t clients who may be struggling in this area as it can indicate where the struggle lies. this can possibly quicken the time from interview to intervention. as with anything, limitations must always be discussed. a limitation, as noted in toscano (2022), was that the samples used in the creation and validation studies were not representative of the population in terms of eth nicity and race. although this limitation was more about the samples used and less about the measure itself, lack of representation in the creation of a measure can result in a biased measure, leading to misunderstanding of results, inappropriate intervention suggestions, and possible misdiagnoses.  
  journal of lgbtq issues in counseling  41  thus, the need to gather a representative sample in which to further exam ine the of the fsa-sgi. norms group  a norms group or normative sample is a representative sample from the population in which a measure was designed to be used. at one point, cre ating a norms group and normative data for a measure was seen as stand ard practice but has now become more optional, questionable, and even opposed by some. a norms group is necessary for many statistical proce dures, including the creation of standard scores. according to geisinger (2001) “without standardization, measurement is only an informal process that varies from examiner to examiner” (p. 1769). although times have changed and this statement is no longer adhered to as an absolute, the value of standardization and normative samples have not faded. according to hays and wood (2017), “norms as comparative data can provide greater efficiency in predicting phenomena or selecting counseling interventions, making individual score interpretation more meaningful” (p. 282). further, it can assist researchers with being able to quickly and accurately determine if the results from their study are in line with or contrary to those in the literature. as noted previously, there is some concern with the process of norming data. for example, some may argue that queer theory seems to be against normative samples and standardized scores, and possibly quantitative research in general, as quantitative research focuses on categorization (i.e. labeling), instead of fluidity (browne, 2010; frank ii & cannon, 2010). although this is an accurate criticism and limitation of quantitative research, it does not need to be in conflict with queer theory. browne (2010) noted about the use of categorization and reliance on norms that “where ticking the boxes, and the interpretation of data, contest objection able and exclusionary social norms, they can also offer anti-normative (and thus in some ways queer) politics and potentials” (parentheses included in original, p. 249). incremental validity  the value of any scale is in its ability to tell us more about the outcome variable than what can already be explained by existing measures. in order to do this, one must first identify a measure or measures to be used as a control. the control measure(s) would need to be able to evidence relation ship to the outcomes of interest as well as the measure that is being exam ined. due to the breadth and depth of information that can be explained  
  42  m. e. toscano  by the five factor model of personality (mccrae, 2010), and because it has been shown in previous literature to have predictive ability on psychological well-being, including all the outcome variables selected for this study within this population (stroud et al., 2015; toscano, 2018), it makes sense to utilize personality, as explained by the international personality item pool (ipip-50), as the basis for testing the fsa-sgi. in addition to the ipip-50, the revised internalized homophobia scale (ihp-r) was utilized as a control. this deci sion was made for two reasons: first, the ihp-r is a well-used measure of internalized homophobia which toscano (2022) noted was the converse of self-acceptance; thus, the use of the measure for convergent validity. second, because the results from the confirmation study indicated that the relationship between the ihp-r and fsa-sgi was considered large (r 1⁄4 .50) based on cohen’s (1992) indicators (toscano, 2022). thus, the ihp-r seemed an appro priate measure to use in examining the incremental validity of the fsa-sgi. current research  this study was conducted for several reasons. first, to obtain a representa tive sample of lgbtq t adults in order to create a norms group whose data (means and standard deviations) will be utilized to create standardized scores for comparison in future research and clinical settings. second, to examine the following hypotheses: (a) that the fsa-sgi’s factor structure and convergent validity found in the creation and validation studies by toscano (2022) would be replicated in a more diverse representative sam ple; and (b) that the fsa-sgi will evidence incremental validity over per sonality and internalized homophobia when predicting the psychological outcomes of depression, anxiety, stress, self-compassion, and resilience. research design and procedures participants  a sample of 1237 lgbtq t adults (18 years old and older), who self-identi fied as lgbq t regardless of gender identity and reside in the united states of america, were solicited utilizing convenience sampling techniques, including snowball, direct contact, and amazon mechanical turk (mturk). each mturk participant received a small monetary compensation ( 0.50) for the completion of the survey, with the money being directly credited to their respective amazon accounts by amazon. participants in this study ranged in age from 18 to 78 years of age, with a mean age of 31.24 (s 1⁄4 9.16). in terms of gender identity and sexual orientation, the largest groups were cisgender females (n 1⁄4 613) and participants who identified as bisexual (49.10%). further, 53.90% of the participants identified as  
  journal of lgbtq issues in counseling  43  table 1. participants’ demographics with number and frequency including racial comparison to the 2010 and 2020 us census data. item age (m 1⁄4 31.24, s 1⁄4 9.16) age groups: 18–39 40–78 race & ethnicity: african american/black american indian/indigenous or alaska native arab asian caucasian/white hispanic/latino (a)(x) native hawaiian or pacific islander multiethnic other gender identity groups: cisgender females cisgender males transgender individuals transgender male transgender female gender queer/gender fluid other sexual orientation: gay man lesbian bisexual pansexual asexual queer other residential setting: rural suburban urban  frequency  percent (%)  1166 71  94.30 5.70  193 12 1 71 667 225 4 53 11  15.60 1.00 .10 5.70 53.90 18.20 .30 4.30 .90  613 466 158 53 27 60 18  40.60 37.70 12.80 4.30 2.20 4.90 1.50  237 210 607 72 37 52 22  19.20 17.00 49.10 5.80 3.00 4.20 1.80  227 598 412  18.40 48.30 33.30  2010 us censusa  2020 us censusb  12.60 .90  13.40 1.30  4.80 62.50 16.30 0.20 2.90  5.90 60.10 18.50 0.20 2.80  note. n 1⁄4 1237. a 1⁄4 percentage of the population based upon the 2010 us census data (humes et al., 2011), b 1⁄4 percentage of the population based upon the 2020 us census data (united states census bureau, 2022).  caucasian/white. participants represented every state in the united states and the district of columbia, with the exception of montana, and 48.30% reported living in suburban areas. see table 1 for demographics and the racial comparisons to the 2010 us census (humes et al., 2011) as well as the 2020 us census (united states census bureau, 2022). for additional demographics information, see supplemental table s1. measures the factors of self-acceptance - sexual and gender identities (fsa-sgi).  developed by toscano (2022), the fsa-sgi is an 18-item three-factor measure assessing self-acceptance of sexual and gender identities. selfacceptance (sa) is comprised of seven items, safety (s) has five items, and connection (c) has six items. it is scored on a 5-point likert-type scale, ranging from strongly disagree (1) to strongly agree (5). example items are “i  
  44  m. e. toscano  accept my sexual/gender identity is part of who i am” (sa), “accepting my sexual/gender identity makes me a target for bullying/harassment” (s), and “if i accept my sexual/gender identity, i will be isolated from those i care about” (c). after the appropriate items are recoded, scores are summed by factor, with higher factor scores indicating higher levels of acceptance by the individ ual on that factor. a high overall or total score indicated high overall selfacceptance. according to toscano (2022), the measure indicates strong internal consistency with an a 1⁄4 .90 for self-acceptance, a 1⁄4 .83 for safety, a 1⁄4 .87 for connection, and total score alpha of .91. international personality item pool (ipip-50). developed by goldberg (1992), the ipip-50 is a 50-item scale with 10 items assessing each of the five domains of personality: emotional stability (es), extraversion (e), openness to experience (o), agreeableness (a), and conscientiousness (c). it is scored on a 5-point likert-type scale, ranging from very inaccurate (1) to very accurate (5). examples of personality items are “am the life of the party,” “seldom feel blue,” and “follow a schedule.” raw scores are summed by factor with high scores, indicating that the individual exhibits more of that trait than others. according to goldberg (1992), the reported alpha reliabilities for the domains were es 1⁄4 .86, e 1⁄4 .87, o 1⁄4 .84, a 1⁄4 .82, and c 1⁄4 .79, with an overall alpha reliability of .84. convergent validity of the ipip-50 has been tested by com paring the measure to the neo pi-r, which indicated domain correlations ranging from .88 to .93 (goldberg, 1992). depression anxiety stress scale 21 (dass-21). developed by lovibond and lovibond (1995) as a short form to their dass measure, this is a three fac tor, 21-item (7 items per factor) scale measuring depression (dep; e.g. “i couldn’t seem to experience any positive feeling at all”), anxiety (anx; e.g. “i was aware of dryness of my mouth”), and stress (str; e.g. “i found it hard to wind down”). all items are scored on a 4-point likert-type scale ranging from never (0) to almost always (3). subscale scores range from 0–21, with the maximum total score of 63. after the appropriate items are recoded, scores are summed by factor, with higher factor scores indicating higher levels of depression, anxiety, and/or stress. the alpha reliabilities for the scales were found to be acceptable with dep 1⁄4 .90, anx 1⁄4 .79, and str 1⁄4 .89 (crawford et al., 2011, p. 8). as for validity, henry and crawford (2005) tested several models and found that the three-factor was seen to be the best fit in a non-clinical sample. revised internalized homophobia scale (ihp-r). developed by herek et al. (1998) as a revision to the ihp, the ihp-r looks at gay men’s, lesbians’, and bisexual individuals’ attitudes toward their sexual identity; whereas, the  
  journal of lgbtq issues in counseling  45  ihp was only for use with gay men. the ihp-r is a 5-item scale with items scored on a 5-point likert-type scale ranging from strongly disagree (1) to strongly agree (5). an example of items on this measure are “i have tried to stop being attracted to women [men] in general” and “i feel that being lesbian/gay/bisexual is a personal shortcoming for me.” scores are summed with higher scores, indicating higher levels of internalized negative feelings relating to their sexual/affectional orientation. in their study, herek et al. (2009) noted that the ihp-r had an alpha reliability of .82. brief resilience scale (brs). smith et al. (2008) developed the brs as a meas ure of resilience in terms of ability to recover from a trauma or stressful experiences. the brs is a balanced keyed 6-item scale with items scored on a 5-point likert-type scale ranging from strongly disagree (1) to strongly agree (5). items on the brs include “i tend to bounce back quickly after hard times” and “it is hard for me to snap back when something bad hap pens.” after the appropriate items are recoded, scores are summed with higher scores, indicating more ability to bounce back after a hardship. the developers of the test indicated that the alpha of the brs have been accept able and have ranged between .80–.91. using four different samples within one study, smith et al. (2008) was able to determine the validity of the measure by comparing it to other resilience measure as well as outcome variables. self-compassion scale – short form (scs-sf). raes et al. (2011) developed the scale to measure six components of self-compassion: self-kindness, selfjudgment, common humanity, isolation, mindfulness, and over-identifica tion. the scs-sf is comprised of 12 items, two from each of the areas of interest (e.g. “when i fail at something important to me i become con sumed by feelings of inadequacy,” “when something upsets me i try to keep my emotions in balance,” and “i’m intolerant and impatient towards those aspects of my personality i don’t like”). the items are scored on a 5point likert-type scale ranging from almost never (1) to almost always (5). after the appropriate items are recoded, scores are summed with higher scores, indicating higher levels of overall self-compassion. the alpha reliabilities for the subscales have ranged from .55 (self-kindness) to .81 (self-judgement), with the overall alpha equaling .87 (raes et al., 2011). data collection  prior to the collection of any data, permission to conduct this study was granted by the author’s university institutional review board (irb). the sampling process was purposely done in two stages. stage one was an open call using convenience sampling techniques, including snowball, direct  
  46  m. e. toscano  contact, and solicitation from mturk workers. although these are similar sampling techniques to those in previous studies (toscano, 2018; toscano, 2022), all data are original. the inclusionary criteria for this study were that the individual must be 18 years of age or older, identify as lgbq t regardless of gender identity, reside within the us, and read and understand english at a 7th grade level. due to concerns about the quality and reliability of the use of mturk in counseling research (see chmielewski & kucker, 2020), when constructing the survey in qualtrics, kickouts or triggers were embedded into the demo graphics that would end the survey if a participant did not meet inclusion ary criteria (e.g. heterosexual or under the age of 18) or if they failed the validation questions (e.g. selected true to the question “i can lift 10 times my body weight over my head”). though qualtrics records all survey results, partial assessments that are caused by these triggers are easily rec ognizable and thus can be removed once the data is moved to spss. these precautions and later screening methods, such as rechecking answers on validation question and checking for univariate and multivariate outliers, is done to increase confidence in the study findings and increase accuracy of conclusions. as chmielewski and kucker (2020) noted “the use of unscreened data could lead to improper scale development decisions or inaccurate conclusions about performance of existing measures” (p. 470). the first data collection was intended to gain a large percentage of the respondents (n 1⁄4 1813) and after the removal of cases with missing data and failed validation questions, the collection resulted in the retention of 1006 cases. missing data was primarily due to the survey automatically end ing if participants did not meet inclusionary criteria. at this point, the demographics were examined for representativeness. due to the fact that there is no specific data on how many lgbtq t individuals reside in the united states and no specific racial or ethnic breakdown of the lgbtq t communities, the 2010 us census (humes et al., 2011) results were used to determine representation. the 2010 data were utilized because the 2020 us census data was not available at the time of data collection. however, that data has since been included (see table 1). based upon the comparison of the sample to the 2010 census, it was determined that the sample did not accurately represent african american/black individuals or hispanic/latinx. this led to the initiation of stage two. in stage two, the researcher, with irb approval, revised the collection method to specifically recruit members of the african american/black and hispanic/latinx communities in order to increase the percentages of these populations within the overall sample. these additional data collections were conducted similarly to the original collection, with the added inclu sionary criteria of race and ethnicity. these data collections garnered an  
  journal of lgbtq issues in counseling  47  additional 939 cases. after the removal of cases with missing data (from cases that did not meet inclusionary criteria) and failed validation ques tions, this procedure rendered an additional 245 retained cases (99 african american/black and 146 hispanic/latinx). procedures  once data collection was completed, all data was aggregated. although cases with missing data were removed prior to this step, the data still needed to be examined for univariate and multivariate outliers as well as the assumption of linearity. linearity was evaluated using a plot and it was determined that there was no violation. the univariate normality was deemed acceptable because the standard error of skewness (.070) and the standard error of kurtosis (.139) were in normal rage (less than 5; warner, 2020). an examination of multivariate normality was conducted, following procedures described by tabachnick and fidell (2007). utilizing this method, 14 outliers were identified and removed, resulting in a final study with n 1⁄4 1237. prior to further analyses, the data was checked again for representation and to match to the 2010 census. as seen in table 1, the racial and ethnic percentages in final data matched or exceeded the 2010 census data with the exception of caucasian/white individuals (current study 1⁄4 53.90%, 2010 census 1⁄4 62.50%). results descriptive statistics  the means, standard deviations, and alpha reliabilities for the study varia bles are listed in table 2. the alphas reliabilities indicate strong internal consistency for all measures. the ipip-50 is the only scale with us norms and has been converted into t-scores, meaning average scores on this meas ure range between 45 and 55. all of the ipip-50 domains were seen as average, with the exception of extraversion (m 1⁄4 43.64, s 1⁄4 11.97); however, this variant was expected, as it replicated results from previous studies with lgbtq t participants recruited via mturk (toscano, 2018; toscano et al., 2017). though the means and standard deviations have evidenced some stability over time, chmielewski and kucker (2020) note that reliability of personality measures may be reduced in mturk samples. to evaluate this, the alphas of the ipip-50 were visually compared to the alphas reported in toscano (2018). although the alphas were slightly lower in this sample than the toscano (2018) sample (e.g. in toscano, 2018 for extraversion a 1⁄4 .89 and this study a 1⁄4 .86), they were still within the acceptable range (greater or equal to .70; tavakol & dennick, 2011). additionally, as with  
  note. n 1⁄4 1237. a 1⁄4 scores presented as t-scores with m 1⁄4 50 and s 1⁄4 10 based on robertson et al. (2015). p .05. p .01. p .001, two-tailed.  m s a 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 1. age 31.24 9.16 – .04 .02 .13 .06 .15 .15 .15 .07 .16 .14 .16 .14 .12 .05  .18 .22 .19 2. ethnicity – – – – .00 .02 .06 .12 .08 .04  .09 .06 .04  .06 .12 .08 .05  .01  .07 .029 3. gender identity – – – – .12 .20 .12 .18 .18 .044 .14 .05  .18 .015 .026 .08 .14 .13 .10 4. self-acceptance 27.72 5.37 .87 – .22 .48 .72 .18 .09 .43 .32 .31 .25 .26 .50 .27 .25 .20 5. safety 15.39 4.85 .84 – .58 .74 .30 .10 .14 .09 .15 .16 .21 .33 .34 .40 .36 6. connection 21.37 6.21 .89 – .89 .24 .03 .36 .33 .29 .16 .21 .59 .42 .54 .42 7. fsa-sgi total 64.48 13.00 .90 – .30 .06 .40 .32 .32 .24 .28 .61 .44 .51 .42 8. emotional stabilitya 47.55 10.10 .74 – .30 .11 .16 .42 .68 .62 .16 .63 .51 .65 9. extraversiona 43.64 11.97 .86 – .11 .14 .08 .42 .37 .02  .23 .09 .15 10. opennessa 53.74 12.32 .79 – .54 .38 .15 .19 .34 .23 .33 .20 11. agreeablenessa 45.78 11.98 .82 – .37 .24 .20 .29 .28 .31 .27 12. conscientiousnessa 48.82 12.23 .81 – .41 .39 .22 .46 .41 .40 13. self-compassion 36.24 8.33 .84 – .62 .09 .54 .35 .46 14. resilience 18.82 5.20 .85 –  .18 .52 .38 .45 15. internalized 11.19 5.83 .91 – .34 .38 .31 homophobia 16. depression 15.40 11.22 .92 – .77 .79 17. anxiety 13.05 10.28 .87 – .80 18. stress 16.32 9.81 .88 –  variables  table 2. descriptive statistics, alpha reliabilities, and zero-order correlations of the fsa-sgi scores, age, ethnicity, gender identity, and study variables for lgbtq t individuals.  48 m. e. toscano  
  journal of lgbtq issues in counseling  49  the expected differences between the alphas for the fsa-sgi in this study, which are minimally lower than those reported by toscano (2022), the dif ference in the alphas in the ipip-50 may be due in part to the fact that this sample is more diverse than the toscano (2018) or toscano (2022) samples. factor analysis  although the factor structure of the fsa-sgi was confirmed in the toscano (2022) article, those studies did not have representative samples. therefore, a principal components analysis (pca) selecting three factors with the resulting dimensions obliquely rotated was conducted in order to examine if the factor structure found in previous studies would hold with a more diverse sample as well to provide future researchers with a factor structure based on the norms groups for comparison purposes. prior to doing so, suitability for analysis was examined using the kaiser-meyerolkin (kmo) measure of sampling adequacy and bartlett’s test of sphericity. according to williams et al. (2010), these indexes were utilized to determine if the grouped items are suitable for factor analysis. the kmo is considered acceptable at 0.50; whereas, suitability is determined by bartlett’s with the obtained v2 indicating significance. for the fsa-sgi scale, the kmo 1⁄4 .92 and the bartlett’s was found to be significant [v2 (153, n 1⁄4 1237) 1⁄4 10748.46, p .001]. the results of the pca, pattern loadings, and factor correlations are reported in table 3. as can be seen in the table, all items load strongly on their expected factor with no double or mis-loadings. the total explained variance for all factors is 60.69%. selfacceptance had an explained variance of 37.78% with an eigenvalue (k) of 6.80, safety explained 15.84% (k 1⁄4 2.85), and connection explained 7.07% (k 1⁄4 1.27). correlations  in addition to obtaining descriptive statistics and a factor structure, zeroorder correlations were run examining the relationship between the fsasgi and all variables, including selected demographics (see table 2). with results similar to what was found in toscano (2022), the measure continues to demonstrate convergent validity in its relationships to internalized homophobia [r (1235) 1⁄4 .61, p .001 (total score)] and selfcompassion [r (1235) 1⁄4 .24, p .001 (total score)]. table 2 also shows that all factors of the fsa-sgi, including the total scale score, correlated to both age [r (1235) 1⁄4 .15, p .001 (total score)] and gender identity [r (1235) 1⁄4 .18, p .001 (total score)]. because of  
  50  m. e. toscano  table 3. pattern loadings from principal components analysis and component correlation matrix of the factors of self-acceptance - sexual and gender identities (fsa-sgi) scale items. fsa-sgi self-acceptance1 self-acceptance2 self-acceptance3 self-acceptance4 self-acceptance5 self-acceptance6 self-acceptance7 safety1 safety2 safety3 safety4 safety5 connection1 connection2 connection3 connection4 connection5 connection6 component correlations self-acceptance safety connection  self-acceptance .62 .77 .81 .72 .72 .74 .81  .07 .06  .02 .00 .07 .15 .04  .03 .04 .08  .10  safety  .01  .01  .00  .14  .04 .10 .05 .81 .83 .81 .75 .60 .02 .15  .05 .15 .06  .05  connection .01 .02  .04 .18 .04  .09  .04  .07  .03 .05 .11 .14 .70 .71 .85 .71 .71 .88  1.00  .19 1.00  .45 .50 1.00  note. n 1⁄4 1237. loadings above .40 are in boldface.  these relationships, additional analyses were conducted to better understand these connections. in terms of gender identity, an analysis of variance (anova) with a tukey’s honestly significant difference (hsd) post-hoc test was run to see what identities differed. based on the results, it was determined that cisgender females were different from cisgender males, and they were both different from transgender males, transgender females, gender queer/gender fluid, and other. however, these identities did not show a difference and thus, the data was collapsed into a singular group, transgender individuals. in terms of age, after testing several differ ent groups structures, including the age grouping for the ipip-50, it was determined that the difference laid at age 40. therefore, two groups were created, 18–39 and 40t. table 4 shows the descriptive statistics of the fsa-sgi broken down by both age and gender identity groups. incremental validity  to determine the incremental validity or value added by the use of the fsa-sgi, a series of two-step hierarchical regression analyses were con ducted. in the first step, the five personality domain scores of the ipip-50 and the ihp-r scores were entered and on the second step, the three fsasgi scales scores were entered (see table 5 for results). however, as noted previously, there are gender effects within the scale; thus, the regressions were also run separating the results by gender group. these results are  
  journal of lgbtq issues in counseling  51  table 4. descriptive information for the factors of self-acceptance – sexual and gender identi ties (fsa-sgi), facet scales and overall total scores separated by gender and age group. self-acceptance gender identity cisgender male (n 1⁄4 466) cisgender female (n 1⁄4 613) transgender identities (n 1⁄4 158) overall  safety  connection  total score  age  m  s  m  s  m  s  m  s  18-39 (n 1⁄4 437) 40t (n 1⁄4 29) 18-39 (n 1⁄4 574) 40t (n 1⁄4 39) 18-39 (n 1⁄4 155) 40t (n 1⁄4 3) n 1⁄4 1237  26.68 31.14 28.49 29.49 26.65 29.33 27.72  5.44 2.91 5.22 5.24 5.27 4.04 5.37  14.76 16.38 16.35 17.26 12.94 16.67 15.39  4.74 4.27 4.87 4.89 4.02 4.04 4.85  20.06 24.72 22.34 24.85 19.94 22.00 21.37  6.22 5.19 6.08 5.41 5.83 7.21 6.21  61.50 72.24 68.19 71.59 59.54 68.00 64.48  12.64 9.04 13.14 12.27 10.32 15.13 13.00  note: age (18-39) n 1⁄4 1166; age (40t) n 1⁄4 71.  presented in the supplemental materials: supplemental table s2 (cisgender females), 3 (cisgender males), and 4 (transgender individuals). as can be seen in table 5, the fsa-sgi explained a significant amount of unique variance (1%–8%) across all outcomes. in addition, all factors were indicated as being predictors, both individually and in conjunction to the other factors. for example, by itself, self-acceptance predicted resilience, whereas for anxiety, self-acceptance was a co-predictor with both safety and connection. although all factors predicted, there was one result that seemed to stand out; positive self-acceptance was seen as a predictor of anxiety, and stress. on the surface, these results may seem counterintuitive in that many researchers note that self-acceptance is negatively related to these variables (carson & langer, 2006; macinnes, 2006; popov et al., 2016; stitt, 2014). however, when examining the result by gender group (supplemental tables s2–s4), it was noted that these results were only seen in cisgender females, which make up 40.60% of the current sample. a possible explan ation of this variance is that in the united states, women often receive messages to value others over the self, especially when the valuing of the self is in direct contrast to the cultural, familial, and/or societal values of the group. in the case of acceptance of sexual and gender identities, it is not unreasonable to assume that the acceptance of these identities may mean a rebuttal of familial or cultural norms and the assumption of heterosexism. discussion  the results obtained from this study are relevant in several ways. first, the creation of the norms group allows for the means and standard deviations to be used to create standardized scores. the standardized scores in turn will allow future researchers to better understand and make interpretations of their samples and/or clinicians to be able to see how their clients com pare to their peers in terms of self-acceptance, safety, and connection.  
  52  m. e. toscano  table 5. incremental validity of the fsa-sgi controlling for personality domains and internalized homophobia for identified lgbtq t individuals. 95 % ci for b variable self-compassion step 1 constant internalized homophobia emotional stability extraversion openness agreeableness conscientiousness step 2 constant internalized homophobia emotional stability extraversion openness agreeableness conscientiousness self-acceptance safety connection resilience step 1 constant internalized homophobia emotional stability extraversion openness agreeableness conscientiousness step 2 constant internalized homophobia emotional stability extraversion openness agreeableness conscientiousness self-acceptance safety connection depression step 1 constant internalized homophobia emotional stability extraversion openness agreeableness conscientiousness step 2 constant internalized homophobia emotional stability extraversion openness agreeableness conscientiousness self-acceptance  b  ll  ul  se b  .63 .07 .45 .16  .01 .06 .09   1.87 .01 .41 .13  .05 .03 .06  3.13 .13 .48 .19 .02 .10 .13  1.27 .03 .02 .01 .02 .02 .02   2.74 .12 .45 .15  .03 .06 .08 .22  .08  .01   5.82 .04 .42 .13  .06 .03 .05 .14  .17  .09  .34 .19 .49 .18 .00 .09 .12 .29  .00 .06  1.57 .04 .02 .01 .02 .02 .02 .04 .04 .04  .00  .05 .25 .09 .02  .00 .06   1.70  .09 .22 .07 .00  .03 .04  1.71  .01 .27 .11 .04 .02 .08  .87 .02 .01 .01 .01 .01 .01   1.84  .02 .25 .09 .01  .00 .05 .09 .00  .01   3.97  .07 .22 .07  .01  .03 .03 .04  .06  .07  .28 .03 .27 .11 .04 .02 .07 .15 .06 .04  1.08 .03 .01 .01 .01 .01 .01 .03 .03 .03  50.12 .38  .55  .06  .00  .06  .16  46.61 .30  .60  .10  .05  .10  .20  53.62 .47  .50  .02 .05  .01  .11  1.79 .04 .03 .02 .02 .02 .02  55.18 .23  .52  .08 .01  .05  .15 .08  50.89 .13  .57  .12  .03  .09  .20  .02  59.48 .33  .46  .04 .06 .00  .11 .18  2.19 .05 .03 .02 .02 .02 .02 .05  b  r2  dr2  .53  .53  .55  .01  .44  .44  .45  .01  .49  .49  .52  .02  .05 .54 .23  .02 .09 .14 .08 .55 .22  .05 .08 .13 .14  .05  .01   .06 .48 .21 .05  .01 .13  .02 .48 .20 .03  .01 .13 .10 .00  .01  .20  .50  .07  .00  .06  .18 .12  .47  .08 .01  .05  .17 .04  (continued)  
  journal of lgbtq issues in counseling  53  table 5. continued. 95 % ci for b variable safety connection anxiety step 1 constant internalized homophobia emotional stability extraversion openness agreeableness conscientiousness step 2 constant internalized homophobia emotional stability extraversion openness agreeableness conscientiousness self-acceptance safety connection stress step 1 constant internalized homophobia emotional stability extraversion openness agreeableness conscientiousness step 2 constant internalized homophobia emotional stability extraversion openness agreeableness conscientiousness self-acceptance safety connection  b  .08  .32  ll  .20  .43  ul .04  .21  se b .06 .06  39.29 .38  .43 .06  .10  .07  .10  35.79 .30  .48 .02  .14  .11  .14  42.79 .47  .38 .10  .05  .02  .05  1.79 .04 .03 .02 .02 .02 .02  45.73 .16  .37 .03  .09  .05  .09 .25  .16  .54  41.66 .06  .42  .01  .13  .09  .13 .15  .27  .64  49.80 .25  .32 .07  .04  .01  .05 .35  .05  .44  2.07 .05 .03 .02 .02 .02 .02 .05 .06 .05  44.96 .28  .56 .04 .01  .09  .07  41.86 .21  .60 .00  .03  .13  .11  48.07 .36  .51 .07 .05  .05  .03  1.58 .04 .02 .02 .02 .02 .02  48.06 .16  .52 .02 .01  .08  .07 .19  .10  .33  44.30 .07  .57  .02  .03  .12  .11 .10  .20  .42  b  .04  .18  r2  dr2  .40  .40  .48  .08  .48  .48  .22  .42 .07  .12  .07  .11 .09  .36 .04  .10  .06  .11 .13  .07  .33  .17  .58 .04 .01  .11  .09  .52 .04 51.81 1.91 .24 .04 .09  .48 .02  .54 .05 .02 .02 .05 .02 .02  .04 .02  .10  .04 .02  .09 .28 .05 .11 .00 .05  .05  .23 .05  .21   note. ci 1⁄4 confidence interval; ll 1⁄4 lower limit; ul 1⁄4 upper limit. p .05. p .01. p .001.  second, in addition to the descriptive statistics, the presentation of the fac tor structure and alpha reliabilities will be used as future touchstones to continue to assess the internal consistency of the fsa-sgi. third, that the individual factor of the fsa-sgi can tell us more about the psychological well-being of lgbtq t individuals than what could be previously known by existing measures. specifically, that it was the socially-focused facets, safety and connection, which were the main drivers for the fsa-sgi when predicting the negative outcome variables of depression, anxiety, and stress. finally, it is the combination of all these results that quantify and support the long-held contention in the qualitative literature that the psychological  
  54  m. e. toscano  well-being of these communities is not just contingent on a single personal issue, but is influenced by other broader social realities (duncan & hatzenbuehler, 2014; klein & golub, 2016; moody et al., 2015). implications for counselors and researchers  the creation of a normative and representative sample with the collection of means and standard deviations allows for the creation of t-scores and thus, the ability for researchers and clinicians to make more accurate com parison between different samples. for the researcher, that allows them to make more specific interpretations and recommendations. it also provides a foundation in the literature in which future studies can utilize as a moor ing for understanding current situations and behaviors or as a springboard for the examination of new avenues of study, more in-depth work on out comes or with lgbtq t groups and individuals. for counselors, in add ition to the value already noted by toscano (2022) about providing a starting point for conversation and clinical exploration, having a measure with a comparative normative sample can help the clinician understand where the individual stands in relationship to their peers. for example, if the client’s score is lower (a t-score below 45) than the average, then the clinician can help the client identify why they feel that if they accept them selves they will be socially isolated. the clinician can then work with the client in being able to identify sources of possible social connection that are welcoming and accepting. these sources may then act as protective agents against or possible replacements to other social connections that are less accepting or lost if the client fully accepts their sexual and gender iden tities. this can be especially helpful for a clinician or counselor-in-training that has limited experience with working with lgbtq t individuals. limitations & future research  as with any study, limitations must be identified and addressed. all partici pants needed access to a device (computer, mobile phone, or tablet) with online capability, and all self-identified as lgbtqt. the majority of the study participants were mturk workers, who have been screened and accepted by amazon prior to being allowed to participate in studies. although there are not clear numbers on how many persons in the us identify as lgbtqt, it must be noted that the numbers for transgender individuals were small, causing multiple groups to be collapsed for statis tical analyses. thus, it is recommended that further studies are conducted using the fsa-sgi with transgender males, transgender females, gender queer/gender fluid, and non-binary individuals in order to examine the  
  journal of lgbtq issues in counseling  55  differences between these communities. further studies are also needed to fully examine the relationship of self-acceptance and depression, stress, and anxiety within cisgender females. another area for future research is to test the functionality of the fsa-sgi in clinical settings so that counse lors can have access to a quick measure that assesses both intra- and inter personal aspects of self-acceptance in their lgbtq t clients. finally, due to the correlational design, causality cannot be inferred from the results. conclusion  this purpose of this paper was to continue to examine and validate the fsa-sgi, a new measure of self-acceptance for lgbtq t individuals. in service of this, a representative normative sample was collected in order to compile means and standard deviation for use in creating t-scores for com parison purposed for future research. additionally, the factor structure and internal consistency for this normative sample was presented and compared to the results of the fsa-sgi creation and validation study by toscano (2022). finally, incremental validity was assessed by examining the predict ability of the fsa-sgi after controlling for personality and internalized homophobia. overall, these results continue to evidence the reliability and validity of the fsa-sgi as well as add to the current literature on this measure specifically, and self-acceptance within the lgbtq t communities in general. acknowledgments i would like to thank drs. tamekia bell, christian chan, and tameeka hunter for their support and consultation during the conception and evaluation of this project.  disclosure statement no potential conflict of interest was reported by the authors.  references carson, s. h., & langer, e. j. (2006). mindfulness and self-acceptance. journal of rationalemotive & cognitive-behavior therapy, 24(1), 29–43. https://doi.org/10.1007/s10942-0060022-5 chmielewski, m., & kucker, s. c. (2020). an mturk crisis? shifts in data quality and the impact on study results. social psychological and personality science, 11(4), 464–473. https://doi.org/10.1177/1948550619875149 cohen, j. (1992). a power primer. psychological bulletin, 112(1), 155–159. https://doi.org/ 10.1037/0033-2909.112.1.155 crawford, j., cayley, c., lovibond, p. f., wilson, p. h., & hartley, c. (2011). percentile norms and accompanying interval estimates from an australian general adult population  
  56  m. e. toscano  sample for self-report mood scales (bai, bdi, crsd, ces-d, dass, dass-21, stai-x, stai-y, srds, and sras). australian psychologist, 46(1), 3–14. https://doi.org/10.1111/j. 1742-9544.2010.00003.x duncan, d. t., & hatzenbuehler, m. l. (2014). lesbian, gay, bisexual, and transgender hate crimes and suicidality among a population-based sample of sexual-minority adolescents in boston. american journal of public health, 104(2), 272–278. https://doi.org/10.2105/ ajph.2013.301424 boateng, g. o., neilands, t. b., frongillo, e. a., melgar-qui nonez, h. r., & young, s. l. (2018). best practices for developing and validating scales for health, social, and behav ioral research: a primer. frontiers in public health, 6(, 149. https://doi.org/10.3389/ fpubh.2018.00149 browne, k. (2010). queer quantification or queer(y)ing quantification: creating lesbian, gay, bisexual or heterosexual citizens through governmental social research. in k. browne & c. j. nash (eds.) queer methods and methodologies: intersecting queer theories and social science research. (pp. 231–249). routledge. frank, d. a., ii., & cannon, e. p. (2010). queer theory as pedagogy in counselor educa tion: a framework for diversity training. journal of lgbt issues in counseling, 4(1), 18– 31. https://doi.org/10.1080/15538600903552731 geisinger, k. f. (2001). test standardization. in w. e. craighead & c. b. nemcroff (eds.), the corsini encyclopedia of psychology and behavioral science (3rd ed., vol. 4, pp. 1796– 1770). john wiley & sons. goldberg, l. r. (1992). the development of markers for the big-five factor structure. psychological assessment, 4(1), 26–42. https://doi.org/10.1037/1040-3590.4.1.26 hays, d. g., & wood, c. (2017). stepping outside the normed sample: implications for val idity. measurement and evaluation in counseling and development, 50(4), 282–288. https://doi.org/10.1080/07481756.2017.1339565 henry, j. d., & crawford, j. r. (2005). the short-form version of the depression anxiety stress scales (dass-21): construct validity and normative data in a large non-clinical sample. the british journal of clinical psychology, 44(pt 2), 227–239. https://doi.org/10. 1348/014466505x29657 herek, g. m., cogan, j. c., gillis, j. r., & glunt, e. k. (1998). correlates of internalized homophobia in a community sample of lesbians and gay men. journal of the gay and lesbian medical association, 2, 17–25. http://psychology.ucdavis.edu/rainbow/html/ jglma_1998_pre.pdf herek, g. m., gillis, j. r., & cogan, j. c. (2009). internalized stigma among sexual minor ity adults: insights from a social psychological perspective. journal of counseling psychology, 56(1), 32–43. https://doi.org/10.1037/a0014672 humes, k. r., jones, n. a., & ramirez, r. r. (2011). overview of race and hispanic origin: 2010, 2010 census briefs. u.s. department of commerce economics and statistics administration u.s. census bureau. https://www.census.gov/prod/cen2010/briefs/ c2010br-02.pdf klein, a., & golub, s. a. (2016). family rejection as a predictor of suicide attempts and substance misuse among transgender and gender nonconforming adults. lgbt health, 3(3), 193–199. https://doi.org/10.1089/lgbt.2015.01111 lovibond, s. h., & lovibond, p. f. (1995). manual for the depression anxiety stress scales. (2nd. ed.). psychology foundation. macinnes, d. l. (2006). self-esteem and self-acceptance: an examination into their rela tionship and their effect on psychological health. journal of psychiatric and mental health nursing, 13(5), 483–489. https://doi.org/10.1111/j.1365-2850.2006.00959.x  
  journal of lgbtq issues in counseling  57  moody, c., fuks, n., pel aez, s., & smith, n. g. (2015). “without this, i would for sure already be dead”: a qualitative inquiry regarding suicide protective factors among trans adults. psychology of sexual orientation and gender diversity, 2(3), 266–280. https://doi. org/10.1037/sgd0000130 popov, s., radanovi c, j., & biro, m. (2016). unconditional self-acceptance and mental health in ego-provoking experimental context. suvremena psihologija, 19(1), 71–80. https://doi.org/10.21465/2016-sp-191-06 raes, f., pommier, e., neff, k. d., & van gucht, d. (2011). construction and factorial val idation of a short form of the self-compassion scale. clinical psychology & psychotherapy, 18(3), 250–255. https://doi.org/10.1002/cpp.702 robertson, t. m., jangha, a., piedmont, r. l., sherman, m. f., & williams, j. e. g. (2015). factor structure and personality disorder correlates of responses to the 50-item ipip big five factor marker scale department of pastoral counseling, loyola university maryland. [unpublished manuscript] smith, b. w., dalen, j., wiggins, k., tooley, e., christopher, p., & bernard, j. (2008). the brief resilience scale: assessing the ability to bounce back. international journal of behavioral medicine, 15(3), 194–200. https://doi.org/10.1080/10705500802222972 stitt, a. l. (2014). the cat and the cloud: act for lgbt locus of control, responsibility, and acceptance. journal of lgbt issues in counseling, 8(3), 282–297. https://doi.org/10. 1080/15538605.2014.933469 stroud, c. h., cramer, r. j., la guardia, a. c., crosby, j. w., & henderson, c. e. (2015). personality, spirituality, suicide, and self-injury proneness among lesbian, gay, and bisex ual adults. mental health, religion & culture, 18(9), 777–788. https://doi.org/10.1080/ 13674676.2015.1096240 tabachnick, b. g., & fidell, l. s. (2007). using multivariate statistics. (5th ed.). allyn and bacon. tavakol, m., & dennick, r. (2011). making sense of cronbach’s alpha. international journal of medical education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd toscano, m. e. (2018). [examining the role of spirituality and self-acceptance on psychological well-being in the lgbtq community]. [unpublished doctoral dissertation]. loyola university maryland. toscano, m. e. (2022). reconceptualization and validation of the measure of self-accept ance within the lgbtq t communities. journal of lgbtq issues in counseling, 16(3), 285–299. https://doi.org/10.1080/26924951.2022.2057389 toscano, m. e., piedmont, r. l., & wilkins, t. a. (2017). measuring spirituality in the lgbtq community using the assessment of spirituality and religious sentiments (aspires) scale department of pastoral counseling, loyola university maryland. [unpublished manuscript] warner, r. m. (2020). applied statistics ii: multivariable and multivariate techniques. (3rd ed.). sage publications inc. williams, b., onsman, a., & brown, t. (2010). exploratory factor analysis: a five-step guide for novices. australasian journal of paramedicine, 8(3), 1–13. retrieved from http://ro.ecu.edu.au/jephc/vol8/iss3/1 https://doi.org/10.33151/ajp.8.3.93 worthington, r. l., & whittaker, t. a. (2006). scale development research: a content analysis and recommendations for best practices. the counseling psychologist, 34(6), 806–838. https://doi.org/10.1177/0011000006288127)  
       


authors_index: ghorashi, halleh
secondary_title_index: feminist review
abstract_index:
       the christian church in the u.s. has historically played the roles of both advocator and oppressor of the disenfranchised. christian views on advocacy, justice, social welfare, and social services are often intertwined with the mainstream opinions of the time. yet, some groups of christians have also been at the forefront of human rights movements, convicted by their religious beliefs.
  currently, in the wake of 9/11, mass migration, and the rise of populism in the u.s., there is a collective anxiety in the u.s. against immigrants, refugees, and muslims. attitudes of othering—“us versus them”—have targeted these internationally vulnerable groups. this study examined whether othering attitudes and humanitarian attitudes could predict christian religious affiliation, using the 2012 chicago council survey on american public opinion and foreign policy, a representative national sample of adults (n = 1702). logistic regression analysis examined the relationships between othering attitudes (belief that the u.s. is inherently greater than other nations and attitudes towards immigrants, refugees, and muslims), humanitarian attitudes (importance of defending and promoting human rights and combatting world hunger), and religious affiliation (christian or non-christian). results showed humanitarian variables were not significant predictors of religious affiliation. however, othering variables
  were significant predictors of religious affiliation. this study offers insight into the increased need to address the subtle discrimination that may be negatively affecting christian community engagement with disenfranchised immigrant,
  refugee, and muslim groups.     

title_index:
       from absolute invisibility to extreme visibility: emancipation trajectory of migrant women in the netherlands     

full_text_index:
       associations between humanitarianism, othering, and religious affiliation lindsey r. disney the christian church in the u.s. has historically played the roles of both advocator and oppressor of the disenfranchised. christian views on advocacy, justice, social welfare, and social services are often intertwined with the mainstream opinions of the time. yet, some groups of christians have also been at the forefront of human rights movements, convicted by their religious beliefs. currently, in the wake of 9/11, mass migration, and the rise of populism in the u.s., there is a collective anxiety in the u.s. against immigrants, refugees, and muslims. attitudes of othering—“us versus them”—have targeted these internationally vulnerable groups. this study examined whether othering attitudes and humanitarian attitudes could predict christian religious affiliation, using the 2012 chicago council survey on american public opinion and foreign policy, a representative national sample of adults (n 1702). logistic regression analysis examined the relationships between othering attitudes (belief that the u.s. is inherently greater than other nations and attitudes towards immigrants, refugees, and muslims), humanitarian attitudes (importance of defending and promoting human rights and combatting world hunger), and religious affiliation (christian or non-christian). results showed humanitarian variables were not significant predictors of religious affiliation. however, othering variables were significant predictors of religious affiliation. this study offers insight into the increased need to address the subtle discrimination that may be negatively affecting christian community engagement with disenfranchised immigrant, refugee, and muslim groups. case scenario  t  he world has become increasingly global (ellwood, 2006;  united nations, 2012; united nations, 2015), and social workers and faith-based organizations both inside and outside the u.s. are more likely to see an increasingly international clientele. the united nations (2015) reported that international migration has grown rapidly social work & christianity, vol. 44, no. 3 (2017), 60–74 journal of the north american association of christians in social work  
  humanitarianism, othering, and religious affiliation  over the past fifteen years, from 173 million in 2000 to 244 million in 2015. for social workers, advocating for the marginalized and oppressed has unique challenges in the global world. in the wake of 9/11 and the current sociopolitical climate, attitudes of othering—“us versus them”— have targeted the internationally disenfranchised groups of immigrants, refugees, and muslims living in the u.s. and the west (ghorashi, 2010; hanson-easey, augoustinos, & moloney, 2014; nurullah, 2010; pehrson, brown, & zagefka, 2009). othering occurs when one automatically and conclusively determines that someone else is fundamentally different than oneself (said, 1979). the other person or groups of persons are mentally categorized as “not like me” or “not like us.” the humanity of the “them” group is lessened. an example of othering is the creation of an “in group” and an “out group,” each of which has innate or biological (essential) qualities that make them inherently better or worse (ghorashi, 2010; hanson-easey, augoustinos, & moloney, 2014). othering creates anxiety that my better group will “lose” to the other worse group. this anxiety can be used to manipulate in the social or political arenas—for example, to create drama in television, or political fodder in elections. nurullah (2010) used the othering process to critically examine the hollywood tv series “24” and the ways that a negative stereotype of “the arab” was created for the purpose of a dramatic story. u.s. politicians campaigning in the 2016 presidential election used emotional appeals (beydoun, 2016) about muslim terrorists or a flood of undocumented mexican immigrants (healy & haberman, 2015) both to create a threat and to present themselves as the savior to that threat (allen, 2004). meanwhile, the christian church exists within the influence of the current sociopolitical atmosphere. the history of the american church has included divergences between christian groups and views on advocacy, justice, social welfare, and social services. parts of the christian church have been at the forefront of humanitarian movements such as the abolitionist movement, the civil rights movement, prison reforms, and supporting humane treatment for those with disabilities and mental illness (ferris, 2011; lee & barrett, 2007; simon, 1994). yet, other parts of the christian church have endorsed and promoted social injustices, such as the biblical justification of slavery (elkins, 2013) and the abuse of women (miles, 1999). history suggests that the divergences between christian groups split between what is the dominant discourse and belief system, and the counterculture to the dominant discourse and belief system. for example, in the case of slavery, the sociopolitical atmosphere at the time believed that slavery was “right” and a marker of a civilized society, and the view of slavery as an injustice was not conceivable by most (miles, 1999). yet, at the same time, some smaller groups of christians (i.e. the quakers) were a key part of the abolitionist movement (miles, 1999). thus, while the  61  
  social work & christianity  62  christian and mainstream view of slavery were generally aligned at the time, there was also a vocal and active minority of christians who held an oppositional view on slavery, convicted by their religious beliefs that slavery was a human rights violation. one of the current discourses in the united states is that immigrants, refugees, and muslims are not only bad for america, but also dangerous (cooper, 2016 kopan, 2016). the purpose of this study is to examine the relationship between humanitarianism attitudes, othering attitudes, and religious affiliation. the hypothesis is that there will be no significant relationship between humanitarianism attitudes and religious affiliation, but that due to the prevalent othering of immigrants, refugees, and muslims in the u.s. mainstream discourse, there will be a significant relationship between othering attitudes and religious affiliation, with greater othering attitudes being a predictor of christian religious affiliation. method participants the data analyzed in this study were drawn from the 2012 chicago council survey on american public opinion and foreign policy, which is a representative national sample of 1,702 adults. the 2012 chicago council survey collected a host of information on participants’ opinions and perspectives related to domestic and foreign policy issues, as well as participants’ backgrounds and characteristics. data were collected using a survey via phone interviews. measures othering variables: three questions were identified as having othering attitudes intrinsic in the questions. these three questions assessed for othering attitudes towards muslims, immigrants, and refugees, and for essentialist attitudes of the u.s. being inherently better than any other country. attitudes towards muslims. attitudes towards muslims were assessed through a dichotomous, single item question—“please tell me which of the following statements is closer to your own view: (1) because muslim religious, social and political traditions are incompatible with western ways, violent conflict between the two civilizations is inevitable. (2) because most muslims are like people everywhere, we can find common ground and violent conflict between the civilizations is not inevitable.” this question was chosen as an othering measure because language such as “incompatible” and  
  humanitarianism, othering, and religious affiliation  “inevitable” suggest that the respondent believes in absolute difference and rigid categories between entire groups of people (west versus muslim). the other groups of persons (in this case, muslims) are mentally categorized as fundamentally different and “not like us.” attitudes towards immigrants and refugees. attitudes towards immigrants and refugees were assessed through a single item question—“please select whether you see large numbers of immigrants and refugees coming into the u.s. as a critical threat, an important but not critical threat, or not an important threat at all.” responses were dichotomized into those who responded that immigrants and refugees were some kind of threat (critical or important) and those were responded that immigrants and refugees were not a threat. this question was chosen as an othering measure because fear of a group of people based on one identifying feature—in this case, status as an immigrant or a refugee—is a characteristic of the othering process. essentialist attitudes. essentialist attitudes were assessed through a dichotomous, single item question—“some people say the united states has a unique character that makes it the greatest country in the world. others say that every country is unique, and the united states is no greater than other nations. which view is closer to your own? (1) the greatest country in the world (2) no greater than other nations. this question was chosen as an othering measure because language such as “unique character” suggests that the respondent believes that the u.s. is inherently better than other countries. a belief in inherent qualities that make one country better than another is an essentialist belief. humanitarianism variables two questions were identified as having a global humanitarian nature to them. these two questions assessed views of both foreign policy and humanitarian intervention. assessing for both is an appropriate indicator of views on global humanitarianism because u.s. intervention and humanitarianism are closely connected and intertwined (holzgrefe & keohane, 2003). humanitarianism implies some type of concrete intervention, such as a foreign policy or an organization’s mission (i.e. the red cross). war interventions have often been justified by the need to protect or liberate civilians from their government (tirman, 2003). however, a limitation to the validity of this measure is the overlap between foreign policy beliefs and humanitarianism beliefs. world hunger. importance of combatting world hunger was assessed through a single item question—“please select whether you think that combating world hunger should be a very important foreign policy goal of the united states, a somewhat important foreign policy goal, or not an important goal at all.”  63  
  social work & christianity  64  human rights. importance of advocating for human rights was assessed through a single item question—“please select whether you think that promoting and defending human rights in other countries should be a very important foreign policy goal of the united states, a somewhat important foreign policy goal, or not an important goal at all.” religious affiliation religious affiliation was assessed through a single item question that asked whether the respondent identified as either christian (n 1351, 72.0%), jewish (n 33, 1.8%), muslim (n 7, 0.4%), hindi (n 9, 0.5%), other (n 122, 6.5%), or no religion (n 323, 17.2%). nearly all participants responded to the question, with only 1.7% who refused. seventy-two percent chose christian, and all other responses were coded as non-christian (n 494, 26.4%). sociodemographic measures participants were asked to report their gender, racial identity (white/ non-white), education, geographical region, and annual household income. analysis plan all analyses were completed using spss, version 21. bivariate analyses examined correlates of christian and non-christian affiliation. a logistic regression analysis examined whether the association between othering and humanitarianism was similar among christian and non-christian groups, controlling for other sociodemographic measures. the model examined whether the main effects of othering and humanitarianism were associated with religious affiliation. limitations one limitation of this study was that the variables used to measure othering attitudes and humanitarian attitudes were single item questions and could not have captured the completeness of attitudes. further research should examine these specific constructs more closely. additionally, dichotomous variables can be susceptible to misinterpretation. dichotomized variables force the respondent to make a choice between two opposing views—when his or her actual view may be more nuanced. although databased, this study cannot be more than exploratory. results univariate descriptives of othering and humanitarianism variables are reported in table 1 and table 2. bivariate comparisons among participants  
  humanitarianism, othering, and religious affiliation  65  showed a significant difference in all othering variables and humanitarianism variables, gender, racial identity, and geographical region between those who identified as christian versus those who identified with another religion or no religion. those with a christian affiliation were more likely to report othering attitudes, more likely to be female, and more likely to identify as white. they were more likely to live in the south and less likely to live in the west. for the humanitarianism variables, those with a christian affiliation were more likely to report “somewhat important,” while non-christians were more likely to report “very important” or “not important” than christians. see table 3 for sample characteristics and bivariate comparisons among participants. table 1: univariate descriptives of othering variables which statement is closer to your own view?  yes  n 1877  because muslim religious, social and political traditions are incompatible with western ways, violent conflict between the two civilizations is inevitable.  43.4%  815  because most muslims are like people everywhere, we can find common ground and violent conflict between the civilizations is not inevitable.  54.0%  1014  large numbers of immigrants and refugees coming into the u.s. is a critical or important threat.  83.2%  1562  large numbers of immigrants and refugees coming into the u.s. is not an important threat at all.  16.4%  307  the united states has a unique character that makes it the greatest country in the world.  70.4%  1321  every country is unique, and the united states is no greater than other nations.  29.0%  545  attitudes towards muslims  attitudes towards immigrants and refugees  essentialist attitudes  table 2: univariate descriptives of humanitarianism variables foreign policy goala very important  somewhat important  not important  yes  n  yes  n  yes  n  combating world hunger  41.1%  771  49.3%  926  8.8%  166  promoting and defending human rights in other countries  27.3%  513  58.9%  1105  13.4%  251  variable  an 1877  
  social work & christianity  66  table 3: sample characteristics and bivariate associations with religious affiliation overall n 1877 %  christian n 1351 %  non-christian n 526 %  x2  incompatible  44.6  47.9  36.0  21.15**  like people everywhere  55.4  52.1  64.0  critical or important threat  83.6  88.2  71.7  not a threat  16.4  11.8  28.3  u.s. is greatest country in the world, has unique character  70.8  76.8  55.3  u.s. is no greater, all countries are unique  29.2  23.2  44.7  very important  41.4  39.5  46.3  somewhat important  49.7  51.9  44.2  not important  8.9  8.7  9.6  very important  27.4  25.6  32.1  somewhat important  59.1  61.6  52.8  not important  othering attitudes towards muslims  attitudes towards immigrants and refugees 74.57**  essentialist attitudes 84.15**  humanitarianism combating world hunger 9.02*  promoting and defending human rights in other countries 12.25**  13.4  12.8  15.1  gender (female)  51.1  53.9  44.1  14.49**  racial identity (white)  71.0  74.6  61.6  31.12**  northeast  17.6  17.7  17.3  41.73**  midwest  23.6  25.0  20.0  south  35.9  38.2  30.0  west  22.9  19.1  32.7  less than high school  11.5  10.5  13.9  high school/ged/equivalent  31.9  32.9  29.3  more than high school  56.6  56.6  56.8  region  education 5.34  
  humanitarianism, othering, and religious affiliation  67  overall n 1877 %  christian n 1351 %  non-christian n 526 %  x2  less than 19,999  14.5  13.5  17.1  6.99   20,000 to 49,999  28.1  28.4  27.2   50,000 to 99,999  33.0  34.3  29.7   100,000 or more  24.4  23.8  26.0  annual household income  * p .05, ** p .01  table 4: logistic regression model predicting christian religious affiliation  variable  or  se  95% ci  incompatible  1.36**  0.12  1.07 – 1.72  like people everywhere  -  -  -  critical or important threat  2.28*  0.14  1.72 – 3.02  not a threat  -  -  -  u.s. has unique character that makes her greater than other countries  2.41**  0.12  1.91 – 3.05  all countries are unique, u.s. is no greater  -  -  -  very important  -  -  -  somewhat important  1.12  0.14  0.86 – 1.45  not important  1.02  0.24  0.63 – 1.61  very important  -  -  -  somewhat important  1.24  0.14  0.93 – 1.64  not important  0.95  0.21  0.63 – 1.45  gender (female)  1.60**  0.11  1.28 – 2.01  racial identity (white)  1.59**  0.12  1.25 – 2.03  othering attitudes towards muslims  attitudes towards immigrants and refugees  essentialist attitudes  humanitarianism combating world hunger  promoting and defending human rights in other countries  
  social work & christianity  68  variable  or  se  95% ci  south  -  -  -  northeast  0.81  0.17  0.58 – 1.12  midwest  1.00  0.16  0.74 – 1.37  west  0.47**  0.15  0.36 – 0.63  less than high school  0.79  0.19  0.54 – 1.14  high school/ged/equivalent  1.06  0.14  0.81 – 1.38  more than high school  -  -  -  less than 19,999  0.73  0.18  0.51 – 1.05   20,000 to 49,999  0.90  0.15  0.67 – 1.20   50,000 to 99,999  -  -  -   100,000 or more  0.83  0.15  0.62 – 1.12  region  education  annual household income  * p .05, ** p .01  the main effects of the multivariate logistic regression model (see table 4) revealed that othering attitudes were a significant predictor of religious affiliation, while humanitarianism attitudes were not a significant predictor of religious affiliation. participants were more than twice as likely to identify as christian if they reported that the u.s. had a unique character that made her the greatest country in the world (or 2.41, p .00) and that immigrants and refugees were a threat to the u.s. (or 2.28, p .00). participants who reported that muslims were incompatible with western ways were nearly 1.4 times more likely to identify as christian (or 1.36, p .01). humanitarian variables of combating world hunger (p .68) and defending human rights in other countries (p .17) were not significant predictors of religious affiliation. geographical region (p .00), white identity (or 1.59, p .00), and being female (or 1.60, p .00) were significant predictors of christian religious affiliation, while income (p .33) and education (p .31) were not significant predictors of religious affiliation. the model correctly predicted 74.5% of cases and was much better at predicting those who identified as christian (93.9%) than those who identified as non-christian (23.7%). the model explained 16.4% of the variance (nagelkerke r2 0.164). discussion the logistic regression model suggested that individuals who reported more othering attitudes and beliefs were significantly more likely to identify  
  humanitarianism, othering, and religious affiliation  as christian than non-christian, while there was no significant effect between christians and non-christians in regard to concern for international humanitarian needs. while the christian church in the u.s. has, at times and in some congregations, been a place of social advocacy and social services for the disenfranchised, this study suggests that othering attitudes towards muslims, immigrants, and refugees are generally more likely to occur in the christian community than the non-christian community. that humanitarianism was not a significant predictor of religious affiliation is not surprising since the imperative to help each other is a command in all religions and is considered a universal characteristic of humans (ferris, 2011). this study suggests that attitudes of othering may be an impediment to enlisting the helping natures of many congregants within faith-based organizations. for example, if christian congregations believe that immigrants are a threat to the u.s., then how likely are christian congregations to extend help to immigrant groups struggling with poor social support, poverty, and other human needs? yet, faith-based organizations and faithbased communities such as catholic charities and lutheran services help to resettle approximately seventy per cent of all refugees to the u.s. today (eby, iverson, smyers, & kekic, 2011). while the majority of christian individuals may hold discriminatory views, a minority of christian groups is actively engaged in supporting the refugee population. one explanation for why christians are more associated with othering attitudes than non-christians could be that christians are demographically different than immigrants, refugees, and muslims in several respects, and these demographic differences are what separate people into categories when met with “us versus them” anxiety. for example, christians are less likely to be immigrants themselves or have parents who were immigrants—approximately 73% of christians have an immigration status of third generation or higher, compared with those from other religions (buddhism, hinduism, judaism, and islam) who have a collective average of only 35% being third generation or higher (pew research center, 2014). thus, christians may have less first-hand experience or sympathy for the causes of immigration and the financial and social struggles of being an immigrant in the u.s., while additionally placing themselves in a different category of people than the “immigrant.” another possible explanation for the higher association between othering attitudes and christians could be that the respondents who make up “non-christians” may themselves include more immigrants, refugees, muslims, or other non-majority groups, making this group more aware of and sensitized to subtle discrimination. attitudes of othering may be easily justified or dismissed by majority groups, and not recognized as an act of discrimination (verkuyten, 2013). qualitative analyses of interviews in britian (phillips, 2006), sweden (borell & gerdner, 2011) and austria,  69  
  70  social work & christianity  bulgaria and italy (kutay, 2015) have revealed a societal discourse that arab and muslim immigrants are unable to acculturate to the west, and uninterested in acculturating. according to the othering process, this is an example of attributing intrinsic and fixed characteristics to a person based on one aspect of his or her identity. however, borell & gerdner’s study (2011) in sweden found that while the discourse among social service agencies was that muslim congregations were uninterested in acculturating and wanted to provide their own social services, interviews with muslim congregations revealed that they were indeed interested in partnering with swedish services, or were actually already making referrals or partnering with swedish social service agencies. in other words, the muslim congregations were perceived by the community as self-segregating due to the swedish community’s attitudes of othering, when in actuality, the muslim congregations felt they were pushed away or discouraged from engaging in cross-cultural partnerships. by engaging in othering attitudes, christians in the u.s. may be perpetuating—rather than advocating against—the discrimination towards immigrants, refugees, and muslims that exists in u.s. society. research has documented that since 9/11, muslims and refugees in the west have experienced verbal threats, physical attacks, social exclusion, economic marginalization, and restricted civil rights (amawi, 2012 baker, stockton, howell, jamal, lin, shryock, & tessler, 2006; faragallah, schumm, & webb, 1997). discrimination has negative effects on nearly every facet of human life, from poor mental health and unemployment to political instability and terrorism (caprioli & trumbore, 2003; goodwin, chenhall, & bilbrey, 2015; kessler, mickelson, & williams, 1999; piazza, 2011). recommendations for social work, this study offers insight into the increased need to address the subtle discrimination that may be negatively affecting christian community engagement with disenfranchised immigrant, refugee, and muslim groups. for social work educators, this research provides a springboard for discussion in the classroom regarding the integration of the christian faith and the social work profession, such as how students’ own religious affiliations may interrelate with their social and political worldviews. social work educators should teach and discuss theory in the classroom in order to increase students’ awareness of the othering process and subtle discrimination. theories such as categorization (verkuyten, 2013), out-group psychology (verkuyten, 2013), new realism (prins, 2002), and essentialism (borell & gerdner, 2011) can help students to critically consider the justification of discrimination in political and societal discourses. students from privileged  
  humanitarianism, othering, and religious affiliation  or majority group backgrounds may be less sensitized to the more subtle forms of discrimination, and social work educators can nurture awareness in these students through discussions about the othering process and the ways that it lessens humanity. for example, discussions could explore beliefs about whether one culture is able to adapt or modify to another culture, or whether identity can be multifaceted (i.e. arab and western). social work educators and practitioners also need to examine their own assumptions and essentialist beliefs about immigrants, refugees, and muslims. for example, social workers are more liberal than the general u.s. population (hodge, 2002; hodge, 2003), but have an ethical duty to provide services that are sensitive to clients’ cultures and to differences among people and cultural groups (reamer, 2003). being more liberal, social workers may need to examine their own assumptions about the compatibility of muslims living in the west. hodge, baughman, & cummings (2006) examined social work textbooks for religious sensitivity and found that muslims were nearly invisible in textbooks, and when they were mentioned in texts, they were depicted in a negative light. outside of the classroom, social work practitioners can increase crosscultural understanding through organizing cross-cultural interchanges within their places of worship or within faith-based community agencies. for example, a christian church could partner with a muslim mosque, or a predominantly american congregation could partner with a christian church that has a predominately immigrant congregation and host a cross-cultural interchange. cross-cultural interchanges can take a variety of forms such as a monthly meal together or a joint community service project. finally, while christianity in the u.s. has a history of being at the forefront of the modern humanitarian movement, it also has a more egregious history of validating colonialism when working cross-culturally (ferris, 2011). being mindful of the othering process can help buffer against unintentional colonialism for christian social workers and faithbased agencies crossing cultural boundaries, both in the u.s. and abroad. conclusion the current sociopolitical atmosphere and collective anxieties have an effect on all people. this paper aimed to uncover some of the perhaps unconscious bias within the faith community that may be preventing a greater response of helping behaviors with the immigrant, muslim, and refugee populations. as the world becomes increasingly interconnected, social workers and christians will have to continually examine their attitudes towards people who may look, speak, and act very different from them if they are to fulfill their professional and spiritual missions. v  71  
  72  social work & christianity  references allen, c. (2004). justifying islamophobia: a post-9/11 consideration of the european union and british contexts. american journal of islamic social sciences, 21(3), 1–25. amawi, n. (2012). the experience of iraqi refugees in jordan and the us; a comparative study (doctoral dissertation, massachusetts school of professional psychology). baker, w., stockton, r., howell, s., jamal, a., lin, a. c., shryock, a., & tessler, m. (2006). detroit arab american study (daas), 2003. ann arbor, mi: interuniversity consortium for political and social research. beydoun, k. a. (2016). electing islamophobia. available at ssrn. borell, k., & gerdner, a. (2011). hidden voluntary social work: a nationally representative survey of muslim congregations in sweden. british journal of social work, 41(5), 968–979. caprioli, m., & trumbore, p. f. (2003). ethnic discrimination and interstate violence: testing the international impact of domestic behavior. journal of peace research, 40(1), 5–23. cooper, a. (host.) (2016, march 9). anderson cooper 360 degrees. exclusive interview with donald trump. transcript retrieved from http://transcripts.cnn. com/transcripts/1603/09/acd.01.html eby, j., iverson, e., smyers, j., & kekic, e. (2011). the faith community’s role in refugee resettlement in the united states. journal of refugee studies, 24(3): 586–605. elkins, s. m. (2013). slavery: a problem in american institutional and intellectual life. university of chicago press. ellwood, w. (2006). the no-nonsense guide to globalization. cornwall, uk: new internationalist publications ltd. faragallah, m. h., schumm, w. r., & webb, f. j. (1997). acculturation of arabamerican immigrants: an exploratory study. journal of comparative family studies, 182–203. ferris, e. (2011). faith and humanitarianism: it’s complicated. journal of refugee studies, 24(3), 606–625. ghorashi, h. (2010). from absolute invisibility to extreme visibility: emancipation trajectory of migrant women in the netherlands. palgrave macmillan. doi:10.1057/ fr.2009.38 goodwin, d., chenhall, e., & bilbrey jr, j. (2015). statistical analysis of religious discrimination complaints through the equal employment opportunity commission. journal of legal issues and cases in business, 3, 1. hanson-easey, s., augoustinos, m., & moloney, g. (2014). ‘they’re all tribals’: essentialism, context and the discursive representation of sudanese refugees. discourse & society, 25(3), 362–382. doi:10.1177/0957926513519536 healy, p. & haberman, m. (2015, december 5). 95,000 words, many of them ominous, from donald trump’s tongue. the new york times, retrieved from http://www.nytimes.com/2015/12/ 06/us/politics/95000-words-many-of-themominous-from-donald-trumps-tongue.html hodge, d. (2002). equally devout, but do they speak the same language? comparing the religious beliefs and practices of social workers and the general public. families in society: the journal of contemporary social services, 83(5), 573–584.  
  humanitarianism, othering, and religious affiliation hodge, d. (2003). differences in worldviews between social workers and people of faith. families in society: the journal of contemporary social services, 84(2), 285–295. hodge, d. r., baughman, l. m., & cummings, j. a. (2006). moving toward spiritual competency: deconstructing religious stereotypes and spiritual prejudices in social work literature. journal of social service research, 32(4), 211–231. holzgrefe, j. l., & keohane, r. o. (2003). humanitarian intervention: ethical, legal and political dilemmas. cambridge university press. kopan, t. (2016, august 31) what donald trump has said about mexicans and vice versa. cnn politics. retrieved from http://cnn.com/2016/08/31/politics/ donald-trump-mexico-statements/ kessler, r. c., mickelson, k. d., & williams, d. r. (1999). the prevalence, distribution, and mental health correlates of perceived discrimination in the united states. journal of health and social behavior, 208–230. kutay, a. (2015). dominant pluralism and discursive strategies of contemporary racism against muslim minorities in europe. journal of muslim minority affairs, 35(1), 1–16. lee, e. k. o., & barrett, c. (2007). integrating spirituality, faith, and social justice in social work practice and education: a pilot study. journal of religion & spirituality in social work: social thought, 26(2), 1–21. miles, a. (1999). when faith is used to justify abuse. ajn the american journal of nursing, 99(5), 32–35. nurullah, a. s. (2010). portrayal of muslims in the media: “24” and the ‘othering’ process. international journal of human sciences, 7(1), 1020–1046. pehrson, s., brown, r., & zagefka, h. (2009). when does national identification lead to the rejection of immigrants? cross-sectional and longitudinal evidence for the role of essentialist in-group definitions. british journal of social psychology, 48(1), 61–76. doi:10.1348/014466608x288827 pew research center. (2014). immigrant status by religious group [data file]. retrieved from http://www.pewforum.org/religious-landscape-study/immigrantstatus/#religious-tradition-trend. retrieved 7/2016 phillips, d. (2006). parallel lives? challenging discourses of british muslim selfsegregation. environment and planning d-society & space, 24(1), 25–40. piazza, j. a. (2011). poverty, minority economic discrimination, and domestic terrorism. journal of peace research, 48(3), 339–353. prins, baukje. (2002). the nerve to break taboos: new realism in the dutch discourse on multiculturalism. journal of international migration and integration/revue de l’integration et de la migration internationale 3(3–4): 363–379. reamer, f. g. (2003). social work, evangelical christians, and values. social work, 48(3), 428–431. said, e. w. (1979). orientalism. new york, ny: vintage. simon, b. l. (1994). the empowerment tradition in american social work: a history. columbia university press. tirman, j. (2003, december). the new humanitarianism: how military intervention became the norm. boston review. retrieved from https://www.globalpolicy.org/ component/content/article/154/26042.html#author  73  
  74  social work & christianity united nations conference on trade and development. (2012). development and globalization: facts and figures 2012. retrieved from http://unctad.org/en/ publicationslibrary/webgdsdsi2012d2_en.pdf. united nations. (2015). international migration report 2015. retrieved from http:// www.un.org/en/development/desa/population/migration/publications/migrationreport/docs/migrationreport2015_highlights.pdf. verkuyten, m. (2013). justifying discrimination against muslim immigrants: out‐ group ideology and the five‐step social identity model. british journal of social psychology, 52(2), 345–360.  lindsey r. disney, lcsw, ph.d. student, university of georgia school of social work, 279 williams street, athens, ga 30605. phone: (404) 987–5412. email: lindsey.disney25@uga.edu. keywords: religious affiliation, humanitarianism, othering, discrimination  editor’s note: we are pleased to publish the above article, which won the inaugural social work & christianity graduate student paper award competition. the purpose of this award is to encourage and recognize excellence in scholarly work by a graduate student on issues related to the integration of christian faith and professional social work practice or other professional concerns that have relevance to christianity. a panel of anonymous reviewers judged the submitted papers on several criteria—relevance for integration, significance, originality and innovation, research methodology, implications for social work practice and/or education, and writing style—and offered editorial feedback. subsequently, the winning author used this feedback to revise her manuscript for publication. we hope this award with encourage other graduate students to conduct research related to their interests and the mission of social work & christianity, make the extra effort to write it up, and take the risk of submitting it for evaluation. this is how scholars and their scholarship develop.  
  copyright of social work & christianity is the property of north american association of christians in social work and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. however, users may print, download, or email articles for individual use.  
       


primary_title_index: plos one
secondary_title_index: plos one
abstract_index:
       president trump and his administration have been regarded by news outlets and scholars as one of the most hostile administrations towards scientists and their work. however, no study to-date has empirically measured how federal scientists perceive the trump administration with respect to their scientific work. in 2018, we distributed a survey to over 63,000 federal scientists from 16 federal agencies to assess their perception of scientific integrity. here we discuss the results of this survey for a subset of these agencies: department of interior (doi) agencies (the us fish and wildlife service (fws), the us geological survey, and the national park service); the centers for disease control and prevention (cdc); the us environmental protection agency (epa); the food and drug administration (fda); and the national oceanic and atmospheric administration (noaa). we focus our analysis to 10 key questions fitting within three core categories that relate to perceptions of integrity in science. additionally, we analyzed responses across agencies and compare responses in the 2018 survey to prior year surveys of federal scientists with similar survey questions. our results indicate that federal scientists perceive losses of scientific integrity under the trump administration. perceived loss of integrity in science was greater at the doi and epa where federal scientists ranked incompetent and untrustworthy leadership as top barriers to science-based decision-making, but this was not the case at the cdc, fda, and noaa where scientists positively associated leadership with scientific integrity. we also find that reports of political interference in scientific work and adverse work environments were higher at epa and fws in 2018 than in prior years. we did not find similar results at the cdc and fda. these results suggest that leadership, positive work environments, and clear and comprehensive scientific integrity policies and infrastructure within agencies play important roles in how federal scientists perceive their agency's scientific integrity.     

affiliation_index:
       center for survey statistics and methodology, iowa state university, ames, ia, united states of america.     

keywords_index:
       federal government     humans     perception     policy     science     surveys and questionnaires     trust     united states     

title_index:
       perceived losses of scientific integrity under the trump administration: a survey of federal scientists.     

full_text_index:
       plos one research article  perceived losses of scientific integrity under the trump administration: a survey of federal scientists gretchen t. goldman id1*, jacob m. carter1*, yun wang2, janice m. larson2 1 center for science and democracy, union of concerned scientists, washington, dc, united states of america, 2 center for survey statistics and methodology, iowa state university, ames, ia, united states of america  a1111111111 a1111111111 a1111111111 a1111111111 a1111111111  open access citation: goldman gt, carter jm, wang y, larson jm (2020) perceived losses of scientific integrity under the trump administration: a survey of federal scientists. plos one 15(4): e0231929. https://doi.org/10.1371/journal.pone.0231929 editor: mojtaba vaismoradi, nord university, norway received: november 15, 2019 accepted: april 3, 2020 published: april 23, 2020 peer review history: plos recognizes the benefits of transparency in the peer review process; therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. the editorial history of this article is available here: https://doi.org/10.1371/journal.pone.0231929 copyright: 2020 goldman et al. this is an open access article distributed under the terms of the creative commons attribution license, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.  * ggoldman@ucsusa.org (gtg); jcarter@ucsusa.org (jmc)  abstract president trump and his administration have been regarded by news outlets and scholars as one of the most hostile administrations towards scientists and their work. however, no study to-date has empirically measured how federal scientists perceive the trump administration with respect to their scientific work. in 2018, we distributed a survey to over 63,000 federal scientists from 16 federal agencies to assess their perception of scientific integrity. here we discuss the results of this survey for a subset of these agencies: department of interior (doi) agencies (the us fish and wildlife service (fws), the us geological survey, and the national park service); the centers for disease control and prevention (cdc); the us environmental protection agency (epa); the food and drug administration (fda); and the national oceanic and atmospheric administration (noaa). we focus our analysis to 10 key questions fitting within three core categories that relate to perceptions of integrity in science. additionally, we analyzed responses across agencies and compare responses in the 2018 survey to prior year surveys of federal scientists with similar survey questions. our results indicate that federal scientists perceive losses of scientific integrity under the trump administration. perceived loss of integrity in science was greater at the doi and epa where federal scientists ranked incompetent and untrustworthy leadership as top barriers to science-based decision-making, but this was not the case at the cdc, fda, and noaa where scientists positively associated leadership with scientific integrity. we also find that reports of political interference in scientific work and adverse work environments were higher at epa and fws in 2018 than in prior years. we did not find similar results at the cdc and fda. these results suggest that leadership, positive work environments, and clear and comprehensive scientific integrity policies and infrastructure within agencies play important roles in how federal scientists perceive their agency’s scientific integrity.  data availability statement: all relevant data are within the paper and its supporting information files. funding: the authors received no specific funding for this work.  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  1 / 26  
  plos one  competing interests: the authors have declared that no competing interests exist.  survey of federal scientists and perceived loss of scientific integrity under trump leadership  introduction the use and misuse of science in policy development has spurred a long-standing discussion about the proper role of science and scientists in government decision-making. one important facet of this issue is the role of scientists who feed their knowledge and expertise into policy decision-making processes [1]. scientists working for the federal government conduct, synthesize, and communicate scientific information that informs, guides, and directs policy action on issues with implications for public health and safety (e.g., the effects of climate change, the safety and efficacy of new drugs). this process has allowed government decisions to be better informed by scientific evidence producing great societal benefits [2–8]. the process by which science informs policy decisions is a complex “value-laden social process” and is affected by both scientists and non-scientist actors, as well as the environment in which it occurs [9, 10]. thus, it is critical that postures (e.g., attitudes, stances, or perspectives) of all actors working at the science-policy interface are considered when they are contributing to policy decisions [11]. such transparency can shield science-policy decisions from being overburdened by personal preferences that are presented as scientific fact and allow the public and other officials to observe and react to decisionmakers’ actions [11–13]. in addition to transparency, scientific input from experts such as scientific advisory committees, academic scholars, or interdepartmental consultations, along with input from the public, can lead to robust and informed policy decisions. however, such expert-informed and participatory structures prove difficult in systems of government that have high concentrations of power in the executive branch, such as the us [14, 15]. in such a system, federal scientists and their work can be influenced by the values, culture, beliefs, and interests of those who hold more power in the u.s. bureaucracy. these elite actors may inappropriately undermine, alter, or otherwise interfere with the scientific process or scientific conclusions for political, financial, or ideological reasons. this is political interference in science, which often results in decisions more heavily influenced by political ideologies rather than scientific knowledge [16]. political decisions that are not informed by science are not as likely to benefit society and, in some cases, may cause undue harm to the public. political interference has occurred in both republican and democratic administrations dating back as far as the eisenhower administration [17]. however, political interference in science policy processes was first widely recognized under the george w. bush administration [18]. the union of concerned scientists, by its measure, counted nearly 100 incidents in which the administration sidelined scientists or their work when making science-based policy decisions [19]. the george w. bush administration’s political interference in the science-policy process prompted thousands of scientists to pen a letter entitled “scientist statement on restoring scientific integrity to federal policymaking” to the administration [20]. this letter gave rise to a larger movement within the scientific community to push for integrity in processes at agencies where science informs decisions—referred to as "scientific integrity.” ultimately, this push led to the development of scientific integrity policies across federal science agencies [21–23]. these policies are maintained and implemented by 28 federal entities that conduct and/or use science to inform decisions, and the provisions therein vary widely across agencies. the policies govern actions by staff who work at the science-policy interface and the provisions are designed to ensure scientific integrity is maintained and to provide a mechanism to report a violation of scientific integrity if it occurs. these provisions may include protections for scientists to communicate their work to the public without political interference, rights to submit work to an outside peer-reviewed journal, or the right of last review on agency statements that refer to a scientist’s work.  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  2 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  while the us government science-policy apparatus is set-up to receive input on sciencepolicy decisions from multiple stakeholders, ultimately science-policy decisions often lie in the hands of elite bureaucratic officials [24]. for example, the administrator of the environmental protection agency (epa) has a responsibility to consider scientific information provided from scientific advisory committees, stakeholder groups, the epa’s own scientific staff, and public opinion, but final decisions on the laws epa is charged with carrying out (e.g. the clean air act, the safe drinking water act, the toxic substances control act) often lie with the administrator. as mentioned previously, science policy is a value-laden social process and certainly the values of elite bureaucratic officials play a role in their science-policy decisions. if their decisions rely heavily on their own values and judgments and not on scientific information, this may lead to scientific work being deprioritized. the stifling of scientific work because it is viewed as politically contentious may demoralize scientific staff, make them feel ineffective, or result in self-censorship of their own work. indeed, leadership, job effectiveness, and morale are often strongly correlated and can affect worker retention and production [25–27]. research has indicated that an employee’s leader has the most influence on their job satisfaction [27]. while research focusing on scientists in the workplace is scarce, existing research shows that they can be de-motivated by lack of support from their leaders [26, 27]. while few studies have assessed federal scientific integrity specifically, the us office of personnel management conducts annual studies of federal employees, known as the federal employee viewpoint survey [28]. these surveys are inclusive of scientists and other technical experts in the federal workforce and assess employee satisfaction, engagement level, and allow for comparisons over time and between agencies. they have been used to assess the engagement of specific groups (e.g., women, millennials in the federal service) as well as specific challenges or events (e.g., impact of the 2019 federal government shutdown, education within the federal workforce) [29]. some work has found correlations between the level of engagement found on opm surveys and agency performance [30]. additionally, workplace satisfaction surveys in technical fields have provided insights into scientists’ job experiences. on its 2018 biannual survey of its readers with graduate scientific training, nature found that 68% of the government scientists in its survey of 4,334 eligible respondents globally reported being satisfied with their job, and 21% of all respondents reported personally experiencing harassment or discrimination at their current job [31]. the american association for the advancement of science has conducted similar surveys of its us-based technical readers and found an average job satisfaction rate of 3.7 on a five-point scale [32]. both of these surveys were inclusive of us federal government scientists; however, results were not stratified by federal agency and scientific integrity at work was not a focus of the surveys. other surveys have focused on the job satisfaction, job stress, and career trajectories of subpopulations and specific scientific disciplines, but respondent numbers and scope have been limited [33–34]. some work has focused on job satisfaction and perceived ethics and fairness in the federal government context. choi (2011) used the merit system protection board survey to assess correlations between perceived organizational justice and federal employees’ work-related attitudes including job satisfaction, trust toward their supervisors and management, and intention to leave their agency [35]. the study found that higher levels of three types of organizational justice (distributive, procedural, and interpersonal) are positively related to job satisfaction and trust in supervisor and management, whereas they are negatively associated with turnover intentions of employees. other work has found correlations between job satisfaction and perceptions of reciprocity between employees and employers [36]. these studies are suggestive of relationships between employees’ workplace experiences and perceptions of integrity of organizational management.  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  3 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  limited work has focused on organizational justice, ethics, and integrity in the federal workforce. the best predictors for maintaining scientific integrity in the workplace are not definitively known. however, based on prior relevant studies [17, 36–37], surveys [38–44], and federal scientific integrity policies [45], maintaining scientific integrity in the workplace depends on multiple factors that fall within the following three categories: 1) the existence and implementation of a clear comprehensive scientific integrity policy at the agency or department level; 2) the perception of competent and trustworthy agency leadership; and 3) an environment where employees feel effective and valued. a better understanding of what factors are most critical to maintain scientific integrity could provide government leaders and staff with information identifying the means by which political interference occurs within their agency. such information may help to better inform the agency’s scientific integrity policy, which could lead to more effective scientific work and policy decisions informed by science. one way to assess the degree and dimensions of scientific integrity at federal agencies is through surveys of government scientists. the nonprofit union of concerned scientists, working with iowa state university’s center for survey statistics & methodology, has conducted nine such surveys every 2–3 years since 2005 to assess the status of policies and practices at agencies relating to scientific integrity and to identify problem areas that might not otherwise be made public [38]. in the most recent study, conducted in 2018, we surveyed federal agency scientists’ perceptions of scientific integrity under the donald j. trump administration, which is largely viewed by the scientific community and scholars as an administration with an unprecedented frequency of scientific integrity violations relative to prior administrations [17, 37, 46]. little research has focused on perceptions of government scientists on scientific integrity over time and across agencies, particularly under the trump administration. therefore, surveying under the trump administration offers a rare opportunity to gauge scientists’ perceptions of how scientific integrity is best maintained at their agency. moreover, our study provides insights into how these perceptions compare across multiple agencies and administrations using prior surveys with similar items addressing scientist’s perceptions of scientific integrity. using survey data collected under multiple administrations, we addressed the following questions: 1) what scientific integrity factors (e.g., adherence to scientific integrity policy, perception of effective leadership, censorship) are perceived to be most at threat in 2018 (i.e., under the trump administration)? 2) do perceptions of scientific integrity factors vary across federal agencies? 3) have perceptions of scientific integrity changed over time?  materials and methods the survey instrument the 2018 survey instrument consisted of 52 core questions, most of which were used on previous surveys, with a few revisions made to accommodate the wide range of agencies being surveyed (see supplementary material). initial screening questions verified the respondent’s primary agency and sub-agency (office) affiliation as well as the percentage of job duties involving science. if respondents indicated their job duties did not involve science, they were not permitted to complete the remaining questions. in addition to a core set of items answered by all agencies, the survey also included agency-specific questions that were answered based on the primary agency selected in the initial screening. with the exception of the screening items, survey questions were considered response optional and, in most cases, provided a “prefer not to disclose” answer option. in this analysis, we identified and analyzed a set of ten questions that fit within three categories likely to be predictive of the state of scientific integrity within federal agencies (see table 1  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  4 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  table 1. factors contributing to scientific integrity perceptions and relevant survey questions. factor contributing to scientific integrity perceptions  relevant survey questions  competent and trustworthy leadership  1. in your opinion, what are the greatest barriers to science-based decisions at your agency? (choose up to 3.) absence of leadership with needed scientific expertise influence of political appointees in your agency or department 2. the level of consideration of political interests hinders the ability of my agency to make science-based decisions. (agree/disagree scale) 3. the level of consideration of business interests hinders the ability of my agency to make science-based decisions. (agree/disagree scale)  feeling effective and valued  4. over the past year, i have noticed that resource allocations (e.g., funding, staff time) have been distributed away from programs and offices whose work is viewed as politically contentious. (agree/disagree scale) 5. in the past year, i have been excluded from discussions or decisions related to my scientific work that i normally would expect to be a part of. (agree/disagree scale) 6. i have avoided working on climate change or using the phrase “climate change,” though i was not explicitly told to avoid them. (agree/disagree scale) 7. compared to one year ago, the effectiveness of your division or office has: (decreased/stayed the same/increased) 8. the presence of top agency decision makers who come from regulated industry inappropriately influences the decisions made by the agency. (agree/disagree scale) 9. currently, i can openly express any concerns about the mission-driven work of my agency without fear of retaliation. (agree/disagree scale)  clear, comprehensive scientific integrity policy   10. my agency adheres to its scientific integrity policy. (agree/disagree scale)  on question 1 survey respondents were given 14 options; however, for the purposes of assessing perceptions of  scientific integrity, only the two options listed below were included in this analysis. https://doi.org/10.1371/journal.pone.0231929.t001  for questions and categories). other questions from this survey instrument and frequency data are provided in supplementary materials. this table highlights the ten survey items (out of 52 core items total) discussed in this paper. these ten items were chosen as they are representative of three categorical factors evidenced to affect perceptions of scientific integrity in the workplace. the items are numbered by the order they are discussed in this paper, and do not correspond to their number or the order that they were addressed in the original administered survey instrument.  the survey sample sixteen federal agencies within six departments were identified for the 2018 survey sample based on their science-based missions, commitment to scientific integrity, or history of past scientific integrity challenges. these agencies included the us environmental protection agency (epa); the national oceanic and atmospheric administration (noaa) and the us census bureau within the department of commerce; the centers for disease control and prevention (cdc) and the food and drug administration (fda) within the department of health and human services; the energy efficiency and renewable energy office (eere) at the department of energy (doe); the bureau of ocean energy management (boem), bureau of safety and environmental enforcement (bsee), the national park service (nps), us fish and wildlife service (fws), and the us geological survey (usgs) in the department of the interior (doi); the agricultural research services (ars), the economic research service (ers),  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  5 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  the national institute of food and agriculture (nifa) and the national agricultural statistics service (nass) in the us department of agriculture (usda); and the national highway traffic safety administration (nhtsa) at the department of transportation. staff email addresses for these agencies were obtained through publicly available online staff directories and freedom of information act (foia) requests. online directories for the fda, cdc, usgs, nifa, and ars included both job titles as well as contact information. for government agencies with an incomplete online employee directory or no directory at all, foia requests were filed. the offices where a foia request resulted in full staff lists were boem, bsee, nps, fws and nhtsa. for agencies with job title information available, employees were identified as scientific or nonscientific based on job title and office within their department. for the purposes of this survey, scientists were considered anyone whose job involved a significant level of science, including but not limited to research, analysis, modeling, inspection and oversight, and science policy. full-time federal employees, contractors, and associates were included in the survey but fellows, students, and interns were excluded. when available, the specific office the employee worked in was used to exclude large amounts of people that were unlikely to perform the above scientific functions. common non-scientific offices such as administration, finance, information technology, and facility maintenance were consistently excluded from lists. the epa, doe, and census bureau were unresponsive to foia requests within six months. for these agencies, lists were obtained either from inside or external sources. a website called federalpay.org was used to collect information on federal employees earning salaries exceeding 100k, allowing some contact information for nass employees to be obtained. in early 2017, the epa and doe removed their employee directory from their website. however, the data refuge project preserved the full epa staff list in a publicly accessible database scraped in february 2017, though without job title information. ultimately no job title screening information was available for the epa, doe, and the census bureau; thus, email addresses for all employees in relevant divisions were included in the sample. as a result, it is acknowledged that the sample likely included many ineligible individuals from these agencies. the final sample for the 2018 survey consisted of 63,248 email addresses. survey contacts were made exclusively by email. as an additional quality control mechanism, the survey instrument asked respondents to indicate what percentage of their time was spent on science. respondents answering zero percent were routed to the final survey question and excluded from aggregate survey statistics.  survey administration prior to launching the survey, researchers contacted each of the sampled federal agencies, notifying them that the survey was being conducted. cssm staff obtained survey approval from the iowa state university institutional review board (irb #18–017). the survey was prepared and made available to participants in one of three response modes: online web survey, on paper using a downloadable pdf, or by telephone. unique access codes were assigned to each person within the sample to ensure the integrity of the survey and its results. the online survey was programmed using qualtrics software. respondents could access the survey login page either by clicking on the embedded link in their email invitation or by manually entering the url. respondents then entered their assigned access code to access their personal survey. as an additional layer of security, cssm used qualtrics’ “anonymizing responses” feature, which removes all identifying respondent information (including email address, ip address, location data, and password/external data reference codes) upon  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  6 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  completion, resulting in data that cannot be linked to a particular respondent or email address. both the survey and the data were stored on secure iowa state university servers. a pdf version of the survey was created by cssm, which mirrored the online survey as much as possible. agency-specific information embedded in the online survey was changed to be more generic for the pdf version, and routing or skip instructions were added. the survey email invitations and reminders provided links to both the cssm and ucs federal scientists survey webpages, which housed the pdf file for downloading. recipients who wished to complete the pdf version were invited to print a copy and complete it on paper, returning it either by mail or by emailing a scanned file. for survey integrity purposes and to avoid duplicate responses, pdf respondents were asked to write their assigned access code on their completed survey. once received, cssm staff verified that the access code hadn’t previously been used and then recorded the data. once the pdf responses were entered, paper copies were securely destroyed and scanned forms were deleted. confidential phone interviews were also made available to survey recipients. a toll-free phone number was provided in the email invitations/reminders and on the cssm and ucs survey webpages. respondents were invited to call the number and schedule a time for the interview. interviews were conducted by cssm’s professional survey staff. for verification and survey integrity purposes, phone respondents were asked to provide their assigned access code before completing the interview. the email invitation and four email reminders each included an embedded survey link and the respondent’s access code, as well as instructions for each of the three survey modes (see supplementary material). in addition, each email described the survey’s purpose, the organizations involved, participant selection criteria, and anticipated time requirement of 15–20 minutes. the emails stated that sampled agencies had been notified about the survey and encouraged participants to complete the survey on their own personal time and equipment. an “opt-out” link was included for those who preferred not to participate or be contacted again. cssm and ucs website links were provided for those who desired more information. the cssm and ucs websites both contained additional resources, including a frequently asked questions document, a link to the survey pdf, instructions for completing the survey by phone, and information about past research conducted by the organizations. in the field of survey research, it is generally acknowledged that offering multiple response methods simultaneously does not necessarily increase response rates and can even lead to a decline in some situations. in this situation, however, given the subject matter and the anticipated concerns regarding confidentiality and anonymity, it was decided to allow respondents to choose the response option that best fit their comfort level. multiple contact methods were not possible since only email addresses were available. due to response collection limitations and other facters raised in the discussion section, results should not be considered representative, but rather a window into perspectives of a subset of scientists at each agency.  data collection the data collection period was february 12, 2018 –march 26, 2018. email invitations were sent to a quarter of the sample per day on four successive days, with reminder emails sent approximately 6 to 9 days apart. throughout data collection, respondents could enter their online survey using their access code as often as they wished until the survey was completed. completed surveys were closed and immediately stripped of all personal identifiers, including email address, ip address and access code. during data collection cssm and ucs staff received and responded to questions, comments, or concerns about the survey. issues included  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  7 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  verification of survey legitimacy, questions about procedures, ineligibility (e.g., no scientific work), and technical problems. survey response varied by agency. of the 63,248 email invitations sent, 1843 were undeliverable, 122 were classified as ineligible, and 1951 refused or selected the “opt-out” option. data was obtained from 4211 respondents, including 3783 complete and 428 partially complete surveys. of the 4211 surveys in the data set, 4178 were completed online, 23 on paper, and 10 by phone. the overall response rate was 6.67%, based on 4211 surveys out of 63,126 potentially eligible contacts. response rates by agency ranged from 2.17% to 19.12% (see supplementary materials).  statistical analysis procedures statistical analyses were conducted across agencies for the 2018 survey and over time within agencies by referencing past survey data. all procedures were accomplished by using sas and r. the assessments across agencies were restricted to eight survey items for five agencies with the higher survey response rates: cdc, fda, epa, noaa, and doi (usgs, fws, and nps combined), hereafter referred to as the “five agencies analyzed.” one of these ten items of interest uses nominal responses (yes/no). the others have ordinal scale responses, excluding “prefer not to disclose,” “don’t know,” and “not available” options. the nominal response item (question 1 in table 1) has 16 sub-questions, or options, to be selected (yes) or not selected (no). for each sub-question, the association between agencies and the response was assessed by using the pearson chi-square test. an overall p-value for each sub-question was also calculated by simulating data according to the observed probability within each agency and its fixed sample size. a post-hoc test was conducted after significant overall test and tukey adjustment [47] was enhanced to control family-wise type i error (fwe). for questions with ordinal responses, pearson chi-square and mantel-haenszel chi-square tests were utilized to assess the overall association and linear trend. a cochran-armitage trend test was applied for post-hoc pair-wise comparisons and a tukey adjustment was used to control fwe. the analyses of responses over time were restricted to the agencies: cdc, epa, fws, fda, and noaa for several comparable question items. most agencies only had survey data available from 2015 and 2018, except for the fda which had data from earlier surveys for some items. pearson chi-square tests were used for evaluating overall association between time and responses, and mantel-haenszel chi-square tests were applied for ordinal responses. since the fda had available data from more than two surveys, post-hoc comparisons were conducted when needed and hochberg (step-up bonferroni) was applied for fwe control.  results and discussion we examined the degree to which over 3,700 federal scientists working across five federal agencies perceive scientific integrity to be upheld at their agencies by analyzing responses to survey questions that inform the three categories identified as contributing to maintaining scientific integrity in the workplace (see table 1 for questions and categories analyzed). we also compared responses over time using survey data collected under prior administrations for the cdc, fda, epa, noaa, and the fws. overall, we show that (a) federal scientists in some agencies perceived the influence of political officials and the absence of leadership with needed scientific expertise as major barriers to fulfilling their agencies’ science-based mission, (b) federal scientists vary in the degree to which they feel valued and effective in their work varied  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  8 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  across agencies, and (c) in comparison to prior survey data, federal scientists perceived that the effectiveness of their offices/divisions decreased under the trump administration. however, federal scientists from some agencies also perceived that their agency adheres to its scientific integrity policies under the trump administration.  competent and trustworthy leadership federal scientists from the five agencies analyzed largely perceived that their current agency leadership lacks technical expertise and is not perceived as trustworthy with respect to scientific work. across the five agencies, 2026 survey respondents reported “influence of political appointees” and “absence of leadership with needed scientific expertise” as major barriers to science-based decisions made at their agency (fig 1). this is an important finding given current research that shows leadership’s technical competence and employees’ trust in leadership can affect employee’s job satisfaction, engagement in work, and productivity [48, 49]. for example, in a study that surveyed 35,000 us and uk employees, technical competence of leaders was a strong predictor of job satisfaction [49]. in another study of 204 employees working in various organizations in south africa, worker perception of leaders as trustworthy and who maintain integrity in their work was positively associated with worker engagement and productivity [50]. if leadership cannot fully relate to the tasks their employees undertake and are not viewed as trustworthy to their employees and the work they perform (e.g., politically interfering in scientific work), we would expect a high number of employees to view the absence of these types of leaders as a barrier to their work. we also would expect lower job satisfaction  fig 1. barriers to science-based decisions at agencies. figure shows the results from survey respondents’ selections to the survey item, “in your opinion, what are the greatest barriers to science-based decisions in your agency?” survey respondents could choose up to three barriers out of 14 options: delay in leadership making a decision; absence of leadership with needed scientific expertise; uncertainty or disagreement with the science; influence of political appointees in your agency or department; influence of the white house; influence of congress; influence of other agencies; influence of industry stakeholders; influence of nongovernmental interests (such as advocacy groups); inefficient decision making process within the agency; potential discrepancy with existing rules and regulations; uncertainty of agency jurisdiction; complexity of the issue; limited staff capacity; other, and prefer not to disclose. the top five barriers selected were: absence of leadership with needed scientific expertise, influence of political appointees in your agency or department, influence of the white house, limited staff capacity, and delay in leadership making a decision. https://doi.org/10.1371/journal.pone.0231929.g001  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  9 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  and effectiveness as a byproduct if such leadership were perceived to be lacking (discussed in below section on perception of being valued). among the five agencies analyzed in this study, we also found qualitative evidence from open-ended response questions that some federal scientists perceive their agency leadership to be technically incompetent and untrustworthy to their work. respondents were asked, “how have actions taken or changes made by the current administration related to science (positive or negative) helped or harmed your agency’s mission?” in response, an anonymous employee from doi stated, “appointing a secretary with little understanding or appreciation of our agency’s function or structure has been detrimental to work productivity.” when prompted with the same question, an anonymous epa employee replied, “top levels of leadership don’t fundamentally believe in the value and mission of the agency and are actively planning to hit the agency, while lying about their intentions. top leadership appoints political hacks who utterly lack scientific credentials, and who don’t understand or value the scientific process.” while most respondents across the five agencies analyzed here chose “influence of political appointees” and “absence of leadership with needed scientific expertise” as a major barrier to science-based agency decisions, there was variation across agencies in what federal scientists considered the greatest barriers to their work (fig 1). a greater proportion of respondents from the epa than other agencies cited “influence of political appointees” as a top barrier to science-based decisions, with 60% of epa respondents choosing this option. at doi, this was also the most frequently chosen barrier, selected by 40% of respondents. at the fda and noaa this option was chosen in lesser proportions, with 10% of fda respondents and 17% of noaa respondents. both the chi-square p-value and simulated p-value were 0.0001 and all five agencies responded differently except cdc vs. noaa (21% vs. 17%, tukey adjusted pvalue 0.097). the option “absence of leadership with needed scientific expertise” was chosen at higher proportions at epa (33%, 147 respondents) and doi agencies (31%, 361 respondents) compared to the fda (18%, 65 respondents) and noaa (23%, 260 respondents). other potential barriers selected most often by respondents included: (1) “influence of the white house.” the average selection rate for all five agencies was 23%. the fda (11.9%) and noaa (19%) chose this option at a significantly lower rate than other agencies (31% for cdc, 27% for epa, and 26% for doi) with p-values 0.02. (2) “limited staff capacity.” the average rate among the five agencies for this option was 35%. the rate at which this option was chosen for the cdc, epa, fda, noaa and doi were 25%, 22%, 25%, 45%, and 40%, respectively. noaa had a significantly higher selection rate for this option compared to other agencies, except versus the fda (p-value 0.27). the epa had a significantly lower rate for this choice compared to the doi (p-value 0.04). (3) “delay in leadership making a decision.” the average selection rate among all five agencies for this chosen option was 22% with no significant differences in rate between agencies (p-value 0.25). other survey items also provide evidence that some federal scientists do not currently perceive agency leadership as trustworthy. many respondents did not feel that they could trust agency leadership to keep political interests from influencing their works’ role in decisions. for example, 83% of respondents (345 respondents) at the epa and 58% of respondents (621 respondents) at the doi agreed or strongly agreed that consideration of political interests hindered their agency’s ability to make science-based decisions, while 48% of respondents (255 respondents) at the cdc, 32% of respondents (101 respondents) at the fda, and 42% of respondents (462 respondents) at noaa agreed with this statement (fig 2). similar results were found with respect to levels of consideration of business interests with 75% of respondents (317 respondents) at epa, 46% of respondents (457 respondents) at doi agencies, 36% of respondents (114 respondents) at the fda, and 38% of respondents (416 respondents (38%) at noaa indicating that these interests serve as a barrier to science-based work.  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  10 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  fig 2. consideration of political interests in science-based decisions. figure shows the results from survey respondents’ responses to the survey item, “the level of consideration of political interests hinders the ability of my agency to make science-based decisions.” survey respondents could choose one of six options: strongly disagree, disagree, do not agree or disagree, agree, strongly agree, or prefer not to disclose. responses varied across agencies with more scientists agreeing that political interests hinder their agency’s science-based decisions at the epa and doi as compared to noaa and the fda. https://doi.org/10.1371/journal.pone.0231929.g002  the items discussed above provide a measure of scientific expert’s perception of their scientific work being politicized. the responses to these items indicate that survey respondents at the epa and doi do not perceive agency leadership to be trusting of scientific work when compared to political interests, which scientists perceive to be dominating decisions. additionally, survey respondents perceive leadership to lack the technical expertise to be involved in science-policy work. prior work has shown that lack of technically competent and trustworthy leadership can strongly affect employee’s job satisfaction and engagement and these two factors often affect employee work productivity. while a majority of respondents at the epa and doi reported that technically incompetent leadership as well as political interests were major barriers to their work, there was variation in responses across the five agencies analyzed. at the cdc, the fda, and noaa scientists appear to be more trusting of agency leadership to defend their work against political interests and to be more technically competent. the perception of leadership at the cdc, fda, and noaa having technical competence and behaving in a way that scientists could trust their leaders to defend their work was evidenced in open-ended responses. qualitative data was in-line with survey frequency data regarding federal scientists’ perceptions of leadership and how those perceptions varied across agencies. for example, when prompted with the question, “is there anything else you would like to share with us regarding scientific integrity at your agency?” an anonymous fda employee stated, “i believe that at fda we have been exceedingly fortunate with dr. scott gottlieb. he is an ethical scientist steeped in policy formulation. he has been tasked with continuing a tradition of upholding scientific principles and procedures and i believe he is doing an outstanding job despite the current congressional climate.” responding to the same prompt,  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  11 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  another anonymous fda employee said, “i’ve been pleasantly surprised by commissioner gottlieb’s knowledge and focus on fda science. i was at a brief with him and he was interested in the science and less focused on the legal and political effects than i would have guessed.” the technical competence of fda commissioner gottlieb was signaled as positive across many open-ended responses in our survey, and many scientists expressed they trusted the commissioner to defend and use their scientific work. similar open-ended responses were provided by noaa employees, for example, “i have the sense that our noaa leaders are working so hard to keep our agency focused on science and mission.” at the time that the survey was administered, cdc lacked a politically appointed leader, but scientists generally reported positively on how career leadership handled the science policy process. qualitative data was in-line with survey frequency data regarding federal scientific experts’ perceptions of leadership and how those perceptions varied across agencies. we would expect that such variation would also be reflected in workers’ perceptions of their value and effectiveness.  perception of value in the workplace while results varied across agencies, we find that scientists reported feeling less valued and effective in the workplace under the trump administration. there is increasing evidence that employees’ perceptions of their value in the workplace strongly affect their engagement [51]. while there is no single definition of employee engagement, the term can be generally defined as the emotional attachment of employees to their place of work [51–53]. if employees are more engaged in their work, it is likely that this will lead to a more effective and productive workforce [54]. prior research has identified several workplace attributes that contribute significantly to employee engagement. one attribute common across many studies is feeling valued and involved. in a survey that asked 1,714 employees about the perception of their value and engagement in their work, of the 93 percent of employees reporting they felt valued by their employer, 88 percent of those employees also reported being engaged in their work [55]. another identified driver of employee engagement is being kept abreast of what is going on in the organization, especially regarding communication of issues related to their specific work [56]. employee finances are also relevant to the perception of value in the workplace. while this includes the personal financial aspect of an organization’s employees (e.g., adequate pay, performance-based raises), it also includes the allocation of adequate financial resources to an employee’s work projects. when individuals receive resources from their organization to conduct their work, they feel obliged to respond in kind and “repay” the organization [57]. based on the above research, we analyzed four items in our survey that relate to attributes of employee engagement and employee effectiveness. we included a measure of effectiveness in this category, since we would expect, based on prior studies, that effectiveness is strongly related to employee engagement. the four items we analyzed were: 1) the employee receiving resources to do their work, 2) feeling included in team discussions when the employee’s work is involved, 3) the employee being able to speak freely about their scientific work even when it is viewed as politically contentious (e.g., ‘climate change”), and 4) direct reporting of how the employee perceives the effectiveness of their office or division (questions 4, 5, 6, and 7, respectively, listed in table 1). overall, across the four items analyzed, we find evidence that employee engagement of federal scientists varied across the surveyed agencies. on question 4, some respondents reported that over the past year they have noticed that resource allocations (e.g., funding, staff time) have been distributed away from programs and offices whose work is viewed as politically  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  12 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  fig 3. resource allocations for politically contentious work. figure shows the results from survey respondents’ responses to the survey item, “over the past year, i have noticed that resource allocations (e.g., funding, staff time) have been distributed away from programs and offices whose work is viewed as politically contentious.” survey respondents could choose one of six options: strongly disagree, disagree, do not agree or disagree, agree, strongly agree, or prefer not to disclose. responses varied by agency with a larger percentage of scientists agreeing that resource allocations have been distributed away from programs viewed as politically contentious at the epa and doi as compared to noaa and the fda. https://doi.org/10.1371/journal.pone.0231929.g003  contentious. a higher percentage of respondents from the cdc, doi and epa perceived a lack of resources for their work, with 47% (264 cdc respondents), 57% (646 doi respondents) and 65% (284 epa respondents), agreeing or strongly agreeing that this was the case (fig 3). in comparison, 20% of fda respondents and 28% of noaa respondents either agreed or strongly agreed that allocations have been distributed away from these programs. these responses were significantly different with p-values 0.0001 (doi vs. fda, doi vs. noaa, epa vs. fda, and epa vs. noaa, cdc vs. fda, cdc vs. epa, cdc vs. noaa). agreement was significantly higher for epa respondents vs. respondents at doi (p-value 0.007) and cdc respondents vs. doi respondents (p-value 0.005). on being provided adequate financial resources to do their jobs, many respondents, in their qualitative responses, cited real or anticipated budget cuts as inhibiting their job effectiveness. at the cdc, for example, several respondents cited potential budget cuts and resource constraints around infectious diseases and international cooperation as limiting their ability to do their jobs. one cdc respondent stated, “talk of defunding global health initiatives has caused my division to start terminating research collaborations with international laboratories that function as infectious disease surveillance sites in africa and asia.” another cdc respondent echoed similar budget concerns, “proposed funding cuts limits our [agency’s] capacity for responding to infectious disease overseas and domestically. these cuts don’t just affect our ability to prepare and respond at a federal level either. these will disproportionately affect smaller state and local health departments and grant funded programs.” some respondents also reported being excluded from conversations they would normally be a part of (table 1, question 5); however, this was not the case for most survey respondents. the percentage of respondents who disagreed with this statement varied by agency, with noaa (61%, 671 respondents), fda (61%, 193 respondents), and cdc (58%, 311  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  13 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  fig 4. exclusion of scientists from discussions and decisions. figure shows the results from survey respondents’ responses to the survey item, “in the past year, i have been excluded from discussions or decisions related to my scientific work that i normally would expect to be a part of.” survey respondents could choose one of six options: strongly disagree, disagree, do not agree or disagree, agree, strongly agree, or prefer not to disclose. most respondents across the fda, doi, and noaa disagreed with this statement. respondents from epa showed more agreement that they have been excluded from conversations that are related to their scientific work. https://doi.org/10.1371/journal.pone.0231929.g004  respondents) demonstrating the greatest disagreement among the agencies analyzed in this study (fig 4). respondents at the doi (54%, 593 respondents) and epa (45%, 188 respondents) disagreed or strongly disagreed to a slightly lesser extent. responses at the epa were significantly different from other agencies (p-values 0.03), and responses from noaa were significantly different from doi as well (p-value 0.0001) (fig 4). some scientists also reported self-censorship with respect to climate change-related work (question 6 in table 1, fig 5). as observed with the other items in this category, responses varied across agencies with a higher perception of self-censorship at the epa and doi. at the epa, 31% of respondents (124 respondents) agreed or strongly agreed that they had avoided working on climate change or using the phrase “climate change,” though they were not explicitly told to avoid them, however a similar proportion of those surveyed at the epa (34%, 134 respondents) also disagreed or strongly disagreed with the same statement. at the doi, 26% of respondents (279 respondents) agreed or strongly agreed, and 48% of respondents (512 respondents) disagreed or strongly disagreed with the same statement. the distribution of responses was significantly different between epa and doi (p-value 0.002). although the percentage of respondents in agreement was not as high for noaa respondents as at the epa or doi, 16% of respondents at noaa (179 respondents) were in agreement that they avoided working on climate change issues. while the percentage of respondents reporting self-censorship is low, it is unusual for hundreds of federal scientists to report censoring science-based information. therefore, we expect this to be a signal of devaluing employees at the epa and doi.  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  14 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  fig 5. scientist self-censorship on climate change. figure shows the results from survey respondents’ responses to the survey item, “i have avoided working on climate change or using the phrase “climate change,” though i was not explicitly told to avoid them.” survey respondents could choose one of six options: strongly disagree, disagree, do not agree or disagree, agree, strongly agree, or prefer not to disclose. respondents at the fda were not asked to respond to this item as climate change is not a priority of their work. more survey respondents at the epa and doi agreed that they were selfcensoring the phrase “climate change” in their work compared to noaa respondents. the percentage of respondents reporting self-censorship was generally low across agencies; however, hundreds of scientists reported self-censoring a scientific term. https://doi.org/10.1371/journal.pone.0231929.g005  we did not see a similar pattern at the cdc with only 5% of respondents (25 respondents) agreeing that they had avoided working on climate change or using the phrase “climate change,” though they were not explicitly told to avoid them. this response could be a product of employees feeling more valued at the cdc, or that climate change is not as broadly researched across the federal agency as compared to other agencies like the epa or doi. lastly, we analyzed the perception of office effectiveness by asking, “compared to one year ago, the effectiveness of your division or office has: increased, stayed the same, or decreased.” (table 1, question 7; fig 6) respondents had the option to choose one response (“don’t know,” “not applicable,” and “prefer not to disclose,” answer options were also available). perceived effectiveness and value are often intertwined as workers who are valued feel more engaged in their work and, therefore, are often more effective at their jobs. therefore, it is possible that perceived effectiveness is an indicator of employees’ perception of their own value and/or the value of other employees in the workplace. perceived effectiveness significantly varied by agency with a majority of respondents reporting decreases at the epa (68%, 284 respondents) and doi (58%, 624 respondents), whereas the largest portion of respondents at noaa (50%, 541 respondents), the cdc (43%, 228 respondents), and the fda (54%, 170 respondents) reported that their office’s effectiveness had largely not changed over the prior year (fig 6). the cdc was different from fda statistically, with a p-value of 0.001, even though the largest portion from both agencies reported no change in effectiveness. the epa and doi were also statistically different (p-value 0.004) for all responses, even though they both reported a decrease in effectiveness.  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  15 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  fig 6. office effectiveness over time. figure shows the results from survey respondents’ responses to the survey item, “compared to one year ago, the effectiveness of your division or office has.” survey respondents could choose one of six options: increased, stayed the same, decreased, don’t know, not applicable, or prefer not to disclose. responses varied by agency with respondents at epa and doi largely reporting decreased job effectiveness, whereas most respondents at the fda and noaa reported their effectiveness had stayed the same. https://doi.org/10.1371/journal.pone.0231929.g006  these results are in-line with other items that show more responses from doi and epa respondents indicating that they may not feel valued by their leadership, while the opposite is true for cdc, fda, and noaa. therefore, it would be expected that federal scientists at the epa and doi would perceive their agencies as not very effective compared to scientists at the cdc, fda and noaa.  comparisons over time to provide insight into whether the perception of scientific integrity being upheld varied over time across political administrations, we compared 2018 results to prior survey studies for cdc, epa, fws, fda, and noaa. ideally, we would make these comparisons for all survey items analyzed in this study (see table 1). however, the language used in survey items has changed over time to ensure that we are asking respondents’ questions that most effectively gauge their perception of scientific integrity. additionally, many agencies surveyed in our current study were not included in prior surveys. therefore, our comparative analyses are restricted to three questions that have the same or very similar language across surveys, and they also are restricted to agencies that have been surveyed at least once in a prior survey. to determine how perceptions of leadership being competent and trustworthy have changed over time, we analyzed responses to the item, “the presence of top agency decision makers who come from regulated industry inappropriately influences the decisions made by the agency” (table 1, question 8: fig 7). we were able to compare responses across three agencies: the cdc (comparison across surveys implemented in 2015 and 2018), fws (comparison across surveys implemented in 2015 and 2018), and fda (comparison across surveys  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  16 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  fig 7. presence of decisionmakers from regulated industry. figure shows the results from survey respondents’ responses to the survey item, “the presence of top agency decision makers who come from regulated industry inappropriately influences the decisions made by the agency.” survey respondents could choose one of seven options: strongly disagree, disagree, do not agree or disagree, agree, strongly agree, don’t know, or prefer not to disclose. while responses significantly varied by agency, most responses skewed more positive over time according to mantel-haenszel chi-square tests. thus, more scientists perceive the presence of decision makers who come from regulated industries inappropriately influencing agency decisions. https://doi.org/10.1371/journal.pone.0231929.g007  implemented in 2010, 2015, and 2018). responses varied over time for the cdc, fws, and fda with responses showing more agreement (agree or strongly agree) in the 2018 survey (i.e., indicating a greater level of inappropriate influence in 2018). at the fws, the proportion of respondents who agreed with this item was significantly lower in 2015 (31%, 254 respondents) than in 2018 (40%, 137 respondents; chi-square test p 0.0012; mantel-haenszel chisquare, p 0.0012). we also found significant differences among responses at the fda (p 0.0064). a mantel-haenszel chi-square test indicated more positive results in later surveys for this item at fda (p 0.0054), with agree/strongly agree responses slightly varying between 2010 (26%, 131 respondents), 2015 (25%, 346 respondents), and 2018 (23%, 70 respondents). responses also varied significantly at the cdc (p 0.0001). results skewed statistically more towards agreement (mantel-haenszel chi-square, p 0.0001) in 2018 (21%, 111 respondents) than in 2015 (17%, 198 respondents).  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  17 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  fig 8. fear of retaliation. figure shows the results from survey respondents’ responses to the survey item, “currently, i can openly express any concerns about the mission-driven work of my agency without fear of retaliation.” survey respondents could choose one of seven options: strongly disagree, disagree, do not agree or disagree, agree, strongly agree, don’t know, or prefer not to disclose. mantel-haenszel chi-square tests indicated survey responses skewed more positive in later surveys both for the fws and epa but skewed more negative for later surveys at the fda. https://doi.org/10.1371/journal.pone.0231929.g008  to assess if federal scientists perceived their value to have changed over time, we compared responses across surveys to the item, “currently, i can openly express any concerns about the mission-driven work of my agency without fear of retaliation.” (table 1, question 9; fig 8) chi-square tests showed significant differences in responses for the epa, fws, and fda; and mantel-haenszel chi-tests revealed that responses were more negative in later surveys as compared to earlier surveys (i.e., indicating a greater fear of retaliation in 2018). at the fws, for example, 29% of respondents disagreed or strongly disagreed with this statement in 2015 whereas 36% of respondents disagreed or strongly disagreed with this statement in 2018 (mantel-haenszel chi-square, p 0.0011). at the epa, disagree/strongly disagree responses also were significantly different in 2015 (24%, 382 respondents) versus 2018 (44%, 191 respondents; mantel-haenszel chi-square, p 0.0001). while responses were significantly different among surveys implemented at the fda (2006, 2010, 2012, 2015, and 2018), responses skewed more negative in earlier versus later surveys. in 2006, a higher percentage of respondents disagreed  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  18 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  fig 9. adherence to agency scientific integrity policy. figure shows the results from survey respondents’ responses to the survey item, “my agency adheres to its scientific integrity policy.” survey respondents could choose one of seven options: strongly disagree, disagree, do not agree or disagree, agree, strongly agree, don’t know, or prefer not to disclose. as compared to a survey conducted in 2015, agreement among scientists at the cdc and noaa increased in 2018 regarding the agencies adherence to their scientific integrity policy, whereas responses trended towards a neutral/don’t know response for the fws. https://doi.org/10.1371/journal.pone.0231929.g009  or strongly disagreed (36%, 357 respondents) with this survey item as compared to all other years when the survey was implemented including 2018 (22%, 75 respondents; mantel-haenszel chi-square, p 0.0001). only the cdc, fws, and noaa had questions from previous surveys that we could compare to determine how their agencies adhere to scientific integrity policies (table 1, question 10; fig 9). the cdc, fws, and noaa all responded positively over time to how their agencies uphold scientific integrity policies with those either agreeing or strongly agreeing changing from 66% to 76% in the cdc from 2015 to 2018 (p-value 0.0001), 50% to 63% in the fws from 2015 to 2018 (p-value 0.0001), and 57% to 64% in noaa (p-value 0.0001; fig 9). while it is not possible to compare responses for all survey items across time, the comparisons we are able to analyze provide further validation that our 2018 results, showing variation by agency, are sound. across many agencies, we find that federal scientists perceive some aspects of scientific integrity (ethical leadership and feeling valued) to have decreased in 2018.  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  19 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  however, this is not the case at all agencies. for example, federal scientists at the fda perceive scientific integrity to have been under greater assault during an earlier time. at other agencies, such as at the epa, federal scientists are in significant agreement that some aspects of scientific integrity are at a loss in 2018.  adherence to agency scientific integrity policy the 2018 survey asked respondents the extent to which they agreed or disagreed that “my agency adheres to its scientific integrity policy.” (table 1, question 10; fig 9) most scientists at the doi, cdc, fda, and noaa reported that they agreed or strongly agreed that their agency adhered to its scientific integrity policy. specifically, 76.1% at doi (752 respondents), 87% at the cdc (389 respondents), 80% at the fda (209 respondents), and 79% at noaa (696 respondents) agreed or strongly agreed that their agency adheres to its scientific integrity policy. this compares to 12% (115 respondents), 7% (33 respondents), 7% (17 respondents), and 9% (78 respondents) at the doi, cdc, fda, and noaa, respectively, disagreeing or strongly disagreeing that their agency adheres to its scientific integrity policy. among epa respondents, however, 46% (144 respondents) agreed or strongly agreed that the agency adhered to its scientific integrity policy, while 31% (100 respondents) disagreed or strongly disagreed. this response from epa scientists was significantly different from the other four agencies (p-values 0.0001). the fda was not different from the cdc, doi and noaa regarding the response proportions (p-values 0.19). these findings may appear to be at odds with other responses that suggest scientists do not perceive their agency to maintain scientific integrity (e.g., the perception of leadership politicizing science used in decisionmaking). we believe, however, that responses to other quantitative survey questions, as well as to the open-ended questions of the survey, provide an answer to this conundrum. federal scientists perceive the adherence to scientific integrity differently across groups of workers within their agency. specifically, there is evidence to suggest federal scientists distinguish scientific integrity of career staff and those working immediately around them from those in politically appointed positions. within their own community of careerlevel scientists, it seems that federal scientists perceive those individuals adhere to the policy. for example, respondents across agencies gave generally positive responses to the question, “my direct supervisor consistently stands behind scientists who put forth scientifically defensible positions that may be politically contentious.” specifically, 363 cdc respondents (63.9%), 830 doi respondents (73.6%), 285 epa respondents (64.9%), 222 fda respondents (65.5%), and 765 noaa respondents (67.4%) agreed or strongly agreed with this statement. as an anonymous respondent from the national park service responded, “from my perspective, the employees i work with have the highest standards of scientific integrity, remain focused on their work and clearly accept climate change as the primary issue in the future of our public lands.” however, across many agencies respondents noted that they did not feel political appointees were adhering to the policy, as this usgs respondent wrote, “scientific integrity within the scientists at the usgs is impeccable. this does not seem to be the case for people in appointed leadership positions within the doi.” in its own survey on scientific integrity, the epa also found that its staff’s perception of adherence to the agency’s scientific integrity policy varied depending on the group of individuals being referenced [58]. for example, a significant majority of epa survey respondents (88%, 3,338 responses) reported that they would feel comfortable reporting allegations to supervisors, despite some respondents expressing concern about political interference and loss of scientific integrity by agency leadership. when asked about the culture of scientific integrity at the epa in the three years prior to the survey, respondents to the epa-conducted survey most prevalently reported positive experiences and  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  20 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  viewpoints but also expressed concern about agency management and leadership with respect to scientific integrity. another potential explanation for this discrepancy is the distinction between the provisions of scientific integrity policies and the scientific integrity violations observed. in many cases, federal scientists expressed concerns about the actions of appointed leadership that fall outside of the scope of agency scientific integrity policies, which tend to focus instead on the rights and responsibility of agency scientists and other career level staff. therefore, it may be that federal agencies are adhering to the agency’s scientific integrity policies, but federal scientists observed science-policy actions perceived as ethically problematic that fall outside of the scope of the policy.  response rate and potential bias the survey response rate was notably lower than prior year surveys conducted using similar methods. several factors may have contributed to this. both active and passive discouragement from agency officials for taking the survey may have played a role. while leadership at several agencies, including the epa and noaa, encouraged employees to take the survey, some potential participants received communications from agency officials instructing them not to take the survey. the authors are aware of such communications being received by subsets of potential participants at the epa, fws, and noaa. relatedly, fear of retaliation for taking the survey from agency leadership, either with or without explicit instruction to not take the survey, may have contributed to lower response rates [59]. individual open-ended responses to the survey suggested that some agency scientists perceived that either they or their colleagues would not feel comfortable filling out the survey and/or disclosing details in the current political environment. lower response rates compared to similar prior surveys may also be a function of greater concerns about phishing scams in recent years. the frequency and prominence of such scams where people fell victim by clicking links in emails from unknown addresses has increased greatly since the years when prior online surveys were conducted [39, 40]. finally, survey fatigue may be partially responsible for lower response rates, as federal employees, particularly between 2016 and 2018, received a high level of outside inquiries from journalists and nongovernmental organizations; and high frequency of survey invitations has been shown to decrease response rates. there is potential for bias in the sample of individuals who responded to the survey. though the survey was administered via iowa state university, level of familiarity and agreement with the union of concerned scientists and its policy positions may have influenced who responded to the survey and the responses given. while the language of most survey questions was identical to that of prior surveys asked under different administrations, there is potential for response bias as is true for all survey work [60]. as the survey was sent to all potentially eligible participants, the sample was not random. further, it is likely that many ineligible people received invitations to take the survey, particularly for agencies with no available job titles to target potential participants. for these reasons, survey results should not be considered representative of views of federal scientists at large; rather, survey results provide a window into perceptions that a select set of agency scientists hold and how such views vary across agencies and time. further, this is the first time to our knowledge that it has been attempted to assess scientist perceptions of scientific integrity at federal government agencies using quantitative statistical methods. potential for validation of results was limited in this study design; however, internal results were found to be consistent, and this work can be considered a starting point for further work to quantitatively assess perceptions of scientific integrity at federal agencies.  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  21 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  conclusion our results indicate that federal scientists perceive losses of scientific integrity under the trump administration, given responses to key questions on the 2018 survey and comparison to surveys conducted prior to 2016. respondent perceptions of scientific integrity appear to be influenced by their perceptions of the trustworthiness and competence of agency and administration leadership, the degree to which they feel valued and effective in the workplace, and the presence of clear and comprehensive scientific integrity policies and infrastructure. differences in respondents’ perceptions of scientific integrity between federal agencies can be explained by these three factors identified as contributing to government scientists’ perceptions of agency scientific integrity. the existence and implementation of a clear comprehensive scientific integrity policy at the agency or department level can explain some of the variability in responses observed. prior to 2011, most government science agencies did not have scientific integrity policies in place. this lack of guidance, procedure, and infrastructure is apparent in surveys prior to 2011 where respondents across agencies indicated scientific integrity challenges in the workplace. at the time of the 2015 and 2018 surveys, the cdc, doi, epa, fda, and noaa all had comprehensive scientific integrity policies in place, as well as scientific integrity infrastructure including scientific integrity officers, trainings, and committees within agencies [37, 46, 61]. such improvement in scientific integrity policies and processes at agencies is reflected in the 2018 survey results, where respondents reported generally high marks in response to question 10 on their agency’s adherence to its scientific integrity policy. additionally, survey results indicate the significance of competent and trustworthy agency leadership as a contributing factor to federal scientists’ perception of agency scientific integrity. at the time the survey was conducted, only noaa didn’t yet have a senate-confirmed head. the doi, its agencies included in this analysis, the fda, and the epa all had senate-confirmed political appointees leading the agency. survey results, in addition to external evidence from independent reporting of agency policy actions, indicate that between 2016 and 2018, the epa and doi agencies experienced a higher degree of leadership changes, new policy directions, and divergence from business as usual on science policy issues compared to the fda and noaa. this could explain why respondents at the fda and noaa reported generally more positive results that were significantly different from those at the doi and epa. finally, survey results suggest that agency scientists’ perceptions of scientific integrity may be influenced by the degree to which employees feel effective and valued in their jobs. at the doi and the epa, respondents reported higher degrees of censorship and self-censorship, notably around climate change related work. such restrictions on production or communication of scientific work affect government scientists’ ability to be effective at their job and influence the degree to which they feel their role is valued. respondents at doi agencies and the epa also reported higher proportions of resource allocations away from politically contentious topics and being excluded from conversations they would normally be a part of than respondents from the fda and noaa—measures that also inform the degree to which employees feel valued and effective at work. these factors suggest that elements of the work environment at doi agencies and the epa may be contributing to scientists feeling less valued and effective at these agencies. overall, these results indicate that the presence of agency-wide scientific integrity policies and infrastructure alone do not predict government scientists’ perceptions of their agency’s degree of effectiveness or adherence to those policies. there are a variety of nuances under the umbrella of scientific integrity. this suggests that at the time the survey was conducted, agency scientific integrity policies may either not have been put into effective practice at all levels of  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  22 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  agencies or may not be sufficiently comprehensive to ensure agency scientists perceive scientific integrity to be a high priority in their workplace. results suggest that perceptions of competent and trustworthy leadership and feeling effective, well-resourced, and valued at work seem to be contributing factors. these findings suggest that enhancing scientific integrity at federal science agencies may necessitate competent and trustworthy leadership, a positive work environment for federal scientists and comprehensive scientific integrity policies and infrastructure. future work should consider ways of increasing response rates in order to increase the number, variety, and representativeness of perspectives collected across federal agencies. additional surveys or incorporation of questions related to scientific integrity into scheduled surveys of federal agencies would provide comparative data to further assess government scientists’ perceptions of the multiple components that together make up scientific integrity.  supporting information s1 appendix. survey of federal scientists response rates, 2018. (docx) s2 appendix. survey e-mail invitation. (docx) s3 appendix. 2018 survey of federal scientists (paper version). (docx) s4 appendix. quantitative survey results. (pdf) s5 appendix. open-ended survey results. (xlsx) s6 appendix. quantitative statistical assessment code. (zip)  acknowledgments the authors would like to thank those who put in lots of time and effort in developing the survey instrument, organizing and analyzing vast amounts of survey data, and providing feedback on earlier versions of this manuscript including charise johnson, anita desikan, emily berman, andrew a. rosenberg, michael halpern, liz borkowski, and susan wood. we’d also like to thank the staff at the center for survey statistics and methodology at iowa state university for their help in implementing the survey to thousands of federal scientists. lastly, we thank the federal scientists themselves for participating in this survey and for the work they do every day.  author contributions conceptualization: gretchen t. goldman, jacob m. carter. data curation: yun wang, janice m. larson. formal analysis: gretchen t. goldman, jacob m. carter, yun wang, janice m. larson. investigation: gretchen t. goldman, jacob m. carter. methodology: gretchen t. goldman, jacob m. carter, yun wang, janice m. larson.  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  23 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  project administration: gretchen t. goldman, jacob m. carter, yun wang, janice m. larson. resources: janice m. larson. software: yun wang. supervision: gretchen t. goldman, janice m. larson. validation: gretchen t. goldman, jacob m. carter, yun wang, janice m. larson. visualization: jacob m. carter, yun wang. writing – original draft: gretchen t. goldman, jacob m. carter. writing – review & editing: yun wang, janice m. larson.  references 1.  turnhout e, stuiver m, klostermann j, harms b, leeuwis c. new roles of science in society: different repertoires of knowledge brokering. science and public policy. 2013; 40(3):354–65. https://doi.org/10. 1093/scipol/scs114  2.  pope ca, cropper m, coggins j, cohen a. health benefits of air pollution abatement policy: role of the shape of the concentration–response function. j of the air & waste management association. 2015; 65 (5):516–22. https://doi.org/10.1080/10962247.2014.993004  3.  driscoll ct, buonocore jj, levy ji, lambert kf, burtraw d, reid sb, et al. us power plant carbon standards and clean air and health co-benefits. nature climate change. 2015; 5:535.  4.  kerkvliet j, langpap c. learning from endangered and threatened species recovery programs: a case study using u.s. endangered species act recovery scores. ecological economics. 2007; 63(2):499– 510. https://doi.org/10.1016/j.ecolecon.2006.12.007.  5.  male td, bean mj. measuring progress in us endangered species conservation. ecology letters. 2005; 8(9):986–92. https://doi.org/10.1111/j.1461-0248.2005.00806.x  6.  currie j, greenstone m, moretti e. superfund cleanups and infant health. american economic review. 2011; 101(3):435–41. https://doi.org/10.1257/aer.101.3.435 pmid: 25152535  7.  gamper-rabindran s, timmins c. does cleanup of hazardous waste sites raise housing values? evidence of spatially localized benefits. j of env. econ and mgmt. 2013; 65(3):345–60. https://doi.org/10. 1016/j.jeem.2012.12.001.  8.  williamson ra, hertzfeld hr, cordes j, logsdon jm. the socioeconomic benefits of earth science and applications research: reducing the risks and costs of natural disasters in the usa. space policy. 2002; 18(1):57–65. https://doi.org/10.1016/s0265-9646(01)00057-1.  9.  van den hove s. a rationale for science–policy interfaces. futures. 2007; 39(7):807–26. https://doi.org/ 10.1016/j.futures.2006.12.004.  10.  jorstad e, skogen k. the norwegian red list between science and policy. env. sci. & policy. 2010; 13 (2):115–22. https://doi.org/10.1016/j.envsci.2009.12.003.  11.  crouzat e, arpin i, brunet l, colloff mj, turkelboom f, lavorel s. researchers must be aware of their roles at the interface of ecosystem services science and policy. ambio. 2018; 47(1):97–105. https://doi. org/10.1007/s13280-017-0939-1 pmid: 28913614  12.  ferna ndez rj. how to be a more effective environmental scientist in management and policy contexts. env. sci. & policy. 2016; 64:171–6. https://doi.org/10.1016/j.envsci.2016.07.006.  13.  gupta a. transparency in global environmental governance: a coming of age? global environmental politics. 2010; 10(3):1–9. https://doi.org/10.1162/glep_e_00011  14.  stewart j. the meaning of strategy in the public sector. australian j of public admin. 2004; 63(4):16– 21. https://doi.org/10.1111/j.1467-8500.2004.00409.x  15.  lalor bm, hickey gm. strengthening the role of science in the environmental decision-making processes of executive government. organization & environment. 2014; 27(2):161–80.  16.  mcgarity to, waggoner we. bending science: how special interests corrupt public health research. cambridge, massachusetts, and london, england: harvard university press; 2008.  17.  berman e, carter j. scientific integrity in federal policymaking under past and present administrations. j of science policy and governance. 2018; 13(1).  18.  mooney c. the republican war on science. new york, ny: basic books; 2005.  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  24 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  19.  union of concerned scientists. abuses of science: case studies. 2009 [cited 2019 jun 11]. available from: https://www.ucsusa.org/our-work/center-science-and-democracy/promoting-scientific-integrity/ abuses-science-case-studies  20.  union of concerned scientists. 2004 scientist statement on restoring scientific integrity to federal policy making. 2004 [cited 2019 jun 11]. available from: https://www.ucsusa.org/our-work/center-scienceand-democracy/promoting-scientific-integrity/scientists-sign-on-statement.html.  21.  environmental protection agency. u.s. environmental protection agency scientific integrity policy. 2012.  22.  department of interior. integrity of scientific and scholarly activities. 2014.  23.  national oceanic and atmospheric administration. administrative order 202-735d. 2011.  24.  gilens m, page bi. testing theories of american politics: elites, interest groups, and average citizens. perspectives on politics. 2014; 12(3):564–81. https://doi.org/10.1017/s1537592714001595  25.  raziq a, maulabakhsh r. impact of working environment on job satisfaction. procedia economics and finance. 2015; 23:717–25. https://doi.org/10.1016/s2212-5671(15)00524-9  26.  jindal-snape d, snape jb. motivation of scientists in a government research institute: scientists’ perceptions and the role of management. management decision. 2006. 44(10):1325–43. https://doi.org/ 10.1108/00251740610715678  27.  mirza s. job satisfaction among research and development scientists: the relationship of leadership practices and job characteristics: capella university; 2005.  28.  us office of personnel management. federal employee viewpoint survey. [cited 22 march 2020]. available from: https://www.opm.gov/fevs/  29.  us office of personnel management. federal employee viewpoint survey: special reports [cited 22 march 2020]. available from: https://www.opm.gov/fevs/reports/special-reports/  30.  hameduddin t, fernandez s. employee engagement as administrative reform: testing the efficacy of the opm’s employee engagement initiative. public administration review. 2019 feb 21. https://doi.org/ 10.1111/puar.13134 pmid: 32025058  31.  woolston c. satisfaction in science. nature. 562, 611–614. 2018 https://doi.org/10.1038/d41586-01807111-8 pmid: 30356203  32.  austin j. u.s. life scientists report rising salaries and high job satisfaction. science 2006 nov 3. https:// doi.org/10.1126/science.314.5800.842  33.  faupel-badger jm, nelson de, izmirlian g. career satisfaction and perceived salary competitiveness among individuals who completed postdoctoral research training in cancer prevention plos one. 2017. https://doi.org/10.1371/journal.pone.0169859  34.  holt tj, blevins kr, smith rw. examining the impact of organizational and individual characteristics on forensic scientists’ job stress and satisfaction. j. crime justice. 2017. https://doi.org/10.1080/ 0735648x.2016.1216731  35.  choi s. organizational justice and employee work attitudes: the federal case. the american review of public admin. 2011. https://doi.org/10.1177/0275074010373275  36.  ahmad a. the relationship among job characteristics organizational commitment and employee turnover intentions: a reciprocation perspective. j of work-applied mgmt. 2018 jun 4. issn: 2205-2062  37.  goldman gt, berman e, halpern m, johnson c, kothari y, reed g, et al. ensuring scientific integrity in the age of trump. science. 2017; 355(6326):696. https://doi.org/10.1126/science.aam5733 pmid: 28209862  38.  union of concerned scientists. survey of scientists at federal agencies 2005 [cited 2019 6/11/19]. available from: https://www.ucsusa.org/our-work/center-science-and-democracy/promoting-scientificintegrity/surveys-scientists-federal.  39.  union of concerned scientists. survey: u.s. fish and wildlife service scientists. washington, d.c. : union of concerned scientists, 2005.  40.  union of concerned scientists. voices of scientists at the fda. washington, d.c.: 2006.  41.  union of concerned scientists. voices of epa scientists. washington, d.c. : 2007.  42.  union of concerned scientists. voices of federal climate scientists. washington, d.c.: 2006.  43.  union of concerned scientists. voices of scientists at the fda: measuring progress on scientific integrity. washington, d.c. : 2012.  44.  union of concerned scientists. progress and problems: government scientists report on scientific integrity at four agencies. washington, d.c. : 2015.  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  25 / 26  
  plos one  survey of federal scientists and perceived loss of scientific integrity under trump leadership  45.  koizumi k, sheehan j. washington, d.c. : white house office of science and technology policy. 2016. [cited 2019]. available from: https://obamawhitehouse.archives.gov/blog/2016/12/19/scientificintegrity-policies-update.  46.  goldman g, reed g, carter j. risks to science-based policy under the trump administration. stetson law review. 2017; 47:267–93.  47.  jin m. implementing multiple comparisons on pearson chi-square test for an r x c contingency table in sas. sas global forum; washington, dc: sas; 2014.  48.  oswald aj, proto e, sgroi d. happiness and productivity. j of labor economics. 2015; 33(4):789–822. https://doi.org/10.1086/681096  49.  artz bm, goodall ah, oswald aj. boss competence and worker well-being. ilr review. 2017; 70 (2):419–50. https://doi.org/10.1177/0019793916650451  50.  seto s, sarros j. servant leadership influence on trust and quality relationship in organizational settings. international leadership j. 2016; 8(3):23–33.  51.  kompaso s, sridevi sm. employee engagement: the key to improving performance. international j of business and management. 2010; 5(12):89–96.  52.  shuck b, wollard k. employee engagement and hrd: a seminal review of the foundations. human resource dev. review. 2010; 9(1):89–110. https://doi.org/10.1177/1534484309353560  53.  maslach c, schaufeli wb, leiter mp. job burnout. annual review of psychology. 2001; 52(1):397– 422. https://doi.org/10.1146/annurev.psych.52.1.397 pmid: 11148311.  54.  shmailan as. the relationship between job satisfaction, job performance and employee engagement: an explorative study. issues in business management and economics. 2016; 4(1):1–8.  55.  apa survey finds feeling valued at work linked to well-being and performance [internet]. 2012. available from: https://www.apa.org/news/press/releases/2012/03/well-being  56.  vogelgesang gr, leroy h, avolio bj. the mediating effects of leader integrity with transparency in communication and work engagement/performance. the leadership quarterly. 2013; 24(3):405–13. https://doi.org/10.1016/j.leaqua.2013.01.004.  57.  cropanzano r, mitchell ms. social exchange theory: an interdisciplinary review. j of management. 2005; 31(6):874–900. https://doi.org/10.1177/0149206305279602  58.  environmental protection agency. scientific integrity at the epa: results of the 2016 epa employee survey. washington, dc: environmental protection agency; 2016.  59.  skibell a. scientist survey flagged as spam. e&e news. 2018 mar 9 [cited 2020 mar 22]. available from: https://www.eenews.net/greenwire/2018/03/09/stories/1060075941.  60.  lavrakas pj. encyclopedia of survey research methods. 2008. https://doi.org/10.4135/9781412963947  61.  carroll c, hartl b, goldman gt, rohlf dj, treves a, kerr jt, et al. defending scientific integrity in conservation policy processes: lessons from canada, australia, and the united states. con. bio. 2017 jul 25. https://doi.org/10.1111/cobi.12958  plos one https://doi.org/10.1371/journal.pone.0231929 april 23, 2020  26 / 26  
       


secondary_title_index: research &amp; politics
title_index: political shock and international students: estimating the “trump effect”
full_text_index:
       research article  political shock and international students: estimating the “trump effect”  research and politics october-december 2022: 1–9  the author(s) 2022 article reuse guidelines: sagepub.com/journals-permissions doi: 10.1177/20531680221141526 journals.sagepub.com/home/rap  mingsi song and quan li†  abstract the negative “trump effect” on international students has attracted wide media and scholarly attention. surprisingly, the best existing evidence remains anecdotal and case-based. in this study, we fill this important gap. we employ a difference-indifferences (did) design to estimate the trump effect for the us vis-a-vis various control groups: top 5, top 10, top 20, and all other countries that compete with the us. we find a statistically significant and negative trump effect that drives international students from the us to competing destinations. relative to the top five competitors, about 12% fewer students came to the us during the first 3 years of the trump presidency. the average treatment effect is statistically significant across the top 5, top 10, and top 20 destination groups but not for the group of all other destinations as a whole. pairwise did estimates between the us and 91 individual countries further indicate that the trump effect is primarily driven by 26 host nations. these findings contribute to our understanding of trump effects, student flows, and migration. keywords trump effect, international students, political shock, difference-in-differences  introduction higher education is one of the united states sectors with a vital global comparative advantage. in the 2019–2020 academic year, more than one million international students came to the us to study. they contributed 38.96 billion in education exports and created over 415,990 jobs; higher education ranked 6th among united states service exports in 2020.1 the united states has been the most popular and largest destination for international students seeking higher education abroad. according to the unesco statistics,2 during the past two decades, the united states accounts for around 20% of all student flows. the number of international students enrolled in american colleges and universities rose from 0.4 million in 2001 to more than 0.9 million in 2019. however, after taking office in 2017, president trump adopted anti-immigration rhetoric and various policies that many considered to have dampened the appeal of the united states to international students.3 these policies included the muslim travel ban and more stringent student and work  visa restrictions. many scholars (e.g. hacker and bellmore, 2020; laws and ammigan, 2020; van de walker and slate, 2019; todoran and peterson, 2020) argue that there is a strong negative “trump effect” on international student flows into the us. despite receiving wide attention from both media and academia, the effect has never been tested and estimated empirically, as one researcher (mckivigan, 2020) points out in a recent article. nearly all studies employ anecdotal and case-based evidence. rigorous  department of political science, texas a&m university, college station, tx, usa †  an earlier version of this paper was presented at the 2022 mpsa annual meeting. we thank ping xu, alexander kustov, two anonymous reviewers, and christian grose for their helpful comments and suggestions. authorship is shared equally. corresponding author: mingsi song, department of political science, texas a&m university, 4220 tamu, college station, tx 77843-4220, usa. email: mingsi@tamu.edu  creative commons non commercial cc by-nc: this article is distributed under the terms of the creative commons attribution-noncommercial 4.0 license (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the sage and open access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).  
  2 systematic statistical analysis and evidence are lacking. in this study, we fill this gap. in this study, we address the following research questions. is there any solid statistical evidence substantiating the “trump effect” that deters international students from coming to the us? if the effect exists, which countries benefit as substitute destinations? we employ a differencein-differences (did) design to estimate the effect for the u.s. vis-a-vis various control groups: top 5, top 10, top 20, and all other host countries that compete with the u.s. for international students. we find a statistically significant and negative “trump effect” that drives international students to top competing destinations. to ascertain which countries substitute the us, we obtain and test the did estimate between the us and each competing host. among 119 other countries, 91 have sufficient data to produce pairwise did estimates, and only 26 seem to have clearly benefited from the decline of student flows into the u.s. our research makes several contributions. first, the trump effects on various outcomes have received wide attention in the news, social media, and academic research. the unexpected victory of trump in the 2016 united states presidential election has generated an increasing number of studies exploring outcomes such as racial discrimination (newman et al., 2021), public health (smith, 2022), eu’s popularity (minkus et al., 2019), the reputation of the united states abroad (carreras et al., 2021; haman and s kolni k, 2021; bateson and weintraub, 2022), trade protectionist sentiment and policy (coupe and shepotylo, 2021; essig et al., 2021), and the financial and currency markets (cunha and kern, 2022; slaski, 2021). our research provides a useful addition to that literature by analyzing a different type of outcome. second, a growing body of literature on international student flows focuses on social, economic, and geographic determinants, including distance, common official language, shared religion, historical colonial ties, and relative economic development between origin and destination countries (e.g. beine et al., 2014; perkins and neumayer, 2014; tay, 2014; abbott and silles, 2016; perkins and neumayer, 2014). the impact of political shocks on international students, though widely discussed, is rarely analyzed empirically. our research contributes to that literature by demonstrating a trump effect and highlighting the impact of political shocks. third, student flows are closely related to migration because today’s students often become tomorrow’s legal migrants. for example, dreher and poutvaara (2011) find that international student inflows into the us significantly predict future emigration from those home countries. since the trump administration increased immigration restrictions significantly, one might likely find that the negative trump effect on student flows also applies to migration. our research suggests a rigorous approach to estimating that  research and politics effect. by demonstrating the substitution effect, our research also echoes and complements the findings in (glennon, 2020) that restrictions on the united states h-1b immigration cause multinational firms to offshore jobs to other countries such as canada, india, and china, without increasing the employment of natives in the united states. finally, as noted earlier, american colleges and universities appeal to students worldwide. education exports constitute a significant component of the u.s. and global economies. demonstrating how the trump effect influences the movement of international students among various countries is of interest to a wide range of stakeholders, such as colleges and universities, students, and policymakers in different countries.  “trump effect” on international student flows the “trump effect” is based on rhetoric and policies with negative implications for international students regarding racial profiling, migration, and visa restrictions. first, prejudiced elite speech during the trump campaign encouraged people with racial prejudices to express and act upon those prejudices actively (newman et al., 2021). on january 27, 2017, president trump issued an executive order restricting the entry of people from seven muslim majority countries—iran, iraq, libya, somalia, sudan, syria, and yemen—into the united states. following several legal challenges during the year, the trump administration removed iraq and sudan from the list but added chad, north korea, and venezuela. in 2018, chad was removed from the list. finally, on january 31, 2020, six more countries—eritrea, kyrgyzstan, myanmar, nigeria, sudan, and tanzania—were added. although the restrictions were placed on banned countries, the “trump ban” reportedly also caused a chilling effect on international students more generally.4 many students who planned to study abroad became terrified by the stressful and hostile political climate in the us and changed their plans. second, the trump administration consistently tried to curtail both legal and illegal immigration. on april 18, 2017, president trump signed the “buy american and hire american” executive order.5 the policy sought to strengthen the enforcement of immigration laws, particularly the h-1b visa program that directly pertains to international students who intend to stay and work in the us after getting a degree. also, in 2017, president trump decided to end the daca program (deferred action for childhood arrivals)—the obama-era program allowing undocumented immigrants who arrived in the country as children to stay.6 meanwhile, the trump administration significantly reduced the number of non-immigrant visas for tourists, temporary workers, and international students.7  
  song and li third, in 2018, the trump administration imposed new rules on international students regarding part-time on-campus jobs, course load, abilities to maintain legal status, and penalties for violations of conditions of visas.8 for example, united states citizenship and immigration services (uscis) changed the calculation of “unlawful presence” that results in immediate deportation of students. near the end of the trump presidency, more stringent restrictions were imposed on student and work visas.9 these controversial policies, often challenged, sometimes rescinded, often re-introduced, and even escalated, created an atmosphere and environment that made the u.s. less attractive for many international students. the media reported the chilling “trump effect,” and researchers also explored the phenomenon. laws and ammigan (2020) show that since 2016, the number of international students enrolled in the us began to decrease. hacker and bellmore (2020) find that in the 2017–2018 academic year, international student enrollment in the us decreased by 6.6% from the 2016–2017 academic year. van de walker and slate (2019) show that from fall 2016 to fall 2018, in two of the largest public texas institutions, international graduate applicants declined by 33.34% and 18.306% from muslim-majority countries and non-muslim majority countries, respectively. using data collected from four groups of international doctoral students in january 2017, todoran and peterson (2020) find that the “trump ban” affected not only students from those banned countries but also students from other countries who feel the united states climate has become stressful and hostile. the “trump effect” has implications for the us as a destination and other destination countries that compete for international students. students spend a lot of time and money preparing to study abroad. when the us becomes less appealing, these students will likely select other destinations rather than cancel going abroad due to the “sunk cost.” meanwhile, as the victory of trump caused global public opinions toward the us to decline dramatically (haman and s kolni k, 2021; carreras et al., 2021; bateson and weintraub, 2022), it led to a significant increase in the eu’s popularity in europe (minkus et al., 2019). as other top and popular destination countries also provide highquality educational venues and maintain the same level of student restrictions in terms of visa policies regarding entry, re-entry, and post-graduation employment, these countries benefit as alternative destinations. university administrators are acutely aware of this substitution effect. university of california president janet napolitano commented in 2018, “where the united states retreats, there’s a vacuum, and other countries will rush to fill it.”10 in contrast, universities canada, a membership organization of canadian universities and colleges, started their news release on november 15, 2017, with the  3 following statement, “at a time of closing borders and closing minds, students from around the world are choosing canada.”11 canada was not the only one that benefited from the “trump effect.” australia, new zealand, spain, japan, and even china all reported an increase in international enrollment.12 in sum, the “trump effect” on international student flows consists of two components: a chilling effect on those who want to but decide not to come to the us and a substitution effect of american colleges being replaced by foreign universities as new destinations. thus, estimating the effect requires both components to be accounted for empirically, an issue we turn to in the next section.  research design the dependent variable is the number of students from an origin country enrolled in higher education institutions in a destination country. we collect data from the unesco institute for statistics (uis).13 compared with other sources like oecd statistics, unesco data have the broadest spatial and temporal coverage. since the “trump effect” consists of two components, we employ a difference-in-differences (did) design for identification and estimation (angrist and pischke, 2008). in our design, we compare the us against various control groups: top 5, top 10, top 20, and all other host countries that compete with the u.s. for international students, and we also compare the changes in student flows between 3 years before the trump presidency (2013–2015) and 3 years during the trump presidency (2017–2019). we exclude the election year 2016 because there is too much uncertainty for international students to anticipate who will be the next president and united states policies regarding international students. for robustness checks, we re-estimated all the models including observations from 2016. the findings reported in the supplemental appendix remain robust. we exclude the year 2020 because it coincides with the covid19 pandemic, thus confounding our estimation. the equation below illustrates our did specification yijt 1⁄4 β0 t β1 usj t β2 trumpt t β3 usj trumpt t βxijt t εijt  (1)  yijt represents student flows from home country i to host country j in year t; xijt represents control variables in year t εijt represents the independently and identically distributed random error. usj indicates a dummy variable for the u.s. as a host country, which one may consider as a “treated” host country that experiences trump policies at some point relative to the control group (competing destinations); β1 represents the time-invariant difference (fixed effect) between the treated (u.s.) and the control group (competing destinations) in the absence of trump policies. trumpt  
  4 indicates a time dummy that equals one after trump takes office and his policies come into effect and zero otherwise; β2 represents the trump presidency period-specific effect common to both the treated and control groups. thus, usj trumpt is the interaction between the treated country dummy and the trump presidency period dummy, which equals one if a host country is the us and the value of the year is after 2016, and zero otherwise; β3 denotes the did effect of interest, that is, the average trump effect on student flows. according to our discussion in the previous section, β3 should be negative and statistically significant, providing a clear signal of the trump effect. it is worth noting that a statistically insignificant β3, while not a significant sign of a strong trump effect, could imply a weak trump effect if the control group catches up in the growth of student inflows with the previously faster growing u.s. we further adapt the did specification above to suit our analysis. first, because unobserved heterogeneity is associated with various home and host countries and country pairs or dyads, we control for home and host country fixed effects or dyad fixed effects to avoid spurious findings. these fixed effects end up absorbing the time-invariant difference (fixed effect) between the treated (us) and the control group (competing destinations) that is captured by the dummy variable usj. second, we control for possible year-specific effects common to all countries by including year dummies, which absorb the trump presidency periodspecific effect common to all countries as reflected by trumpt. we include various control variables according to the literature on the drivers of bilateral student flows. almost all studies find that bilateral distance reduces student flow by raising the cost of studying abroad. scholars also find that common official language, shared religion, and historical colonial ties promote bilateral student flows (e.g. beine et al., 2014; perkins and neumayer, 2014; tay, 2014). we control for geographical distance weighted by the populations of origin and destination countries, geographic contiguity, common official language, and historical colonial ties. data on these variables are from the cepii project (mayer and zignago, 2011). studies find that relative economic development between countries matters to student flows. different wage rates and work opportunities could drive students to move from poor to rich countries (abbott and silles, 2016; perkins and neumayer, 2014). hence, we control for the ratio of gdp per capita (in 2015 constant u.s. dollars) between the home and the host country in a dyad. we also control for home and host countries’ populations (log-transformed) as standard gravity model variables. we expect the gdp per capita ratio between home and host countries to decrease student flows and the population size to increase student flows. data on these variables are from the world bank’s world development indicators.  research and politics in terms of the estimation technique, we follow recent studies of student flows (e.g. abbott and silles, 2016; beine et al., 2014) and use the poisson pseudo maximum likelihood (ppml) estimator. as shown by silva and tenreyro (2006), when the dependent variable has a disproportionately large number of zero values and the error variance is heteroskedastic, ppml produces consistent statistical estimates and is superior to ordinary least squares (ols) or negative binomial. we follow previous studies and estimate robust standard errors clustered over dyad to deal with heteroskedastic error variance and possible correlation within dyad.  findings before presenting our estimation results, we provide a descriptive comparison of annual total student inflows into the top six destination countries—united states, united kingdom, australia, germany, france, and canada—between 2013 and 2019. as shown in figure 1, before 2016, the number of international students increased rapidly in nearly all six countries except the united kingdom. however, after 2016, the number of student flows into the u.s. stopped increasing rapidly and decreased slightly in 2018. in contrast, student flows into the other five destinations, except france, grew rapidly after 2016. figure 1 also presents the aggregate trend lines for the top 5, top 10, top 20, and all other competing destinations. relative to the us, the trend line is upward and rising continuously for these competitor groups. there appears to be some prima facie evidence for the “trump effect”: a chilling effect preventing students from coming to the us and a substitution effect of the us being replaced by other top destinations. table 1 presents the results based on the samples in which the host countries are the us and the other top 5, top 10, top 20, and all other destinations. models 1–2 include home country, host country, and year fixed effects, and models 3–4 control for dyad and year fixed effects. models 1 and 3 include the did design relevant variables, while models 2 and 4 add substantive control variables. table 1 reports the did estimates, and full estimation results are reported in the supplemental appendix. based on the various samples in which the host countries are the us and the other top 5, top 10, top 20, or all other destinations, the coefficient of the did term us trump is negative across all four models, as expected. it is statistically different from zero in all but three of the all other destination models.14 the size of the coefficient of the did term is the largest in the top 10 destination models, similar between the top 5 and top 20 destination models, and the smallest in the all other destination models. overall, the “trump effect” deters some international students from  
  song and li  5  figure 1. note: international student enrollment from 2013 to 2019. author’s calculation based on unesco data. table 1. did coefficient estimates of the “trump effect.” (1)  (2)  (3)  (4)  top 5 (se.) (n) top 10 (se.) (n) top 20 (se.) (n) all others (se.) (n)   0.12* (0.06) (6369)  0.15** (0.06) (10188)  0.11* (0.06) (18847)  0.08 (0.05) (51156)   0.12* (0.07) (6073)  0.17** (0.07) (9695)  0.12** (0.06) (17961)  0.09 (0.06) (47991)   0.11* (0.06) (6356)  0.14** (0.06) (10136)  0.11* (0.06) (18693)  0.09* (0.05) (49597)   0.12* (0.07) (6038)  0.15** (0.06) (9649)  0.11* (0.06) (17828)  0.08 (0.05) (46582)  controls home fe host fe dyadic fe year fe  no yes yes no yes  yes yes yes no yes  no no no yes yes  yes no no yes yes  two-tailed test: *p 0.10, **p 0.05, ***p 0.01 robust standard error and number of observations in parentheses. control variables include home-host gdp per capita ratio, logged home and host populations, contiguity, common language, colonial ties, and logged bilateral distance.  coming to the us for higher education and causes a substitution of the united states colleges by competitor universities as destinations for these students. to further comprehend the size of the effect, we follow silva and tenreyro (2006) and interpret the coefficient of the did term, which is a dummy variable in the ppml b model, as 1⁄2deβ 1t 100 percent change in y as the dummy turns from zero to one. thus, relative to other top five destinations, the trump effect based on the four did model estimates reduces student inflows into the us by about 11.3%, 11.3%, 10.4%, and 12.2%, respectively, during the 2017–2019 period. relative to other top 10 destinations, the substantive effects based on the four did model estimates are larger, being 13.9%, 15.6%, 13.1%, and 13.9%, respectively. interestingly, relative to the top 20 destinations, the substantive effects resemble those for the top five destinations. overall, these estimates demonstrate a substantial “trump effect.” in contrast, for the sample in which the host countries are the u.s. and all other destinations, the did model estimates remain negative but are statistically significant in model 3 only. these results suggest that the trump effect does not apply to all destinations equally.  
  6  research and politics  figure 2. pairwise did estimates as clear evidence of trump effect.  figure 3. pairwise did estimates as ambiguous evidence of trump effect.  pairwise did estimates of the “trump effect” our estimates so far represent average treatment effects. the results in the last section suggest that not all destinations benefit from the trump effect, and select countries could drive the phenomenon. we take an extra step and estimate the effect for the us vis-a-vis each host country. indeed, in their effort to attract international students, each host country might benefit from, lose out to, or be unaffected by the “trump effect.” we obtain pairwise did estimates based on model 4 in table 1. for 119 host countries,  pairwise did coefficients are estimable for 91 countries due to data availability. we categorize these pairwise estimates into three groups: negative and statistically significant estimates that present clear evidence of the effect; statistically insignificant estimates that represent ambiguous evidence of the effect (e.g. the slowing down of the united states growth in student inflows and the catching up of the competitor’s growth make them indistinguishable); positive and statistically significant estimates that provide clear evidence of opposite trump effect. figure 2, figure 3, and figure 4 display individual did point estimates and their 95% confidence intervals for the three groups, respectively. since  
  song and li  7  figure 4. pairwise did estimates as clear evidence of opposite trump effect.  table 2. top destinations during 2013–2019 and their classifications.  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22  the host country  sum of inflows  united states of america united kingdom australia* france germany* canada russia japan italy turkey* malaysia netherlands austria south korea* saudi arabia argentinae ukraine new zealand switzerland poland* spain czechia  6,241,506 2,917,963 2,342,219 1,571,738 1,500,271 1,348,810 1,307,853 1,093,533 600,193 598,427 508,061 478,471 470,577 470,122 469,903 374,166 351,585 346,146 341,889 329,482 316,720 298,080  note: * clear trump effect; ambiguous trump effect; opposite trump effect; e excluded from pairwise estimation due to the lack of data before 2016.  the top destinations account for most student flows, it is illuminating to examine their classification. table 2 shows the top 22 destinations during 2013–2019, including the us as the baseline treatment group and argentina, whose  pairwise did is not estimable due to its lack of data before 2016. among the 91 countries, 26 of them belong to the clear trump effect group (significant negative did coefficients), 51 countries belong to the ambiguous trump effect group (insignificant did coefficients), and 14 countries belong to the opposite trump effect group (significant positive did coefficients). therefore, our findings demonstrate that the commonly suggested negative “trump effect” does exist, but only for a significant subset of countries. these findings also provide other insights into the socalled “trump effect.” first, in figure 2, the effect size varies dramatically. countries that witness the most considerable percent change, such as croatia, moldova, rwanda, niger, and grenada, have small absolute numbers of student inflows. in contrast, countries like australia and germany have relatively smaller estimates, but they are the strongest competitors against the us and receive large numbers of international students. among the top 20 destinations, as shown in table 2, turkey, poland, australia, germany, and south korea benefit the most. they experience faster growth in student inflows during the trump administration. second, figure 3 shows that in the ambiguous trump effect group, the 51 countries are very diverse. about half of them have insignificant and expected negative estimates. some of these countries are good examples of the possibility that the trump effect is most likely at work. for example, canada’s did estimate is 0.2395, with t statistics 1.84 and p value 0.066. among the top united states competitors, as shown in table 2, spain, japan, russia, and the netherlands all have the expected negative estimates. on the other hand, the other half of the countries have insignificant but positive estimates, including major destinations such as the uk. for the uk, the negative “brexit effect” dampened its attraction to  
  8 international, especially european, students. had it not for brexit, the uk would certainly have benefited from the trump effect. finally, in figure 4, 14 host countries exhibit patterns contrary to the expected trump effect and experience even slower growth than the us. except for italy, malaysia, and saudi arabia, as shown in table 2, these countries do not appear to be major destinations for international students. their impact on global student flows is likely to be small.  conclusion since president trump took office, policies became more restrictive for migrants and international students. the negative “trump effect” has attracted wide media and scholarly attention. however, the evidence to date is anecdotal and case-based. using a difference-in-differences (did) design, we estimated the impact of the trump presidency on international student flows. we demonstrated that following trump’s election, international student flows into the u.s. declined relative to various groupings of peer countries. we also decomposed this decline and demonstrated the substitution effect among countries. the average negative treatment effect is statistically significant across the top 5, top 10, and top 20 destination groups but not for the group of all other destinations. pairwise did estimates between the u.s. and 91 individual host countries further indicate that the trump effect is driven mainly by 26 nations. as noted, our research makes several valuable contributions. we provide the first empirical test of the so-called “trump effect” on student flows and offer a valuable addition to the growing number of studies on the trump effect. we show that instead of coming to the us, many international students go to competing destination countries. more importantly, our research highlights the limited scope of the trump effect, that is, it does not apply to all host countries indiscriminately. our research also contributes to the literature on student flows by highlighting the impact of political shocks and government policies and exposing the interconnected nature of student flows among competing destination countries. our analysis complements the vast literature on migration to the extent that international students could be future legal migrants. one implication is that the decline of high-quality international students today could reduce high-quality legal migrants tomorrow. finally, our research informs stakeholders in education service exports and higher education abroad about the uneven impact of political shocks and the possibility of substitution. our study has its limitations that call for future research. first, we do not directly test what specific policy action caused the trump effect. in our conjecture, the trump effect consists of both chilling and substitution effects. many behaviors of the trump administration could have  research and politics produced those effects, such as trump’s anti-immigration rhetoric, the muslim ban, the stringent student and work visa restrictions, and the unwelcome political climate for international students during his presidency. future research may seek to disentangle these possible causes. second, our research uncovers heterogeneity of the trump effect among competing destination countries, but we do not investigate the sources of these variations. they may have resulted from the possibility that the rise of populism and the push for de-globalization unfolded unevenly among countries. brexit in the u.k. is a good example. future research may examine this possibility. finally, our study focuses on student enrollment, but the trump effect could also apply to applications to united states institutions. declining applications may signal decreasing attractiveness to international students. unaffected applications may suggest the absence of the trump effect. the survey of 294 u.s. institutions in the fall of 2017 by aacrao (american association of collegiate registrars and admissions officers) shows a mixed picture. among 294 american colleges and universities, 77% were worried about application yield; 38% experienced a decline in the number of international applications, 27% experienced no change, and 35% witnessed an increase (aacrao, 2017). the chilling effect on applications is not homogeneous, and there could be a substitution effect even among united states institutions. future research should explore the variations among different colleges and universities.  declaration of conflicting interests the author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.  funding the author(s) received no financial support for the research, authorship, and/or publication of this article.  orcid ids mingsi song https://orcid.org/0000-0001-5018-5742 quan li https://orcid.org/0000-0002-0510-3258  notes 1. https://www.trade.gov/education-service-exports 2. source: unesco institute for statistics (uis), http://data.uis. unesco.org/index.aspx 3. sophia tareen (2020, october 25). foreign students show less zeal for us since trump took over. ap news 4. sophia tareen (2020, october 25). foreign students show less zeal for us since trump took over. ap news 5. uscis (2017). buy american, hire american: putting american workers first.  
  song and li 6. adam edelman (2017, september 5). trump ends daca program, no new applications accepted. nbc news. 7. nahal toosi, ted hesson, and sarah frostenson (2018, april 3). foreign visas plunge under trump. politico. 8. marnette federis (2019, june 20). visa rules are restricting the future of international students in the us. the world. 9. catherine e. shoichet and geneva sands (2020, september 24). trump administration takes aim at international students again. cnn. also, britta glennon (2020, july 20). why the trump administration’s anti-immigration policies are the united states’ loss and the rest of the world’s gain. brookings. 10. study international staff (2018, april 24). trump is ruining us universities’ ‘brand america’. study international. 11. universities canada (2017, november 15). canada’s global moment: students from around the world choose canada. 12. benjamin wermund (2018, april 23). trump blamed as u.s. colleges lure fewer foreign students. politico. 13. we downloaded the data from http://data.uis.unesco.org/ index.aspx in february 2022. 14. interestingly, as shown in the supplemental appendix, the coefficient is negative and statistically significant in all models when observations from 2016 are included in the estimation.  references aacrao (2017) trending topics survey: international applicants for fall 2017 - institutional & applicant perceptions. washington, dc: american association of collegiate registrars and admissions officers. abbott a and silles m (2016) determinants of international student migration. the world economy 39(5): 621–635. angrist jd and pischke js (2008) mostly harmless econometrics. new jersey: princeton university press. bateson r and weintraub m (2022) the 2016 election and america’s standing abroad: quasi-experimental evidence of a trump effect. the journal of politics 84(4): 2300–2304. beine m, noe l r and ragot l (2014) determinants of the international mobility of students. economics of education review 41: 40–54. carreras m, visconti g and aca cio i (2021) the trump election and attitudes toward the united states in latin america. public opinion quarterly 85(4): 1092–1102. coupe t and shepotylo o (2021) popular support for trade agreements and partner country characteristics: evidence from an unexpected election outcome. economic inquiry 59(1): 549–566. cunha raphael and kern andreas (2022) global banking and the spillovers from political shocks at the core of the world economy. the review of international organizations 17(4): 717–749.  9 dreher a and poutvaara p (2011) foreign students and migration to the united states. world development 39(8): 1294–1307. essig j, xu p, garand jc, et al. (2021) the “trump” effect: political elite and support for free trade in america. american politics research 49(3): 328–342. glennon britta (2020) how do restrictions on high-skilled immigration affect offshoring? evidence from the h-1b program. nber working paper 27538. hacker nl and bellmore e (2020) “the trump effect”: how does it impact international student enrollment in us colleges? journal of critical thought and praxis 10(1): 1–11. haman m and s kolni k m (2021) trump and the image of the united states in latin america. central european journal of international & security studies 15(1): 58–85. laws k and ammigan r (2020) international students in the trump era: a narrative view. journal of international students 10(3): xviii–xxii. mayer thierry and zignago soledad (2011) notes on cepii’s distances measures: the geodist database. paris: cepii. mckivigan jm (2020) effect of federal policy changes on international students pursuing higher education studies in the united states. higher education research 5(2): 60. minkus l, deutschmann e and jan d (2019) atrump effect on the eu’s popularity? the us presidential election as a natural experiment. perspectives on politics 17(2): 399–416. newman b, merolla jl, shah s, et al. (2021) the trump effect: an experimental investigation of the emboldening effect of racially inflammatory elite communication. british journal of political science 51(3): 1138–1159. perkins r and neumayer e (2014) geographies of educational mobilities: exploring the uneven flows of international students. the geographical journal 180(3): 246–259. silva jmcs and tenreyro s (2006) the log of gravity. the review of economics and statistics 88(4): 641–658. slaski a (2021) policy signaling and foreign electoral uncertainty: implications for currency markets. international studies quarterly 65(4): 1124–1136. smith kb (2022) politics is making us sick: the negative impact of political engagement on public health during the trump administration. plos one 17(1): e0262022. tay c (2014) an econometric model on bilateral trade in education using an augmented gravity model. journal of industrial engineering and management (jiem) 7(2): 401–412. todoran c and peterson c (2020) should they stay or should they go? how the 2017 us travel ban affects international doctoral students. journal of studies in international education 24(4): 440–455. van de walker d and slate jr (2019) the 2017 trump administration travel ban and international graduate applications at two texas public universities. higher education politics & economics 5(1): 1–14.  
       


authors_index: bujang, mohamad adam omar, evi diana baharum, nur akmal
secondary_title_index: malaysian journal of medical sciences
title_index:
       a review on sample size determination for cronbach’s alpha test: a simple guide for researchers     

full_text_index:
       original article  a review on sample size determination for cronbach’s alpha test: a simple guide for researchers mohamad adam bujang1, evi diana omar2, nur akmal baharum3  submitted: 26 mar 2018 accepted: 11 nov 2018 online: 28 dec 2018  1  clinical research centre, sarawak general hospital, ministry of health, sarawak, malaysia  2  clinical research centre, serdang hospital, ministry of health, selangor, malaysia  3  national clinical research centre, ministry of health, kuala lumpur, malaysia  to cite this article: mohamad adam b, evi diana o, nur akmal b. a review on sample size determination for cronbach’s alpha test: a simple guide for researchers. malays j med sci. 2018;25(6):85–99. https://doi.org/10. 21315/mjms2018.25.6.9 to link to this article: https://doi.org/10.21315/mjms2018.25.6.9  abstract background: reliability studies are commonly used in questionnaire development studies and questionnaire validation studies. this study reviews the sample size guideline for cronbach’s alpha test. methods: manual sample size calculation using microsoft excel software and sample size tables were tabulated based on a single coefficient alpha and the comparison of two coefficients alpha. results: for a single coefficient alpha test, the approach by assuming the cronbach’s alpha coefficient equals to zero in the null hypothesis will yield a smaller sample size of less than 30 to achieve a minimum desired effect size of 0.7. however, setting the coefficient of cronbach’s alpha larger than zero in the null hypothesis could be necessary and this will yield larger sample size. for comparison of two coefficients of cronbach’s alpha, a larger sample size is needed when testing for smaller effect sizes. conclusions: in the assessment of the internal consistency of an instrument, the present study proposed the cronbach’s alpha’s coefficient to be set at 0.5 in the null hypothesis and hence larger sample size is needed. for comparison of two coefficients’ of cronbach’s alpha, justification is needed whether testing for extremely low and extremely large effect sizes are scientifically necessary. keywords: cronbach’s alpha, internal consistency, reliability, sample size  introduction cronbach’s alpha is a measure of the internal consistency or reliability between several items, measurements or ratings. in other words, it estimates how reliable are the responses of a questionnaire (or domain of a questionnaire), an instrumentation or rating evaluated by subjects which will indicate the stability of  the tools. alpha was developed by cronbach (1), which was originally used to measure the reliability of a psychometric instrument. the value of cronbach’s alpha ranges from zero to one with the higher values implying the items are measuring the same dimension. in contrary, if the cronbach’s alpha value is low (near to 0), it means some or all of the items are not measuring the same dimension (2–3).  malays j med sci. nov–dec 2018; 25(6): 85–99  www.mjms.usm.my penerbit universiti sains malaysia, 2018  this work is licensed under the terms of the creative commons attribution (cc by) (http://creativecommons.org/licenses/by/4.0/).  85  
  malays j med sci. nov–dec 2018; 25(6): 85–99  for example, the depression anxiety and stress scale (dass-21) that is used to measure the magnitude of respondents’ stress level condition for over the past week, has seven items measured by using a four-point likert scale (ranging from zero to three) (4). to ascertain whether the items are reliable in measuring the same dimension, a test for cronbach’s alpha may be used. the cronbach’s alpha test is usually applied to test the consistency and stability of the questionnaires which measure latent variables. although cronbach’s alpha test may be applied in situations other than questionnaire development or validation, there is limited literature of its application in such scenarios. cronbach’s alpha has been applied in research to develop clinical tool. for instance, a study by berg et al. (5) developed an instrument to measure the balancing ability in elderly patients. the study used a scale consisting of 14 movements of patients such as sitting unsupported, change of position and etc. the movements were evaluated by five physical therapists and they were given a score from zero to four for each movement. the high degree of internal consistency in this study showed that the scale measures the same dimension one of the common issues in inferential studies is to determine the sufficient sample size. the lack of knowledge in sample size determination and unfamiliarity with sample size softwares are usually the challenges encountered by researchers especially among those who do not have adequate knowledge in statistics. sufficient sample size is needed so that research conducted can provide reliable and reproducible evidence that can detect the desired consistency or stability of an instrument (questionnaire). it is an important aspect in research to avoid lack of power of the test due to underestimation of sample size and also prevent the waste of resources due to overestimation of sample size (6). the purpose of this present paper is to provide a simple guide for medical researchers to plan sample size estimation for their studies that involve cronbach’s alpha test. the sample sizes are determined and presented in the form of tables to estimate the minimum number of samples needed to obtain the desired value of cronbach’s alpha. following that, the discussion of this paper emphasizes on the application of the formula and the sample size tables.  86  www.mjms.usm.my  cronbach's alpha test discussion on cronbach’s alpha test in terms of concept, applications, statistical test and sample size determination have been widely discussed in the literatures (7–11). some of the discussions related to sample size are summarised in table 1 (6, 12–16). the present study emphasizes the review of sample size determination for cronbach’s alpha and focuses on sample size determination introduced by bonett (6, 12). hypothesis testing for a single cronbach’s alpha test the null hypothesis is h0: ca0 ca1 and the possible selection of alternative hypotheses are: ha: ca0 ca1 (this option yields a two-tailed test) ha: ca0 ca1 (this option yields a one-tailed test) ha: ca0 ca1 (this option also yields a one-tailed test) ca0 refers to the value of cronbach’s alpha in null hypothesis and ca1 refers to the value of cronbach’s alpha in alternative hypothesis. most commonly ca0 is assumed and set at 0 in which smaller sample sizes are derived since the gap between ca0 and ca1 is larger. setting up the ca0 as not equal to 0 is uncommon in research but it is very useful in testing and comparing the hypothesis of two different alpha coefficients (13). hypothesis testing for comparison of two cronbach’s alpha test the null hypothesis is h0: ρa ρb and the possible selection of alternative hypotheses are: ha: ρa ρb (this option yields a twotailed test) ha: ρa ρb (this option yields a onetailed test) ha: ρa ρb (this option also yields a one-tailed test) ρa refers to the value of cronbach’s alpha in population “a” and ρb refers to the value of cronbach’s alpha in population “b”.  
  original article sample size for cronbach's alpha table 1. the summarise on previous publication related sample size estimation for cronbach’s alpha no  title  author  remarks  1  sample size requirements for testing and estimating coefficient alpha.  bonett dg (2002)  the sample size formulas closely approximate the sample size requirements for an exact confidence interval or an exact test.  2  sample size requirements for comparing two alpha reliability coefficients  bonett dg (2003)  introduce sample size determination for comparing two alpha coefficients.  3  minimum sample size for cronbach's coefficient alpha: a monte carlo  yurdugul h (2008)  this paper suggests the sample size of 30 is sufficient on condition that first (largest) eigenvalue obtained from principal component analysis (pca) of the sample data set is higher than 6.00. however, if that first eigenvalues are between 3.00 and 6.00, then required minimum sample size is 100.  4  cronbach’s alpha reliability: interval estimation, hypothesis testing, and sample size planning  bonett dg, wright t (2015)  the results of a simulation study demonstrated that the proposed method performed better than alternative methods.  5.  statistical methods–scale reliability analysis with small samples, birmingham city university, centre for academic success  samuels p (2015)  provides guideline for reliability analysis considering small samples.  6.  the rcsi sample size handbook: a rough guide  conroy r (2016)  this study summarized that sample size of 30 can measure reliability using cronbach’s alpha considering the scale items have strong correlations.  methods sample size calculation based on formula by bonett (6) sample size for coefficient alpha or cronbach’s alpha was calculated using microsoft excel. the formulation was from bonett (6) based on formula given; 2 2 2k n (c k - 1 ma z a 2 zb k 2 ln _du i h 2  (i)  where 1 - ca0  d 1 - ca1  (ii)  sample size was calculated based on power of 0.80 and 0.90 (power 1–β) while probability of type i error (α) was set at 0.05 at all time. there are options on whether to use two-tailed  test or one-tailed test settings. in this paper, the more-commonly used two-tailed test was chosen with the aim to detect the difference between the two cronbach’s alpha. there are three things that need to be considered when determining the sample size for cronbach’s alpha test: the number of items or raters (k), the value of cronbach’s alpha at null hypothesis (ca0) and the expected value of cronbach’s alpha (ca1). the value for ca0 and ca1 could be any value ranges from 1 to 1, however ca1 should not be equal to ca0 (ca1 ca0). in order to illustrate the above formula with an example; there are 15 items in a questionnaire of which the reliability of its measurements need to be measured (ca0 and ca1 are identified at 0.0 and 0.7, respectively). power is set at 90% and the value of alpha at 0.05. the minimum sample size required based on formula (i) and (ii) is as shown below:  www.mjms.usm.my  87  
  malays j med sci. nov–dec 2018; 25(6): 85–99  calculations: α 0.05 β 0.1 k 15 ca0 0.0 ca1 0.7  ten items in a questionnaire in group 1 and 2 for which the reliability of its measurements need to be measured (ρa and ρb are identified at 0.3 and 0.7, respectively). power is set at 90% and the value of alpha at 0.05. the minimum sample size required based on formula (iii) and (iv) is as shown below:  1 - 0.0 d 1 - 0.7 3.333  calculations:  rs vw ss 2 _15 i 2 w ww     j z 1.96 z 1.282 * 4 e o s 0.025 0.1 ww 2 n ss 15 - 1 ss ww 2 ln _3.333 i s w t x n 17.53 . 18  α 0.05 β 0.1 k 10 ρa 0.3 ρb 0.7  therefore, the minimum sample size required for this case study is approximately 18 samples. sample size calculation based on formula by bonett (12) sample size for comparing two coefficient alpha or cronbach’s alpha was also calculated using microsoft excel. the formulation was from bonett (12) based on formula given; v srs k1 k2 z z j 2 ww m ss2 c - ww a b 2 n s k1 1 k2 - 1 ww 2 ss 2 ww ln _du i s t x where d  _1 - ta i _1 - tb i  (iii)  (iv)  sample size was calculated based on power of 0.80 and 0.90 (power 1 – β) while probability of type i error (α) was set at 0.05 at all time. there are options on whether to use two-tailed test or one-tailed test settings. in this paper, the more-commonly used two-tailed test was chosen with the aim to detect the difference between the two cronbach’s alpha. there are four things that need to be considered when determining the sample size for cronbach’s alpha test: the number of items or raters in group one (k1), the number of items or raters in group two (k2), the value of cronbach’s alpha at null hypothesis or group one (ρa) and the expected or group two value of cronbach’s alpha (ρb). the value of k1 and k2 are set to be similar and present as k. the value for ρa and ρb could be any value ranges from 1 to 1, however ρb should not be equal to ρa (ρb ρa ). in order to illustrate the above formula with an example; there are  88  www.mjms.usm.my  d  _1 - 0.3 i  2.3 _1 - 0.7 i  rs v ss2 c 10 10 m_1.96 1.282 i 2 www ww 2 n ss 10 - 1 10 - 1 2 ss ww _ i 2.3 ln s w t x n 67.049 . 68 therefore, the minimum sample size required for this case study is approximately 68 samples per group.  results based on pre-specified alpha, power and effect size, the minimum sample size requirements are shown in table 2 to table 3. the pre-specified alpha and power are usually known (0.05 and 80.0% or 90.0%, respectively) and thus, the minimum sample size required is mainly determined by the value of k and the differences of the coefficients in null hypothesis and alternative hypothesis. the minimum requirement for sample size remains relatively unchanged when the value of k increases. larger sample size was observed when the differences between the coefficients in the null and alternative hypotheses were smaller. for one coefficient of cronbach’s alpha test, if the coefficient of cronbach’s alpha equals to zero in the null hypothesis, it is possible to derive a smaller sample size of less than 30 to achieve a minimum desired effect size of 0.7 or more (table 2). however, setting the coefficient of cronbach’s alpha larger than zero (e.g., ca0 0.50) in the null hypothesis could be necessary in some cases and this will yield a larger sample size. (table 2). for a comparison of two coefficients of cronbach’s alpha in different  
  original article sample size for cronbach's alpha groups, a larger sample size is needed to detect smaller effect size when the coefficients in the null hypothesis (ρa) and alternative hypothesis (ρb) are nearly about the same (table 3).  discussions for a single coefficient of cronbach’s alpha test, the application of the sample size tables can be used into two situations, which are sample size determination when ca0 can be assumed equal to zero and sample size determination when ca0 is assumed to be more than zero. sample size determination for a single coefficient of cronbach’s alpha test when ca0 can be assumed equal to zero in the questionnaire reliability studies, the coefficient of cronbach’s alpha in the null hypothesis (ca0) is always assumed to be equaled to zero, implying that researchers assumed that there is no internal consistency of the whole items or for a particular domain of a questionnaire. then, the coefficient of cronbach’s alpha in the alternative hypothesis (ca1) will assume that there is a magnitude of internal consistency (either low, moderate or strong) of the questionnaire or a particular domain of a questionnaire. the minimum sample size requirement depends on the desired magnitude of internal consistency in the alternative hypothesis. higher differences of cronbach’s alpha coefficients in the null hypothesis and alternative hypothesis will yield the lower sample size. based on table 2, in order to achieve an acceptable cronbach’s alpha of 0.7, when the values of alpha and power are fixed at 0.05 and 80.0%, respectively, the sample size ranges from four (k 100 and ca0 0.0 versus ca1 0.95) to 52 (k 3 and ca0 0.0 versus ca1 0.5), depending on the number of k (number of items). hence, cronbach’s alpha test is suitable for conducting pilot studies when the ca0 is usually assumed to be zero. in studies related to questionnaire development and validation, cronbach’s alpha is a common indicator for internal consistency and has always been evaluated in the pilot studies (17–20). based on the calculation, it is possible that the sample size can be as small as four with assumption of a very high internal consistency such as 0.95 or more (k 100 and ca0 0.0 versus ca1 0.95). although researchers may  target a high coefficient of cronbach’s alpha, however, it is impractical to do so as there is a possibility that it may not be achieved in an actual study. it is more reasonable to target a desired cronbach’s alpha between 0.7 to 0.8 with an acceptable sample size. an example of a sample size statement is as followed: “the aim of the study is to evaluate the internal consistency of particular questionnaire “a”. questionnaire “a” has 15 items with a fivepoint likert scale for every item. the coefficient of cronbach’s alpha in the null hypothesis and alternative hypothesis are assumed to be equal to 0.0 and 0.7, respectively. based on alpha value fixed at 0.05, the minimum sample size requirement is 14 in order to achieve power of 80.0%. the calculation is based on the formula introduced by bonett (6).” if the questionnaire has a different number of items, researchers will need to calculate the sample size based on the formula in equation (i). sample size determination for a single coefficient of cronbach’s alpha test when ca0 is assumed to be more than zero hypothesis testing when ca0 0 is rare in literature. however, the concept of testing the null hypothesis with ca0 0 may be useful in cases when researchers set a high target of internal consistency of an instrument. some statistical soft-wares do not provide statistical test to assess internal consistency when ca0 is assumed to be larger than zero. however, with the sample size calculation, researchers can calculate and determine the sufficient sample size to assess the internal consistency when ca0 is assumed to be larger than zero. for example, when k 3 and ca0 0.50, based on alpha 0.05 and power of at least 80.0%, a minimum sample size of 31 is sufficient to detect ca1 at 0.80. a larger minimum sample size is needed (n 93) to detect ca1 at 0.70 (table 2). an example of a sample size statement is as followed: “the aim of the study is to determine whether a particular questionnaire “b” has high magnitude of internal inconsistency. questionnaire “b” has 10 items with a 5-point likert scale for every item. the null hypothesis has set the coefficient of cronbach’s alpha is at 0.50 while in the alternative hypothesis, the coefficient of cronbach’s alpha is set at 0.80. based on alpha of 0.05, the minimum sample size requirement is 23 to be able to detect at least 80.0% power of the test. the calculation is based on the formula introduced by bonett (6).”  www.mjms.usm.my  89  
  malays j med sci. nov–dec 2018; 25(6): 85–99  table 2. sample size tables for cronbach’s alpha test with various effect sizes, alpha 0.05 and power are set to 80.0% (na) and 90.0% (nb), respectively k  ca0  ca1  na  nb  k  ca0  ca1  na  nb  k  ca0  ca1  na  nb  3  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  52 39 24 19 15 12 9 7 5  68 52 31 24 19 15 11 8 6  5  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  43 33 20 16 13 10 8 6 5  57 44 26 21 16 13 10 7 5  10  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  39 30 18 15 12 9 7 6 4  51 39 24 19 15 12 9 7 5  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  188 93 52 31 19 12 7  250 123 68 40 24 15 8  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  157 78 43 26 16 10 6  209 103 57 34 21 13 7  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  140 69 39 23 15 9 6  186 92 51 30 19 12 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  375 146 71 38 22 13 7  502 194 94 50 29 16 9  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  313 122 59 32 19 11 7  418 162 79 42 24 14 8  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  279 109 53 29 17 10 6  372 145 70 38 22 13 7  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  1,323 287 109 52 27 15 8  1,770 383 145 68 35 19 10  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  1,103 240 91 43 23 13 7  1,476 320 121 57 30 16 9  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  981 213 81 39 21 12 7  1,312 285 108 51 27 15 8  0.65  0.70 0.75 0.80 0.85 0.90 0.95  993 210 78 35 18 9  1,329 281 103 46 23 11  0.65  0.70 0.75 0.80 0.85 0.90 0.95  828 176 65 30 15 8  1,108 235 86 39 19 9  0.65  0.70 0.75 0.80 0.85 0.90 0.95  737 157 58 27 14 7  985 209 77 35 17 9  0.70  0.75 0.80 0.85 0.90 0.95  711 146 52 22 10  951 194 68 29 12  0.70  0.75 0.80 0.85 0.90 0.95  593 122 43 19 9  793 162 57 24 11  0.70  0.75 0.80 0.85 0.90 0.95  527 109 39 17 8  705 145 51 22 10  0.75  0.80 0.85 0.90 0.95  475 93 31 12  636 123 40 15  0.75  0.80 0.85 0.90 0.95  397 78 26 10  530 103 34 13  0.75  0.80 0.85 0.90 0.95  353 69 23 9  471 92 30 12  0.80  0.85 0.90 0.95 0.90 0.95  287 52 15 146 22  383 68 19 194 29  0.80  0.85 0.90 0.95 0.90 0.95  240 43 13 122 19  320 57 16 162 24  0.80  0.85 0.90 0.95 0.90 0.95  213 39 12 109 17  285 51 15 145 22  0.85  0.85  0.85  (continued on next page)  90  www.mjms.usm.my  
  original article sample size for cronbach's alpha table 2. (continued) k  ca0  ca1  na  nb  k  ca0  ca1  na  nb  k  ca0  ca1  na  nb  15  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  38 29 18 14 11 9 7 6 4  49 38 23 18 14 11 9 7 5  20  0.00 0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  37 28 17 14 11 9 7 6 4  49 37 23 18 14 11 9 7 5  25  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  37 28 17 14 11 9 7 6 4  48 37 22 18 14 11 9 7 5  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  135 67 38 23 14 9 6  179 89 49 29 18 11 7  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  132 66 37 22 14 9 6  176 87 49 29 18 11 7  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  131 65 37 22 14 9 6  175 86 48 29 18 11 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  269 105 51 28 16 10 6  359 139 68 37 21 12 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  264 103 50 28 16 10 6  353 137 67 36 21 12 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  261 102 50 27 16 10 6  349 136 66 36 21 12 7  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  946 206 79 38 20 11 6  1,265 275 104 49 26 14 8  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  929 202 77 37 20 11 6  1,243 270 103 49 25 14 8  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  920 200 77 37 19 11 6  1,230 267 102 48 25 14 8  0.65  0.70 0.75 0.80 0.85 0.90 0.95  710 151 56 26 13 7  950 201 74 34 17 8  0.65  0.70 0.75 0.80 0.85 0.90 0.95  698 148 55 26 13 7  933 198 73 33 17 8  0.65  0.70 0.75 0.80 0.85 0.90 0.95  691 147 55 25 13 7  924 196 72 33 16 8  0.70  0.75 0.80 0.85 0.90 0.95  508 105 38 16 8  680 139 49 21 10  0.70  0.75 0.80 0.85 0.90 0.95  500 103 37 16 8  668 137 49 21 9  0.70  0.75 0.80 0.85 0.90 0.95  494 102 37 16 8  661 136 48 21 9  0.75  0.80 0.85 0.90 0.95  340 67 23 9  455 89 29 11  0.75  0.80 0.85 0.90 0.95  334 66 22 9  447 87 29 11  0.75  0.80 0.85 0.90 0.95  331 65 22 9  442 86 29 11  0.80  0.85 0.90 0.95  206 38 11  275 49 14  0.80  0.85 0.90 0.95  202 37 11  270 49 14  0.80  0.85 0.90 0.95  200 37 11  267 48 14  0.85  0.90 0.95  105 16  139 21  0.85  0.90 0.95  103 16  137 21  0.85  0.90 0.95  102 16  136 21  (continued on next page)  www.mjms.usm.my  91  
  malays j med sci. nov–dec 2018; 25(6): 85–99  table 2. (continued) k  ca0  ca1  na  nb  k  ca0  ca1  na  nb  k  ca0  ca1  na  nb  30  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  36 28 17 14 11 9 7 6 4  48 37 22 17 14 11 9 7 5  35  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  36 28 17 14 11 9 7 6 4  48 36 22 17 14 11 9 7 5  40  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  36 28 17 14 11 9 7 6 4  47 36 22 17 14 11 8 7 5  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  130 65 36 22 14 9 6  173 86 48 28 17 11 7  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  130 64 36 22 14 9 6  173 85 48 28 17 11 7  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  129 64 36 22 14 9 6  172 85 47 28 17 11 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  260 101 50 27 16 10 6  347 135 65 36 21 12 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  258 101 49 27 16 10 6  345 134 65 35 20 12 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  257 100 49 27 16 10 6  344 134 65 35 20 12 7  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  913 199 76 36 19 11 6  1,222 265 101 48 25 14 8  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  909 198 76 36 19 11 6  1,216 264 100 48 25 14 8  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  905 197 75 36 19 11 6  1,211 263 100 47 25 14 7  0.65  0.70 0.75 0.80 0.85 0.90 0.95  686 146 54 25 13 7  917 195 72 33 16 8  0.65  0.70 0.75 0.80 0.85 0.90 0.95  683 145 54 25 13 7  913 194 72 33 16 8  0.65  0.70 0.75 0.80 0.85 0.90 0.95  680 145 54 25 13 7  910 193 71 33 16 8  0.70  0.75 0.80 0.85 0.90 0.95  491 101 36 16 8  656 135 48 21 9  0.70  0.75 0.80 0.85 0.90 0.95  489 101 36 16 8  653 134 48 20 9  0.70  0.75 0.80 0.85 0.90 0.95  487 100 36 16 8  651 134 47 20 9  0.75  0.80 0.85 0.90 0.95  329 65 22 9  439 86 28 11  0.75  0.80 0.85 0.90 0.95  327 64 22 9  437 85 28 11  0.75  0.80 0.85 0.90 0.95  326 64 22 9  435 85 28 11  0.80  0.85 0.90 0.95  199 36 11  265 48 14  0.80  0.85 0.90 0.95  198 36 11  264 48 14  0.80  0.85 0.90 0.95  197 36 11  263 47 14  0.85  0.90 0.95  101 16  135 21  0.85  0.90 0.95  101 16  134 20  0.85  0.90 0.95  100 16  134 20  (continued on next page)  92  www.mjms.usm.my  
  original article sample size for cronbach's alpha table 2. (continued) k  ca0  ca1  na  nb  k  ca0  ca1  na  nb  k  ca0  ca1  na  nb  45  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  36 28 17 14 11 9 7 6 4  47 36 22 17 14 11 8 7 5  50  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  36 28 17 14 11 9 7 6 4  47 36 22 17 14 11 8 7 5  55  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  36 28 17 14 11 9 7 6 4  47 36 22 17 14 11 8 7 5  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  129 64 36 22 14 9 6  171 85 47 28 17 11 7  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  128 64 36 22 14 9 6  171 85 47 28 17 11 7  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  128 64 36 22 14 9 6  171 85 47 28 17 11 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  257 100 49 27 16 10 6  343 133 65 35 20 12 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  256 100 49 27 16 10 6  342 133 65 35 20 12 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  256 100 49 27 16 10 6  341 133 64 35 20 12 7  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  903 196 75 36 19 11 6  1,208 262 100 47 25 14 7  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  901 196 75 36 19 11 6  1,205 262 100 47 25 14 7  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  899 196 75 36 19 11 6  1,203 261 99 47 25 14 7  0.65  0.70 0.75 0.80 0.85 0.90 0.95  678 144 54 25 13 7  907 192 71 32 16 8  0.65  0.70 0.75 0.80 0.85 0.90 0.95  677 144 54 25 13 7  905 192 71 32 16 8  0.65  0.70 0.75 0.80 0.85 0.90 0.95  675 144 54 25 13 7  903 192 71 32 16 8  0.70  0.75 0.80 0.85 0.90 0.95  485 100 36 16 8  649 133 47 20 9  0.70  0.75 0.80 0.85 0.90 0.95  484 100 36 16 7  648 133 47 20 9  0.70  0.75 0.80 0.85 0.90 0.95  483 100 36 16 7  646 133 47 20 9  0.75  0.80 0.85 0.90 0.95  325 64 22 9  434 85 28 11  0.75  0.80 0.85 0.90 0.95  324 64 22 9  433 85 28 11  0.75  0.80 0.85 0.90 0.95  324 64 22 9  432 85 28 11  0.80  0.85 0.90 0.95  196 36 11  262 47 14  0.80  0.85 0.90 0.95  196 36 11  262 47 14  0.80  0.85 0.90 0.95  196 36 11  261 47 14  0.85  0.90 0.95  100 16  133 20  0.85  0.90 0.95  100 16  133 20  0.85  0.90 0.95  100 16  133 20  (continued on next page)  www.mjms.usm.my  93  
  malays j med sci. nov–dec 2018; 25(6): 85–99  table 2. (continued) k  ca0  ca1  na  nb  k  ca0  ca1  na  nb  k  ca0  ca1  na  nb  60  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  36 28 17 14 11 9 7 6 4  47 36 22 17 14 11 8 7 5  65  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  36 28 17 13 11 9 7 6 4  47 36 22 17 14 11 8 7 5  70  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  36 27 17 13 11 9 7 6 4  47 36 22 17 14 11 8 7 5  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  128 64 36 22 14 9 6  170 84 47 28 17 11 7  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  128 64 36 21 13 9 6  170 84 47 28 17 11 7  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  128 64 36 21 13 9 6  170 84 47 28 17 11 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  255 100 49 27 16 10 6  341 132 64 35 20 12 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  255 99 49 27 16 10 6  340 132 64 35 20 12 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  255 99 49 27 16 10 6  340 132 64 35 20 12 7  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  898 195 75 36 19 11 6  1,201 261 99 47 25 14 7  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  897 195 75 36 19 11 6  1,199 260 99 47 25 14 7  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  896 195 75 36 19 11 6  1,198 260 99 47 25 14 7  0.65  0.70 0.75 0.80 0.85 0.90 0.95  674 144 53 25 13 7  902 191 71 32 16 8  0.65  0.70 0.75 0.80 0.85 0.90 0.95  673 143 53 25 13 7  901 191 71 32 16 8  0.65  0.70 0.75 0.80 0.85 0.90 0.95  673 143 53 25 13 7  900 191 71 32 16 8  0.70  0.75 0.80 0.85 0.90 0.95  483 100 36 16 7  645 132 47 20 9  0.70  0.75 0.80 0.85 0.90 0.95  482 99 36 16 7  645 132 47 20 9  0.70  0.75 0.80 0.85 0.90 0.95  482 99 36 16 7  644 132 47 20 9  0.75  0.80 0.85 0.90 0.95  323 64 22 9  432 84 28 11  0.75  0.80 0.85 0.90 0.95  323 64 21 9  431 84 28 11  0.75  0.80 0.85 0.90 0.95  322 64 21 9  431 84 28 11  0.80  0.85 0.90 0.95  195 36 11  261 47 14  0.80  0.85 0.90 0.95  195 36 11  260 47 14  0.80  0.85 0.90 0.95  195 36 11  260 47 14  0.85  0.90 0.95  100 16  132 20  0.85  0.90 0.95  99 16  132 20  0.85  0.90 0.95  99 16  132 20  (continued on next page)  94  www.mjms.usm.my  
  original article sample size for cronbach's alpha table 2. (continued) k  ca0  ca1  na  nb  k  ca0  ca1  na  nb  k  ca0  ca1  na  nb  75  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  36 27 17 13 11 9 7 6 4  47 36 22 17 14 11 8 7 5  80  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  36 27 17 13 11 9 7 5 4  47 36 22 17 14 11 8 7 5  85  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  36 27 17 13 11 9 7 5 4  47 36 22 17 14 11 8 7 5  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  128 63 36 21 13 9 6  170 84 47 28 17 11 7  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  127 63 36 21 13 9 5  170 84 47 28 17 11 7  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  127 63 36 21 13 9 5  170 84 47 28 17 11 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  254 99 49 27 16 10 6  340 132 64 35 20 12 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  254 99 49 27 16 10 6  339 132 64 35 20 12 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  254 99 48 27 16 10 6  339 132 64 35 20 12 7  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  895 195 75 36 19 11 6  1,197 260 99 47 25 14 7  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  894 195 74 36 19 11 6  1,196 260 99 47 25 14 7  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  893 194 74 36 19 11 6  1,195 259 99 47 25 14 7  0.65  0.70 0.75 0.80 0.85 0.90 0.95  672 143 53 25 13 7  899 191 71 32 16 8  0.65  0.70 0.75 0.80 0.85 0.90 0.95  671 143 53 25 13 7  898 190 70 32 16 8  0.65  0.70 0.75 0.80 0.85 0.90 0.95  671 143 53 25 13 7  897 190 70 32 16 8  0.70  0.75 0.80 0.85 0.90 0.95  481 99 36 16 7  643 132 47 20 9  0.70  0.75 0.80 0.85 0.90 0.95  481 99 36 16 7  643 132 47 20 9  0.70  0.75 0.80 0.85 0.90 0.95  480 99 36 16 7  642 132 47 20 9  0.75  0.80 0.85 0.90 0.95  322 63 21 9  430 84 28 11  0.75  0.80 0.85 0.90 0.95  322 63 21 9  430 84 28 11  0.75  0.80 0.85 0.90 0.95  322 63 21 9  430 84 28 11  0.80  0.85 0.90 0.95  195 36 11  260 47 14  0.80  0.85 0.90 0.95  195 36 11  260 47 14  0.80  0.85 0.90 0.95  194 36 11  259 47 14  0.85  0.90 0.95  99 16  132 20  0.85  0.90 0.95  99 16  132 20  0.85  0.90 0.95  99 16  132 20  (continued on next page)  www.mjms.usm.my  95  
  malays j med sci. nov–dec 2018; 25(6): 85–99  table 2. (continued) k  ca0  ca1  na  nb  k  ca0  ca1  na  nb  k  ca0  ca1  na  nb  90  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  36 27 17 13 11 9 7 5 4  47 36 22 17 14 11 8 7 5  95  0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  36 27 17 13 11 9 7 5 4  47 36 22 17 14 11 8 7 5  100 0.00  0.50 0.55 0.65 0.70 0.75 0.80 0.85 0.90 0.95  36 27 17 13 11 9 7 5 4  47 36 22 17 14 11 8 7 5  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  127 63 36 21 13 9 5  170 84 47 28 17 11 7  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  127 63 36 21 13 9 5  169 84 47 28 17 11 7  0.50  0.65 0.70 0.75 0.80 0.85 0.90 0.95  127 63 36 21 13 9 5  169 84 47 28 17 11 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  254 99 48 27 16 10 6  339 132 64 35 20 12 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  254 99 48 27 16 10 6  339 132 64 35 20 12 7  0.55  0.65 0.70 0.75 0.80 0.85 0.90 0.95  254 99 48 27 16 10 6  339 132 64 35 20 12 7  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  893 194 74 36 19 11 6  1,194 259 99 47 25 14 7  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  892 194 74 36 19 11 6  1,194 259 99 47 25 14 7  0.60  0.65 0.70 0.75 0.80 0.85 0.90 0.95  892 194 74 36 19 11 6  1,193 259 99 47 25 14 7  0.65  0.70 0.75 0.80 0.85 0.90 0.95  671 143 53 25 13 7  897 190 70 32 16 8  0.65  0.70 0.75 0.80 0.85 0.90 0.95  670 143 53 25 13 7  896 190 70 32 16 8  0.65  0.70 0.75 0.80 0.85 0.90 0.95  670 143 53 25 13 7  896 190 70 32 16 8  0.70  0.75 0.80 0.85 0.90 0.95  480 99 36 16 7  642 132 47 20 9  0.70  0.75 0.80 0.85 0.90 0.95  480 99 36 16 7  641 132 47 20 9  0.70  0.75 0.80 0.85 0.90 0.95  480 99 36 16 7  641 132 47 20 9  0.75  0.80 0.85 0.90 0.95  321 63 21 9  429 84 28 11  0.75  0.80 0.85 0.90 0.95  321 63 21 9  429 84 28 11  0.75  0.80 0.85 0.90 0.95  321 63 21 9  429 84 28 11  0.80  0.85 0.90 0.95  194 36 11  259 47 14  0.80  0.85 0.90 0.95  194 36 11  259 47 14  0.80  0.85 0.90 0.95  194 36 11  259 47 14  0.85  0.90 0.95  99 16  132 20  0.85  0.90 0.95  99 16  132 20  0.85  0.90 0.95  99 16  132 20  96  www.mjms.usm.my  
  original article sample size for cronbach's alpha table 3. sample size tables for comparisons of two cronbach’s alpha test with various effect sizes, alpha 0.05 and power are set to 80.0% (na) and 90.0% (nb), respectively k  ρa  ρb  na  nb  k  ρa  ρb  na  nb  k  ρa  ρb  na  nb  2  0.5  0.6 0.7 0.8 0.9 0.8 0.9  1264 243 77 27 384 55  1691 325 103 35 514 72  8  0.5  0.6 0.7 0.8 0.9 0.8 0.9  723 140 45 16 221 32  967 187 60 21 295 42  30  0.5  0.6 0.7 0.8 0.9 0.8 0.9  655 127 41 15 200 29  876 169 54 19 267 39  0.6 0.7 0.8 0.9 0.8 0.9  948 183 59 21 289 42  1269 244 78 27 386 55  9  0.6 0.7 0.8 0.9 0.8 0.9  712 138 45 16 217 32  952 184 59 21 290 42  40  0.6 0.7 0.8 0.9 0.8 0.9  649 126 41 15 198 29  868 168 54 19 265 38  0.6 0.7 0.8 0.9 0.8 0.9  843 163 52 19 257 37  1128 217 69 24 343 49  10  0.6 0.7 0.8 0.9 0.8 0.9  703 136 44 16 215 31  940 181 58 21 287 41  50  0.6 0.7 0.8 0.9 0.8 0.9  646 125 41 15 197 29  864 167 54 19 263 38  0.6 0.7 0.8 0.9 0.8 0.9  791 153 49 18 241 35  1058 204 65 23 322 46  15  0.6 0.7 0.8 0.9 0.8 0.9  678 131 43 15 207 30  907 175 56 20 276 40  60  0.6 0.7 0.8 0.9 0.8 0.9  644 125 41 15 197 29  861 166 53 19 262 38  0.6 0.7 0.8 0.9 0.8 0.9  759 147 47 17 232 34  1015 196 63 22 309 44  20  0.6 0.7 0.8 0.9 0.8 0.9  666 129 42 15 204 30  891 172 55 20 272 39  70  0.6 0.7 0.8 0.9 0.8 0.9  642 125 40 15 196 29  859 166 53 19 262 38  0.5  0.6 0.7 0.8 0.9  738 143 46 17  987 190 61 21  25  0.5  0.6 0.7 0.8 0.9  659 128 41 15  882 170 55 19  80  0.5  0.6 0.7 0.8 0.9  641 124 40 15  857 166 53 19  0.7  0.8 0.9  225 33  301 43  0.7  0.8 0.9  201 30  269 39  0.7  0.8 0.9  196 29  261 38  0.7 3  0.5  0.7 4  0.5  0.7 5  0.5  0.7 6  0.5  0.7 7  0.7 0.5  0.7 0.5  0.7 0.5  0.7 0.5  0.7  the present article recommends testing hypothesis with ca0 0.5 is necessary to test whether an instrument has an excellent internal consistency. however, it is a choice by the researchers to set the value of ca0, including setting ca0 as equals to zero. but, solely targeting a large difference of cronbach’s alphas between ca0 and ca1 to get a lower sample size is not recommended. therefore, justification is needed whether testing such effect size is scientifically necessary.  0.7 0.5  0.7 0.5  0.7 0.5  0.7 0.5  0.7  sample size determination when to compare two cronbach’s alpha in two different groups hypothesis testing to compare two cronbach’s alphas in two different groups is also rare in literature. however, this hypothesis testing is useful if the researcher aims to compare two coefficients of cronbach’s alphas in two different groups. although some statistical softwares might not provide statistical test to assess such hypothesis, researchers can calculate and determine sufficient sample size for the planned study to compare two cronbach’s alphas in two different groups. www.mjms.usm.my  97  
  malays j med sci. nov–dec 2018; 25(6): 85–99  for example, when k 3 and ρa 0.50, based on alpha 0.05 and power 80.0%, a minimum sample size of 183 is sufficient to detect ρb at 0.70. a lower minimum sample size is needed (n 59) to detect ρb at 0.80 (table 2). an example of a sample size statement is as followed: “the aim of the study is to determine whether the internal consistency of particular questionnaire “b” in group “two” is higher than in group “one”. questionnaire “b” has 15 items with a 5-point likert scale for every item. the coefficient of cronbach’s alpha in group “one” was reported at 0.70 while the coefficient of cronbach’s alpha in group “two” is hypothesised to be at least 0.80. based on alpha of 0.05, the minimum sample size requirement is 207 to be able to detect at least 80.0% power of the test. the calculation is based on the formula introduced by bonett (12).”  conclusion this article provides sample size tables and simple guides for sample size estimation for cronbach’s alpha. this guideline is useful for researchers especially in medical field where the recommended tables will ease them in estimating sample size for reliability studies. for one coefficient of cronbach’s alpha test, the assumption of “the ca0 equals to zero” will usually yield a smaller sample size, and that is appropriate for pilot studies. however, to ensure an instrument has an excellent internal consistency, testing the hypothesis with ca0 larger than zero (ca0 0), such as ca0 0.5 is recommended. besides that, the calculation of sample size based on comparing two cronbach’s alpha values in two different groups is also presented. justifications are needed to decide whether the testing for extremely low or extremely large effect sizes are scientifically necessary. hopefully, this guideline could help the researchers in sample size estimation for their researches to assess the reliability of an instrument.  conflicts of interest none.  funds none.  authors’ contributions conception and design: mab analysis and interpretation of the data: mab, edo, nab drafting of the article: mab, edo, nab critical revision of the article for important intellectual content: mab, edo, nab final approval of the article: mab, edo, nab statistical expertise: mab, edo, nab  correspondence dr mohamad adam bujang research officer bsc (hons) statistics (universiti teknologi mara), mba (universiti teknologi mara), phd (it and quantitative sciences) (universiti teknologi mara) clinical research centre, sarawak general hospital, ministry of health, jalan tun ahmad zaidi adruce, 93586 kuching, sarawak, malaysia. tel: 6082 276820 fax: 6082 276823 e-mail: adam@crc.gov.my  references 1.  cronbach, lj. coefficient alpha and the internal structure of tests. psychometrika. 1951;16(3):297–334. https://doi.org/10.1007/ bf02310555  2.  gliem ja, gliem rr. calculating, interpreting, and reporting cronbach’s alpha reliability coefficient for likert-type scales. midwest research-to-practice conference in adult, continuing, and community education. 2003. retrieved from https://scholarworks.iupui.edu/ bitstream/handle/1805/344/gliem & gliem.pdf? sequence 1.  3.  leontitsis a, pagge j. a simulation approach on cronbach's alpha statistical significance. math comput simul. 2007;73(5):336–340. https:// doi.org/10.1016/j.matcom.2006.08.001  acknowledgements we would like to thank the director general of health malaysia for his permission to publish this article. we would also like to thank dr lee keng yee for her effort in proofreading the manuscript and ms chien joo lim for processing the manuscript for publication.  98  www.mjms.usm.my  
  original article sample size for cronbach's alpha 4.  lovibond sh, lovibond pf. manual for the depression anxiety stress scales. 2nd ed. sydney: psychology foundation; 1995.  5.  berg ko, wood-dauphinee sl, williams jt, et al. measuring balance in the elderly: validation of an instrument. can j public health. 1992;83(2):s7– s11.  6.  7.  8.  9.  bonett, dg. sample size requirements for testing and estimating coefficient alpha. j educ behav stat. 2002;27(4):335–340. https://doi. org/10.3102/10769986027004335 green s, lissitz r, mulaik s. limitations of coefficient alpha as an index of test unidimensionlity. educ psychol meas. 1977;37(4):827–838. https://doi.org/10.1177/ 001316447703700403 cortina j. what is coefficient alpha: an examination of theory and applications. j appl psychol. 1993;78(1):98–104. https://doi.org/10. 1037/0021-9010.78.1.98 schmitt n. uses and abuses of coefficient alpha. psychol assess. 1996;8(4):350–353. https://doi. org/10.1037/1040-3590.8.4.350  10. bland j, altman d. statistics notes: cronbach's alpha. bmj. 1997;314:572. https://doi.org/10. 1136/bmj.314.7080.572 11. streiner d. starting at the beginning: an introduction to coefficient alpha and internal consistency. j pers assess. 2003;80(1):99–103. https://doi.org/10.1207/s15327752jpa8001_18 12. bonett, dg. sample size requirements for comparing two alpha reliability coefficients. appl psychol meas. 2003;27(1):235–240. https://doi. org/10.1177/0146621602239477  14. bonett dg, wright ta. cronbach’s alpha reliability: interval estimation, hypothesis testing, and sample size planning. j organ behav. 2015;36(1):3–15. https://doi.org/10.1002/job.1960 15. conroy, r. sample size: a rough guide. ethics (medical research) committee; 2015. retrieved from: http://www.beaumontethics.ie/docs/ application/samplesizecalculation.pdf. 16. samuels p. statistical methods–scale reliability analysis with small samples. birmingham: birmingham city university, centre for academic success; 2015. 17. jalaludin my, fuziah mz, hong jyh, mohamad adam b, jamaiyah h. reliability and validity of the revised summary of diabetes self-care activities (sdsca) for malaysian children and adolescents. malaysian family physician. 2012;7(2–3):10–20. 18. jalaludin my, fuziah mz, hadhrami mz, hong jyh, mohamad adam b, jamaiyah h. reliability and validity of the malay translated version of diabetes quality of life for youth questionnaire. malaysian family physician. 2013;8(1):12–19. 19. bujang ma, ismail m, mohd hatta nkb, baharum n, othman ss, mat lazim ss, shah sa. malaysian journal of public health medicine. 2016;16(3):227–234. 20. bujang ma, ismail m, mohd hatta nkb, othman sh. baharum n, mat lazim ss. validation of the malay version of diabetes quality of life (dqol) questionnaire for adult population with type 2 diabetes mellitus. malays j med sci. 2017;24(4):86–96. https://doi.org/10.21315/ mjms2017.24.4.10  13. yurdugul h. minimum sample size for cronbach's coefficient alpha: a monte carlo study. hacettepe egitim dergisi. 2008;35:397–405.  www.mjms.usm.my  99  
       


