- Social Sciences
- African Studies
- American Studies
- Asian Studies
- Communication Sciences
- Ethnic Studies
- European Studies
- Gender Studies
- Physical Sciences
- Life Sciences
- Animal Communications
- Cell Biology
- Evolutionary Biology
- Food Science and Technology
- Human Anatomy
Motivated Scientific Reasoning Biases, Epistemological Beliefs, and Theory Polarization: A Two-Process Approach to Adolescent Cognition
by Paul A. Klaczynski
Motivated Scientific Reasoning Biases, Epistemological Beliefs, and Theory Polarization: A Two-Process Approach to Adolescent Cognition
Paul A. Klaczynski
Updated: March 4th, 2013
Child Development, September /October 2000, Volume 71, Number 5, Pages 1347-1366
Motivated Scientific Reasoning Biases, Epistemological Beliefs, and Theory Polarization: A Two-Process Approach to Adolescent Cognition
Paul A. Klaczynski
Theory-motivated reasoning biases arise when different reasoning skills are invoked to evaluate evidence that is congruent or incongruent with individuals' belief systems. To explore this phenomenon, 66 early and 73 middle adolescents evaluated evidence relevant to their theories of social class or religion. In both conditions, reasoning biases were found, but in-group biases were evident only in the religion condition. In both condi- tions, higher order scientific reasoning was used to reject theory-incongruent evidence and judgmental heuris- tics (i.e., cognitive rules of thumb) were used to evaluate theory-congruent evidence. In both conditions, subse- quent to the evidence presentation, adolescents' theories became more extreme (i.e., polarized) than at the outset of the experiment. Beliefs regarding the origin, acquisition, and certainty of knowledge, however, ap- peared to moderate reasoning biases and theory polarization. Age differences emerged on only one index of bias: In the religion condition, middle adolescents were more likely to treat theory-incongruent evidence as im- plausible. These findings are pertinent to theories of cognitive development, decision making, rationality, and
Cognitive development has traditionally been con- ceived as a unidirectional progression from relatively intuitive reasoning to computationally complex, ratio- nal, and objective reasoning (e.g., Case, 1998; Mosh- man, 1998). Analytic reasoning competence, involv- ing the abilities to weigh decision-relevant costs and benefits appropriately, to deduce logical relationships among goals, decisions, and plans, and to reason sci- entifically, has long been considered the pinnacle of adolescent cognitive development (Furby & Beyth-Marom, 1992; Inhelder & Piaget, 1958). These abili- ties, in concert with the ability to "decontextualize" (Stanovich & West, 1997) prior knowledge and beliefs from a task's logical demands, are believed to consti- tute the core of adolescents' emerging rationality.
Decontextualization is particularly important in everyday reasoning because it increases the like- lihood that analytic reasoning competencies-higher order, content-independent reasoning abilities, such as those described above-will be used to evaluate social data. However, in contrast to traditional views of cognitive development, two-process theories assume that a heuristic reasoning system, which relies on "self-evident truths" (Epstein, 1994), stereotypes (Slo- man, 1996), intuitions (Reyna & Brainerd, 1995), and vivid, readily available memories and on judgmental heuristics (see Kahneman, Slovic, & Tversky, 1982), co-develops with analytic reasoning. Heuristic rea- soning often predominates when beliefs or "personal theories" interfere with the processing of relevant ev- idence (Stanovich, 1999), in part because such theo-
ries are not treated as cognitive entities open to criti- 1988).
cal introspection (Kuhn, Lmsel, & ~'~oughlin,
In the present context, personal theories are con- ceived as networks of interrelated propositions that speak to the causal effects of belonging to specific social categories. For example, one might believe that the current welfare system promotes laziness, child abuse, and religious fundamentalism. Like scientific theories, personal theories of the causal and correlational rela- tionships in a social domain serve as explanatory mech- anisms that are used to predict behaviors and attitudes, generate and test hypotheses, and guide evidence in- terpretation. Theory-motivated reasoning arises when individuals fail to reason independently from their theories. Consequently, different reasoning strategies and principles are used as a function of theorylevi- dence congruence to preserve a theory's integrity. Theory-motivated reasoning thus entails holding log- ically isomorphic evidence to qualitatively different standards.
Only recently have researchers become concerned with the development of motivated reasoning. For in- stance, Klaczynski and Narasirnham (1998) found that adolescents used sophisticated scientific reason- ing skills to dismiss evidence that threatened their the- ories of religion. Presented evidence that supported their theories, adolescents relied on cursory justifica- tions that enabled uncritical assimilation of the evi- dence. In their study of adolescents' theories of ho- mosexuality, Kardash and Scholes (1996) found
2000 by the Society for Research in Child Development, Inc. All rights reserved. 0009-3920/2000/7105-0019
similar fluctuations between analytic reasoning on theory-incongruent data and heuristic reasoning on theory-congruent data.
The goals of the present research were both empiri- cal and theoretical. Below, I describe a model intended to illustrate the cognitive-motivational mechanisms underlying theory-biased reasoning and the relation- ship of motivated reasoning to analytic and heuristic information processing. Although various facets of this model have received empirical support, other model-based hypotheses have not yet been tested. The present research was thus intended to expand the model's empirical basis.
A General Model of Theory-Motivated Reasoning
Two-process theorists (e.g., Epstein, 1994; Evans & Over, 1996; Sloman, 1996; Stanovich, 1999) speculate that decision making and reasoning are dependent on two information-processing systems. Analytic process- ing is consciously controlled, effortful, and deliberate and relies on the abilities (e.g., scientific reasoning) that are frequently considered the apex of cognitive devel- opment and that are often normatively prescribed for sound decision making.
Heuristic system processing is preconscious, rapid, and effortless. Judgments derived through the func- tioning of this system are subjectively appealing be- cause they "feel" intuitively correct, are consistent with stereotype-based and theory-based beliefs, and therefore appear self-evidently true (Epstein, 1994). This system also relies on an assortment of judgmen- tal heuristics, including the "implausibility" heuristic (described subsequently), as a basis for reasoning and problem solving. Although the heuristic and analytic processing systems are interdependent, contextual variables (e.g., time constraints) determine which sys- tem will be predominant in a given situation.
The model in Figure 1illustrates the relevance of the analytic/ heuristic distinction to motivated reasoning. The strength and resilience of theories is dependent on ego and emotional investments in existing beliefs (Klaczynski & Narasirnham, 1998), the extent to which theoretical propositions are treated as "truths" rather than as testable hypotheses (Kuhn et al., 1988), and the general motivation to conserve cognitive en- ergy. At the first processing step, relevant information activates theoretical beliefs. A preconscious check then ensues during which theory-data congruence is examined. The outcome of this examination deter- mines which of three paths subsequent information processing will follow.
Incongruent evidence typically leads to analytic processing because the dominant motivation for eval- uating such evidence is discerning means by which it can be refuted, rejected, or reinterpreted. The adoles- cent theorist seeks the most accurate refutation he or she can construct to protect the assaulted belief system and, in some cases, identity and self-esteem. Conse- quently, processing is thorough, problem representa- tions are based on logical structure (Reyna & Brainerd, 1991), and context-independent reasoning abilities are activated. That principle deemed most appropri- ate is used to reject the evidence in a seemingly ratio- nal manner (Kunda, 1990).
Evidence depicting adolescents, the groups to which they belong (e.g., a specific religion), or their belief systems favorably is processed heuristically. Typically, theory-congruent data are assimilated into existing theories with minimal cognitive expenditure. The acceptance of such evidence as "correct" is often justified by means of readily available memories, ste- reotypes, and heuristics. Theories thus serve as infor- mation filters (Krull & Anderson, 1997) that interfere with the consistency of analytic reasoning and, conse- quently, inhibit objectivity.
Both processing paths function to preserve and strengthen existing belief systems. Having "discov- ered" supportive evidence and having rejected logi- cally isomorphic falsifying evidence, adolescents may construct the unwarranted belief that their theories are stronger than they were prior to evidence evalua- tion (see, however, Kuhn & Lao, 1996). This phenom- enon, belief polarization (Lord, Ross, & Lepper, 1979), may be particularly likely among those adolescents whose reasoning is most biased (i.e, who defend their theories most tenaciously).
As depicted in Figure 1, there is a third path by which evidence can be evaluated. Koslowski (1996) has argued that incongruent evidence may be legiti- mately rejected when it is theoretically implausible. However, arguments that data are implausible may be selective. Specifically, because individuals in gen- eral and adolescents in particular are oriented toward theory preservation and toward the maximization of cognitive resources (Reyna & Brainerd, 1995), theory- incongruent data may sometimes be processed heu- ristically and may be rejected on the basis of superfi- cial claims of implausibility.
For instance, given the belief that low socioeco- nomic status leads to criminal activity, evidence that higher socioeconomic status (SES) persons commit more crimes than lower SES persons may be rejected because "it just doesn't make sense" (see also Perkins, 1985). The reasoner thus goes beyond the evidence, does not analyze the data for its logical quality, and nonetheless preserves his or her theory with the un- principled assertion that the evidence is inconceiv-
Yes 4 -b No
Heuristic processing Analytic processing Heuristic processing
Gist-based Logical Gist-based
representation, representation, representation,
low-accuracy high accuracy low accuracy
motivation, low motivation, high motivation, low
Search for/activation Activation of
of higher-order heuristics
evidence into theory
reasoning strategies (e.g.,implausibility)
Valid rejection of Rejection of evidence
evidence based on based on unprincipled,
principled, scientific cognitively "cheap"
Figure 1 A model of theory-motivated reasoning.
able. Unless supported by a theoretically meaning-Individual Differences in Reasoning Biases ful rationale, implausibility claims provide reasoners a "quick, cheap, and dirty" heuristic that enables the Thus far, adolescents have been portrayed as cog-rejection of unfavorable data at minimal cognitive nitive conservationists who rely heavily on their per-cost. sonal theories to evaluate information. This approach
assumes that most adolescents subordinate epistemic belief systems to that domain-specific personal theory activated by a given data set. However, this character- ization is probably inaccurate for some adolescents. Specifically, for some adolescents, beliefs regarding the nature, certainty, and acquisition of knowledge may be more influential than personal theories in evidence evaluation. Such beliefs are largely metacognitive be- cause the course of one's own reasoning must be mon- itored and self-regulated to achieve various epistemic goals. Metacognitive dispositions related to intellec- tual self-regulation include reflectiveness, openmindedness, and willingness to scrutinize one's knowl- edge, reevaluate one's opinions, postpone closure, and recognize that theories must sometimes be relin- quished or revised to acquire knowledge (Kruglanski, Webster, & Klem, 1993; Perkins, Jay, & Tishrnan, 1993). These characteristics constitute a thinking style in which the goal of theory preservation is subordi- nated to the goal of knowledge acquisition.
Epistemic beliefs are distinct from personal theories
in several ways, the foremost of which are scope and content. Social theories are domain specific; epistemic beliefs are typically domain general (Schommer, 1994). Individual differences in epistemic beliefs may be linked to differences in reasoning biases because of differences in the priorities assigned to theory preser- vation and to knowledge acquisition. For instance, close-minded adolescents are likely to treat their beliefs as if they are factual (see Kuhn et al., 1988). Adolescents who believe that they should "stick to their guns" when their beliefs are threatened, who devalue objectivity, and whose self-esteem is tied to the truth of their the- ories may be particularly vulnerable to theory-biased reasoning. Opposed to these "belief-driven" adoles- cents are "knowledge-driven" adolescents, whose be- liefs in logical data analysis, the uncertainty of knowl- edge, and the importance of scrutinizing their beliefs should lead to relatively unbiased reasoning. As indi- cated in Figure 2, these adolescents are likely to pro- cess both theory-congruent and theory-incongruent evidence analytically.
|No||t||,||Knowledge acquisition goals stronger than theory preservation goals?||)||Yes|
|Selective heuristic and A analytic processing||Analytic processing as in Figure 1|
Acceptance /refutation based on evidence quality
Figure 2 Data processing path for "knowledge-driven" adolescents.
The Present Investigation
The present research was intended to test the model presented in Figure 1and the foregoing hypoth- esis regarding epistemic beliefs and reasoning biases and to examine age differences in reasoning biases in two domains, social class and religion. In each domain, biased use of the implausibility heuristic and of two analytic reasoning competencies was explored. The ability to apply the "law of large numbers" (LLN)- the statistical precept that the certainty of an infer- ence about a population increases as the size of evi- dential samples drawn from that population increase (Kahneman & Tversky, 1972; Piaget & Inhelder, 1951 / 1975)-was one analytic competency. The ability to evaluate and interpret scientific evidence was the sec- ond analytic competency (Inhelder & Piaget, 1958). This competence was operationalized as the extent to which threats to the internal validity of hypothetical re- search were detected (subsequently called "experiment- evaluation reasoning"). Both competencies are, to some extent, developed by early adolescence (Koslowski, 1996; Piaget & Inhelder, 1975).
The utility of the model would be enhanced con- siderably if each of the following questions is an- swered affirmatively. Do theories have similar biasing effects on experiment-evaluation reasoning, LLN rea- soning, and implausibility claims in both social do- mains? Are these forms of motivated reasoning posi- tively correlated with each other and, if so, does a common factor underlie biases in experiment-evaluation reasoning, LLN reasoning, and the implausibility heu- ristic? Are biases moderated by epistemological dispo- sitions? Following the evaluation of evidence, do theo- ries polarize? If so, is polarization related to age, epistemological beliefs, and reasoning biases? Are middle adolescents as biased as early adolescents?
Although affirmation of this last question would support the null hypothesis, its importance derives from theories in which the decontextualization of be- liefs and content from reasoning is considered a cen- tral characteristic of cognitive maturity (e.g., Donald- son, 1978; Inhelder & Piaget, 1958). Repeated findings that motivated reasoning biases are prevalent and do not diminish with age could be taken to indicate that cognitive maturity-in terms of the proclivity to rea- son independently from beliefs-is not often attained.
The research presented herein examined the rela- tionship between adolescents' theories of social classes and reasoning biases and between theories of religious affiliations and reasoning biases. By investi- gating two distinct social domains, the generalizability of theory/ reasoning relationships to groups to which adolescents' did and did not belong could be deter-
mined. Specifically, participating adolescents were members of specific religions. However, although they had wide-ranging theories regarding the effects of belonging to different social classes, participants were generally not members of either the upper mid- dle class or the working class. As such, theory-biased reasoning was anticipated in both domains, but in- group favoritist reasoning was anticipated only in the religion condition.
For each social domain, problems were either fa- vorable toward one social group (the upper middle class or each participant's religion), unfavorable toward one group, or neutral toward the groups involved. To assess experiment-evaluation and LLN reasoning, both ratings and justifications of the quality of hypo- thetical experiments and of hypothetical arguments (in the LLN problems) were obtained.
Biases in each ability and in the implausibility heu- ristic are indicated by "task variability" (Reyna, 1992, 1995): moment-to-moment changes in the complexity of reasoning on tasks that are logically isomorphic but which have different superficial properties. Biases, therefore, were operationalized by difference scores. First, upper-middle-class favorable and own-religion favorable rating biases were calculated by subtracting ratings of upper-middle-class unfavorable or religion- unfavorable evidence from ratings of favorable evidence. Positive difference scores would indicate biases toward the upper-middle class or toward par- ticipants' religions.
Because implausibility responses and complex jus- tifications result in data rejection, biases favoring the upper middle class or participants' religions were in- dicated by more complex justifications and more im- plausibility responses on unfavorable problems than on favorable problems. Therefore, justification biases and implausibility biases were computed by subtract- ing scores on favorable problems from scores on un- favorable problems. For both types of bias, positive difference scores indicated biases toward the upper middle class or toward participants' religions. Partic- ipants thus had six bias scores-LLN rating, LLN jus- tification, and LLN implausibility bias scores; and ex- periment-evaluation rating, experiment-evaluation justification, and experiment-evaluation implausibil- ity bias scores.
Many of the study's hypotheses can be derived di- rectly from Figure 1; only less readily apparent hy- potheses are presented here. First, because the same mechanisms are believed to underlie the production of each type of bias, significant intercorrelations were ex- pected among the bias indices; bias indices were there- fore expected to load on a single factor. Second, the theories of adolescents who more vigorously defended their beliefs, as indicated by more biased reasoning, were expected to polarize more than the theories of other adolescents. Third, because "knowledge-driven" adolescents subordinate theory maintenance to knowl- edge acquisition (see Figure 2), both reasoning biases and belief polarization were expected to relate nega- tively to epistemological beliefs.
In addition, the competence to reason scientifically was expected to be greater among the older adoles- cents. Because the conclusions drawn in each prob- lem were based on weak evidence (sample size in the LLN problems and validity threats in the experiment- evaluation problems), total competence scores could be computed for both the LLN and the experiment- evaluation problems. In each case, lower ratings of evidence quality and more complex justifications in- dicate greater competence than higher ratings and less complex justifications. Rating competence scores and justification competence scores, then, were oper- ationalized by adding ratings and by adding justifica- tion scores across the favorable, unfavorable, and neutral problems. For example, on a 9-point scale, a competent reasoner might provide an evidence qual- ity rating of 3 on a favorable problem and a quality rating of 1on an unfavorable problem. A less compe- tent reasoner might give the same favorable problem a rating of 9 and the unfavorable problem a rating of
7. Note that the rating bias score-indicated by the difference between ratings on favorable and unfavor- able problems-is the same for both individuals (i.e., 2), despite differences in competence scores (i.e., 4 versus 16). As this example illustrates, reasoning competence and reasoning biases may be indepen- dent from one another.
Reasoning biases are assumed to result from in- creased analytic processing on theory-incongruent problems, rather than the suppression of analytic pro- cessing on theory-congruent problems. Performance on neutral problems provides a baseline against which this assumption can be tested (see Klaczynski & Gordon, 1996b). Specifically, reasoning on neutral problems should be less analytic than reasoning on incongruent problems.
Thirty-one early adolescents (14 male, 17 female; M = 13.4 years, SD = 1.2) and 35 middle adolescents (17 male, 18 female; M = 16.8 years, SD = 1.5) were in the social-class condition. Thirty-five early (16 male, 19 female; M = 13.9 years, SD = 1.3) and 38 middle adolescents (20 male, 18 female; M = 16.6 years, SD = 1.4) were in the religion condition. All participants were European American. The early adolescents were seventh- and eighth-grade students from a public junior high school; the middle adolescents were tenth- and eleventh-grade students from a public high school. Nine participants expressed suspicion (four in the social-class condition; five in the religion condition) and were dropped from the analyses. Sam- ple sizes were thus 62 for the social-class condition and 68 for the religion condition. Following debrief- ing, each participant was paid $10.
Mean self-reported social class (1 = low social class, 2 = lower middle class, 3 = middle class, 4 = upper middle class, 5 = upper social class) was 3.34, SD = 1.36, in the social-class condition and 3.41, SD = 0.84, in the reli- gion condition. In the social-class condition, mean levels of maternal and paternal education (indicated on a 7-point scale: 1 = less than high school; 7 = advanced degree, e.g., MD) were: 4.79, SD = 1.46, range = 2-7 and 5.24, SD = 1.39, range = 2-7. In the religion condition, maternal and paternal educational levels were 4.62, SD = 1.38, and 4.55, SD = 1.69, respec- tively. In neither condition did the age groups differ significantly on these variables.
Testing was conducted with groups of two to seven adolescents. On the first day of testing, partici- pants completed self-report measures of epistemo- logical dispositions (ED), social class and religion the- ories, and religious affiliations. To reduce suspicion, the Primary Mental Abilities Vocabulary and Letter Sets tests (Thurstone, 1962) were also administered (correlations between reasoning biases and ability were not significant; these may be obtained from the author). Participants were told that the study in- volved the relationship between personality and intelligence. To further avoid suspicion, the second session was conducted 5 to 8 days after the initial ses- sion. Participants were then told that they were talung part in a second experiment that dealt with thinking about social classes or about religion.
In each condition, nine experiment-evaluation problems and nine LLN problems were administered. Half of the participants received the LLN problems first and half received the experiment-evaluation problems first; presentation order had no significant effects. In the social-class condition, six problems (three LLN, three experiment-evaluation) were upper middle class favorable, six were upper middle class unfavorable, and six were social class neutral. In the religion condition, six problems (three LLN, three experiment-evaluation) were religion favorable, six were religion unfavorable, and six were religion neu- tral. Within the experiment-evaluation and LLN problem packets, problems were administered in ran- dom order to each participant.
Following each set of problems, the theory question- naires were administered again. Questionnaire items were presented in one of three randomly determined orders counterbalanced across the initial session, fol- lowing the experiment-evaluation problems, and following the LLN problems. Initial sessions lasted between 1 hr and 1%hrs; second sessions lasted ap- proximately 1hr 20 min.
Theoretical beliefs. Two 10-item scales assessed theo- retical beliefs. On the social-class theory questionnaire, participants rated the extent to which belonging to the upper-middle class, relative to the working class, "makes" people "good parents," "happy with life," "enjoy work," "know the morally right thing to do," and "self-confident." The questionnaire also contained five negatively valenced items (have bad marriages, commit crimes, less creative and artistic, depressed, and use illegal drugs). The religion questionnaire as- sessed the extent to which participants believed that belonging to their religions, relative to other religions, increased responsibility, life satisfaction, getting along with other people, a sense of right and wrong, and good parenting. The remaining five items (de- pression, anxiety, divorce, lack of willpower, and ag- gressiveness) were negatively valenced.
On both measures, items were rated on 11-point scales (-5 = has an extremely negative efect; 0 = has no ef- fect; +5 = has an extremely positive efect). To make all scores positive, a constant of 5 was added to each rat- ing. Both social-class and religion scores, therefore, could range from 0 to 110 (actual range, social-class the- ories = 22-102, M = 59.83, SD = 19.81; actual range, re- ligion theories = 14-101, M = 71.49, SD = 25.68). For social-class theories, a = 27, at first administration, .83 at second administration, and .85 at third administra- tion. For religion theories, a = .95, .78, and .76 at first, second, and third administrations, respectively.
Participants in both conditions completed both the- ory questionnaires to provide a control against which polarization of social-class theories (or religion theo- ries) could be compared. If polarization is due to re- peated testing or to mere exposure to evidence, then in the social-class condition, religion theories should be- come as polarized as social-class theories (and vice versa in the religion condition). Similarly, if polariza- tion is due to repeated testing or to mere exposure to evidence, social-class theories in the religion condition
should change as much as social-class theories in the social-class condition (and religion theories in the social- class condition should change as much as religion the- ories in the religion condition). Statistical analyses failed to support these alternative explanations.
Epistemological dispositions. Three measures were used to index ED. The 18-item (a = .90) short form of the Need for Cognition (NFC) scale (Cacioppo & Petty, 1982) measures tendencies to seek challenging intellectual experiences and to enjoy the pursuit of knowledge. A sample item is, "I prefer my life to be filled with puzzles that I must solve." Each item is rated on a 9-point scale (1 = strongly disagree; 9 = strongly agree).
The Belief Defensiveness scale (BD) contains 17 items (a = 36) designed to assess openness to belief revision (Klaczynski, Fauth, & Swanger, 1998). For ex- ample, "Keeping my beliefs even if most other people don't believe them." Items are rated on a 1 (not at all important) to 10 (extremely important) scale.
The 42-item Need for Closure scale (NFCL) (Kruglanski et al., 1993) assesses tendencies to avoid uncertain states of knowledge and to be uncomfort- able with contradictory "truths" and ill-defined prob- lems (a = 232). A sample (reversed-scored) item is, "Even after I've made up my mind about something, I am always eager to consider a different position." Items are rated on a 6-point scale (1 = strongly dis- agree; 6 = strongly agree). The NFCL scale was admin- istered only in the social-class condition. Some items on the NFCL were modified slightly to ensure that the younger adolescents would understand them. For in- stance, the first sample item in the adult version of the scale, "I'd rather know bad news than stay in a state of uncertainty," was changed to, "I'd rather know bad news than not know if the news is good or bad." NFCL scores correlate significantly with biases.
In the religion condition, the Head over Heart scale (Epstein, Pacini, Denes-Raj, & Heier, 1995) was the third measure of epistemological beliefs. The HOH scale contains 16 items designed to measure an indi- vidual's report of relying on rationally or intuitively acquired knowledge (a = 32). A sample item is, "I'd rather rely on my own way of thinking than learn new ways of thinking" (scoring reversed on this item). Each item is rated on a 5-point scale (1 = completelyfalse; 5 = completely true).
For adolescents, previously reported test-retest re- liabilities for the NFC, BD, and HOH scales are .82, .68, and 33, respectively (Klaczynski et al., 1998).
Experiment-evaluation problems. Nine experiment-evaluation problems were presented. Each problem contained a description of the participants in an ex- periment, the social classes (or religions) of the partic- ipants, the methods used to conduct the research, the findings, and the researchers' conclusions. Each hy- pothetical study involved a comparison between the upper middle class (or each participant's religion) and the working class (or religions other than the par- ticipant's) on various social, intellectual, and moral behaviors.
One of three types of conclusions was presented. In the social-class condition, upper middle class favorable (UMC-F) conclusions were drawn when the research indicated that the upper middle class was "superior" in some way to the working class. Upper middle class unfavorable (UMC-U) conclusions were drawn when the research indicated that the upper middle class was "inferior" to the working class. The remaining problems were social class neutral (SC-N). As in the UMC-F and UMC-U problems, the upper middle and working classes were compared. It was stated, how- ever, that social class had no effects on the character- istic or behavior in question.
In the religion condition, three conclusions were re- ligion favorable (i.e., depicted participants' religions as superior to other religions), three were religion unfa- vorable, and three were religion neutral. The neutral problems involved comparisons of two or more reli- gions; however, participants' religions were not in- cluded in these comparisons.
The conclusions drawn in the favorable and un- favorable problems always involved the claim that the causal force underlying between-groups differ- ences was either social class or religion. In each prob- lem, the two correlated variables were social class (or religion) and a specific attribute. All conclusions could be criticized on the grounds that because indi- viduals are not randomly assigned to social classes (or to religions), they could differ in ways other than social class (or religion). Although selection was an implicit validity threat in each problem, each problem also contained one of three more explicit validity threats. Thus, an explicit selection bias, a construct with questionable validity, or an experimental con- found was built into each problem.
Although research within each problem was open to several criticisms, the most explicit threat (e.g., con- struct validity) varied between problems. Therefore, on each problem, the evidence and the researchers' conclusions could be effectively dismissed by identi- fying and explaining the validity threat. Within each validity type in the social-class condition, there was one UMC-F problem, one UMC-U problem, and one SC-N problem. Within each validity type in the reli- gion condition, there was one religion-favorable, one religion-unfavorable, and one religion-neutral prob- lem. Examples of each validity type are presented in the Appendix.
Three forms were created such that each problem appeared with equal frequency as favorable, unfavor- able, and neutral. For example, a problem that por- trayed the upper middle class positively on one form portrayed it negatively on the second form. On the third form, the same problem indicated that no differ- ences existed between social classes. Because no effects involving form were significant (all ps > .lo), form was not included in subsequent analyses.
Following each problem, participants indicated the strength of the conclusion (1= very weak; 9 = very strong) and how valid or well conducted the research was (1 = extremely poorly conducted; 9 = extremely well- conducted) on 9-point scales. Total scores on each scale were calculated separately for favorable, unfavor- able, and neutral problems. Scores on both the "strength and the "validity" scales, therefore, could range from 3 to 27 for each problem type.
Participants also wrote justifications of why they believed the research was valid or invalid. Justifica- tions were assigned scores ranging from 0 to 2 by two trained research assistants who were blind to the study's hypotheses and to all participant information. Justifications were scored 0 when no indication was given that a validity threat existed. Such responses occurred when participants claimed that the social class-behavior correlation or that the religion-behavior correlation was implausible, referred to per- sonal experiences that seemed to support or invali- date the research, or simply accepted or rejected the evidence, for example, "This seems like it's okay re- search." Justifications were given scores of 1 when participants indicated awareness of a validity threat but did not indicate that this threat made straightfor- ward interpretation of the findings impossible, for ex- ample, "[The research] is not good at all. Some people were businessmen and the others were at a rally." Scores of 2 were given only when participants indi- cated that the threat made it impossible to interpret the research and that a legitimate case could be made that conclusions other than the researchers' could be drawn, for example "It's terrible research because the real reason that the working class showed less stress was because they didn't have to steal anything. You can't tell if it's [social class differences] because of social class that they were less nervous or because the situa- tion wasn't fair." Scores of 1 and 2 were also awarded when participants detected other validity threats (e.g., selection). Exact interrater agreement, based on the responses of 30 adolescents from each age group, was 89.3%in the social-class condition and 88.9% in the reli- gion condition. Disagreements were resolved through discussion. Justification scores on each problem type (e.g., favorable) could range from 0 to 6.
Justifications were also coded for "implausibility" responses. A score of 0 was awarded when no indica- tion was given that a relationship was implausible. Scores of 1 were awarded when participants expressed clear disbelief that the relation between socio- economic class or religion and the second variable was possible. Examples are, "There's no way that social class has anything to do with how you do in school! It's your personality that makes a difference, not social class!" and "I don't see why they did this research, be- cause my religion can't make me a bad parent." Inter- rater agreement was 84.8% and 95.0% in the social class and religion conditions, respectively. Implausi- bility scores on each problem type could range from 0 to 3. Note that implausibility scores could co-occur with scientific justifications because students could write as many responses to the problems as they desired.
In the social-class condition, to ensure that partici- pants understood the distinction between the upper- middle and working classes, brief lists of occupations and approximate average earnings for upper-middle- class (e.g., lawyer) and working-class (e.g., custodian) occupations were provided. In both conditions, in- structions included a statement that the research in the 10-page packets was part of a large scientific study conducted nationally. Participants were asked to read each description carefully, think carefully, and answer the questions that followed each problem.
Law of large numbers problems. Nine problems, each of which presented a hypothetical discussion of the effects of belonging to different social classes (or to different religions) were administered. In each problem, individuals violated the LLN by making generaliza- tions from small evidential samples. The largest sample compared three people from one social category to three people from a different category. Examples are provided in the Appendix.
In both conditions, three problems were favorable, three were unfavorable, and three were neutral. Three forms were created such that each problem appeared with equal frequency as favorable, unfavorable, and neutral. No effects involving form were significant, largest F = 1.31.
Following each problem, participants rated the strength of the conclusion (1 = very weak; 9 = very strong) and the persuasiveness of the argument (1 = extremely unconvincing; 9 = extremely convincing). Scores on each rating scale were calculated separately for favorable, unfavorable, and neutral problems. For each problem type, total scores on the "strength" and the "persuasiveness" scales could range from 3 to 27.
Participants also provided justifications of their
ratings. These were assigned scores ranging from 0 to 2 by two research assistants blind to the study's objec- tives and to participant information. A response was scored 0 when it contained no mention of statistical concepts (e.g., sample size, random variation, gener- alizability). Such responses often entailed simple as- sertions, references to the virtues of direct experience, or claims of implausibility. A response was scored 1 when it contained reference to a statistical principle but was vague and poorly defined, for example "Jen- nie's argument isn't very strong because she only saw three parents." Responses were scored 2 only when the final judgment was clearly based on the LLN prin- ciple, for example, "There aren't enough people for her to make that conclusion. With just six people, she can't say that all working [class] people abuse their kids." Interrater reliability was 87.8% for the social- class condition and 85.6% for the religion condition. Justification scores on each problem type could range from 0 to 6.
Interrater agreement for "implausibility" responses was 90.7% and 83.3% in the social-class and religion conditions, respectively. Implausibility scores for each problem type could range from 0 to 3.
The analyses presented first are those conducted to determine, regardless of theoretical orientation, the extent to which reasoning was biased in each condi- tion, whether the strength of biases was age related, and whether the middle adolescents were more ana- lytically competent, but not less biased, scientific rea- soners than the early adolescents. Second, analyses of theory polarization are presented. The relationships among experiment-evaluation reasoning biases, LLN- reasoning biases, and implausibility biases are pre- sented next, as is a factor analysis intended to explore the hypothesis that the same latent variable underlies the observed correlations. Correlations of factor scores to epistemological dispositions, theories, and theory polarization are presented last. At the conclu- sion of each of these subsections, a brief summary of the statistical analyses is provided.
Developmental Trends in Reasoning Biases and Reasoning Competence
In the analyses that follow, age differences in biases are indicated by significant Age X Problem type (i.e., favorable, neutral, unfavorable) interactions; devel- opmental differences in scientific reasoning compe- tence are indicated by main effects of age. Analyses in this section were conducted separately for the two
domains to examine the hypothesis that in-group fa- voritism would be evident in the religion condition but would not be found in the social-class condition.
Social-class condition. In Table 1, mean rating, justi- fication, and implausibility scores for the experiment- evaluation and LLN problems are presented. A set of 2 (age) X 3 (problem type: UMC-F, SC-N, and UMC-U) ANOVAs indicated that on the experiment-evaluation problems, the middle adolescents rated the research as weaker and as less valid than the early adolescents, Fs(1, 60) = 11.20, 13.07, ps = .002, .001, respectively, and gave more complex justifications than the early adolescents, F(1, 60) = 13.53, p = .001. The justifica- tions of both age groups were more complex on UMC-U problems than on SC-N and UMC-F prob- lems, F(2,120) = 6.89, p = .001, which indicates a jus- tification bias toward the upper middle class. Justifi- cations on SC-N and UMC-F problems did not differ, F<1.
Analyses of LLN reasoning showed that the mid- dle adolescents rated the arguments as weaker and as less persuasive than the early adolescents, Fs(l,60) = 4.22,6.27, ps = .044, .015, respectively, and had higher justification scores than early adolescents, F(1, 60) = 9.09, p = .004. In addition, justifications were more complex on UMC-U and UMC-F problems than on the SC-N problems, Fs(1, 120) = 5.13, 9.55, ps = .027, .003, respectively. Justifications on UMC-U and UMC-F problems did not differ (p > .20).
Implausibility responses did not significantly dif- fer by age or problem type, largest F(2,120) = 1.98, p = .14, on the experiment-evaluation or the LLN problems.
The following conclusions may be drawn from the social-class condition.
- The most consistent finding, indicated by main effects of age on ratings and on justifications, was that the scientific reasoning competence of the middle ad- olescents was superior to that of the early adoles- cents. No age-related differences were found in the fre- quency with which the implausibility heuristic was used. These observations held for both the experiment- evaluation problems and the LLN problems.
- The lack of significant Age X Problem type in- teractions indicates that, despite their greater compe- tence, the middle adolescents were no less biased than the early adolescents.
(3) Bias was nonetheless evident in the experiment-
Table 1 Mean Ratings, Justification Scores, and Implausibility Scores for Experiment-Evaluation and LLN Problems, Social-Class Condition
|Law of large numbers|
Note: Standard deviations are in parentheses. UMC-F = upper-middle class favorable; SC-N = social class neutral; UMC-U = upper-middle class unfavorable.
Table 2 Mean Ratings, Justification Scores, and Implausibility Scores for Experiment-Evaluation
and LLNProblems, Religion Condition
Age Group Favorable
Early adolescent Strength Validity Justification Implausibility
Middle adolescent Strength Validity Justification Implausibility
Law of large numbers
Early adolescent Strength Persuasiveness Justification Implausibility
Middle adolescent Strength Persuasiveness Justification Implausibility
Note: Standard deviations are in parentheses.
evaluation and LLN justifications but not in the rat- ings or implausibility responses. These findings- that on some measures adolescents were unbiased, but on other measures some adolescents were biased toward the upper-middle class and some were biased toward the working class-disallow general conclu- sions concerning social-class favoritism among ado- lescents. That is, before consideration of theoretical beliefs, coherent patterns of biases in the data are dif- ficult to discern.
Religion condition. Mean rating, justification, and implausibility scores for the experiment-evaluation and the LLN problems are presented in Table 2.
Analyses of experiment-evaluation ratings revealed significant effects of problem type on per- ceived strength and validity, Fs (2, 132) = 9.07, 8.86, respectively, ps < .001. In each case, favorable prob- lems were given higher ratings than neutral problems which, in turn, were rated more highly than unfavor- able problems. In-group favoritist reasoning was also apparent in the experiment-evaluation justifications. Follow-up analyses of a significant main effect of problem type, F(2, 132) = 12.10, p < .001, indicated that justifications were more complex on unfavorable problems than on favorable and neutral problems (ps < .01). Also, the experiment-evaluation justifications of
Problem Type Neutral Unfavorable
the middle adolescents were superior to those of the early adolescents, F(1,66) = 23.05, p < .001.
Analyses of "implausibility" responses on the ex- periment evaluation problems revealed a significant main effect of problem type, F(2, 132) = 15.12, p < .001, which was qualified by a significant Age X Problem type interaction, F(2, 132) = 3.57, p = .031. Thus, implausibility responses did not differ by problem type among the early adolescents (p = .23). By con- trast, the middle adolescents made more implausibil- ity claims on unfavorable problems than on favorable and neutral problems (ps < .001), which indicates an in-group implausibility bias.
Similar findings were observed in the analyses of LLN reasoning. Ratings were higher on favorable problems than on unfavorable and neutral problems (ps < .01), and justifications were more complex on unfavorable problems than on favorable and neutral problems (ps < .005). The middle adolescents gave lower strength and persuasiveness ratings than the early adolescents, Fs(1, 66) = 5.19, 5.66, ps = .018, .02, respectively, and provided more complex justifica- tions, F(1,66) = 9.42, p = .003. Analyses of implausibil- ity responses on the LLN problems again indicated bias among the middle adolescents. Specifically, follow-up analyses of the significant Age x Problem type interaction, F(2, 132) = 5.38, p = .006, showed that the middle adolescents, but not the early adoles- cents, made more implausibility claims on unfavor- able problems than on neutral and favorable prob- lems (ps < .01).
Several conclusions can be drawn from the religion condition:
- Main effects of age were observed on experiment-evaluation and LLN justifications and on LLN ratings. The similarity of these findings to those in the social-class condition indicates that mid- dle adolescents are typically more competent scien- tific reasoners than early adolescents.
- As in the social-class condition, in neither their ratings nor their experiment-evaluation and LLN jus- tifications were the middle adolescents more or less biased than the early adolescents. In their implausi- bility responses, however, the middle adolescents were more biased.
- In contrast to the social-class condition, in- group favoritism was exhibited by members of both age groups. These biases were indicated by higher ratings, fewer scientific justifications, and (for the middle adolescents only) fewer implausibility responses on own-religion favorable problems than on own-religion unfavorable problems.
In sum, results from both conditions indicate that the competence for sound scientific reasoning is greater among middle than among early adolescents. Nonetheless, middle adolescents are no less likely than early adolescents to use their analytic reasoning competencies in a bias-free manner. The consistency of these cross-condition findings with those from re- search in which different social domains and different reasoning competencies were investigated suggests a domain-general progression in analytic reasoning competence between 13 and 16 years of age; the lack of significant differences in motivated analytic reasoning biases, at least from early to middle adolescence, also appears to be generalizable across numerous domains (see Klaczynski, 1997; Klaczynski & Gordon, 1996a; Klaczynski & Narasirnham, 1998).
The same cannot be said for the implausibility heu- ristic. Older participants were more biased than younger participants when they invoked this heuris- tic, but only in the religion condition. Although this finding is far from conclusive, it may be the case that age-related increases in implausibility biases are domain specific.
Theory Polarization and Development
Two polarization scores were computed: one to ex- amine whether theories polarized after the experiment-
evaluation problems and the second to determine whether theories polarized after the LLN problems. Scores were computed as follows: If initial theory scords were above the group mean, then initial scores were subtracted from later theory scores. If initial scores were below the group mean, then later theory scores were subtracted from initial scores. In each case, positive values indicated polarization (i.e., posi- tive scores would indicate that theories became more extreme following the evaluation of relevant evidence).
Initial analyses indicated that theories became more polarized in the religion condition than in the social- class condition (ps < .025). Nonetheless, because fur- ther analyses indicated that the associations of polar- ization scores to age, epistemological beliefs, and bi- ases were essentially identical in the two conditions, subsequent analyses were collapsed over conditions.
These analyses indicated a nonsignificant trend for the theories of the middle adolescents to become more polarized than those of the early adolescents (p = .094). Across ages, from initial theory assessment to second theory assessment, mean polarization scores were 7.48 (SD = 13.93) for the experiment- evaluation problems and 7.33 (SD = 14.45) for the LLN problems. Each mean was significantly different from zero, ps < .001. In contrast, mean religion polar- ization scores in the social-class condition were not significantly different from zero (ps > .lo). Likewise, mean social class polarization scores in the religion condition did not deviate significantly from zero (p > .20), which indicates that neither mere exposure to scientific reasoning problems nor repeated testing caused polarization.
In sum, subsequent to reasoning about a series of theory-congruent and incongruent problems, the the- ories of both early and middle adolescents became more extreme. As anticipated by intergroup relations theory (e.g., Tajfel & Turner, 1979), theories polarized more in the religion condition than in the social-class condition. Polarization occurred only between the first and second theory assessments, but not between the second and the third assessments (ps > .20).
Theory-Based Biases in Reasoning
Preliminary analyses indicated that neither age nor self-reported social class were related to social-class or religion theories (us < .lo) and that self-reported social class was not significantly related to any of the bias indicators (rs ranged from -.I4 to .15). In each condition, ratings on the two experiment-evaluation scales and ratings on the two LLN scales were highly correlated (rs >.90). Therefore, composite experiment- evaluation rating scores and composite LLN-rating scores were created by summing ratings across the strength and validity ratings for the experiment- evaluation problems and across the strength and per- suasiveness ratings for the LLN problems. In the follow- ing analyses, then, there were six bias scores. For both the experiment-evaluation and the LLN problems, a composite-rating bias score, a justification-bias score, and an implausibility-bias score were calculated for each participant in the manner described previously (i.e., as differences scores between favorable and un- favorable problems).
The model predicts that the strength of social theories should be positively related to biases. This prediction was supported: All six correlations between the bias scores and theories were significant (rs ranged from .48 to .62, all ps < .001).
Further analyses indicated that the bias measures were significantly interrelated (rs ranged from .18 to .67, all ps < .05), which suggests that the same latent variable could account for each type of bias. To test this possibility, a principal components analysis of the six bias scores was conducted.
The principle components analysis yielded two factors with eigenvalues greater than 1.Loadings for each factor are presented in Table 3. The analytic-bias factor was so-named because it was derived entirely from the four measures of analytic reasoning biases (i.e., composite rating biases and justification biases on the experiment-evaluation and LLN problems). The implausibility-bias factor was derived from im- plausibility-bias scores on the experiment-evaluation and LLN problems. Together, these factors accounted for 65.4% of the variance in the correlations among measures (analytic-bias factor, 49.9%; implausibility- bias factor, 15.5%).
The correlations presented in Table 4 indicate that theories explained 38% of the variance in analytic-bias factor scores and 24% of the variance in implausibility- bias factor scores. Additional analyses showed that, to somewhat different degrees, both factors were related
Table 3 Principal Components Analysis on Bias Measures
|Experiment evaluation ratings LLN ratings LLN justifications Experiment evaluation justifications LLN implausibility Experiment evaluation implausibility||.75 .73 .72 .71||.83 .82|
|Note: LLN = law of large numbers.|
Table 4 Correlations of Analytic-Bias and Implausibility-Bias Factor Scores to Theories, Age, Epistemological Dispositions,
|and Theory Polarization|
Note: ED = Composite epistemological dispositions. LLN = law of large numbers. Polar-LLN = polarization score following com- pletion of the LLN problems. Polar-EXP = polarization score fol- lowing completion of the experiment-evaluation problems. *p < .05; **p < .01; **p < ,001.
to theory polarization. Thus, the theories of the most analytically-biased adolescents polarized more than the theories of less biased adolescents.
As predicted, adolescents who held stronger theo- ries of social class and of religion demonstrated more extensive analytic biases and implausibility biases on both the experiment-evaluation and the LLN prob- lems than adolescents whose theories were relatively more neutral. Reasoning biases, in turn, predicted the extent to which theories polarized after evidence evaluation. Each of these findings supports the hy- potheses advanced earlier and the model from which these hypotheses originated.
Epistemological dispositions and reasoning biases. Be- cause most of the individual measures of epistemo- logical dispositions were significantly correlated with each other, a composite measure was created by add- ing standardized NFC and BD scores to reflected standardized NFCL scores (social-class condition) or to standardized HOH scores (religion condition). In Table 4, correlations between the two bias factors and the composite ED measure are presented (correla- tions between individual measures of epistemic be- liefs and biases may be obtained from the author).
As indicated in the table, the composite ED mea- sure predicted analytic reasoning biases. Because ED scores were not correlated significantly to theory scores (rs < .14), most of the variance explained by ED is unique. Together, epistemological dispositions and theories explained 43% and 28% of the variance in analytic and implausibility biases, respectively.
Composite ED scores were negatively related to polarization after both the experiment-evaluation problems (r = -.31, p < .01) and the LLN problems (r = -.24, p < .05). ED also predicted scientific rea- soning competence (rs = .29, .23, and .22, all ps < .05, for LLN ratings, LLN justifications, and experiment- evaluation ratings, respectively) and was correlated negatively with implausibility responses on the experiment-evaluation problems (r = -.29, p < .05).
In sum, "knowledge-driven" adolescents were less likely to rely on biased analytic reasoning, less likely to polarize their theories, and more likely to have well-developed scientific reasoning competencies than "belief-driven" adolescents.
Correlations between reasoning competence and reason- ing biases. There were no significant correlations be- tween the four indices of overall scientific reasoning competence and the six measures of bias (rs ranged from -.08 to .16, smallest p = .06), which indicates that high levels of scientific reasoning competence are nei- ther necessary nor sufficient for bias-free reasoning.
The results from this investigation replicate and extend previous research on motivated reasoning. The princi- pal findings, summarized below, expand the empirical basis for the models of motivated reasoning presented in Figures 1and 2, suggest modifications that would increase the predictive value of the models, and indi- cate several similarities and differences in the reason- ing of early and middle adolescents.
First, in the social-class condition, biases toward the working class indicated proletarianism in the rea- soning of some adolescents, biases toward the upper middle class suggested elitism in the reasoning of other adolescents, and the reasoning of a third group of adolescents was relatively unbiased. It therefore appears that generalizations regarding the nature of adolescents' social-class biases cannot be made unless the effects of theories are taken into consideration. That is, had theoretical beliefs not been assessed, patterns in adolescents' social-class reasoning would have been difficult to extract. The theory-related findings indi- cate that adolescents may be biased toward particular social groups even when they are not directly affili- ated with those groups. In the present case, theories predicted which middle class adolescents displayed reasoning biases toward the working class and which middle class adolescents were biased toward the upper middle class.
Second, in-group favoritism was apparent in the religion condition. Belonging to a distinct and clearly demarcated social group (e.g., religious, ethnic) with which adolescents can readily identify may increase awareness of their group membership (Phinney, Fer- guson, & Tate, 1997), lead to the construction of group- favorable theories, and increase group-affiliative ten- dencies (Tajfel & Turner, 1979). Motivated reasoning is one way in which these tendencies are manifested.
Third, in both conditions, theories became more extreme following evidence evaluation. The extent of theory polarization was, however, greater in the reli-
gion condition than in the social-class condition, a dif- ference that may be attributed to heightened group- affiliative tendencies when one's religion is the object of scrutiny.
Fourth, although the scientific reasoning abilities of the middle adolescents were superior to those of their younger counterparts, biases in the use of these abilities did not diminish with age. However, theoreti- cal beliefs were related to both experiment-evaluation and LLN reasoning biases. Analytic reasoning, the use of which led to evidence rejection, was predomi- nant on theory-incongruent problems, presumably because of the dissonance created by theory-data mismatches.
The claim that the relationships depicted in the ev- idence were implausible represented a second tactic by which incongruent data were rejected. Although biases in implausibility responding increased with age, this age-related increase was limited to the reli- gion condition. This evidence that age differences in implausibility biases are domain-specific is consistent with other findings that age-related biases in heuristic processing differ by domain. Motivated heuristic bi- ases may increase with age in some domains (e.g., re- ligion), may decrease with age in other domains (e.g., occupational aspirations, see Klaczynski & Fauth, 1997), and may be unrelated to age in still other do- mains (e.g., social class). Whereas the conclusion that motivated analytic biases diminish little, if at all, from late childhood to early adulthood appears to be appli- cable across numerous social domains, age differ- ences in motivated heuristic biases may be found only in some domains and only with some heuristics (e.g., implausibility).
Finally, epistemological dispositions were nega- tively associated with analytic reasoning biases and with theory polarization but were positively related to reasoning competence, as indexed by total reason- ing scores. "Knowledge-driven" adolescents-those who placed primacy on logical (versus intuitive) knowledge acquisition, and who appeared to subor- dinate the goal of theory preservation to the goal of learning-were more competent thinkers than belief- driven adolescents and showed greater balance in the use of their abilities. Yet, these adolescents had theo- ries as wide-ranging as those of belief-driven adoles- cents. It is not, therefore, the relative neutrality of their theories that separated knowledge-driven from belief-driven adolescents; rather, the competence-performance gap was smaller for adolescents moti- vated to understand knowledge and its origins.
Knowledge-driven adolescents may have been more metacognitively careful than their counterparts and may have consistently monitored the quality of their justifications as they moved from problem to prob- lem. Regardless of the relation between evidence and their theoretical beliefs, these adolescents engaged in predominantly analytical processing and rarely lapsed into heuristic processing, as shown in Figure 2. Presented a challenging task, knowledge-driven adoles- cents, motivated by the desire to look beyond superficial task characteristics, were more likely to decontextualize beliefs from reasoning than adolescents whose predom- inant goal was theory preservation.
In the following section, the relationship between age and reasoning biases is discussed. Next examined is the relevance of the findings to the previously de- scribed analyticlheuristic model of motivated reasoning. The concluding section explores the implica- tions of this research for debates regarding rationality.
The Development of Theory-Based Reasoning Biases
A number of domain-general and domain-specific trends in the development of scientific reasoning and reasoning biases were revealed by this research. Specif- ically, the findings are consistent with those reported in previous investigations (Baron, Granato, Spranca, & Teubal, 1993; Klaczynski, 1997; Klaczynski & Gordon, 1996a; Klaczynski & Narasimham, 1998; Perkins, 1985) in that no age-related differences were observed in motivated analytic-reasoning biases. When coupled with these investigations, the following conclusions may be tentatively put forth: (1)From middle child- hood to early adulthood, theory-motivated biases in analytic reasoning do not dramatically change in strength, despite domain-general increases in analytic reasoning competence; (2) when the metacognitivel epistemological dispositions that moderate reasoning biases are strong, they subordinate domain-specific theories; and (3) neither general intellectual ability nor scientific reasoning competence is related to the strength of reasoning biases.
These conclusions must be tempered for several reasons. First, Stanovich and West (1997, 1998) found that general ability is positively related to decontextu- alization. These disparate findings may have arisen from subtle methodological differences. For instance, the problems used by Stanovich and West may not have been as personally engaging as those used here. Also, participants in that research, but not in the present research, were instructed to ignore prior beliefs as they evaluated evidence (Sa', West, & Stanovich, 1999). Other research (Schauble, 1996) indicates that adults are better decontextualizers than children and adolescents, at least in asocial domains in which per- sonal stakes and self-esteem are not relevant issues. The conclusion that the strength of motivated reason-
ing biases is affected minimally by age and by intelli- gence may therefore be restricted to domains in which beliefs are deeply held and emotionally valued. Even in emotionally sterile domains, however, age-related declines in biases are surprisingly small (Kuhn et al., 1988).
Finally, the conclusion that age-related differences in bias are small may be limited to domains in which younger and older participants hold theories that are approximately equal in strength and diversity. It seems likely that age-related increases and decreases in motivated reasoning biases will be found when dif- ferent age groups have different beliefs.
The present research indicates that biased use of judgmental heuristics, such as implausibility claims, increases with age in some social arenas. One possible explanation for this unexpected finding is that the cognitions of older adolescents are more flexible and are more geared toward cognitive economy than those of younger adolescents. Implausibility responses to in- congruent evidence may be a reflection of this flexibil- ity (i.e., the more frequent use of both the heuristic and the analytic paths for data rejection), the motiva- tion to conserve cognitive energy, and the conviction that religion (but not social class) can have only posi- tive effects on behavior. The heuristic quality of im- plausibility responses is supported by the failure of adolescents to provide clear rationales for assertions of implausibility and by negative correlations to indi- ces of intelligence and to widely-accepted scientific reasoning abilities (see, however, Koslowski, 1996). Note that in this study LLN reasoning competence and experiment-evaluation reasoning competence were both positively correlated to the PMA Letter Sets test (a measure of fluid intelligence) and to the PMA Verbal Meaning test (a measure of crystallized intelligence) (us ranged from .28 to .43, all ps < .01). Implausibility responses, however, were given pri- marily by lower ability adolescents (rs ranged from -22 to -32, all ps <.05). This finding lends further cre- dence to the argument that, at least in this study, im- plausibility responses were unprincipled.
The finding that implausibility biases in the reli- gion condition were more prevalent among older par- ticipants is consistent with other research indicating age-related increases in heuristic reasoning under other conditions. For example, in their false belief research, Mitchell, Robinson, Isaacs, and Nye (1996) found that adults were more prone to a "realist bias" and, therefore, had more difficulty decoupling rea- soning from prior knowledge than children. Reyna and Ellis (1994) reported that framing biases do not emerge until late childhood. Appreciation for statisti- cal base rates and reliance on the representativeness heuristic are both positively correlated with age (Davidson, 1995; Jacobs & Potenza, 1991).
Rather than the traditional view that "intuitive" reasoning eventually yields to analytic reasoning as the predominant mode of understanding the social world (Inhelder & Piaget, 1958; Piaget & Inhelder, 1951/ 1975), these findings support the two-process hypothesis that analytic and heuristic processes co- develop. As they age, children acquire highly effortful analytic reasoning abilities and heuristics that enable cognitive conservation and rapid data assimilation and interpretation. Although biases supported by shifting between these two modes of reasoning may sometimes be maladaptive (e.g., promote erroneous stereotypes and unrealistic optimism), in general the flexibility afforded by two processing systems is a necessary prerequisite for the successful completion of cultural-developmental tasks.
Heuristic Reasoning, Analytic Reasoning, and Motivated Reasoning
Presented theory-relevant data, adolescents oscil- late between analytic and heuristic processing; thus, the two processes collude to produce reasoning bi- ases. Heuristic processing enables the acceptance of theory-congruent evidence and, less often, the rejec- tion of incongruent evidence. Generally, analytic pro- cessing is used to reduce the dissonance created by theory-data incompatibility because such processing enables the construction of arguments that appear ra- tional, well-conceived, and justifiable by appeal to nor- mative standards. Even when evidence is only tangen- tially relevant to original theories, the end result of data processing is that theories-which may be based on erroneous or untested propositions (e.g., if a person is Catholic, then that person is likely to have poor parenting skills)-become more polarized.
The significant correlations among reasoning biases suggest that different forms of motivated reasoning could be accounted for by the same mechanism. However, the principal components analysis-which identified statistically independent factors-indicated that considerable variance in these intercorrelations could be explained by a two-factor solution. This implies that different processes are involved in generating analytic reasoning biases and heuristic/ implausibil- ity reasoning biases.
To an extent, both of these suppositions may be correct. That is, there may be some overlap between the mechanisms responsible for analytic and heuristic reasoning biases. In both cases, evidence is precon- sciously matched to relevant theories. As a function of theory-evidence congruence, reasoning strategies change on a problem-to-problem basis. The outcome of theory-evidence checks determines when these reasoning shifts take place. From this point, however, different processes produce the two types of bias. In the case of analytic biases, reasoning shifts are from analytic processing to heuristic processing and vice versa. In contrast, shifts in heuristic reasoning biases may occur entirely within the heuristic processing system. That is, justifications shift from implausibility responses when evidence is rejected to other heuris- tics, such as those based on stereotypes (e.g., repre- sentativeness), when evidence is accepted.
The two-process model as originally presented must therefore by refined to include instances of motivated reasoning in which vacillation between different heu- ristics enables theory preservation. Klaczynski and Fauth (1997) report supportive evidence: Reasoning on their theory-congruent problems was sometimes based on stereotype-based schemata and sometimes based on exemplars derived from personal experiences. On incongruent problems, the most common route to data rejection was through analytic processing. As in this study, however, some adolescents in their research re- lied on heuristics to deal with theory-incongruent evi- dence. The incorporation of a heuristic processing path that illustrates shifts from one heuristic to other heuris- tics as, for example, when adolescents use the represen- tativeness and availability heuristics on theory-congruent problems and the implausibility heuristic on theory-incongruent problems, would further increase the predictive scope of the model.
The Development of Rationality, Motivated Reasoning, and Computational Capacity
Rational thinking has often been defined as cognitive activity that increases the probability that goals will be attained (see Evans & Over, 1996). The difficulty with this approach is that it is (at best) tangentially open to falsification because arguments can be (and have been) made that almost any form of thinking is rational. Findings taken by many psychologists as ev- idence for pervasive irrationality in human thought have been taken by others as evidence for the inher- ent rationality of human cognition (Stanovich, 1999). Perceiving correlations where none exist, drawing different conclusions from logically isomorphic evi- dence, and believing that conjunctions (e.g., Linda is both a bank teller and a feminist) are more probable than the components that comprise them (e.g., Linda is only a bank teller) number among the processes that some claim are "cognitive illusions" (e.g., Kahne- man & Tversky, 1996; Nisbett & Ross, 1980; Piatelli- Palmarini, 1994) that open the door to poor decision making. Others (e.g., Evans & Over, 1996; Gigerenzer, 1993, 1996) claim that these same cognitions increase the likelihood of successful adaptation.
Included in this latter camp are "bounded rational- ity" theorists (e.g., Evans, 1993; Gigerenzer, 1993; Oaksford & Chater, 1993; Simon, 1993). The core pos- tulate of these theorists is that humans are maximally rational within limits imposed by cognitive constraints (e.g., on working memory) or cognitive capacity. It has been argued that motivated reasoning is irrational be- cause, although it facilitates the attainment of some short-term goals (e.g., theory preservation), it impedes progress toward other goals (e.g., knowledge acquisi- tion) that may be more important to long-term adapta- tion. If this is indeed the case, then bounded rational- ity arguments cannot account for the apparent irrationality of the adolescents in this investigation.
First, cognitive capacity is thought to increase with age (Case, 1998). If reasoning biases among adolescents and adults are due to computational capacity limita- tions, then it theoretically must be the case that, as task complexity increases, the reasoning of children and early adolescents becomes more biased and irrational. The present research, however, indicates that older ado- lescents are not more objective than younger adoles- cents. This null finding has now been replicated in several studies (e.g., Baron et al., 1993; Klaczynski & Fauth, 1997). Neither the present findings nor findings (e.g., Ja- cobs & Potenza, 1991; Mitchell et al., 1996; Reyna &Ellis, 1994) that other apparently irrational cognitive tenden- cies (e.g., framing effects) and biases increase with age can be accounted for by theories of bounded rationality.
An additional argument against computational capacity or "bounded rationality" arguments comes from repeated findings of task variability. Even as sci- entific reasoning competencies improve, applications of those competencies remain selective. Consequently, theory-based and group-favoritist biases cannot be ac- counted for by information-processing constraints be- cause standards of evidence evaluation fluctuate on problems that impose the same information-processing demands. A similar argument can be derived from findings that individuals with greater intellectual ability (and presumably greater cognitive capacity) are no less biased by their theories than less intelli- gent persons (Klaczynski & Gordon, 1996a, 1996b).
Evolutionary psychologists, many of whom are also bounded rationality theorists, have argued that if the cognitive processes that produce beliefs and judgments-such as unrealistic optimism, conjunc- tion fallacies, deductive reasoning fallacies, base rate miscalculations, motivated reasoning biases, and so forth-had no adaptational value, they would not oc- cur as frequently as research has shown (i.e., they
would not have been "selected" by nature). This ar- gument seems misbegotten for a number of reasons. First, it fails to recognize the opinion of numerous bi- ologists that not all evolutionary changes result directly from selection pressures (e.g., the female hy- ena "penis") but rather are artifacts of other changes for which nature did select or are "accidents of nature." Second, the evolutionary argument ignores the fact that cultural evolution proceeds more rapidly than bio- logical evolution. As such, cognitive tendencies that may have had some adaptive value in the distant past may be maladaptive in an increasingly complex, tech- nological society (Stanovich, 1999). The finding that motivated scientific reasoning does not become less prevalent with increasing age and that reasoning biases are particularly prevalent among adolescents with narrow epistemic goals implies that such reasoning is one of these tendencies.
This research was supported, in part, by a research grant from the Western Carolina University Graduate School, Cullowhee, NC.
Selection: Upper-middle-class favorable. Two well-known researchers have conducted several studies on the relation- ship between social class and the tendency to conform to authority. Members of the upper-middle classes were com- pared to members of the working class in one study. In this study, 50 members of the upper-middle class were con- tacted at local businesses and agreed to be in the study. At the time of the study, members of the working class were hard to locate and contact. Thus, the researchers recruited 50 members of the working class from a political rally that was taking place in a nearby city. Next, each person was brought to the researchers' laboratory. At the laboratory, both the upper-middle class people and the working class people were given several orders to complete menial tasks. For example, they were ordered to clean the floor of several rooms. At the completion of the experiment, the researchers found that the people from the working class were far more likely to obey the commands than the people from the up- per-middle class. The people from the upper-middle class were much more likely to question the commands and to refuse. The researchers concluded that the experience of being from the upper-middle class results in a greater sense of independence and of inner strength than does being from the working class.
Construct validity: Social-class neutral. Bill R. is a psychol- ogist interested in determining whether some economic groups enjoy their work more than others. To conduct his research, he included in his study people from the upper- middle class and from the working class. In each class, he asked 80 people to be in the study. To measure job enjoy- ment, Bill counted the number of times people in each so- cial class were late for work each month. At the end of his study, Bill found that the average upper-middle class per- son was late for work 2.1 times each month. Members of the working class were late for work 2.3 times per month. When he conducted the study a year later, he found that up- per-middle class people were late to work an average of 2.4 times each month and working class people were late an av- erage of 2.2 times a month. Therefore, members of the up- per-middle social class were just as likely to be late for work as people from the working class. Based on this, Dr. R. con- cluded that people from the working class enjoy their jobs just as much as people from the upper-middle class.
Experimental confound: Upper-middle-class unfavorable. Dr.
L. has been studying how people from different economic backgrounds react in stressful situations. She recently com- pleted a study that examined whether upper-middle class students react more positively under stress than students who are in the working class. As the first part of her study, she observed 80 working class students give a short speech to members of their school, a situation which all of the stu- dents thought was very stressful. Because their schools would not allow them to give such speeches, Dr. L. observed 80 students who were from the upper-middle class in a different situation. In this situation, Dr. L. placed the upper-middle class students under stress by having them ask another student to steal something from a shop. The re- sults showed that the working class students held up under stress far better than the upper-middle class students. The working class students were more comfortable, less "jit- tery," had higher blood pressures and higher heart rates than the upper-middle class students. Dr. L. concluded that being from the working class prepares students better for stressful situations than does being from the upper- middle class.
Law of Large Numbers
Upper-middle-class favorable. Jennie has been wondering about the effects of belonging to the upper-middle class ver- sus belonging to the working class. Her interest is in the parenting skills of these two groups. So, she interviewed 3 upper-middle class parents and 3 working class parents. Also, she observed each of the parents playing and talking with their children. After watching the three upper-middle class parents, she saw that they spoke to their children in a more intellectual way and gave them games that involved challenging puzzles, which all their children found enjoy- able. With the three working class parents, she saw no evi- dence that the working class parents had any good parent- ing skills. On this basis, Jennie concluded that upper- middle class people make better parents than working class people. Jennie also concluded that working class people are more likely to neglect and abuse children than upper- middle class people.
Social-class neutral. At her college, Rachel conducted a research project that was designed to determine whether students from the upper-middle class had better manners and were more polite than students from the working class. To do this research, Rachel sent 2 upper-middle class stu- dents and 2 working class students to a dinner party and observed how polite these four people were. She rated each person on a scale where: "0" = poor manners, "1" = moderately good manners, and "2" = very good manners. Be- low is a chart showing her findings.
Upper-Middle Working Manners scores Class Students Class Students
"0" 0 0
"1" 1 1
The findings clearly showed that upper-middle class and working class had equally good manners; therefore, Rachel concluded that social class does not affect how well- mannered and polite a person is.
Upper-middle-class unfavorable. In the following situation, Jill is talking to her friend Amy. They have been dis- cussing Jill's son. Jill's family is from the upper-middle class, but she is arguing that being from the upper-middle class has a negative effect on children's intelligence. Here's what Jill argued:
"All you have to do is look at my son and you can see that he is less intelligent than children from the working class. His grades are poor, he scored below average on an intelligence test. He's just not as smart as working class chil- dren. It's hard to argue against that evidence."
Baron, J., Granato, L., Spranca, M., & Teubal, E. (1993). De-
cision-making biases in children and early adolescence:
Exploratory studies. Merrill-Palmer Quarterly, 39,22-46. Cacioppo, J. T., &Petty, R. E. (1982). The need for cognition. Journal ofPersonality and Social Psychology, 42, 116-131. Case, R. (1998). The development of conceptual structures.
In D. Kuhn & R. Siegler (Eds.), W. Damon (Series Ed.),
Handbook of child psychology: Vol. 2. Cognition, perception,
and language (5th ed., pp. 745-800). New York: Wiley. Davidson, D. (1995). The representativeness heuristic and
the conjunction fallacy in children's decision making.
Merrill-Palmer Quarterly, 41, 328-346. Donaldson, M. (1978). Children's minds. London: Fontana. Epstein, S. (1994). Integration of the cognitive and psycho-
dynamic unconscious. American Psychologist, 49, 709-
Epstein, S., Pacini, R., Denes-Raj, V., & Heier, H. (1995).Individual differences in heuristic processing. Unpublished manuscript. University of Massachusetts, Amherst.
Evans, J. St. B. T. (1993). Bias and rationality. In K. I. Mank- telow & D. E. Over (Eds.), Rationality (pp. 6-30). New York: Routledge.
Evans, J. St. B. T., & Over, D. E. (1996). Reasoning and Ratio- nality. Hove, U.K.: Psychology Press.
Furby, L., & Beyth-Marom, R. (1992). Risk-taking in adoles- cence: A decision-making perspective. Developmental Re- view, 22,l-44.
Gigerenzer, G. (1993). The bounded rationality of probabi- listic mental models. In K. Manktelow & D. Over (Eds.), Rationality: Psychological and philosophical perspectives (pp. 284-313). London: Routledge.
Gigerenzer, G. (1996). On narrow norms and vague heuris- tics: A reply to Kahneman and Tversky. Psychological Re- view, 203, 592-596.
Inhelder, B., & Piaget, J. (1958). The growth oflogical thinking from childhood to adolescence. New York: Basic.
Jacobs, J. E., & Potenza, M. (1991). The use of judgment heu- ristics to make social and object decisions: A develop- mental perspective. Child Development, 62,166-178.
Kahneman, D., Slovic, P., & Tversky, A. (Eds.). (1982). Judgment under uncertainty: Heuristics and biases. Cambridge, MA: Cambridge University Press.
Kahneman, D., & Tversky, A. (1972). Subjective probability: A judgment of representativeness. Cognitive Psychology, 3,430-454.
Kahneman, D., & Tversky, A. (1996). On the reality of cog- nitive Illusions. Psychological Review, 203,582-591. Kardash, C. M., & Scholes, R. J. (1996). Effects of pre-existing beliefs, epistemological beliefs, and need for cognition on interpretation of controversial issues. Journal ofEduca- tional Psychology, 88, 260-271. Klaczynski, P. A. (1997). Bias in adolescents' everyday rea- soning and its relationship with intellectual ability, per- sonal theories, and self-serving motivation. Developmental Psychology, 33,273-283. Klaczynski, P. A,, & Fauth, J. (1997). Developmental differ- ences in memory-based intrusions and self-serving sta- tistical reasoning biases. Merrill-Palmer Quarterly, 43, 539-566. Klaczynski, P. A., Fauth, J., & Swanger, A. (1998). Adoles- cent identity: The roles of rationality, critical thinking dispositions, and formal operations. Journal of Youth and Adolescence, 27, 185-207. Klaczynski, P. A., & Gordon, D. H. (1996a). Goal-directed everyday problem solving: Motivational and general ability influences on adolescent statistical reasoning. Child Development, 67,2873-2891. Klaczynski, P. A., & Gordon, D. H. (199613). Self-serving in- fluences on adolescents' evaluations of belief-relevant evidence. Journal of Experimental Child Psychology, 62, 317-339.
Klaczynski, P. A., & Narasimham, G. (1998). The develop- ment of self-serving reasoning biases: Ego-protective versus cognitive explanations. Developmental Psychology, 34,175-187.
Koslowski, B. (1996). Theory and evidence: The development of scientific reasoning. Cambridge, MA: MIT Press. Kruglanski, A. W., Webster, D. M., & Klem, A. (1993). Moti-
vated resistance and openness to persuasion in the pres- ence or absence of prior information. journal ofPersona1- ity and Social Psychology, 65, 861-876.
Krull, D. S., &Anderson, C. A. (1997). The process of expla- nation. Current Directions in Psychological Science, 6, 1-5.
Kuhn, D., Amsel, E., & O'Loughlin, M. (1988). The develop- ment of scientific thinking skills. Orlando, FL: Academic Press.
Kuhn, D., & Lao, J. (1996). Effects of evidence on attitudes: Is polarization the norm? Psychological Science, 7, 115-
119. Kunda, Z. (1990). The case for motivated reasoning. Psychological Bulletin, 108,480-496.
Lord, C. G., Ross, L., & Lepper, M. R. (1979). Biased assimi- lation and attitude polarization: The effects of prior the- ories on subsequently considered evidence. Journal of Personality and Social Psychology, 37, 2098-2109.
Mitchell, P., Robinson, E. J., Isaacs, R. M., & Nye, R. M. (1996). Contamination in reasoning about false belief: An instance of realist bias in adults but not children. Cognition, 59, 1-21.
Moshman, D. (1998). Cognitive development beyond child- hood. In D. Kuhn & R. Siegler (Eds.), W. Damon (Series Ed.) Handbook ofchild psychology: Vol. 2. Cognition, percep- tion, and language (5th ed.), (pp. 947-978). New York: Wiley.
Nisbett, R. E., & Ross, L. (1980). Human inference: Strategies and shortcomings of social judgment. Hillsdale, NJ: Erlbaum.
Oaksford, M., & Chater, N. (1993). Reasoning theories and bounded rationality. In K. Manktelow & D. Over (Eds.), Rationality: Psychological and philosophical perspectives (pp. 31-60). London: Routledge.
Perkins, D. N. (1985). Postprimary education has little im- pact on reasoning. Journal of Educational Psychology, 77, 562-571.
Perkins, D. N., Jay, E., & Tishman, S. (1993). Beyond abili- ties: A dispositional theory of thinking. Merrill-Palmer Quarterly, 39, 1-21. Phinney, J. S., Ferguson, D. L., & Tate, J. D. (1997). Inter- group attitudes among ethnic minority adolescents: A causal model. Child Development, 68, 955-969. Piaget, J., & Inhelder, B. (1975).The origin of the idea ofchance in children. New York: Norton. (Original work published in 1951)
Piattelli-Palmarini, M. (1994). Cognitive illusions: How mis- takes ofreason rule our lives. New York: Wiley. Reyna, V. F. (1992). Reasoning, remembering, and their rela- tionship: Social, cognitive, and developmental issues. In
M. L. Howe, C. J. Brainerd, & V. F. Reyna (Eds.), Development of long-term retention (pp. 103-132). New York: Springer-Verlag.
Reyna, V. F. (1995). Interference effects in memory and rea- soning. In F. N. Dempster & C. J. Brainerd (Eds.), Interference and inhibition in cognition (pp. 29-59). SanDiego, CA: Academic Press.
Reyna, V. F., & Brainerd, C. J. (1991). Fuzzy-trace theory and framing effects in choice: Gist extraction, truncation, and conversion. Journal of Behavioral Decision Making, 4, 249-
Reyna, V. F., & Brainerd, C. J. (1995). Fuzzy-trace theory: An interim synthesis. Learning and Individual Differences, 7, 1-75.
Reyna, V. F., & Ellis, S. C. (1994). Fuzzy-trace theory and framing effects in children's risky decision making. Psychological Science, 5, 275-279.
Sa', W. C., West, R. F., & Stanovich, K. E. (1999). The domain specificity and generality of belief bias: Searching for a generalizable critical thinking skill. Journal of Educational Psychology, 92.
Schauble, L. (1996). The development of scientific reasoning in knowledge-rich domains. Developmental Psychology, 32, 102-119.
Schommer, M. (1994). Synthesizing epistemological be- lief research: Tentative understandings and provoca- tive conclusions. Educational Psychology Review, 6, 293-319.
Simon, H. A. (1993). Decision making: Rational, nonra-
tional, and irrational. Educational Administration Quar- terly, 29, 392-411. Sloman, S. A. (1996). The empirical case for two systems of reasoning. Psychological Bulletin, 119, 3-22.
Stanovich,K. E. (1999). Who is rational? Mahwah, NJ:Erlbaum.
Stanovich, K. E., & West, R. F. (1997). Reasoning indepen- dently of prior belief and individual differences in ac- tively open-minded thinking. Journal ofEducationa1 Psy- chology, 89, 342-357.
Stanovich, K. E., & West, R. F. (1998). Individual differences in rational thought. Journal of Experimental Psychology: General, 127, 161-188.
Tajfel, H., &Turner,J. C. (1979). An integrative theory of in- tergroup conflict. In W. G. Austin & S. Worchel (Eds.), The social psychology of intergroup relations (pp. 33-47). Monterey, CA: Brookes / Cole.
Thurstone, T. G. (1962).Primary mental abilitiesfor grades 9
12. Chicago: Science Research Associates.