International Psychological Research That Albert ... - apa.org · When the APA s Committee on International Relations in ... Arti-cles based on award addresses are reviewed, but they

support, and practice human rights. Journal of Social Is-sues, 64(4), 857–880.

International Psychological Research ThatMatters for Policy and Practice

Judith V. Torney-PurtaUniversity of Maryland

After a brief history of the Committee on InternationalRelations of the American Psychological Association, 3points are made about international psychological researchthat matters. First, it matters when the definition of theresearch problem area and the findings can potentially bereflected in policy change, in the practice of educators orpsychologists, or in the mindsets of a new generation ofresearchers. Person-centered analysis of adolescents’social and political attitudes has this potential and cancomplement variable-centered analysis. A cluster analysisof the IEA Civic Education Study’s data in 5 WesternEuropean and 5 Eastern European countries illustratesthis. The following 5 clusters of adolescents wereidentified: those supportive of social justice but notparticipative, those active in conventional politics and thecommunity, those indifferent, those disaffected, and aproblematic cluster of alienated adolescents. Second,research that matters is situated in a cultural context. It isproposed that publications using data from any singlecountry be required to include information about thecultural context in which the research was conducted.Finally, it matters that attention be given to the dynamicsof the collaborative international research process, notonly to research results.

Keywords: adolescents’ political attitudes, alienation, clusteranalysis, cultural context, social justice

The mere formulation of a problem is often far more essentialthan its solution, which may be merely a matter of mathemat-ical or experimental skills. To raise new questions, new possi-bilities, to regard old problems from a new angle requirescreative imagination and marks real advances in science.—Albert Einstein

This quotation, from Albert Einstein’s acceptance of theNobel Prize in 1921, appears on the wall of the Nobel Mu-seum in Stockholm, Sweden. It provides the inspiration forthis presentation in which international research about ado-lescents and their connections to the societies and institu-tions of their nations are examined from some new angles.

The history of the American Psychological Associa-tion’s (APA’s) Committee on International Relations inPsychology (CIRP) sets the stage for a presentation of em-pirical results. In the 1940s APA saw its role as acting onthe international scene in cooperation with intergovernmen-tal institutions. This is noteworthy because in subsequentyears social and political institutions have seldom beenexplicit parts of psychological researchers’ concerns. Theearly years of the 21st century are a prime time to thinkmore broadly and to consider domestic and internationalinstitutions as they coconstruct the scenarios for develop-ment and involvement that are reflected in the attitudes andvalues of young people across the world.

Further, a credible criticism of current developmentalresearch is its lack of connection to real young people andthe situations in which they develop. This is not a newproblem. In 1965 Nevitt Sanford complained about “re-search psychologists who . . . can define variables, statehypotheses, . . . get publishable results and miss the per-son” (quoted in Magnusson, 1998, p. 88). A new angle onthis area is to formulate research problems in a person-oriented rather than variable-oriented way. In the first sec-tion of this article, person-oriented analysis is illustratedwith an exploratory examination of adolescents’ attitudestoward social and political institutions conducted with na-tionally representative samples from 10 countries, 5 olderWestern democracies and 5 post-Communist countries. Inthe second section, I discuss the value of explicitly describ-ing contexts for adolescents’ development, which is espe-cially important in studies conducted in one country. In thethird section, I consider the dynamic process of interna-tional collaboration in psychological research that matters.

Young People’s Attitudes Toward Political Institutions:A Research Topic That Matters

The History of APA’s Relations With Governmental andIntergovernmental Institutions

When the APA’s Committee on International Relations inPsychology was established a little more than 60 years ago,psychologists wanted to shape the work of the recentlyestablished United Nations Educational, Scientific and Cul-

Editor’s NoteJudith V. Torney-Purta received the Award for Distin-guished Contributions to the International Advancement ofPsychology. Award winners are invited to deliver an awardaddress at the APA’s annual convention. A version of thisaward address was delivered at the 117th annual meeting,held August 6–9, 2009, in Toronto, Ontario, Canada. Arti-cles based on award addresses are reviewed, but they dif-fer from unsolicited articles in that they are expressions ofthe winners’ reflections on their work and their views ofthe field.

825November 2009 ● American Psychologist

tural Organization in Paris. For example, American psy-chologist Hadley Cantrell worked on an action researchproject on reducing social tensions within Europe. He col-laborated with Czech, Polish, and Hungarian psychologistsat a time when the U.S. government forbade official con-tact with these nations.

Within a few years of its founding, CIRP began to sendjournals to devastated European libraries, to provide assis-tance to those engaged in democratic reeducation in Ger-many, to establish exchanges, and to encourage reviewsdealing with the field of psychology in different countries(e.g., Mintz, 1958). During this period CIRP began to col-laborate with the Society for the Psychological Study ofSocial Issues. Redressing inequality and promoting socialjustice values was part of what mattered to these psycholo-gists; for example, apartheid was a significant concern forthem. These early efforts did not address all types of in-equality, however. For nearly 40 years, until 1982, thelarge majority of CIRP members and all of its chairs weremen.

In the 1940s and 1950s members of disciplines otherthan psychology also conducted research about values andculture. Anthropologists and sociologists led by ClydeKluckhohn (1951) studied Zuni, Mormon, Navajo, SpanishAmerican, and Texan communities to address differencesin values persisting in spite of similar ecological demandsand cross-cultural contact. These studies lacked a frame-work linking the findings to institutions, and the cross-cul-tural empirical study of values was largely abandoned byAmerican anthropologists thereafter (D’Andrade, 2008).

During the next few decades, Ronald Inglehart, a politi-cal scientist, was an intellectual forefather of internationalresearch on adults’ values and attitudes (e.g., Inglehart,1977). In this period several projects to study the politicalattitudes of young people in the United States were initi-ated by political scientists (Jennings & Niemi, 1975) orinvolved collaborations between political scientists andpsychologists (Hess & Torney, 1967/2005). These effortsestablished a subfield called political socialization re-search. The International Association for the Evaluation ofEducational Achievement (IEA), then headquartered inStockholm, conducted a civic education study in 10 coun-tries (Torney, Oppenheim, & Farnen, 1975). This projectdemonstrated that it was possible to measure civic knowl-edge and political attitudes among adolescents cross-na-tionally, but in the next decades IEA made its reputation instudies of literacy and mathematics or science test achieve-ment, not by studying attitudes.

Triandis and Brislin (1984) and Hofstede (1980) wereamong the psychologists who studied adults’ values cross-nationally, and some of this research has mattered for de-signing training programs internationally. However, mostof these scholars ignored young people as members of so-

ciety whose attitudes mattered and focused on differencesbetween group means in variable-centered analyses.

Keeping in mind this history and the desire to addressresearch problems that matter (especially in the area ofsocial justice), in the next section I describe how person-centered approaches such as cluster analysis applied tocross-national data can help in understanding how youngpeople become connected to their society and its institu-tions.

Variable-Centered and Person-Centered Approaches inResearch About Adolescents

For many decades psychologists thought of quantitativemethods such as written questionnaires or surveys as themost appropriate way to study adolescents. Recently therehas been a call to broaden those methods. Garcı́a Collhighlighted the “role of culture in developmental change”(Garcı́a Coll & Magnuson, 1999, p. 6) and called for a par-adigm shift that would combine qualitative and quantitativemethods. Other developmental psychologists have recog-nized the value of multiple methods, especially when oneneeds to place context and culture in the foreground in un-derstanding young people’s development (Haste & Hogan,2006; Yoshikawa, Weisner, Kalil, & Way, 2008).

There are several examples of blending qualitative andquantitative methods to study young people’s attitudes to-ward their society and its governmental institutions. Earlyresearch on political socialization used preliminary inter-views in devising appropriate questions for surveys (Hess& Torney, 1967/2005). Hahn (1998) used focus groupswith young people across nations to supplement surveys.Colby’s current research exemplifies a mixed method ap-proach (e.g., Colby, 2008; Colby, Ehrlich, Beaumont, &Corngold, 2007). In studying 20 programs designed to in-crease political engagement among college students, theseauthors surveyed 600 program participants but also con-ducted interviews with students and faculty members abouthow they saw political engagement and how approachessuch as community-action projects were implemented indifferent contexts. The work of Youniss provides anothermodel for national and international studies that exploreyoung people’s relations to social institutions in depth(Youniss & Hart, 2005) and examine patterns of change inidentity rather than absolute increases or decreases inscores (Reis & Youniss, 2004). These studies address nu-anced research problems and often have social justicethemes.

Magnusson (1998) argued for methods that examineintegrated individuals who are like some individuals andunlike others rather than methods that rely on variable-centered analysis. He developed person-centered modelswith Swedish longitudinal data forming holistic models ofpersonality based on patterns or profiles of developmentover time (Magnusson & Torestad, 1993). In a similar

826 November 2009 ● American Psychologist

vein, Mahoney, Stattin, and Magnusson (2001) conducted apattern analysis of five clusters of Swedish boys with dif-ferent profiles of competencies at age 10 years and com-pared their subsequent participation in youth centers andcriminal involvement.

Other researchers have looked for patterns or clusters ofindividuals at a single time point in the United States.Damon (2008) studied adolescents’ sense of purpose with asurvey of approximately 400 adolescents and young adults(conducted in three U.S. states) and 50 in-depth interviews.The respondents could be divided into four approximatelyequal groups: the purposeful (who have found somethingmeaningful to which they are dedicated), the dabblers (whohave tried various pursuits without becoming committed),the disengaged (who lack both a sense of purpose and aninclination to find one), and the dreamers (who can imag-ine themselves doing great things but are not pursuing anygoals in a meaningful way).

Another example of looking at person-centered and vari-able-centered analyses as complementary comes from inter-national educational research. The Programme for Interna-tional Student Assessment (PISA), a large-scale projectconducted by the Organisation for Economic Co-operationand Development, reports variable-centered analysis inwhich countries’ mean scores are compared. Finland hasconsistently ranked high in PISA assessments and has beenheld up as a model school system. Some educators in theUnited States and Europe have concluded that the solutionto their students’ performance problems is to replicateFinnish schooling. However, when Finnish researchers lookat these variable-oriented analyses, they do not dwell onthe group differences (Simola, 2008). Instead several Finn-ish researchers have chosen to use person-oriented analysisin the form of student profiles of school engagement (Lin-nakyla & Malin, 2008) or preferences for learning environ-ments (Tapola & Niemivirta, 2008).

The remainder of this section examines a person-ori-ented analysis of young people’s orientations to the institu-tions of their societies using data from the IEA Civic Edu-cation Study (CIVED). Here I use cluster analysis toexamine how attitudes cohere within individual young peo-ple who have grown up in different political systems.

The IEA CIVED as a Source of Data

The IEA is a network of research institutes from more than60 countries that is headquartered in Amsterdam (estab-lished in the 1950s). It is best known for studies such asthe Trends in International Mathematics and Science Studythat focus on country ratings and are almost exclusivelyvariable centered (e.g., Mullis et al., 2008). Following astudy in the 1970s in the area of civic education that in-cluded a number of social attitude measures, in the early1990s IEA was prompted by its post-Communist membercountries to explore a study of civic education. From the

mid-1990s until 2006 I acted as chair of the InternationalSteering Committee for this internationally collaborativestudy of civic education in nearly 30 countries.

The IEA General Assembly was persuaded that twophases, one qualitative and one quantitative, were essentialfor CIVED. There was not a clear list of concepts to beinvestigated in this area because civic education was not anestablished subject in most of the countries wanting to takepart. The first phase of this extensive study (1995–1999)consisted of a consensus-building process about whatshould be measured in a test and attitudinal survey, result-ing in the collection of more than 20 qualitative nationalcase studies (Torney-Purta, Schwille, & Amadeo, 1999).The purpose of the case studies was to pinpoint what theaverage 14- or 15-year-old was expected or likely to under-stand about topics such as elections, problems in the com-munity, or the rights of minority groups. In other words,they described expectations for adolescents’ understandingof political and social institutions within each country.

The study’s steering group recognized that politicalthinking does not occur free of contextual influences.When the study began in the mid-1990s, salient aspects ofcontext included the recent collapse of Communism withthe associated social disruption, as well as the structuringof new democracies whose leaders were exploring changesin education. In this period the established democracieswere also concerned about declining interest in politicalparticipation among younger generations. This problemseemed to have slightly different dimensions in the UnitedStates (where most children had social studies courses withsome emphasis on citizenship throughout their school ca-reers) and in England and the Nordic countries (where his-tory was taught but without much attention to its civic di-mensions or participation). Ironically, in countries such asSweden provision was made for students to govern theirschools, but participating in local or national political af-fairs received little attention.

At a meeting in 1995 the individuals appointed by theircountries as CIVED national research coordinators receivedinstruction in methods for preparing national case studies.A sociologist from Poland led a demonstration focus groupin which coordinators from a dozen countries, includingseveral post-Communist countries, were asked to recountchildhood memories of political experiences. One individ-ual mentioned having a picture of Lenin as a young boy onher wall. Others in the group echoed, “that picture of Le-nin, I had it too.” Another described having to whisperwhen the family discussed social issues. Comments such asthese suggested that mistrust and attitudes toward nationalsymbols would be important topics to study.

In the second phase of CIVED (conducted during 1999)nationally representative samples of 14-year-olds in 28countries spent two class periods answering written ques-tions. Students took a test of political conceptual knowl-


edge and skills in interpreting political material such aseditorial cartoons and also a survey of concepts of citizen-ship, political attitudes, and expected behaviors. All itemswere carefully developed to be common across countries.The test specifications included direct quotations from thenational case studies and from Internet conferences withstudents from five countries. Social psychologists from Po-land, Hungary, and Germany suggested revisions for attitu-dinal items from databases in the United States and En-gland.

Since the primary analysis of the 28 country results forapproximately 90,000 fourteen-year-olds was published(Torney-Purta, Lehmann, Oswald, & Schulz, 2001), addi-tional variable-oriented analysis has been undertaken usinglarge groups of countries (e.g., Torney-Purta, Wilkenfeld,& Barber’s 2008 analysis of knowledge of and attitudestoward international human rights) and a more limited setof countries (members of the Organization of AmericanStates; Torney-Purta & Amadeo, 2004). Some of thesevariable-oriented analyses have resulted in findings thatmattered. In Chile, students showed low levels of knowl-edge about democracy compared with those in other coun-tries. The Chilean adolescents also showed little skepticismabout information in the media and little awareness of po-tential threats to democracy (such as political influence onthe judiciary). The military regime had established the cur-riculum that was in place in 1999. The Chilean Ministry ofEducation responded to the CIVED findings by establish-ing a commission to oversee curricular revision. In theUnited States two papers interpreting findings for policy-makers have been issued (Torney-Purta & Vermeer, 2004;Torney-Purta & Wilkenfeld, 2009).

An Exploratory Cluster Analysisof Adolescents From 10 Countries

Examining the attitudinal part of the IEA CIVED instru-ment using person-centered analysis is intriguing for psy-chologists. In the next subsections I report an analysis ofhow more than 30,000 students from 10 countries re-sponded on several attitudinal scales simultaneously, ratherthan comparing mean scores on these attitudes one at atime.

In particular it was of interest to compare clusters ofadolescents from two groups of countries. One group hashad stable democracy for generations (Australia, England,Finland, Sweden, and the United States). The other grouprecently had democratic government established after de-cades of Communist influence (Bulgaria, the Czech Repub-lic, Estonia, Hungary, and Latvia). The contexts for politi-cal and social attitude development differ widely betweenthese regional groups (here referred to as Western Europeand Eastern Europe, even though the first group includesthe United States and Australia and the second group in-cludes Central as well as Eastern Europe). These issues are

considered in an upcoming publication (Barber & Torney-Purta, 2009). Table 1 shows the number of students andschools tested in each country.

The questions of interest to be explored in both regionswere the following:

1. Is it possible to identify groups of adolescents withcoherent and distinctive patterns of social and po-litical attitudes?

2. To what extent are these clusters of adolescentsdifferent in nations with different histories, politi-cal institutions, and social contexts?

3. How do the distributions or proportions of adoles-cents holding these patterns of attitudes differ be-tween countries or regions?

4. What types of beliefs about involvement in societyand politics characterize adolescents in these clus-ters?

5. How do characteristics of the individual and of theschool context relate to membership in these clus-ters?

A series of two-step cluster analyses were conducted toidentify groups of 14-year-olds with distinct patterns ofcivic attitudes. Two-step cluster analysis groups cases intoa series of “preclusters” and then runs a hierarchical clusteranalysis on these preclusters (Norusis, 2007). This analysisis ideal for large-scale data sets such as CIVED. As a pre-liminary step, a missing values analysis was run on each ofthe 12 attitudinal scales chosen for analysis. In no casewere data missing in more than 5% of the cases, and miss-ing data were imputed with means. All cases then hadcomplete data and could be included in the analysis. Thedata file was sorted by student ID within country beforeforming preclusters.

Next, all 12 scales were entered into the cluster analy-sis. In all but one case (Cynicism) these were IRT scales

Table 1Student and School Sample Size in 10 Countries

CountryNo. of students

testedNo. of schools

tested

Western democraciesAustralia 3,331 142England 3,043 128Finland 2,782 146Sweden 3,073 138United States 2,811 124

Post-Communist countriesBulgaria 2,884 148Czech Republic 3,607 148Estonia 3,434 145Hungary 3,167 146Latvia 2,572 130


developed either in the original IEA measurement develop-ment phase (Schulz & Sibberns, 2004) or in subsequentscaling (Husfeldt, Barber, & Torney-Purta, 2005).1 Sepa-rate analyses were run in the five Western European coun-tries and in the five Eastern European countries. It was de-sirable to have the same number of clusters within eachregion. Examining the change in Schwartz’s Bayesian in-formation criterion, with each additional cluster added inboth the Eastern European and the Western Europeangroups, showed that a five-cluster solution provided gooddata fit within each group of countries and comparabilitybetween the regional groups.

Once a five-cluster solution was reached, each case wasassigned to one of these clusters. This created a categoricalcluster-membership variable that could be analyzed likeany other categorical variable. Cluster membership wasused in four analyses. First, average scores for each of the12 attitudinal variables that were included in the clusteranalysis were calculated. This produced a display of simi-larities and differences in attitudes among the five groupsof students in each set of countries. Second, the proportionof students in each cluster in each country was calculated.Third, a descriptive analysis compared the proportion ofstudents in each cluster who planned to participate in vari-ous civic activities and who valued the rule of law.

Fourth, a logistic regression analysis was run to determinewhether certain background variables could predict member-ship in one of the clusters. For this exploration of the useful-ness of person-centered analysis, the focus is on predictingwhether a student is a member of the group with the mostnegative civic-related attitudes (the alienated cluster that ap-peared in all of the countries). In this logistic regression anal-ysis, predictor variables including gender, socioeconomic sta-tus, immigrant status, experiences in classrooms or schools,and experiences with peers predict the log-likelihood of be-longing to the alienated cluster group. These log-likelihoodcoefficients can be transformed into odds ratios for ease ininterpretation, such that an x unit change in the predictor in-creases or decreases the likelihood of cluster membership by afactor of y. All of the analyses take into account the nestedsampling design within each country, weight the data to benationally representative of 14-year-olds attending schoolwithin the countries, and weight each country so that it con-tributes equally to the analysis.

The Five Attitudinal Clusters Foundin Adolescents From 10 Countries

The cluster analyses were run separately for the Western Eu-ropean and the Eastern European countries to identify differ-ences in patterns. Figure 1 presents the pattern of attitudes forthe five clusters in Western Europe, and Figure 2 presents the

parallel patterns for the five clusters in the Eastern Europeancountries. Note that each of the 12 attitudinal scales was set tohave an international mean of 10 and a standard deviation of 2.

The social justice cluster of adolescents contains individu-als who express high levels of support for minority rights,immigrants’ rights, and women’s rights (average of about onestandard deviation above the mean). This cluster is found inboth regional groups. Adolescents in this cluster also believethat the government has some social responsibilities. Studentsin this cluster generally have below-average beliefs in the im-portance of citizens participating in action, either in the con-ventional political domain or through social action in commu-nities or nongovernmental organizations. A motto for thiscluster might be “I believe in rights for everyone but do notfeel obligated to do much about it.”

The conventionally political cluster contains individualswho show high levels of trust in governmental institutions,who hold protectionist and patriotic attitudes (especially inEastern Europe), and who believe that governments havesocial responsibilities. Adolescents in this cluster have rela-tively high levels of political self-efficacy and believe inthe norm that adults should be active in both conventionalpolitics (e.g., voting) and in more socially oriented activi-ties (e.g., volunteering to help the community and joininghuman rights organizations). In Western Europe this groupalso has scores above the international mean on the socialjustice or inclusion scales (e.g., support for minorityrights). The support for social justice for this cluster inEastern Europe is very close to the international average.(See Malak-Minkiewicz, 2007, for a discussion of theseattitudes in post-Communist countries.) The motto forthis group might be phrased as “I believe in my countryand will support the status quo with positive politicaland civic actions that are expected of me.”

The indifferent cluster in both regions contains indi-viduals who have attitudes very close to the mean of 10on the large majority of the scales. In the Western Euro-pean countries, however, they also have strong protec-tionist and patriotic attitudes. Adolescents in the indif-ferent cluster are willing to do the minimum as citizens.Once they have met basic civic requirements they wantto spend their time on pursuits that interest them more.Zukin, Keeter, Andolina, Jenkins, and DelliCarpini(2006) found a similar attitudinal group in the UnitedStates among young adults; their indifferent respondentsbelieved that being a good citizen only required votingand not breaking laws. The disaffected cluster is similarto the indifferent cluster but with more negative beliefsabout women’s and minorities’ rights and about normsof citizenship related to involvement in the community.The motto for those in both the indifferent and the dis-

1 See these publications for specific items included in scales.


affected clusters might be “I have better ways to spendmy time than thinking about being active in politics, butI won’t do anything rash.”

The final cluster is of concern: the alienated profile ofattitudes. This group holds negative attitudes almost uni-formly across the scales, with scores on trust in govern-ment as much as 1.5 standard deviations below the interna-tional mean. These adolescents are more negative thanthose in any other cluster about the rights of immigrants,minorities, and women. Especially in Western Europe theydo not believe in norms of citizen participation. They seemto be alienated both from the political culture and frombelief in the rights of others. The motto for this groupmight be “I’m angry about the immigrants and minoritygroups in my country, and I don’t trust the government; Ihave the right to do what I want.” This cluster is numeri-cally small, but its members hold problematic attitudes.

In defining these clusters among young people, threeattitudinal dimensions were of importance: attitudes toward

social justice, especially about equality of rights; trust ininstitutions; and beliefs about the norms of adult citizen-ship (according to confidence intervals for the clusters).The central role of social justice attitudes is similar to theconclusions of a set of studies in the United States by psy-chologists interested in ideology (Jost, Federico, & Napier,2009) and also to some themes in the social psychologyliterature in Europe.

Distributions of Attitudinal Clusters Within Countries

In each of the Western European countries (see Figure 3)the most prevalent cluster type is the disaffected cluster(more than one third). Another similarity across thesecountries is that about 7% of students fall into the alien-ated cluster group.

In the United States, the second most prevalent clus-ter is conventional. A smaller proportion falls into thesocial justice cluster (slightly more than 15%), with the feweststudents in the indifferent and the alienated clusters. In most

Figure 1Cluster Profiles for Western European Countries

Note. womrt � Support for Women’s Rights; minor � Support for Minorities’ Rights; immig � Support for Immigrants’ Rights; cynic � Cynicism; effic � Efficacy; protc �Protectionist Attitudes to Country; patri � Patriotism; trustmed � Trust in the Media; trust � Trust in Political Institutions; govso � Support for Government Social Programs;ctsoc � Belief in Adult Citizen Social Action; ctcon � Belief in Adult Citizen Conventional Action.


respects these mirror what Damon (2008) found in his studyof patterns of purpose among adolescents.

Among Swedish and English adolescents the secondmost prevalent cluster (after the disaffected cluster) is thesocial justice cluster (about one quarter of students), with asmaller proportion in the indifferent and conventional clus-ters. To contrast, adolescents in Finland and Australia aremore likely to be found in the indifferent cluster (morethan one fifth of them), with smaller proportions in the so-cial justice and conventionally political clusters.

The patterns of clusters within the Eastern European coun-tries are more difficult to summarize (see Figure 4). The twomost prevalent cluster groups in Bulgaria are the conventionaland alienated clusters. The largest cluster group in the CzechRepublic is the disaffected cluster, with the social justice andindifferent groups also being substantial in size; the two larg-est cluster groups in Estonia are the disaffected and indifferentclusters; the largest cluster groups in Hungary are the conven-tionally political, indifferent, and disaffected clusters, whereas

in Latvia the disaffected cluster is the largest (about one thirdof the surveyed youths). The proportions of student in the alien-ated cluster tend to be higher in Eastern than in Western Europe.Guided by these exploratory results, Barber and Torney-Purta(2009) explore the patterns in more depth within countries.

To summarize, in all 10 countries there is a strikingproportion of students who are indifferent, disaffected, oralienated. In all of the countries these three clusters ac-count for more than half of the adolescents surveyed. Inother words, the absence of a positive connection betweenthe adolescent and his or her society is not confined to thepost-Communist countries.

Differences Between Clusters in Belief About the Rule ofLaw and Expected Participation

Looking at some other beliefs these adolescents hold alongwith their expectations for future participation helps to fur-ther clarify the meaning of these clusters. Across the 10countries the very large majority of adolescents support the

Figure 2Cluster Profiles for Eastern European Countries

Note. womrt � Support for Women’s Rights; minor � Support for Minorities’ Rights; immig � Support for Immigrants’ Rights; cynic � Cynicism; effic � Efficacy; protc �Protectionist Attitudes to Country; patri � Patriotism; trustmed � Trust in the Media; trust � Trust in Political Institutions; govso � Support for Government Social Programs;ctsoc � Belief in Adult Citizen Social Action; ctcon � Belief in Adult Citizen Conventional Action.


rule of law. However, adolescents who respond that it is“not important” or “somewhat unimportant” to obey thelaw are disproportionately found in the alienated cluster. Inboth regions about one quarter of the alienated adolescentsare disinclined to obey the law, compared with about 1%among those in the conventionally political cluster and 6%among those in the disaffected cluster. The alienated clus-ter is also the group that is most likely to expect to partici-pate in illegal protest activities (e.g., blocking traffic). Be-

cause of the strong anti-immigrant and anti-minorityattitudes that characterize the alienated cluster, the exis-tence of this group of adolescents should be a matter ofconcern in all of these countries. The alienated group issmall proportionally, but nearly 200 students out of ap-proximately 3,000 surveyed fell into this cluster in eachcountry.

Another sobering result concerns the social justice clus-ter. Although this group of adolescents holds the kind of

Figure 3Distribution of Cluster Membership in Western European Countries

Figure 4Distribution of Cluster Membership in Eastern European Countries


supportive intergroup attitudes that many adults would ap-plaud, these young people do not seem inclined to takepolitical action other than voting (presumably for candi-dates who support these positions). In both regions 10% orless of these students say they expect to protest illegally(e.g., block traffic). They are only moderately likely to ex-pect to engage in nonviolent protest (actually less likelythan adolescents in several of the other clusters). In theprevious section I noted that the social justice cluster mem-bers also did not think that participating in nonviolent pro-tests or belonging to human rights organizations was par-ticularly important for citizens. These relatively passiveyouth are hardly the inheritors of the Civil Rights Move-ment in the United States or the student protest movementsof Europe.

Predicting Which Adolescents Arein the Alienated Cluster

This exploratory person-centered analysis identified analienated cluster group in every country, and this is thegroup whose attitudes are of greatest concern. A logisticregression was used to identify characteristics associatedwith membership in this cluster. This analysis was con-ducted within region (rather than within individual coun-tries). See Table 2 for the results for Western and EasternEurope.

Males and students who spend more time outside oftheir homes with peers are more likely to be in the alien-ated cluster. In Eastern Europe low socioeconomic status isassociated with a greater likelihood of alienation. Therewas no association with immigrant status.

There are hopeful signs about the possible role ofschool in reducing the likelihood of alienation, although

the evidence is only correlational. The IEA CIVED hastwo school-based measures of climate, one focused on ado-lescents’ sense that they have power in the school setting(Confidence in the Value of School Participation) and theother measuring the adolescents’ belief that their class-rooms are places where respectful discussion of differentopinions takes place (Openness of Classroom Climate forDiscussion). Adolescents’ reports of positive school andclassroom climates are positively associated with a less-ened likelihood of belonging to the alienated cluster. Theseresults are parallel in the two regions.

Summary of the Person-Centered Analysis

Doing international research that matters requires that psy-chologists pay attention to individuals and not only to vari-able-centered analysis of group differences. Building amultimethod foundation for developmental science is oneway in which this can be accomplished. Another approach,person-centered cluster analysis, was illustrated here. Afterextensive variable-oriented analysis of the CIVED data itwas informative to embark on this exploratory analysis toidentify cluster groups of adolescents who share similarattitudes (which might be called ideologies). This approachlooked at adolescents’ involvement with distal institutionsand also at the kinds of climates in proximal institutions,such as schools, that allow them to become engaged ratherthan alienated. This analysis of data from 1999 across fiveWestern democracies and across five post-Communistcountries produces similar but not identical findings. Stu-dents holding strong patriotic and protectionist attitudes,for example, appear in different clusters in Western versusEastern Europe (see Figures 1 and 2).

Table 2Logistic Regression Analysis Predicting the Likelihood of Belonging to the Alienated Cluster

Predictor

Western Europe Eastern Europe

b SE Odds b SE Odds

Constant 4.31** 0.31 74.44 4.98** 0.24 145.47Gender �0.73** 0.09 0.48 �0.52** 0.06 0.59Immigrant 0.15 0.14 0.16 �0.02 0.16 0.98Socioeconomic status �0.02 0.05 0.98 �0.18* 0.03 0.84Spend time outside

of the home withfriends 0.16** 0.04 1.18 0.30** 0.03 1.34

Confidence in thevalue of schoolparticipation �0.33** 0.03 0.72 �0.46** 0.03 0.63

Openness ofclassroom climatefor discussion �0.34** 0.03 0.71 �0.19** 0.02 0.82

*p � .05. **p � .01.


These person-centered results complement sophisticatedvariable-centered analyses. Adopting a cluster-based ana-lytic strategy such as this also has a practical advantage forresearch that matters. There is a tendency for the generalpublic, practitioners, journalists, and policymakers to beunimpressed by statistical arguments based on mean differ-ences alone. An advantage of research that combines quali-tative and quantitative measures is that audiences outsideof the academic world can often be more readily convincedof a need for action if they can see the research findings aspart of a context rather than as an abstract statistical proce-dure. Calling attention to groups of individuals having par-ticular profiles within and across countries can aid re-searchers in interpreting the information gained from cross-national summary statistics. When it is possible to see acluster of individual adolescents who remind them ofyoung people they know, those involved in policy andpractice are much more likely to take action than whenthey are told only about averages and statistical trends.This research can be a starting point to look at the ways inwhich institutions (schools as well as more distal institu-tions) have an impact on the younger generation. This is acontribution that person-centered cluster analysis makes toenhancing opportunities for young people internationallyand therefore to research that matters.

The Role of Cultural Context in Research That Matters

Psychologists become reflective about the internationalshortcomings of their discipline about every 20 years. In1984, for example, American Psychologist published a spe-cial section titled “The Price of Success: Our MonoculturalScience.” Cole (1984), who was in the process of fullydeveloping his ideas about cultural psychology, had an arti-cle in this issue. He lamented the fact that cross-culturalpsychology, with its concerns about context, rarely in-formed psychological research in general. My article in thisissue was very practical, suggesting specific materials andtopics that could be used to internationalize undergraduatecourses in developmental and social psychology (Torney-Purta, 1984). Holding to this 20-year cycle, in 2004 theAmerican Council on Education involved the APA in across-disciplinary effort to internationalize the undergradu-ate curriculum for students. Adding to the justification forthis effort, in late 2008 Arnett documented the extent towhich articles published in APA journals use samples con-fined to the United States.

Psychologists often seem ambivalent about whether tosearch for depth of understanding of individuals embeddedin their social and cultural contexts or to search for resultsthat can be generalized across regions, countries, politicalinstitutional arrangements, and historical periods. The prob-lem of samples being concentrated in any single countrywould be of less concern if all authors routinely presentedenough contextual material that the reader could decide

how widely the findings could be generalized. An informalreview of articles recently published in Child Developmentand Developmental Psychology shows that studies con-ducted outside of North America were almost as likely asthose conducted in the United States to lack informationabout relevant context specific to the country or region. Forexample, the concern about generalizing inappropriatelyapplies to a study of racial attitudes conducted in Italy thatfailed to note which racial groups reside there or to ac-knowledge that social contexts or types of discriminationin Italy may differ from the social contexts and discrimina-tion that exist in other countries where racial attitudes arestudied (Castelli, Zogmaister, & Tomelleri, 2009). Thereare also problems with studies of academic motivation con-ducted in school systems with competitive selectivity (atleast if one assumes that a study’s results would be thesame in less selective systems). The same criticism couldbe made of studies of cognitive development in China thatfail to give sufficient contextual information about howfamilies relate to their children’s schools and learning (acriticism made by Liu, Wellman, Tardif, & Sabbagh, 2008)or studies of families that fail to take into account culturalnorms and common socialization practices (a criticismmade by Chen & French, 2008).

The more general point may be illustrated with an ex-ample. International research that matters would describethe goals and routines of family life as well as the expecta-tions students face in their schools or neighborhoods. Butthis research would not stop with the proximal contexts fordevelopment provided through interaction with families,peers, and teachers. It would look at schools as institutionsbut also at institutions beyond the school, including thelocal and national governments and their policies. In short,it is depressing to see not only that the large majority ofstudies published in APA journals are conducted with U.S.samples (Arnett, 2008) but also that few authors from anycountry consider distal contexts (e.g., the school’s or uni-versity’s selectivity or the nature of racial diversity in soci-ety).

Taking contexts into account can be framed in relationto the age-old psychological problem of figure and ground.The figure is usually the adolescent, whereas the ground issome mixture of contextual factors. Perhaps there shouldbe a reassessment of the balance between figure andground in psychologists’ research reports. It is not just thatmore diverse samples are needed (from across the world aswell as across regions and ethnic groups). It is also thatmore detail about the contexts in which these young peoplelive needs to be provided. Are they surrounded by otherslike themselves or diverse peers? What is the nature ofschooling to which they are exposed, and what stresses dothey experience? What out-of-school structures allow ado-lescents to explore ideas or to gain personal connection andpower in relation to the institutions of their society?


The Role of International Collaborationin Research That Matters

In recent decades, a characteristic of international psycho-logical research that matters is the collaboration betweenresearchers from different nations. Recently a workshopconducted by the National Academy of Sciences enumer-ated a variety of important contributions of such interna-tional collaborations:

These collaborations enable researchers to go beyond a viewof culture as a static variable to be examined in isolation orcontrolled in an analysis. . . . They can mobilize a global net-work to consider and refine important ideas concerning educa-tion and psychological interventions, as well as social poli-cies. They can give researchers new insights as they solve anunexpected problem. (National Research Council, 2008, p. 1)

International collaborations have a special role to play inenhancing the external validity of results, including deter-mining the appropriateness of interpretations. Understand-ing international collaboration as a process has the poten-tial to contribute to psychological research that mattersbeyond the content of the research findings themselves.Many boundaries are crossed in the process of collabora-tion: culture, language, discipline, and social or politicalinstitutional contexts. Reflection on these experiences canstrengthen both research and researchers’ skills.

During the workshop, a comparison was reported be-tween scientific research teams homogeneous in culturalbackground, discipline, and training and research teamsthat were heterogeneous along these dimensions (NationalResearch Council, 2008). The heterogeneous teams hadless harmonious discussion but generated more discoveriesthan did the homogeneous teams. Members of heteroge-neous teams complained about spending time explainingobvious points, but this process actually generated aware-ness of underlying assumptions and alternative conceptual-izations. This ultimately resulted in better scientific prod-ucts.

The workshop also considered the results of a surveyconducted for the U.S. National Committee for the Interna-tional Union of Psychological Science (IUPsyS). The viewsand recommendations of 26 leaders of internationalprojects on a range of topics in the behavioral and socialsciences were gathered in a survey that combined ratingsand opportunities for written elaborations (Torney-Purta,2008). The IUPsyS survey results suggest that successfulinternational collaborative projects often have three rela-tively distinct phases, and this is confirmed by the experi-ence of the IEA CIVED. The first phase requires buildingtrust and consensus among participants, surfacing assump-tions, and finding both commonalities and differences. Forexample, early in the CIVED study there was mistrust onthe part of some representatives who believed that the U.S.steering group members would propose test items that

would be biased against students in the new democracies.The committee addressed this concern explicitly duringconcrete planning of conceptual frameworks and measuresby attempting to take all countries’ perspectives into ac-count. The second phase is the conduct of the research it-self, facilitated here by IEA’s network of technical special-ists. During the third phase, researchers disseminateresearch findings, publish study results, and create a docu-mented data set available to the larger research community.Dissemination is challenging because of differences in theinterests of audiences in different cultural settings. How-ever, a basic principle is that all collaborators have a rightto reap benefits from the project: building skills, gainingprofessional advancement, developing networks of col-leagues, or seeing ways to apply findings to improve policyand practice in their countries. For the most part, thesewere realized through the IEA CIVED (Hahn, 2006), andthe research has mattered more because of this.

Conclusions

International research is more likely to be reflected in pol-icy or practice and in the mindsets of a new generation ofresearchers if it is possible to visualize real individuals inits reported results. The author of a study conducted in anysingle country should be expected to explicitly address de-tails of the cultural context in presenting and interpretingthe research results. When studies involve more than onecountry, the give and take of international collaborationamong researchers should be an integral and explicit partof the study’s development. In short, person-centered anal-ysis such as cluster analysis (complementing variable-cen-tered analysis), description of the context provided byproximal and distal groups in published studies, and a col-laborative research process are three dimensions operatingtogether and separately that have the potential to bring tointernational research the creative spark of imagination towhich Einstein referred.

Author’s NoteCorrespondence concerning this article should be ad-dressed to Judith V. Torney-Purta, Department of HumanDevelopment, University of Maryland, College Park, MD20742-1131. E-mail: [email protected]

References

Arnett, J. (2008). The neglected 95%. American Psycholo-gist, 63, 602–614.

Barber, C., & Torney-Purta, J. (2009). Cluster analysis ofattitudes in ten countries. Manuscript in preparation.

Castelli, L., Zogmaister, C., & Tomelleri, S. (2009). Thetransmission of racial attitudes within the family. Develop-mental Psychology, 45, 586–591.


Chen, Z., & French, D. (2008). Children’s social compe-tence in cultural context. Annual Review of Psychology, 59,591–616.

Colby, A. (2008). The place of political learning in college.Peer Review, 10(2/3), 4–8.

Colby, A., Ehrlich, T., Beaumont, E., & Corngold, J.(2007). Educating for democracy: Preparing undergradu-ates for responsible political engagement. San Francisco:Jossey-Bass.

Cole, M. (1984). The world beyond our borders: Whatmight our students need to know about it? American Psy-chologist, 39, 998–1005.

Damon, W. (2008). The path to purpose. New York: FreePress.

D’Andrade, R. (2008). A study of personal and culturalvalues: American, Japanese, and Vietnamese. New York:Palgrave Macmillan.

Garcı́a Coll, C., & Magnuson, K. (1999). Cultural influ-ences on child development: Are we ready for a paradigmshift? In A. Marsten (Ed.), The Minnesota Symposia onChild Psychology: Vol. 29. Cultural processes in child de-velopment (pp. 1–21). Mahwah, NJ: Erlbaum.

Hahn, C. (1998). Becoming citizens: Comparative perspec-tives on citizenship education. Albany: State University ofNew York Press.

Hahn, C. (2006). Comparative and international socialstudies research. In K. Barton (Ed.), Research methods insocial studies education: Contemporary issues and per-spectives (pp. 139–158). Greenwich, CT: Information Age.

Haste, H., & Hogan, A. (2006). Beyond conventional civicparticipation, beyond the moral–political divide: Youngpeople and contemporary debates about citizenship. Jour-nal of Moral Education, 35, 473–494.

Hess, R. D., & Torney, J. (2005). The development of po-litical attitudes in children. Chicago: Aldine. (Originalwork published 1967)

Hofstede, G. (1980). Culture’s consequences. BeverlyHills, CA: Sage.

Husfeldt, V., Barber, C., & Torney-Purta, J. (2005). Stu-dents’ social attitudes and expected political participation:New scales in the enhanced database of the IEA Civic Ed-ucation Study. College Park, MD: Civic Education Data

and Researcher Services. Available from http://www.terpconnect.umd.edu/�iea

Inglehart, R. (1977). The silent revolution: Changing val-ues and political styles among Western publics. Princeton,NJ: Princeton University Press.

Jennings, K., & Niemi, R. (1975). Continuity and changein political orientations: A longitudinal study of two gener-ations. The American Political Science Review, 69, 1316–1335.

Jost, J., Federico, C., & Napier, J. (2009). Political ideol-ogy: Its structure, functions, and elective affinities. AnnualReview of Psychology, 60, 307–337.

Kluckhohn, C. (1951). A comparative study of values infive cultures: Foreword in Navaho veterans, a study inchanging values. Papers of the Peabody Museum of Ameri-can Archaeology and Ethnology, 41(1), vii–ix.

Linnakyla, P., & Malin, A. (2008). Finnish students’school engagement profiles in light of PISA. ScandinavianJournal of Educational Research, 52, 583–602.

Liu, D., Wellman, H. M., Tardif, T., & Sabbagh, M. A.(2008). Theory of mind development in Chinese children:A meta-analysis of false-belief understanding across cul-tures and languages. Developmental Psychology, 44, 523–531.

Magnusson, D. (1998). Individual development: Toward adevelopmental science. Proceedings of the American Philo-sophical Society, 143, 86–96.

Magnusson, D., & Torestad, B. (1993). A holistic view ofpersonality: A model revisited. Annual Review of Psychol-ogy, 44, 427–452.

Mahoney, J., Stattin, H., & Magnusson, D. (2001). Youthrecreation centre participation and criminal offending: A20-year longitudinal study of Swedish boys. InternationalJournal of Behavioral Development, 25, 509–520.

Malak-Minkiewicz, B. (2007). Civic education in times ofchange: The post-Communist countries. Citizenship Teach-ing and Learning, 3(2), 58–69.

Mintz, A. (1958). Recent developments in psychology inthe U.S.S.R. Annual Review of Psychology, 9, 453–504.

Mullis, I. V. S., Martin, M. O., & Foy, P. (with Olson,J. F., Preuschoff, C., Erberber, E., Arora, A., & Galia, J.).(2008). TIMSS 2007 international mathematics report:


Findings from IEA’s trends in international mathematicsand science study at the fourth and eighth grades. ChestnutHill, MA: Boston College.

National Research Council. (2008). International collabo-rations in the behavioral and social sciences: Report of aworkshop. Washington, DC: National Academies Press.

Norusis, M. J. (2007). SPSS 15.0 statistical procedures.Upper Saddle River, NJ: Prentice Hall.

Reis, O., & Youniss, J. (2004). Patterns in identity changeand development in relationships with mothers and friends.Journal of Adolescent Research, 19, 31–44.

Schulz, W., & Sibberns, H. (2004). IEA civic educationstudy technical report. Amsterdam: International Associa-tion for the Evaluation of Educational Achievement.

Simola, H. (2008). Understanding the miracle of PISA:Historical and sociological remarks on Finnish teacher-hood. Education and Society, 26(1), 5–23.

Tapola, A., & Niemivirta, M. (2008). The role of achieve-ment goals orientations in students’ perceptions of andpreferences for classroom environment. British Journal ofEducational Psychology, 78, 291–312.

Torney, J., Oppenheim, A., & Farnen, R. (1975). Civic ed-ucation in ten countries: An empirical study. New York:Halsted.

Torney-Purta, J. (1984). Annotated bibliography of materi-als for adding an international dimension to undergraduatecourses in developmental and social psychology. AmericanPsychologist, 39, 1032–1042.

Torney-Purta, J. (2008). Results of a survey of interna-tional collaborative research in psychology: Views and rec-ommendations from twenty-six leaders of projects. In Na-tional Research Council, International collaborations in thebehavioral and social sciences: Report of a workshop (pp.64–78). Washington, DC: National Academies Press.

Torney-Purta, J., & Amadeo, J. (2004). Strengthening de-mocracy in the Americas through civic education: An em-pirical analysis of the views of students and teachers.Washington, DC: Organization of American States.

Torney-Purta, J., Lehmann, R., Oswald, H., & Schulz, W.(2001). Citizenship and education in twenty-eight coun-tries: Civic knowledge and engagement at age fourteen.Amsterdam: International Association for the Evaluation ofEducational Achievement. Available from http://www.terpconnect.umd.edu/�iea

Torney-Purta, J., Schwille, J., & Amadeo, J. (1999). Civiceducation across countries: Twenty-four case studies fromthe IEA Civic Education Study. Amsterdam: InternationalAssociation for the Evaluation of Educational Achieve-ment.

Torney-Purta, J., & Vermeer, S. (2004). Developing citi-zenship competencies from kindergarten through Grade 12:A background paper for policymakers and educators. Den-ver, CO: Education Commission of the States.

Torney-Purta, J., & Wilkenfeld, B. (2009). Pathways totwenty-first century competencies through civic educationclassrooms. College Park: University of Maryland.

Torney-Purta, J., Wilkenfeld, B., & Barber, C. (2008).How adolescents in twenty-seven countries understand,support, and practice human rights. Journal of Social Is-sues, 64, 857–880.

Triandis, H., & Brislin, R. (1984). Cross-cultural psychol-ogy. American Psychologist, 39, 1006–1016.

Yoshikawa, H., Weisner, T., Kalil, A., & Way, N. (2008).Mixing qualitative and quantitative research in develop-mental science: Uses and methodological choices. Develop-mental Psychology, 44, 344–354.

Youniss, J., & Hart, D. (2005). The intersection of socialinstitutions with civic development. In L. A. Jensen &R. W. Larson (Eds.), New directions for child and adoles-cent development: New horizons in developmental theoryand research (No. 109, pp. 73–81). San Francisco: Jossey-Bass.

Zukin, C., Keeter, S., Andolina, M., Jenkins, K., & Delli-Carpini, M. X. (2006). A new engagement? Political par-ticipation, civic life, and the changing American citizen.New York: Oxford University Press.


Documents

International Psychological Research That Albert ... - apa.org · When the APA s Committee on International Relations in ... Arti-cles based on award addresses are reviewed, but they