Rocking the Vote and More - Google Analytics Demystified
Transcription
Rocking the Vote and More - Google Analytics Demystified
Rocking the Vote and More: An Experimental Study ofthe Impact of Youth Political Portals Michael A. Xenos Kyurim Kyoung A B S T R A C T . This study provides a controlled lest of the effects of youth-oriented political portals. Based on existing theories of the individual effects of Interne! u.sc, we hypothesize that these sites may facilitate political engagement among memhers of their target audience in a variety of ways, but we also consider the possibilily thai such effects may ho moderated hy prior levels of political interest. Observations are drawn from a Wcb-hascd experiment administered lo undergraduates at a major midwestern university at the height of the 2006 election season. Findings indicate weak to nonsignificant main effects for exposure to youth-oriented political portals on self-reported cognitive engagement with election information, and a pattern of differential effects of exposure on opinion formation and domain-specific political efficacy, based on users' prior levels of interest in politics. K E Y W O R D S . Internet, online politics, political efficacy, youth The events surrounditig the 2004 presidential election sparked a lively debate over the youth vote. Prior to the election, commentators and academics cited the dramatic growth in efforts to mobilize new, and in particular young, voters. Notably, MTV's familiar "Rock the Vote" campaign was joined by a number of other high-profile efforts, virtually all of which featured a significant onlitie presence atid conducted large portions of their communication and mobilization efforts through the Web, e-mail, and other relatively new forms of comtmuiication technology, After Election Day, early postmortems on campaigns such as Rock the Vote were split. At first, these efforts were colorfully declared fruitless, often on the grounds that the fraction of all ballots that were cast by young people remaitied under 20%, displaying no major chatige sitice the last presidential election, and that the youth vote failed to swing the outcome in favor of John Kerry. Consider, for example. Michael A. Xenos is an assistant professor in the department of Communication Arts at the University of Wisconsin-Madison. He conducts research on the poliiical impacts of new media, particularly as they relate to young people and the electoral process. He is also interested in the broader implications of online communication for processes of public deliberation. Kyurim Kyoung is a Ph.D. student in the Department of Communication Arts, University of WisconsinMadison. Her research focuses on ihe impacts of new communication technologies on priKesses of poliiical stK'ialization among young [icoplc. This research was made possible hy a grant to the first author from the University tii Wisct)nsln-Madison Graduate School Research Committee. The data presented licrc will be deposited at the University of Wisconsin-Madison Data & Information Services Center (DISC) upon publication, and thereafter will be available at the DISC Wch site: http://www.disc.wisc.edu/archive.html. Address correspondence to: Michael A. Xenos, University of Wisconsin-Madison, 6136 Vilas Communication Hall, 821 University Avenue, Madison, Wl 53706. (E-mail: xenos@wisc.edu). -I .^ • Journal of Information Technology & Politics. Vol. 5{2) 2008 Available online at http://www.haworthpress.com © 2008 by The Haworth Press. All rights reserved. doi; 10.1080/19331680802291400 Í7S i 76 JOURNAL OF INFORMA TION TECHNOLOG Y & POLITICS the late Hunter S. Thompson's assessment: "Yeah, we rocked the vote all right. . . . Those little bastards betrayed us again" (Vargas, 2004, p. COI). But this ftame was soon countered by spokespersons for Rock the Vote itself and scholars such as Thomits Patterson, who pointedly contended that these early assessments were "badly off the mark" (n.d., p. 5). The counterframe pointed out that while the youth vote changed little as compared with voting rates among older Americans or when expressed as a proportion of total ballots, the percentage of young people casting ballots had changed substatttially (frotn 42.3% to 51%). Viewed in this light, the 2004 youth vote represented an increase of 4.6 million voters under 30 and the greatest increase in the absoltite number of young voters since 1972 (Coles, 2004; Patterson, n.d.). In addition, young voters were not only ttn'ning out to the polls in record nutnbers, but also paying attention to, as well as reading and talking about, the election markedly more than in previous years (Andolina & Jenkins, 2004). Examining trends from this angle, it thus would appear that youth voter mobilization efforts, including online campaigns such as the one waged through the Rock the Vote Web site, may have made positive contributions to youth civic engagement in 2004. There are, however, few empirical studies concerning how and in what ways online initiatives targeted at young voters may contribute to youth political engagement. To be sure, a good deal of the literature on Internet comtnunication and politics suggests that such Web sites have the potential to increase political interest, attitudinal engagement, and political participation (Kalyanaraman & Sundar, 2(X)6; Lupia & Philpot, 2005; Tolbert & MacNeil, 2003). However, a number of studies foctised on recent trends in youth engagement suggest that the impact is slight or indirect at best. For example, survey data from 2004 suggest that young people weie stimulated to greater levels of political engagetnent in the 2004 catnpaign primarily because of its "issues, personalities and intense partisanship" (Patterson, n.d., p. 7). Aggressive off-litie mobilization and get-out-the-vote efforts also have been identified as partially responsible for healthier youth turnout in 2004 (Lopez, Kirby, & Sagoff, 2004). Examining online political cotnmunication in general and youth-directed political Web sites as well, Xenos and Foot (2007) also suggest that such content is unlikely to reach all but the most interested would-be young voters. Indeed, a variety of studies point to a differential effects model for understanding the impact of online political cotnmunication as tnoderated by political interest or other individual characteristics atid behaviors (Bimber, 2003; Evelatid, Marton, & Seo, 2004; Nisbet & Scheufele, 2004; Prior, 2005; Shah, Kwak, & Holbert, 2001). This article presents results frotn an experimental study of the effects of online youth politics initiatives conducted at the height of the 2006 midterm election season. Guided by recent political communication research on the effects of Internet communication on political engagetnent, we designed an experiment to test the impact of two of the most significant youth politics portals on a variety of political engagemetit indicators, within the context of an ongoing election cycle. The findings identify bolh sites us particularly effective in communicating basic information on the voting process itself (e.g., registration and finding one's polling place), but litnited in their ability to facilitate political engagement in other ways, especially among users with lower levels of political interest. We conclude by discussing the implications of this research for future scholarly and policy-oriented efforts to understand and facilitate the effectiveness of youth-oriented political communication online. The Internet and PoUticai Engagement Researchers have wondered for some time about the potential of the Internet for facilitating political engagement, particularly among the technologically savvy younger generation (Delli Carpini, 2000). Although research in this area covers a wide variety of characteristics of online cotnmunication, we focus on ihree as especially relevant to processes by which young people may be mobilized to greater engagement in electoral politics. Specifying these media characteristics and the possible ways that online political comtnunication Xenos and Kyonng generally may stimulate greater political involvement provides an important theoretical context for considering the effectiveness of youth-oriented political portals that seek to realize this potential for a particular demographic group. In particular, we consider the Internet's unique capabilities for (a) efficient information transmission, (b) interactivity, and (c) targeting specific subsets of the electorate. First, online communication provides an unprecedented conduit for the relatively cheap and efficient communication of large quantities of political information (Bimber, ^003; Rheingold, 2(KX)). In line with an "in.strumenta!" approach to understanding individual-level effects of Internet use on political engagement, this provides the most straightforward set of expectations concerning the potential of youth-oriented political portals (see Bimber, 2003). By reducing the cost of acquiring information on the logistics of registration and voting, as well as information on candidates and issues, such portals may help facilitate youth political engagement by helping young citizens think about elections, form preferences about issues and candidates, and find their way to their local polling stations. A second characteristic of online political communication relevant to understanding the possible effects of youth-oriented political portals is its interactive nature. Considered by some to be the quintessential element of Web communication (e.g.. Buey, 2004; McMillan & Hwang, 2002), interactivity may be another way in which online political communication can serve to facilitate greater engagement in the political arena. Conceptualizing interactivity in a variety of ways, research has demonstrated that interactive content can have a number of positive effects on political engagement. For example, in their study of interactivity on candidate Web sites, Sundar, Kalyanaraman, and Brown (2003) found that greater levels of interactivity (operational i zed as site depth, or the number of clickable pages) stimulated more favorable perceptions of the fictional candidate depicted in their study, as well as greater agreement with the candidate's issue stances. Examining both technical forms of interactivity (consisting of site features enabling two-way 177 exchanges among site visitors and site producers) as well as text-based forms (such as t1rst-person text and photo captions), Warnick, Xenos, Endres, and Gastil (2005) found moderate levels of interactivity on fictional candidate Web sites stimulated significantly greater cognitive engagement with site content. Studying the effects of interactivity on young people during the 2004 campaigns, Tedesco (2006) also found similar effects, as well as increases in political efficacy among participants in his study exposed to more interactive Web materials. Clearly then, it is reasonable to identify interactivity as another possible mechanism through which youth-oriented political content on the Web may facilitate youth engagement. Assuming the presence of useful information about candidates and campaigns, interactivity may serve to draw young voters into greater cognitive as well as affective engagement with election-related issues. In addition to its ability to transmit vast sums of political information efficiently at relatively low cost and its potential for interactive exchanges, a third feature of online political communication useful for understanding the impacts of youth politics sites concerns the targeting of distinct audiences. In part the culmination of the first two factors, the tendency for online content and user behavior to fragment around myriad differences in interest and preferences, is yet another feature of online communication that scholars have identified as one of the defining characteristics of new media (Bimber, 2003; Howard, 2006; Schier, 2000; Sunstein, 2001). Given relatively low production costs, the ability of users to navigate efficiently to sites that refiect their unique perspectives, and the tendency of similar kinds of sites to be nested within hyperlink networks, it is important to consider the tailoring of communication to particular audiences as a third process by which particular kinds of sites might affect the attitudes and behaviors of their users. Although specific re.search on Internet effects in this area is scant, political scientists have long noted the connection between political engagement and targeted recmitment efforts (Verba, Schlozman, & Brady, 1995). Additionally, research on Web customization in general 178 JOURNAL OF INFORMATION TECHNOLOGY a¿ POLITICS suggests that when site producers specifically tailor or target cotnmunications to a particular audience, users are more likely to hold positive attitudes toward the site and its content. Positing that customization and targeting play upon individuals' desire to affirm their individual identities, for example, Kalyanaraiiian atid Sundar (2006) conducted an experiment in which participants were exposed to varying levels of customized content through an all-purpose Web portal, and found that increasing customization was indeed associated with more positive attitudes toward the portal. Further, focusitig in particular on the effects of online political communication consumed by younger users. Lupia and Philpot (2005) found that a critical factor in stitnulating political interest for this group thtough the Web was the extent to which sites are perceived to be effective and efficient for finding information that younger users ate interested in; specifically. Lupia and Philpot argue that "those [Web site producers] who want to increase young adults' interest can succeed if they tnake politics rnore relevant to them" (2005, p. 1138). Youth-Oriented Political Portals With these concepts in mind, it is reasonable to expect that youth-oriented political portal Web sites like RocktheVote.com could help stimulate greater levels of political engagement among youthful site visitors. To begin with, the world of youth-oriented civic and political Web sites has grown dramatically in recent years (Montgomery, Gottlieb-Robles, & Larson, 2(X)4). More important, along with the growth in the number of sites there also has been a steady increase in the amount of political information youth politics sites provide. Examinitig the evolution of the "youth political Web sphere" from 2002 to 2004. Xenos and Bentiett (2007) found that during that period the tutmber of Web sites focusing speciftcally on electoral politics and targeted at American youth swelled ftom 22 to 35 identifiable sites, and that by 2004 the majority of such sites provided "information on the elections in some fonn," hicluding "infonnation on how to register to vote, and information about events and ways to get involved" (p. 451 ). Additionally, youth-oriented political Web sites are targeted speciftcally at young people, with general political inforrnation on candidates and elections framed aroutid youth coticems and presented in formats favored by young people. For example, in addition to documenting the growth in political information available through youth politics sites, Xenos and Bennett (2007) also found youth-oriented Web sites to be substantially more interactive than other online political content, such as sites produced by political candidates; indeed, by 2(X)4 the "youth political Web sphere" had developed a dense network structure with ample hyperlinks contiecting youth sites with each other and, to a lesser extent, with other key sources of political information on the Web (Xenos & Bennett, 2007). Further, as explained directly on the "About RTV" page of the Rock the Vote Web site (http:// www.rockthevote.com). the otganization's goal is explicitly to harness "cutting-edge trends and pop culture to make political participation cool." In other words, youth politics portals offer information about issues and campaigns (and/or links to such infomiation), and do so in an interactive fashion that also potentially engages the identityaffirming process posited by Kalyanaraman and Sundar (2006), which in this case may foster an identity that includes cognitive engagement with political infortnation as well as political participation as key attributes. On the basis of these considerations, we thus hypothesize that youth-oriented political Web sites facilitate greater levels of political engagement atnong their targeted user base. Speciftcally, we identify two specific mechanisms by which youth politics portals may contribute to political engagement among younger users: Hypothesis I. Exposure to youth-oriented political portals will be associated with greater levels of cognitive engagement with election-related infomiation and the formation of election-related opinions. Hypothesis 2. Exposure to youth-oriented political portals will be associated with positive attitudes towatd political engagement. 179 Xenos and Kyottng The Potential for Differential Effects As indicated at the outset, however, there are a variety of reasons to question whether there is a simple positive relationship between use of youth politics Web sites and political engagement. At the surface, survey data from 2004 suggest that recent increases in political engagement among younger people were driven strongly by off-line forces, such as the prevalence of issues especially of interest to young people (e.g., the Iraq war), and traditional fonns of voter contact, rather than the proliferation of online politics (Lopez et al., 2004; Patterson, n.d.). At a deeper level, research on the effects of Internet use on political engagement consistently has found that effects are often moderated by the individual characteristics of users. Dubbed the "psychological approach" by Bimber (2(X)3; in contrast with the instrumental approach discussed earlier), ihis perspective theorizes that political effects of new media are contingent on individuals' levels of sophistication, tnotives, and social context. Studies in this vein suggest that we tiiight better understand the effects of youth politics Web sites by considering the possibility that they may exert differential effects on site visitors based on relevant personal characteristics. For example. in the realm of online news, Eveland et al. (2004) found that the level of experience with using the Web for news moderated gains in knowledge. Further, positing a process by which talk helps to distill and elaborate on information encountered online, Nisbet and Scheufele (2004) found frequency of political talk lo moderate the relationship between Internet use and a variety of political engagement outcomes (i.e., political efficacy, knowledge, and political participation). Finally, research on general Internet use and social capital also suggests that a key determinant of whether this relationship is positive or negative is whether users tend to seek information from the Web, or if they use it more for social-recreational purposes (Shah et al., 2001). Within the context of political portals designed to increase political engagement among American youth, we identify users' pre-existing level of interest in politics as a likely moderator. The rationale behind this is that, as a group, youth traditionally have been at the forefront of technology adoption and are already among the most likely to have experience seeking political information online (Rainie, Cornfield, & Horrigan, 2005). Thus, we assume that variation on this front would be a poor tnoderator lor the youth demographic. Political interest, however, has been identified as a significant predictor of political knowledge (Delli Carpini & Keeter, 19%), as well as political participation in general (Verba et al., 1993). Moreover, political interest is also strongly associated with political discu.ssion, identified earlier as a moderator of Internet effects (McLeod et al., 1999; Pan, Shen, Paek, & Sun. 2006). Indeed, interest itself has been identified as a moderator of Intemet effects in prior research and a significant predictor of online political mobilization (Krueger, 2006; Xenos & Moy, 2007). This suggests that the degree to which online efforts to stimulate political involvement of young people are successful may not only hitige on engaging youth "on their own lertns" (Lupia & Philpot, 2005, pp. 1137-1138), but also on whether site visitors have sufficient motivation and basic political schema to begin with. Although the interactive and targeted communication provided by these Web sites can motivate youth and stimulate their interest, it is reasonable to expect that, particularly in the short term, those who already possess a modicum of interest or motivation may be best positioned to take advantage of the opportunities and information provided by these sites. We thus offer the following research question as a cotnplement lo our two hypotheses: To what extent are the effects of youth-oriented political portals on youth political engagetnent tnoderated by initial levels of political interest? METHODS To provide a controlled test of the effects of youth-oriented political Web sites, we created a three-condition, between-subjects design in which participants engaged in an eight-minute ' T ISO JOURNAL OF INFORMATION TECHNOLOGY ai POLITICS Web-browsing exercise, following a brief pretest, which included a measure of our tnoderator variable: political interest. The browsing session was followed by a post-test survey designed to probe for potential differences in cognitive engagement and attitudes among sttbjects resulting frotn interaction with the variotis treatment Web sites. Panicipants (A' = 197) were recatited from ititroditctory courses in political scietice and coninuinications at a large midwestern university. While certainly not representative of the general population of 18-29 year olds, the participants lie squarely within the target demographics for the sites under analysis. experience. After precisely eight minutes, the ex peri tuen ta I software terminated the browsing sessions and directed participants to the posttest survey, which began with items that asked participants to evaluate their browsing sessions, and tnoved on to questions about their opinions on the official ballot items as well as a variety of other political attitudes atid behaviors. Following the logic outlitied by Lupia and Philpot (2005), we designed the experiment to evaluate the impact of particular Web sites rather than youth Web sites in general or the Web in general for that tnatter. Indeed, given the vast anay of Web sites available to users, it is AH data were collected in a computer labora- important to distinguish analytically between tory between October 30th and November 7th, studies concerning the impact of the Web, and 2006, with the majotity of observations taking those that enable distinct itiferences concerning place tiear or on Election Day. The experimen- the impacts of particular locations in cybertal stimuli and survey instruments were admin- space. Based on a variety of considerations, istered using the MediaLab software platform iticlitding a preliminary network atialysis of the (Empirisoft Corporation, New York, NY). youth political Web sphere cotiducted prior to Upon arrival in the lab, a lab attendant seated fielding the experiment, we selected the Web each participant at a computer tertninal. The sites of Rock the Vote (http://www.rockthevote. first screen on each terminal displayed a state- com) and the New Voters Project (http://www. ment approved by the university's hitman sub- newvotersproject.org) as our primary stitmulus jects review panel informing participants of sites to create two youth-politics conditions. what the study would entail. Consentitig partici- Specifically, we selected Rock the Vote ba.sed pants then began the pretest, which enabled us on its status as the tnost widely known site of its to tap pre-existing levels of interest in politics kind, and the New Voters Project on the basis of among participants prior to their interaction its high level of influence within the hyperlink with the stimulus Web sites. The pretest con- network. These characteristics of the sites are cluded by informing participants that they consistent with a Freeman degree analysis of would be asked a nutnber of qtiestions concern- each site within the larger youth politics Web ing the election after having eight minutes to sphere, which included 134 Web siles during the use the Web to learn about the candidates and fall of 2006. We conducted this analysis using issues that would itppear on official ballots in the UCINET software package (Analytic Techthe area. Participants were also itiformed that nologies, Lexington, KY). Following the logic they would be asked to evaluate their experi- of social tietwork analysis, this procedure ence with the first Web site that appeared in the etiiibles us to examine bolh the influence of each browset\ which was randomly assigned. node in the network, as well as its prestige, by Although paiticipants were able to navigate looking 10 linkages flowing to or from each freely during the browsing session using avail- actor (Hanneman & Riddle, 2005). Based on able hyperlinks on each page, the address bars this analysis, we fmd that Rock the Vole clearly were removed frotii (he otherwise standard outpaces the New Voters Project in terms of browsers, and patticipants were instructed to prestige, receiving 139 links ftotii other points fully explore the first site encountered before in the network, as compared to only 21 such venturing away. This portion of the protocol links for the New Voters Project. Alternatively, was designed to preserve an element of control looking at influence, or outlink patterns, we see over exposure to the stitmulus Web sites while the New Voters Project Web site supplying 279 also retaining elements ofthe traditional browsing linkages to other destinaliotis within the Xenos and Kyoung network, as compared to only 6 for Rock the Vote's Web site. Participants in the third condition were taken to (he Google home page and instructed to use the site as they normally would. In this sense, the design enables us to isolate effects stemming from either or both of the youth politics Web sites, as cotnpared to an undirected or selfdirected session with the Web in general. Ultimately, 74 participants were assigned to the Rock the Vote condition, 64 to the New Voters Project condition, and 59 to the Google condition. Assessing the Impact of Youth-Oriented Political Web Sites: Key Outcomes We chose three sets of post-test items to test our hypotheses and assess the impacts of the Rock the Vote and the New Voters Project Web sites on our undergraduate participants. The first two sets of measures were designed to test Hypothesis I (HI), which concerns cognitive engagement with the candidates and issues surrounding the election, as well as the process of voting itself. The third set, designed to test Hypothesis 2 (H2), deals with attitudes toward political engagement. The first set of dependent variables consists of two self-report items that tap the usefulness of the Web sites for the assigned information task. Specifically, we asked participants whether the site in question was helpful to them "in thinking about the major issues in the various races" and "in understanding how to cast [their] ballot on Election Day (voter registration, polling places, etc.)." Responses to these itetns ranged from 0 (not helpful at all) to 3 (very helpful), and provided a subjective indicator of the degree to which metnbers of the target user-base for such sites found them suitable to one of their primary purposes. The second set of outcome variables we examined was designed to assess the extent to which the browsing sessions may have helped participants form opinions on ballot items that would appear in the geographic area in which the study was conducted. To examine this opinion-fortnation prt)cess, we included items in the post-test survey that asked participants which candidate they preferred in their state's m gubernatorial and Senate races, as well as their di.strict's House race. Additionally, we asked participants to express an opinion on a ballot item concerning gay marriage. For each item, participants could either express a preference or mark a "don't know/no opinion" response. These responses were converted later into a count of up to four ballot items for which participants had a clear preference. These measures enabled us to examine, in more objective terms, the extent to which the sites helped our youthful participants engage cognitively with a critical part of the voting process—forming opinions to express on Election Day. The third set of outcome variables we examined was designed to gauge the effect of exposure to the stimulus Web sites on attitudes young people hold toward political engagetnent or political efficacy. In the classic sense, political efficacy refers to "feeling that individual political action does have, or can have, an impact on the political process . . . and that the individual citizen can play a part in bringing about this change" (Catnpbell, Gurin, & Miller, 1954, p. 187). This concept long has been studied as a critical element in explaining variations in political engagetnent. Indeed, research has shown that feelings of citizen competence and confidence in the political systetn aœ powerful predictors of patiicipation (Abramson, 1983; Rosenstone & Hanson, 1993; Verba et al., 1995). Rather than deploying political efficacy in general as an outcotne measure, however, we decided to narrow our focus to a more contextspecific form, which we call Internet-specific political efficacy. As Bandura (1997) has pointed out. political efficacy in the global sense summarizes a diversity of beliefs and attitudes that span a variety of contexts. In addition, a nutnber of scholars fruitfully have explored particular types of efficacy, iticluding protest-specific, deliberation, and information forms (Boyle, 2004; Mottell, 2003, 2005; Tedesco, 2007; Wollman & Stouder, 1991). For this study, we chose to examine efficacy specifically tailored to the Internet as a medium for learning about, participating in, and affecting politics. As defined by Kyoung (2006), Internetspecific political efficacy refers to "a feeling 182 JOURNAL OF INFORMA TION TECHNOLOGY & POLITICS TABLE 1. Dependent Variables Browsing helped think about issues Browsing heiped understand the voting process Ballot opinions heid Internet Poiiticai Efficacy (Internal) Internet Political Efficacy (External) Min Max 0 0 0 1.83 1.75 3 3 4 7 6.75 Mean SD 0.75 0.95 1.16 0.94 0.95 1.30 3.ie 4.66 4.42 N= 197. that political activity taking place on the Internet can have an influence on political processes and one's confidence in his/her own capabilities" to exercise such infiuence (p. 50). Based on the content and design of youth politics portal sites, we believe this efficacy measure offers a particularly appropriate measure of site effectiveness. Following the evolution of research on political efficacy, which stre.sses the importance of clearly stipulated attitudinal objects, we used two indices. These indices minor the distinction in the literature between internal efficacy, where the object is the self, and external efficacy, for which the object is one's political context (Craig & Maggiotto, 1982). Within the context of the current study, our Internetspecific internal political efficacy index was based on a series of items asking participants their level of agreement (on a scale of I to 7) with statements such as "I consider myself skillful in using the Internet to search for information on politics and public issues" and "I feel well-equipped to use the Internet to express myself politically." The Intern et-s pec i fie external efficacy index used the same format, with statements such as "The Internet allows ordinary people to have more say in government" and "The Internet makes public officials care more about the concerns of ordinary people." In all, the Internet-specific internal political efficacy index included six items (Cronbach's alpha = .71), and the Intemet-specific external political efficacy index included four items (Cronbach's alpha = .70). Prior empirical studies using these measures have revealed these constructs as both distinct from traditional or global efficacy and reasonably predicted by relevant independent variables (Kyoung, 2006). Information on the distributions of all dependent variables used in this study can be found in Table 1. I Political interest To shed light on our research question conceniing the potential moderating role played by political interest, we also included a measure of participants' interest in politics in our analysis. Specifically, we asked participants to estimate how often they follow politics and public affairs, using a format similar to that used in the National Elections Studies and other national survey protocols. To reduce context effects, or the chance that responses to this item would be affected by other questions in the survey (Bishop, 1987), as well as to capture interest prior to administration of the stimulus, we placed this item at the very beginning of the pretest survey instrument. Responses ranged from "hardly at all" to "most of the time," producing a measure ranging from 0 to 3 (AÍ = 1.77, SD = 0.68). RESULTS To assess the potential for exposure to youthoriented political Web sites to facilitate greater political engagement among young site visitors, we began with a simple comparison of frequencies or means for all five of our outcome variables across the three conditions. These results included substantial (though not entirely expected) differences across the conditions for the subjective ratings of site usefulness, but suggested no significant main effects fiowing i83 Xeitos atid Kyoung frotn any of the sites to the number of ballot opinions held or Internet-specific political efftcacy. Examining the subjective reports of cognitive engagetnent provided by our participants, the most useful sites for learning about issues were Rock the Vote.com and Google, which received mean ratings of 1.08, {SD = 0.81) and 1.02 {SD = 0.68). respectively. Surprisingly, the New Voters Project Web site received a signiftcantly lower rating (M - 0.55, SD = 0.64), indicating that participants found this site less useful, relative to either the Rock the Vote Web site or a simple Google search session undirected by the experitiienters. Results of crosstab chi-square test comparing the ordinal ratings across conditions provide support for the notion that these differences in the ratings patterns are significant (p < .001). In conttast, an exatnination of subjective ratings of the sites in terms of their provision of basic information about the voting process yields results that are mote consistent with expectations. On this itetn, the New Voters Project and Rock the Vote sites clearly outperformed Google, receiving mean ratings of 1.41 {SD = 0.79) and 1.61 {SD = 1.06), respectively, as compared to Google's mean rating of 0.83 (SD = 0.81). Again, we assessed the differences in responses to this item with a cross-tab chi-square test, which confirmed that these variations were also significant (p < .(X)l ). For ea.se of interpretation. Figure 1 provides a graphical sumtnary of means for both usefulness ratings across the conditiotis. In all, these ftndings provide only modest support at best for HI, concerning the facilitation of cognitive engagement, and no support for H2, which deals with attitudes toward political engagement. Specifically, our results suggest that sites produced by organizations such as Rock the Vote and the New Voters Project effectively provide information on the voting process, but likely do not have a significant direct impact on young voters' abilities to deal with issues and choices related to voting or their attitudes towatd the Internet as a political tool. Indeed, in the case of learning about candidates and issues, our results indicate that participants found a simple Google search devised on their own was more helpful than visiting the Web site produced by the New Voters Project. To be fair, it is clear that conveying basic infortnation about the voting process is likely the top priority for the producers of such sites (a point we shall retum to later in the discussion). Additionally, it is worth recallitig that in comparing to the Google search condition we are in effect comparing one site to all those available through the rest of the Intertiet. However, the FIGURE 1. Subjective ratings of browsing session usefulness by condition. Rock the Vote New Voters Project B Help«d think aboutissues Google BHetped understand th» votin g process 184 JOURNAL OF INFORMATION TECHNOLOGY & POLITICS absence of main effects for outcomes other than learning about how lo register and where to vote is inconsistent with the expectations we derived earlier concetning the potential for such sites to facilitate greater involvement in politics among the young people they are targeted toward. This raises questions conceming whether the youth politics sites we examined truly have no effect on engagement with issues or attitudes about the Web as a political tool, or whether such effects may be concentrated among a subset of our participants, as suggested by our research question. To address whether the effects of youthoriented political poitals are best understood as moderated by pre-existing levels of political interest, we specified a series of regression models, using dummy coding for exposure to each Web site and the Google condition as the reference category. These regressions enabled us to test for main effects of the stitnulus Web sites in question, as well as interactions with political ititerest. For our subjective Web site ratings variables, as well as the Internet-specific internal and external political efficacy variables, we tested for these effects using ordinary least squares methods. In the case of the issue opinion counts, which may be inteipreted as the number of successful opinion formation processes over a set of four trials (representing each ballot item), we specified a logistic-binomial model (Gelman & Hill, 2007). In tnodeling subjective ratings of cognitive engagement, entering political interest and the site multiplied by interest interactions into our specifications resulted in washing out any main effects for exposure to the youth politics sites, and did not yield significant results for the interaction terms. Perhaps unsurprisingly, in the case of ratings for how helpful lhe browsing session was for helping participants understand the voting process, we even found a significant negative effect for political interest (B = -.39, SE B = A^, p = .04). In other words, these results suggest that participants with high levels of prior political interest were the most likely to rate their browsing session negatively, perhaps because many of them already knew or understood much of the basic information the two youth politics sites are designed to convey. This is consistent with our suspicion that pre-existing levels of political interest are a critical part of understanding how such sites may affect young users, who may vary sub.stantially in the extent to which they already understand basic parts of the political process, such as registering to vote and finding their polling place. The picture becomes even more interesting when we turn to whether participants held opinions on the full slate of ballot items available to them and our Internet-specific political efficacy measures. The full results of these regressions are presented in Tables 2 and 3. Here, the results offer a strongly affirmative answer to the question of whether the effects of youth politics sites may be moderated by political interest. In the case of modeling the number of ballot opinions held, we find that the interaction term New Voters Project x Political Interest is positive and significant (p = .01). Indeed, if we simply estimate the probability of holding a full slate of ballot opinions versus something less (results not shown), the associated odds ratio for the New Voters Project interaction suggests that politically interested participants randomly assigned to this site in particular were over four times more likely to hold an opinion on all four TABLE 2. Logistic Binomial Regression: Number of Ballot Opinions Held Coefficients Experimental Conditions Rock the Vote New Voters Project Moderator Political Interest Site/Interest Interactions Rock the Vote x Political Interest New Voters Project x Political Interest Constant N LL 'p< .05; "p< .01. -.55 (.59) -1.47* (64) 1.62**' (-42) .31 (.33) .98'* (.36) -1.27 (.72) 197 -264.87 44.06*" Xenos and Kyoung TABLE 3. OLS Regressions: Internet-Specific Political Efficacy Experimental Conditions Rock the Vote New Voters Project Adjusted R^ Moderator Political Interest Adjusted R^ Site/Interest Interactions Rock the Vote x Political Interest New Voters Project x Political Interest Final Adjusted R^ Internal Efficacy Externai Efficacy -.05 (.17) -.01 (.17) .00 .01 (.17) .14 (.17) .00 .50'" (.09) .12 .29" (.10) .03 .44* (.23) .53' (.24) .13 .59* (.24) .59" (.25) .06 plots are based on estimated internal and external Internet-specific political efficacy values, using the results from the regressions presented in Table 3, and imputed values of 1 and 3 for political interest. For each set of estimates, we see relatively little difference between ititerested and noninterested participants exposed to the other sites. However, exposure lo the youth politics Web sites introduced differences of a standard deviation or more in the efficacy values of participants at each end of the political interest scale. In other words, for our opinion formation and efficacy outcotnes, we found that the benefits of exposure to youth politics portals appear to be concentrated atnong those already possessing a tnodicutii of intere.st in politics, with tiegligible to negative effects on the participant base as a whole. N= 197. #p<.10;'p<.05;**p<.01. of the ballot items included in the post-test instrument. In the case of Internet-specific political efficacy, we also see positive interaction effects between political interest and exposure to the youth politics potials. A graphical representation of these interactions is found in Figure 2. These DISCUSSION In this study we have provided a rare, controlled exploration of the effects of youth politics Web sites on a critical portion of their intended audience. By and large, our findings are consistent with the conclusions of those who have considered the role played by such sites in the political process using tnore indirect methods (e.g., content analysis and survey FIGURE 2. Interaction plots: youth sites, political interest, and Internet-specific political efficacy. Rock Oit Vote: Internet-Specific Internal EHicacy Rock the Vote: Inl«rnet-S|iecirc Extiinal Etficacy ^ ^ ""^ " "- . , ai«! süBi 1 Naw Voters ProjscI: Itverr^et-Specific Int Lo* tiffiKiU FfDck tuVtita HmhlrtwKl] Nsw Votars PtoJBci: Internet-Specific External Efficacy 186 JOURNAL OF INFORMATION TECHNOLOGYi& POLITICS tnethods); indeed, they suggest that it is in the interplay of off-line and online factors that we see the most interesting developments. However, our findings offer a tnore precise attempt at parsing out the degree to which online political communication can infiuence political engagement atnong young voters than comparisons of broader trends allow. Specifically, the results of our controlled experiment suggest that the Web cotiiponents of campaigns such as Rock the Vote and the New Voters Project have only a slightly positive effect on educating young people about the voting process, bttt that moie substantial effects are concentrated atnong those who are already interested in politics. In other words, our data stiggest that these sites may offer a novel and marginally effective way of introdttcing young people to basic political information and the logistics of the voting process. But, as we see in the cases of opinion formation and efficacy, these sites tnay truly "rock" the vote only for those who already have expressed a noticeable desire to get out on the dance fi oor. Before considering ihe theoretical and practical implications of these findings, it is important to discu.ss briefly some litnitations of the study and the possible effects of these limitations on the results. First, we note an itnportant cost associated with the ptecision afforded by a controlled expetimetit. That is, our observations were gathered in a laboratory and were based on a relatively brief exposure to the .stimulus materials. To make the setting more natural, we did allow participants to more or less browse the Web as they normally would, and we purposefully fielded the study at the height of the election season. However, as wilh any laboratory experiment, conditions were somewhat different from those in which the stimuli are typically encountered. Further, the design does not enable us to make claims about long-term effects of repeated or habitual exposure to the sites, which may afford less interested visitors greater opportunities for realizing gains in political engagement through interactions with site content. A second limitation concerns the participant pool. As a convenience sample recruited from introductory courses at a major university, the participate pool noticeably excludes young voters who are not in college. These noncollege youth represent more than half of the 18 to 24 year old citizens in the United States, and evidence indicates that these individtials vote less frequently than youth enrolled in higher edtication (Lopez, Kirby, Sagoff, & Kolaczkowski, 2005). Because of this, we believe that had we been able to recruit participants from both college and noncollege populations, we may have seen greater variation in political interest, and possibly in patterns of "Don't know" responses to our sample ballot items. In either case, we contend that broadening the pool of participants actually could intensify our findings concerning the moderating role played by political interest and could reveal stronger main effect patterns. However, without specific data from noncollege youth, we at^ forced simply to note this feature of the participant pool and recommend a standard level of caution with regard to generalizing the findings beyond college -studetits. These limitations notwithstanding, we contend that this study makes two important contributions to our knowledge of the role played by youth-oriented political pottals in contemporary politics. The first deals with the impact of variations atnong users, while the second deals with the itnportatice of attending to differences between particular Web sites. We conclude by di.scussing these contributions and their implications for both practitioners interested in using the Internet to facilitate youth political engagement further, as well as researchers interested in further study of the impact of Internet use on political participation among young people. With respect to user characteristics, this study provides direct evidence that differences in user motivations can play a key role in relationships between Internet use and political engagetnent. Even among the subset of young voters included in our experiment, it is clear that at the intersection of youth, politics, and new media, we do not find one mottolithic "youth" audience. Within this group, there are individuals with low levels of interest in politics, who appear to have significantly different informational and practical needs than those with higher levels of political interest. And, Xenos and Kyonng 187 when using a medium noted for its remarkably including youth-oriented political portals and malleable nature (DiMaggio, Hargittai, Neuman, other general interest sites on electoral politics and & Robinson. 2001 ), users appear to come away public affairs (Xenos & Bennett, 2007). Also, with markedly differing experiences, based on participants began their browsing sessions at a this initial level of political interest. Moreover, randomly selected stimulus Web site but were these differences do not appear to move in an still able to travel to other sites via hyperlinks. equalizing direction (in which low-interest In this sen.se, one can think of the design as testusers might come to resemble high-interest ing the effects of starting a browsing session at users after interaction with youth-politics por- some particular point within the youth political tals). Instead, as we see in the case of Internet- Websphere, versus a more open-ended entry specific political efficacy, these interactive into political resources on the Web. Along these effects point to an exacerbation of differences lines, we may interpret differences between the between politically interested and noninterested two youth portal conditions as potLMitialiy stemusers, such that the former are encouraged to ming from structural differences between the pursue further political engagement online (and outlink-rich New Voters Project Web site and to see such engagement as worthwhile), while the relatively outlink-poor Rock the Vote Web the latter actually appear to become less inter- site (Xenos & Bennett, 2007). Recall, for examested in the Internet as an avenue for political ple, the results of our logistic regression model action. of subjects possessing a full slate of ballot opinIn addition to individual user differences, our ions: the strongest site x interest interaction was findings also point to individual site differences for the New Voters Project condition. In this as another critical element in understanding the way, the present study offers an example of potential for online communication to facilitate how behavioral corollaries to the patterns greater political involvement among American revealed in the burgeoning literature on hyperyouth. Although it may seem obvious to note link structui-es might be studied empirically. To that the Internet is not monolithic, as Lupia and be sure, tnore controlled experimentation would Philpot (2005) point out, most research on the be required to parse out the potential effects of political effects of Internet use avoids taking link structure, site features, and, of course, site this fact into consideration, instead looking at content. However, at a more general level, we the effects of Intemet use "in general." Our believe the present study provides a unique findings provide additional evidence in support example of how we might consider data on the of this approach. While the.se findings do not location of a given site (or sites) within a wider directly support Lupia and Philpot's theory of network of sites in future research on the politithe Web site-political interest relationship, they cal effects of Internet use. Together, the insights provided by the do lend important additional support to the present study not only help to develop further argument that individual variations among Web sites can have discernable and significant .scholarship on the political effects of Web communication, particularly among younger voters, impacts on site visitors. As an extension of this perspective, our find- but are also potentially useful for producers of ings also identify a number of ways that political Web sites intended for young people. researchers might further explore the impacts of Specifically, we hope that practitioners may be site differences. For example, while not conclu- encouraged by our findings to intensify even sive, we believe our findings provide sugges- further their use of Ihe unique targeting capabiltive evidence that the immediate network ities of Web communication, so that they may structures surrounding Web sites may be rele- offer more specialized site experiences vant to the ways that particular sites affect designed to provide maximum benefits to users different kinds of users. Recall that our stimulus with varyitig levels of political interest. In this Web sites were selected in part based on a network way, relevant subsets of the youth population analysis of the "youth political Web.sphere," a each may find information that engages them relatively dense hyperlink network of Web sites on their own terms, without either boring or .r ' 18S JOURNAL OF INFORMATION TECHNOLOGY & POLITICS overwhelming them. Given available technologies and the patterns of effects apparent in this and contetnporary studies, this would appear to be an attractive strategy for reducing the differences in political engagement among youth with varying levels of political interest, rather than expanding them. Gelman, A., & Hill. J. (2007). Data analysis using regression and multilevel/hierarchical models. New York: Cambridge University Press. Hanneman. R. A.. & Riddle. M. (2005). Introduction to social network methods. Retrieved August 20, 2006. from hiip://faculty.ucr.edu/-hanneman/. Howard, P. (2006). New media campaigns and the managed citizen. New York; Cambridge University Press. Kalyanaraman. S., & Sundar, S. S. (2006). The psychological REFERENCES appeal of personalized content in Web portals: IX>es customization affect altitudes and behavior? Journal of Abramson, P. R. (1983). Political attitudes in America: Communication, 56(1), 110-132. Formation and dutnge. San Francisco: W.H. Freeman. Krueger, B. S. (2006). A comparison of conventional and Andolina, M. W., & Jenkins, K. (2004, Sepictnber 1). Iniernei poiiticai mobilization. American PoliticsDon 7 write ojf the kids just yet. . . hopeful prospects Research, 34i6), 759-776. for youth in the 2004 election. Paper Presented at the Kyoung, K. (2006). Communicative sources of political Prc-APSA Conference on Political Communication, efficacy: The rotes of traditional ma.ss media and the Chicago, IL. Internet. Unpublished master's thesis. University of Bandura. A. (1997). Self-efficacy: The exercise of control. Wisconsin-Madison. New York: W.H. Freeman. Lopez, M. H.. Kirby, E., & Sagoff, J. (2004, July). The Bimber, B. A. (2003). Infonnation and American democyouth vote 2004 |Cenler for Information and Research racy: Technology in the evolution of political power. on Clvie Learning and Engagement fact sheet]. New York: Cambridge University Press. Retrieved August 20, 2(X)7, from http://www.eivicyouth. Bishop, G. (1987). Context effects on self-perceptions of org/PopUps/FactSheets/FS_Youth_Voiing_72-04.pdf. interesi in government and public aftairs. In H.-J. Hippler, Lopez, M. H.. Kirby, E., Sagoff. J.. & Kolaczkowski, J. P. N. Schwartz, & S. Sudmaii (Kds.), Social information (2(X)5). Electoral engagement among non-college proce.'i.sing and sun-ey methodology (pp. 179-199). attending youlh. Retrieved August 20, 2007, from New York: Springer Verlag. CIRCLE: The Center for Information & Research on Boyle, M. P. (2004). Social networks, mass media and Civic Learning and Engagement Web site: hltp:// efficacy: Implications for protest participation. www.civicyouth.org/PopUps/FactSheeis/FS_04_ tJnpublished doctoral dissertation, University of noncollege_vole.pdf. Wisconsin-^Madison. Lupia. A., & Philpot, T. S. (2005). Views from inside the Buey, E. P. (2004). Interactivity in society: Locating Net: How Web siles affeci young adults' political an elusive concept. Information Society. 20(5), interesi. Journal of Politics, 67(4). I i 22-1142. 373-383. McLeod, J. M., Scheufele, D. A., Moy. P., Horowitz, E. M., Campbell, A., Gurin, G.. & Miller, W. E. (1954). Tlie voter Holben, R. L., Zhang. W. W., el al. (1999). Undersianddecides. Evanston, IL: Row. Peterson and Company. ing deliberation—The effects of discussion networks on Coles, A. (2004). Turning out in droves. Retiieved Augusi 20. participation in a public forum. Cotmnunication 2007, from hltp://www.bk)g.rockthevoie.coni/2(K)4_l 1_ Research, 26(6), 743-774. Ol_archive.html. McMillan, S. J.. & Hwang, J. (2002). Measures of perCraig, S. C . & Maggiolto, M. A. (1982). Measuring politceived interactivity: An exploration of ihe role of ical efficacy. Political Methodology, 8. 85-109. direction of communication, user conirul, and time in Delli Carpini, M. X. (2000). Gen.com: Youth, civic shaping perceptions of inleraetivity. Journal of Adverengagement, and the new information environment. tising. 310), 29-42. Political Communication, V7(4), 341-349. Montgomery, K.. GoUiieb-Robles. B., & Larson. G. Delli Carpini, M. X.. & Keeter, S. (1996). What Americans (2004). Youth as e-citizens: Engaging the digital genknow about politics and why it nuitters. New Haven. eration. Retrieved August 20. 2007, from htlp;// CT: Yale University Press. www.centerforsocialmedia.org/eeitizens/index2.hlm. DiMaggio, P., Hargitlai, E.. Neuman, W. R.. & Robinson. Morrell, M. E. (2003). Survey and experimental eviJ. P. (2001 ). Social implications of the Internet. Annual dence for a reliable and valid measure of internal Review of Sociology, 27, 307-336. poiiticai efficacy. Public Opinion Quarterly. 67. Eveland. W. R., Marton. K., & Sco, M. Y. (2004). Moving 589-602. beyond 'just ihe facts"—the influence of online news Morrell, M. E. (2005). Deliberation, democratic decisionon the contení and structure of public affairs knowlmaking and internal political eftlcaey. Political Behavior, edge. Communication Research, i/{l), 82-108. 27{ i ), 49-69. Xenos and Kyoung Nisbet, M. C , & Scheulele, D. A. (2004). Political talk as a catalysl for online citizenship. Journalism & Mass Communication Quarterly. 81(4), 877-896. Pan. Z. D.. Shen. L. J.. Paek. H. J., & Sun, Y. (2006). Mobilizing politicül lalk in a presidential campaign— An examination of campaign elïecls in a üeliberaiive franicwork. Communication Research, ÍÍ(3), 315-345. Patterson, T. E. (n.d.). Young voters and the 2004 election. Retrieved August 20.2007. truni lntp://www.ksg.liarvard edii/presspol/vanishvoter/Releases/Vanishing_Voter_ Finai_Reporl_2004_ Election, pdf. Prior, M. (2(X)5). News vs. enlerlainment: How increasing mediü choice widens gaps in political knowledge and tui'nout. American Journal of Political Science, 49(3), 577-592. Rainie. L, Cornfield. M.. & Horrigan. J. (2005). The Internet and campaign 2tX)4. Retrieved Augusl 20, 2007. from htip://www.pewiniemet.org/pdIs/pip_2O(>4_ campaign.pdf. Rheingold, H. (2000). The virtual community: Homesteading on lite electronic frontier Cambridge. MA; MIT Press. Rosensionc. S. J., & Hansen, J. M. (1993). Mobilization, participation, and democracy in America. New York: Mücmillan. Sehier, S. E. (2000). ßy invitation only: The rise of exclusive politics in the United States. Pittsburgh, PA: University of Pittsburgb Press. Shah, t). V.. Kwak. N.. & Holbert. R. L. (2001). "Conneeting" and •'disconnecling" with civic life: Patlerns of Inlernet use and lhe prtxluciion of stKial capital. Political Conmunication, 18(2), 14l-t62. Sundar. S. S., Kalyanaraman, S., & Brown, J. (2003), Explicating website interactivity: Impression formation etlects in political campaign sites. Communication Research, 30( 1 ). 30-59. Sunstein. C. R. (2001). Republic.com. Princeton. NJ: Princeton University Press. Tedesco, J. C. (2006). Web interactivity and young adult political eflicacy. In A. P. Williams & J. C. Tedesco (Ed.), Tlie Itmmiet election: Perspectives on the Web in catnpaign 2004 (pp. 187-202). New York: Rowman & I Jlllefield. Tedesco, J. C. (2(X)7). Examining Internet intcractiviiy effects on young adull political intormaiion eíllcacy. American Behavioral Scienlist, .5(J(9), I I «3-1 194. Tolbert, C. J.. & MeNeal. R. S. (2003). Unraveling the effects ofthe Internet on political panicipaiion? Polili' cal Research Quarterly. 56(2), 175-185. Vargas, J. A. (2(X)4. November 9). Vote or die? Well, they did vole; youth ballots up 4.6 million from 2000, in Keriy's favor. Washington Post. p. CO!. Verba. S., Schlozman. K. L., & Brady. H. E. (1995). Voice and equality: Civic voluntarism in American politics. Cambridge, MA: Harvard University Press. Warnick. B.. Xenos. M., Endres. D.. & Gastil, J. (2005). Effects of campaign-to-user and text-based interactivity in political candidate campaign Web sites. Journal of Computer-Mediated Communication, tO{?'), Article 5. Retrieved August 20, 2(X)7. from htlp://jcmc. indiana.edu/vol 10/issue3/wamick.hlnil. Wollman. N.. & Slouder, R. (1991). Believed efficacy and political acliviiy: A lest of lhe specificity hypothesis. Journal of Social Psychology. IJL 557-566. Xenos, M.. & Bennett. W. (2007). The disconnection in online politics: The youth political Web sphere and US election sites. 2002-2004. Information. Communication & Society. 10(4), 44.1^64. Xenos, M. A.. & Fool, K. (2(K)7). Not your father's Internet: The generation gap in online politics. In L. Bennett (Ed.). Civic life online: Learning how digital media can engage youth (pp. 51-70). Boston: MIT Press. Xenos, M.. & Moy, P. (2007). Direct and differential effects of the iniemet on political and civic engagemenL Journal of communication, 57(4). 704-4133.