Introduction Although increasing students' subject-matter understandings and competencies may be the most important goals of instruction, it is widely understood that students' attention, effort, and engagement in academic tasks is a critical intervening variable in determining whether those outcomes are attained. In fact, the widespread appeal of designing computer-based activities for students is at least partly due to teachers' accumulating experience that students are generally more "on-task" and express more positive feelings when they use computers than when they are given other tasks to do. It seems likely, though, that not all computer activities attract the same degree of student interest and effort. (See Note 1.) What uses of computers have effects on student engagement that seem most predictive of important learning? The research reported in this paper provides some evidence about this issue. In particular, we look at empirical associations between the kinds of software that students use, teachers' pedagogical motivations behind their use, and one measure of student engagement: teachers' estimates of the number of students who use computers to do work for their class on their own time–that is, before- and after-school, at lunch, during study hall periods, and at their own home or homes of friends. If teachers who use different software or who have different objectives for students' computer work differ greatly in whether they are likely to report that "most students use computers to do work for class outside of class time," it seems reasonable to infer that the group reporting greater out-of-class computer use is providing a more engaging activity for their students. This is especially so if one controls for differences in the types of students each group of students serve and the subjects they teach. Information to test whether that empirical conjecture holds true comes from a comprehensive national survey of teacher pedagogy and computer use, Teaching, Learning, and Computing, which I conducted along with Ron Anderson from the University of Minnesota. In the 1980's, Stanford professor Mark Lepper pointed to the likely motivational impact of certain uses of computers as classroom learning tools. His examination of the theoretical literature on intrinsic motivation suggested several ways that computer-based learning activities might lead to increased student engagement on academic tasks. First, to the extent that computer activities provide intellectual challenge, they motivate students to seek a solution to a problem. Second, computer activities that stimulate human curiosity or a desire to resolve an incongruity generate similar effort. And third, computer work that provides a sense of independent control and mastery over an environment also provokes sustained and intense effort (Lepper, 1985). Lepper further raised the proposition that active, self-directed, inductive, and exploratory computer activities might result in increased student learning, not just for the best students, but for a broad range of students, although he also cited cautionary warnings in the literature about less-than-satisfactory outcomes for less motivated students or less capable pupils (Lepper and Chabay, 1985). Qualitative research on computer-rich environments have generally supported the idea that project-based work with computers is highly engaging for students. Sandholtz and her associates, studying a rich supply of reflective audiotape journals and written reports of teacher-participants in the Apple Classroom of Tomorrow (ACOT) program (1985-1991), found broad evidence of increased student engagement in academic work. They found that students often went beyond the requirements of their assignments and explored new computer applications and developed application-related skills on their own initiative. They found that students came in before school and stayed after school to work on the class' computers–and the researchers stressed that these were "quite ordinary" students, not those who were otherwise academic stars. Anecdotes included a comment about a student staying after class to discuss a programming language: "Do you know how unusual it is for a student to stay after class to discuss content?" (Sandholtz, Ringstaff, and Dwyer, 1997; p. 93) However, the ACOT researchers found that increased student engagement occurred in certain settings:
In Means' case studies of 17 intensive computer-using classes at nine reform-oriented schools during 1991-93, she found that "the most common–in fact, nearly universal–teacher-reported effect on students was an increase in motivation (Means and Olson, 1995). In some cases, teachers felt the improvement was in terms of students' effort at learning the specific subject-matter of the class. In other cases, the perceived improvement in motivation was more general–a "sense of accomplishment" gained from working with computers. These perceptions were supported by the researchers' own observations during their field visits. As in the ACOT study, this investigation attributed the improved student effort to how computers were being used in the studied classes–for project work in cooperative teams, where the teacher had become a co-learner rather than the primary source of knowledge for students. Consistent with teacher reports and these qualitative case studies, small quantitative project-implementation studies have also found improved motivation on the part of students using computers for product-oriented projects such as designing informative multimedia or hypermedia presentations (e.g., Lehrer, 1993; Liu, 1998). Lehrer, for example, found students volunteering to work on a hypermedia authoring activity during their study hall, after school, and on both Saturday and Sunday (the latter in order to meet a competition deadline). Although case studies and curriculum-development projects such as these often report motivation outcomes for students, there is little nationally descriptive evidence about the relationship between various patterns of computer use and student motivational outcomes. In particular, it would be helpful to know:
Furthermore, there are a number of different ways in which student engagement and effort in academic work might be measured. Typically, a researcher asks teachers about their perceptions of student interest under conditions of active computer use compared with their experience in teaching similar students without that extent or type of computer work. But the anecdotal evidence that Lehrer provided in his account and which was roughly alluded to in some of the case study data is of a different type–namely, that students were engaging in a much higher level of effort outside of class time in order to accomplish work for an academic class. And in these examples the work being done involved using computers and computer software in order to accomplish class assignments or objectives. The ability of teachers to motivate students to do work outside of class time is perhaps one of the great sources of unequal achievement in American schools, particularly in secondary schools. National survey data report that students spend relatively few hours doing homework or similar academically-related productive work. For example, the NELS88 study reported a weekly average of between 5 and 6 hours of homework was done by 8th graders in 1988 (U.S. Department of Education, 1990). Successful secondary students report doing substantially more homework than students who perform poorly in school (Cooper, et al, 1998). Typically that result is interpreted as meaning that doing more homework helps students to get good grades; however, part of that relationship may be because students who are accustomed to being rewarded in school with good grades are more likely to do work that teachers want them to do on their own time. Adolescents' non-school lives are increasingly taken up in informal peer-related recreational activities and part-time employment that is overwhelmingly unrelated to any competencies or understandings learned in their formal school work. In those situations where teachers can break through these competing influences, this is evidence of either a strong extrinsic motivation for high grades for use in college admission or an even more impressive impact of appealing to a rather tenuous and undeveloped motivation for accomplishment. Based on the types of computer work that may be intrinsically motivating to a broad cross-section of adolescent and pre-adolescent students–activities integrated with substantive content objectives, work related to complex projects, "authentic" work done for an audience, and design and construction of multimedia and hypermedia information products–it was postulated that teachers whose students engage in such computer activities are more likely than other teachers to report students being engaged in doing additional computer work for their class outside of class time. Furthermore, teachers whose objectives for having students use computers are consistent with a constructivist view of learning (rather than acquiring a pre-selected and transmitted array of facts and skills) would be more likely to be successful at engaging students in doing computer work for their class outside of class time. Data and Methods The Teaching, Learning, and Computing (TLC) study is comprised of completed questionnaire responses from more than 4,100 teachers and from principals and school technology coordinators at more than 1,000 schools. A majority of respondents represent a national probability sample of schools. The remainder come from schools that were "purposively" sampled from two types of lists: "High-end technology schools" are schools with substantial amounts of computer technology per capita. "Reform Program schools" were long-term (3 year +) participants in one of 54 different national or regional externally-defined "programs" of major school or instructional reform or had originated a technology-oriented program of instructional reform on their own. Teachers were sampled from grades 4-12 and from all subjects except physical education and special education. At each sampled school, three (elementary) or five (middle and high school) teachers were selected with probabilities related to the teacher’s reputed instructional practices and use of technology. A small number of teachers (a maximum of two per school) were selected with certainty based on the principal’s attribution of that teacher having an exemplary instructional practice or based on their known participation in the selected program of instructional reform. Because unequal probabilities were used, at both school and teacher level, all analysis employs weighted data with weights inverse to the probability of selection, as modified by stratum-specific non-response rates and within-school partial completions of teacher rosters. Schools participated in the study at a 75% rate, and individual teachers sampled had a 68% response rate within those schools. Altogether, 2,251 teachers from the probability sample participated in the study and an additional 1,832 teachers from the purposive samples. (Additional information about the sample design and field procedures can be found in Appendix B to Becker, Ravitz, & Wong, 1999; and on the project website: http://www.crito.uci.edu/TLC.) The teacher respondents each completed a survey booklet about their teaching practice and teaching beliefs that was 21 pages in length and required approximately 60-75 minutes. Four different versions of the teacher survey booklet were used, with overlapping sets of questions. Among the questions in the teacher survey were three that form the focus of this paper: Objectives for Computer Use Which of the following are among the objectives you have for student computer use? Which three objectives from the list have been your most important ones?
Proportion of Students Using Computers Outside of Class Time to Do Work for the Class Whether or not students use computers during class time, some students may use computers to do work for this class at other times. How many students in this class have done work for this class using computers in each of these settings on at least several occasions? (choices: none or few; 1/4; 1/2; 3/4; all students) Frequency of Student Use of Different Types of Software For each of the following types of software, please indicate for how many lessons your students have used that type of software this year in ANY of your classes: (choices: No lessons, 1-2 lessons, 3-9 lessons, 10+ lessons)
The first two of those survey items–the question about teachers' objectives (or as phrased here, their pedagogical motivation) and the question about out-of-class computer use for class work–were asked of teachers who had students use computers for a single selected class. The class selected was the one in which the teacher felt she most frequently accomplished her objectives. The third question, about software use, was asked of teachers who had students use computers in any of their classes. In addition, the question about out-of-class computer use appeared in only two of the four versions of the questionnaire (randomly allocated among teachers). Thus, based on the screening question about computer use and the 50% inclusion of the out-of-class use question, the data in this paper that relate to out-of-class computer use come from 753 teachers in the probability sample and an additional 665 teachers in the purposive samples. Descriptive data about the other variables that don't have the question sub-sampling limitation are based on a substantially larger sample. Results Across all teachers in the probability sample who assigned computer work to their selected class, 25% said that all or most (3/4) of their students have done computer work on at least several occasions outside of class time while at school. About the same number (28%) said that all or most students had done class work at home or other places away from school. These two types of out-of-class activities are correlated; a majority of teachers who reported one type of activity also reported the other. Teachers who reported that most students did out-of-class-time computer work differed from other teachers in terms of which subjects they taught, the overall level of student achievement in their class, the socio-economic level of the school’s population, and which objectives they prioritized for student computer use. Thus, student characteristics, teaching responsibilities, and pedagogical motivation for computer use all affect the likelihood that students will supplement their in-class computer work with time before or after school or at home, at least as measured by teacher surveys. Pedagogical Motivation Teachers' objectives for their students' computer use represent their pedagogical motivations– what they hope students will accomplish through the computer activities which they do for their class. The TLC survey gave teachers a list of 10 objectives and asked them to select the three that were most important to them in their use of computers by students. Table 1 compares two groups of teachers for each objective: those who selected that objective in their top three and those who did not. By far the highest level of at-school non-class-time computer work was found for teachers who value computers for helping students to present information to an audience. More than twice as many teachers who valued that objective said that most students did computer work for the class at other times of the school day (44%) as those who listed only other objectives among their top three (21%). Similar differences were found in terms of home computer use between presentation-valuing teachers and other teachers (44% vs 25%).
Seeing computers as valuable for helping students improve their written expression was the second pedagogical motivation associated with greater at-school, out-of-class use of computers for class work (29% reported most students doing this vs. 21% for other teachers). Writing objectives were associated even more strongly with at-home computer use by students (36% vs. 21%). Two other objectives for students' computer use–communicating with other people and finding out about ideas and information–were associated with greater at-home use for class work, but not with use at other times of the school day. (See Note 2.) In contrast, three groups of teachers are much less likely than other teachers to report students using computers out-of-class: those whose objectives are reinforcement of skills, remediation, and students "learning to work independently."(See Note 3.) Frequent Users of Different Types of Software Teachers’ objectives tend to be accomplished by having students use specific types of software. Thus, it is not surprising that teachers who report their classes frequently using certain types of software are the same teachers who report the highest levels of participation in computer work being done outside of class time. The students who are most likely to be doing computer-based classwork at school but outside of class time are those who have had frequent exposure to one of six types of software during class: electronic mail, presentation software such as Powerpoint, multimedia authoring programs such as Hyperstudio, graphics-oriented printing programs, World Wide Web browsers, and CD-ROM reference software. However, it is also true that with the exception of teachers whose students used skill-based games, just about any frequent use of computers during class is associated with greater use of computers outside of class time. Teachers whose students use computers only occasionally (or who use games frequently) are less likely to initiate computer-based activities for class at other times of the school day. (See Table 2; the purposive sample was included in these tabulations in order to increase the sample of frequent software users.)
Outside of school, the students who are most apt to use computers to do school work are those whose teachers gave them frequent opportunities to work with presentation software, email, multimedia authoring programs and word processing. The strong relationship to classroom word processing experience is doubtlessly due to the ease with which that type of software carries over to non-supervised computer time. The fact that students experiencing complex multimedia authoring software in class are among the most likely out-of-school users of computers (for schoolwork) suggests that such in-class experiences develop a wide range of computer skills that can be exploited in a relatively open and yet unsupported environment outside the school. (Of course, non-school efforts may be strongly supported by peer expertise.) To more accurately interpret these associations between student out-of-class computer use and teacher pedagogical motivations and software assignment practices, it is important to consider other factors that might affect both student out-of-class computer use and teacher approaches to instruction. We examine three of these: subject-matter responsibilities, student ability level, and schoolwide socio-economic-status (SES).
Teacher Subject-Matter Responsibility In terms of subject-matter, two-fifths (39%) of all science teachers who assigned computer work during class reported that most or all students did computer work at school outside of class time. Teachers of computer classes and social studies teachers also had higher-than-average rates of reporting before- and after-school computer use. On the other hand, only 10% of computer-assigning math teachers did. Table 3 shows not only the percent of teachers reporting most or all students doing out-of-class school-located computer work, but the percent reporting that no students did this at all. Besides math teachers, three other groups of teachers reported limited out-of-class-time computer work by students–vocational education teachers, business education teachers, and elementary teachers of self-contained classes.
With respect to students using computers to do work for the class at home, more computer-assigning English teachers reported students doing this than any other group of teachers (48%). That is probably due to the wide accessibility of word processing software on home computers. Science and foreign language teachers were also above-average in this regard, also suggesting that word processing is the dominant class-related use of computers at students’ homes. The computer-assigning teachers least likely to report most students engaged in away-from-school computer use for class-related work were fine arts teachers (none of them did), math teachers (5%), and vocational education teachers (10%). For two of those groups, fine arts and vocational education, that is probably due to the specialized and costly nature of the software used in those courses. For math teachers, this appears to be the continuation of the relatively low involvement of math teachers in computer work that we have found evident throughout our reports from the TLC study. (See Becker, 1999; and Becker, Ravitz, and Wong, 1999 for additional details.) It should be noted that while a majority of fine arts and vocational education teachers reported some student computer use outside of school, 71% of computer-assigning secondary math teachers reportedno outside-of-school use by students for math class at all. (See Table 4.)
Student Characteristics Both student ability and family socio-economic status (SES) are likely to be related to out-of-class computer use for class work. Higher-achieving students and students from better off families are much more likely to have a computer at home. (For the latter relationship, see Becker, 2000.) Both variables also appear to affect a student's motivation to use computers at school during their free time (or before- or after-school). However, because the TLC survey is teacher-based rather than student-based, we cannot control perfectly for either ability or SES. Nevertheless, we can show two relationships: the association between teacher-estimated class ability (prior achievement) level and out-of-class computer use and the association between schoolwide average socio-economic status and out-of-class use. In order to provide more stable data, information from teachers in the purposive sample is included in this analysis. Table 5 shows that although the relationship between school-level SES and in-school, out-of-class, computer use is small, the other three relationships are huge. Computer-using classes at schools in the highest quartile on socio-economic status are more than three times as likely to have widespread at-home computer use for class work than computer-using classes at schools in the lowest quartile on SES. Specifically, 50% of the high-SES teachers vs. 14% of the low-SES teachers said that all or most students did computer work for class away from school on at least several occasions during the year.
With respect to student ability, classes of the highest-ability students were about three times as likely as classes of the lowest-ability students to involve widespread use of computers out-of-class, both at home and at school. The differences in computer use at home were somewhat larger than the differences in at-school out-of-class use, probably because of the association between class ability-levels and school SES. Classes of intermediate ability groups are appropriately mid-way between the high- and low-ability classes in terms of out-of-class use of computers for class work although the pattern is quite different for school use than for home use. For home use, the biggest differences are between the upper two quartiles in student ability and the lower two quartiles. For school use, only the top quartile of classes report substantially higher free-time computer use than the others. Effects of Pedagogical Motivation and of Frequent Software Use It is also the case that the kinds of software that teachers use and their objectives for its use are related–not just to their subject-matter responsibilities which would be obvious (e.g., social studies teachers use different software and have different pedagogical motivations than fine arts teachers)–but to school socio-economic status and class ability level as well (Becker, Ravitz, and Wong, 1999; Becker, 2000). For example, 30% of middle school teachers in high-SES schools selected "presenting information to an audience" as a main reason for having students use computers compared to 17% of middle-school teachers in low-SES schools. Among elementary teachers in grades 4-6, those who taught classes average or below in ability were twice as likely to list remediation as one of their major objectives than were teachers who taught classes in the upper-half of the ability scale (39% vs. 19%). (See Note 4.) Thus, some part of the explanation for why teachers with different pedagogical motivations have a correspondingly different likelihood of reporting that their students use computers for class work outside of class time is that they teach different varieties of students in different types of communities. Therefore, a clearer understanding of the actual causal effects of teacher pedagogical motivations and software use on students' out-of-class behavior can only be had by statistically controlling for these other important influences.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||