The Teaching, Learning, and Computing (TLC) study is comprised of completed questionnaire responses from teachers, principals, and school technology coordinators from three separate samples of schools. Somewhat more than one-half of the 1,616 schools sampled for the study (56%) were a stratified national probability sample of elementary (299 schools), middle (253), and high schools (346), including 83 private and parochial schools. Those schools were sampled with probabilities related to both size (estimated number of full-time teachers, grades 4 to 12) and the presence of computer technology (based on an index developed for Quality Education Data, Inc.). The sampling universe was the approximately 108,000 schools in the Quality Education Data (QED) database.
The remaining samples of schools are referred to as “purposive samples” and were based on compiling, refining, and sampling from lists of two basic types of schools: “High-end Technology schools” are schools with substantial amounts of computer technology per capita, including schools selected from the QED technology presence index and schools identified through books, articles in magazines and school web-sites. “Reform Program schools” were compiled by identifying schools or individual teachers who had been long-term (3 year+) participants in one of 54 different national or regional externally-defined “programs” of major school or instructional reform.
In all three school samples, teachers were sampled from grades 4-12 and from all subjects except physical education and special education. At each sampled school, three to five teachers (3, elementary; 5, middle and high school) were selected with probabilities related to the teacher’s reputed instructional practices and use of technology. A small number of teachers (a maximum of 2 per school) were selected with certainty (probability equal to 1) based on the principal’s attribution of that teacher having an exemplary instructional practice or based on their known participation in the selected program of instructional reform. Because unequal probabilities were used, at both school and teacher level, all analysis employs weighted data with weights inverse to the probability of selection, as modified by stratum-specific non-response rates and within-school partial completions of teacher rosters.
The research began in the Spring of 1997 with a validation study of self-report measures of teacher beliefs and practices and exploratory studies of survey measures of changes in teaching practices and technology use and school-level investments in technology hardware, software, and training and teacher support. The validation study provided self-report data from 72 teachers in 24 schools and detailed classroom observation and interview data with those same teachers. At the school level, pilot versions of surveys were used in order to test measurement approaches for studying technology expenditure information, hardware and software acquisition, and investments of time and money in teacher training and support activities.
The data collection itself was the second stage of the project, taking place from January through June of 1998, and conducted by the Battelle Centers for Evaluation and Health Research. Data collection encompassed an initial district contact information letter, followed by a school mailing, in which teachers were rostered and sampled; a subsequent mailing of questionnaires for teachers, the school-level technology coordinator, and the principal; and several waves of mail and telephone followup, editing, coding, data entry, and data cleaning. The teacher respondents were asked to complete a survey booklet about their teaching practice and teaching beliefs that was 21 pages in length and required approximately 60-75 minutes. Four different versions of the teacher survey booklet were used, with overlapping sets of questions. These are called questionnaire versions 1, 2, 3, and 4. The school technology coordinator's booklet was approximately the same length as the teacher survey and principally concerned the investments their school has made in computer hardware, software, and teacher training and support, measured both financially and in units of time, materials, and equipment. The principal's survey booklet was half as long, and inquired about technology-related school policies and efforts in school restructuring and reform.
The third stage of the project involves data analysis, preparation of reports, and the release of national data files for secondary analysis.
Across the three samples, 1,215 of the 1,616 schools selected for participation agreed to participate in the study (75%). They did so by returning a roster of a specifically requested number of teachers (10 in elementary schools; 15 in middle and high schools), providing rough estimates of each teacher’s use of computers, projects, and emphasis on critical thinking and complex problem-solving. The attained probability sample (rostered schools) consists of 598 public and 57 private and parochial schools.
The High-end Technology sample includes 182 rostered schools including 86 entering the sample based on having among the highest technology presence index scores in the QED database. The remainder were believed to have substantial computer and Internet technology, as identified through publicly available information from school Web sites, books, and magazine articles.
The Reform Program sample includes 378 rostered schools that were identified through various sources as being involved in one of 53 different reform efforts. The “reform program” and “high-end technology” samples involve some definitional overlap in that 13 of the reform programs (with 90 rostered schools) appear to have substantial amounts of technology, while 72 rostered high-end technology schools appear to have explicit instructional reform emphases even though they did not participate in any of the major reform programs selected. A majority of Reform Program schools are involved in a schoolwide reform program (e.g., Coalition of Essential Schools, League of Professional Schools, Bay Area School Reform Collaborative, Co-NECT Schools) These total 30 separate programs (200 schools) including four with a technology emphasis and five that are not ‘programs’ per se but schools linked by a common origin (e.g., ‘Charter Schools with a constructivist flavor’). In addition, there are four programs that are limited to math and/or science (26 schools), 17 programs that enrolled individual teacher participants (nine of these are technology-centered), and two programs that recognized individual exemplary teachers.
Lists of participating schools or teachers were obtained directly from the programs in 44 of the cases; in the other 9 they were obtained from public sources—lists of participants on World Wide Web sites or in books. (In some cases, these were not actually programs—just schools identified as exemplary in the public source.) Forty programs provided more schools than were needed so that probability sampling was employed to select the particular schools that would be incorporated into the study. (In some cases, additional selection criteria were used prior to the sampling.)
At each of the 1,616 studied schools, samples of 3 (elementary) or 5 (middle and high school) teachers were drawn through probability sampling methods. A Teacher Roster form was sent to the school principal as the first major mailing to the school (following an introductory letter). That form asked the principal to roster either 10 (elementary) or 15 (secondary) teachers of grade 4 or higher (in some cases limited to the same subject taught by a reform program-participating teacher), starting with teachers with last names beginning with a randomly selected letter of the alphabet and proceeding alphabetically. The roster form asked for 4 additional pieces of information about the rostered teachers that were used to assign sampling weights to each rostered teacher (e.g., subject taught, use of computers, use of projects in teaching).
In addition, two other sources of teachers are incorporated as purposive samples. Approximately 250 teachers were individually selected from the purposive school samples based on reports (public or program-supplied) of their participation in educational reform activities. And finally, approximately 800 teachers were chosen through nominations by principals (as part of the Roster form) as exemplary practitioners of constructivist approaches to teaching.
Response rates of individually selected teachers, principals, and technology coordinators averaged about 70%. This includes 69% of the teachers in the probability sample of schools and 64% in the purposive schools sample. Altogether, responses were obtained from 4,083 teachers of grade 4 and higher in 1,150 schools, as well as 845 technology coordinators and 867 school principals.
Participating teachers completed 20+ page questionnaires. Four versions of the questionnaire were used, with largely overlapping questions, but permitting somewhat greater coverage of topics than a single version of equal length would have permitted. The questionnaires dealt with five principal topics: teaching philosophy and related beliefs about instruction and assessment; teaching practices and strategies followed in the instruction of one class—the class in which the teacher felt most satisfied with achieving teaching objectives—; the ways in which the teacher used computers in teaching and professionally and changes over time in the role of computers in her teaching practice; changes in her general pedagogical approaches made over the previous three years; and a wide range of questions about the teaching environment at the teacher’s school, including formal professional development, support for technology, informal interactions with other teachers, and pressures experienced in their teaching. In addition, a variety of questions about educational and teaching background and current teaching responsibilities were asked at various points in the questionnaire. Nearly all questions were of the fixed-response type. Open-ended questions about college attended, college major, and current set of courses taught were coded into numerical categories. College attended was coded in terms of selectivity, based on SAT and ACT scores of admitted freshmen in the year 1983.
Because unequal probabilities were used, at both school and teacher level, all analysis employs weighted data, with weights inverse to the probability of selection, as modified by stratum-specific non-response rates and within-school partial completions of teacher rosters. Where purposive and national probability samples are combined, weights are adjusted so that the average weight for teachers in the purposive schools sample is equal to the average weight for teachers from schools in the probability sample.