APPENDIX B. SUMMARY OF STUDY METHODOLOGY
The Teaching,
Learning, and Computing (TLC) study is comprised of completed questionnaire responses from teachers, principals, and school technology coordinators from three separate samples of schools. Somewhat more than one-half of the
1,616 schools sampled for the study (56%) were a stratified national probability sample of elementary (299 schools), middle (253), and high schools (346), including 83 private and parochial schools. Those schools were sampled
with probabilities related to both size (estimated number of full-time teachers, grades 4 to 12) and the presence of computer technology (based on an index developed for Quality Education Data, Inc.). The sampling universe was the
approximately 108,000 schools in the Quality Education Data (QED) database. The remaining samples of schools are referred to as "purposive samples" and were based on compiling, refining, and sampling from lists of two
basic types of schools: "High-end Technology schools" are schools with substantial amounts of computer technology per capita, including schools selected from the QED technology presence index and schools identified through books,
articles in magazines and school web-sites. "Reform Program schools" were compiled by identifying schools or individual teachers who had been long-term (3 year+) participants in one of 54 different national or regional
externally-defined "programs" of major school or instructional reform. In all three school samples, teachers were sampled from grades 4-12 and from all subjects except physical education and special education. At each sampled
school, three to five teachers (3, elementary; 5, middle and high school) were selected with probabilities related to the teacher's reputed instructional practices and use of technology. A small number of teachers (a maximum of 2
per school) were selected with certainty (probability equal to 1) based on the principal's attribution of that teacher having an exemplary instructional practice or based on their known participation in the selected program of
instructional reform. Because unequal probabilities were used, at both school and teacher level, all analysis employs weighted data with weights inverse to the probability of selection, as modified by stratum-specific
non-response rates and within-school partial completions of teacher rosters. The research began in the Spring of 1997 with a validation study of self-report measures of teacher beliefs and practices and exploratory studies
of survey measures of changes in teaching practices and technology use and school-level investments in technology hardware, software, and training and teacher support. The validation study provided self-report data from 72
teachers in 24 schools and detailed classroom observation and interview data with those same teachers. At the school level, pilot versions of surveys were used in order to test measurement approaches for studying technology
expenditure information, hardware and software acquisition, and investments of time and money in teacher training and support activities. The data collection itself was the second stage of the project, taking place from January
through June of 1998, and conducted by the Battelle Centers for Evaluation and Health Research. Data collection encompassed an initial district contact information letter, followed by a school mailing, in which teachers were
rostered and sampled; a subsequent mailing of questionnaires for teachers, the school-level technology coordinator, and the principal; and several waves of mail and telephone follow-up, editing, coding, data entry, and data
cleaning. The teacher respondents were asked to complete a survey booklet about their teaching practice and teaching beliefs that was 21 pages in length and required approximately 60-75 minutes. Four different versions of the
teacher survey booklet were used, with overlapping sets of questions. These are called questionnaire versions 1, 2, 3, and 4. The school technology coordinator's booklet was approximately the same length as the teacher
survey and principally concerned the investments their school has made in computer hardware, software, and teacher training and support, measured both financially and in units of time, materials, and equipment. The
principal's survey booklet was half as long, and inquired about technology-related school policies and efforts in school restructuring and reform. The third stage of the project involves data analysis, preparation of reports, and
the release of national data files for secondary analysis. Across the three samples, 1,215 of the 1,616 schools selected for participation agreed to participate in the
study (75%). They did so by returning a roster of a specifically requested number of teachers (10 in elementary schools; 15 in middle and high schools), providing rough estimates of each teacher's use of computers, projects,
and emphasis on critical thinking and complex problem-solving. The attained probability sample (rostered schools) consists of 598 public and 57 private and parochial schools. The High-end Technology sample includes 182 rostered
schools including 86 entering the sample based on having among the highest technology presence index scores in the QED database. The remainder were believed to have substantial computer and Internet technology, as identified
through publicly available information from school Web sites, books, and magazine articles. The Reform Program sample includes 378 rostered schools that were identified through various sources as being involved in one of 53
different reform efforts. The "reform program" and "high-end technology" samples involve some definitional overlap in that 13 of the reform programs (with 90 rostered schools) appear to have substantial amounts of technology,
while 72 rostered high-end technology schools appear to have explicit instructional reform emphases even though they did not participate in any of the major reform programs selected. A majority of Reform Program schools are
involved in a school-wide reform program (e.g., Coalition of Essential Schools, League of Professional Schools, Bay Area School Reform Collaborative, Co-NECT Schools) These total 30 separate programs (200 schools) including four
with a technology emphasis and five that are not 'programs' per se but schools linked by a common origin (e.g., 'Charter Schools with a constructivist flavor'). In addition, there are four programs that are limited to math
and/or science (26 schools), 17 programs that enrolled individual teacher participants (nine of these are technology-centered), and two programs that recognized individual exemplary teachers. Lists of participating schools or
teachers were obtained directly from the programs in 44 of the cases; in the other 9 they were obtained from public sources--lists of participants on World Wide Web sites or in books. (In some cases, these were not actually
programs--just schools identified as exemplary in the public source.) Forty programs provided more schools than were needed so that probability sampling was employed to select the particular schools that would be incorporated
into the study. (In some cases, additional selection criteria were used prior to the sampling.) At each of the 1,616 studied schools, samples of 3 (elementary) or 5
(middle and high school) teachers were drawn through probability sampling methods. A Teacher Roster form was sent to the school principal as the first major mailing to the school (following an introductory letter). That
form asked the principal to roster either 10 (elementary) or 15 (secondary) teachers of grade 4 or higher (in some cases limited to the same subject taught by a reform program-participating teacher), starting with teachers with
last names beginning with a randomly selected letter of the alphabet and proceeding alphabetically. The roster form asked for 4 additional pieces of information about the rostered teachers that were used to assign sampling
weights to each rostered teacher (e.g., subject taught, use of computers, use of projects in teaching). In addition, two other sources of teachers are incorporated as purposive samples. Approximately 250 teachers were
individually selected from the purposive school samples based on reports (public or program-supplied) of their participation in educational reform activities. And finally, approximately 800 teachers were chosen through
nominations by principals (as part of the Roster form) as exemplary practitioners of constructivist approaches to teaching. Response rates of individually selected teachers,
principals, and technology coordinators averaged about 70%. Altogether, responses were obtained from 4,083 teachers of grade 4 and higher in 1,150 schools, as well as 845 technology coordinators and 867 school principals.
END OF REPORT