Problem-Solving and Use of other Meaningful Thinking Strategies

Although in our exploratory factor analyses, the problem-solving items generally loaded on a separate factor from the other items labeled Meaningful Thinking, teachers who use problem-solving strategies are much more likely to also use other thinking strategies as well. Table 12 provides data, by subject and level, on the extent to which users and non-users of each problem-solving activity differ in terms of their use of meaningful thinking strategies.  The measure used in this and subsequent tables is the effect size (E.S.)—the difference in mean scores between two groups of teachers on the criterion measure (in this case, the teacher's Meaningful Thinking score), in standard deviation units.  [10]  In addition to comparing users and non-users of specific problem-solving activities, Table 12 provides comparisons between teachers who were above and below the median on an index that combines frequency-of-use data on all four problem-solving activities, including one not shown in the table. The effect size statistic for the overall problem-solving index measures how much teachers who are above the median on problem solving also report engaging students in meaningful thinking activities compared to other teachers of the same subject and level who do less problem-solving. [11]

Table 12 shows that in every subject, the teachers who report more problem-solving activities are much more likely to report engaging students in meaningful thinking. The effect size is higher than +0.6 for every subject, and greater than +0.9 for the major academic subjects. For example, the middle school math teachers who are above the median on Problem Solving are 1.0 standard deviations higher on Meaningful Thinking than the middle school math teachers below the median on Problem Solving. That translates into a difference of roughly 40 percentile points.  All t hree problem-solving items shown (students deciding own procedures, problems with no obvious solution, designing their own problems to solve) had equally large effect sizes on meaningful thinking. [12]

Table 12: Effect Size of Problem-Solving on Meaningful Thinking

The pattern across the first three columns in Table 12 suggests the issues around which meaningful thinking may be most likely to occur in different subjects.  For example, in middle school social studies, teachers who report giving their students the task of designing their own "problem to solve" had average scores on the Meaningful Thinking index that were 1.6 standard deviations higher than teachers who did not use that approach. This suggests that meaningfulness in middle school social studies revolves around students choosing topics for their work. In vocational classes, on the other hand, teachers' scores on Meaningful Thinking were particularly high for teachers who reported having students decide on their own procedures. This suggests that meaningful thinking in vocational classes may be centered more around procedural activities.

In summary, within every subject, teachers who report doing more problem-solving also appear to engage students in more meaningful thinking. The combination of problem-solving and meaningful tasks provides a cognitively challenging classroom environment. In the remainder of this report, we employ an index of Cognitive Challenge which combines the information from the 13 survey items concerning meaningful thinking and 3 of the 4 items that asked about problem-solving. [13] Because the Cognitive Challenge Index is central to the rest of this report, we present, in Table 13, the average Cognitive Challenge scores by subject and level. Those scores closely match the data on Meaningful Thinking presented in Table 10, except that the high Problem-Solving scores of middle school math teachers and vocational education teachers raise their average Cognitive Challenge scores closer to the midpoint for all teachers.  In contrast, scores of English and social studies teachers are somewhat reduced for this measure of Cognitive Challenge compared to the prior measure.  Again, the score for elementary teachers is not strictly comparable to the others because the vast majority of such teachers teach all-day self-contained classes and many of the survey items are affected by the overall extent of teacher contact with the same students.

Table 13: Frequency of cognitive challenge by subject and level taught
(index mean z-score)

ACTIVE LEARNING STRATEGIES:  Projects and work in small Groups

Discussions of reform teaching practices often focus on students working actively on "authentic" projects and often involve both collaborative work groups and a specific end-product developed for a specific audience.  However, we have introduced constructivist pedagogy in a completely different way—focusing on instruction that is cognitively challenging as a result of the teacher designing tasks that are personally meaningful to students. The obvious research questions are whether these two aspects of teaching practice—active, collaborative project-based work and cognitively challenging tasks—(a) tend to occur among the same teachers and (b) tend to occur at similar relative frequencies across the various secondary subjects and levels. In other words, are the teachers who frequently have students do project and group work the same teachers who provide cognitively challenging instruction? Are the subjects and levels where we find higher rates of cognitively challenging instruction the same subjects and levels where higher-than-average rates of project and group work activities prevail?

One can easily imagine that many teachers whose classes do projects and group assignments do so for reasons other than a belief in a constructivist learning theory. They may use projects and group work as teaching methods because they find that students enjoy those activities more. To the extent that projects and group work strategies are not really being guided by pedagogical beliefs about learning, but instead by simple classroom survivability, the way that teachers use these strategies might differ a great deal and the consequences for student learning may differ as well.  For example, teachers who use projects and group work but who do not believe in constructivist theories of learning may follow a procedurally-scripted approach rather than one in which students are being intellectually challenged.  Group work assignments may simply involve students dividing up their work and sharing answers to worksheet problems. Projects may involve little self-direction and require little reflective thought.

Similarly, teachers who provide students with a cognitively challenging educational experience may teach using direct explanations as much or more than other teachers. If so, this would suggest that teaching through projects and collaborative group work may not be important approaches for providing intellectually enriching school experiences.

First, we address the prevalence of group work and its relationship to cognitive challenge.  Then project work is discussed in the same way.

Group Work – Prevalence and Relation to Cognitive Challenge

To get a rough estimate of how teachers divide up the time they have for classroom instruction, respondents were asked to estimate, out of the most recent five hours of teaching to their sampled class, the amount of time spent on each of four activities: teacher-led whole-class discussion, student-led discussion or presentations, seatwork assignments, and small group work.  Somewhat surprisingly, teachers reported more time spent in group work than in any of the other activities. Based on teacher self-reports regarding the last five hours taught, only computer and business teachers report more seatwork than group work, and only foreign language and math teachers report more whole-class teacher-led instruction than group work.  In fact, a majority of all teachers reported that students worked in groups for more than one hour out of the last five.  Although in answer to another survey question, more teachers reported that their students did seatwork on a daily basis than reported daily "work in small groups to come up with a joint solution or approach to a problem" (20% vs. 14%), if one were to add in other purposes   of small group work, the total daily use of group work might easily exceed the number reporting daily seatwork.

Teachers find that older students' have a greater ability to conduct their activities in groups for extended periods of time.  Elementary teachers generally report doing less group work than do secondary teachers.  More than half (56%) of the secondary academic teachers report more than an hour of group work in the last five, compared to 41% of elementary teachers.

Among secondary teachers, more than two-thirds of science teachers have students work in groups (or pairs), probably in lab settings.  It was business teachers who reported the least time having students work in groups (only 29% for more than an hour in the last five, less than any other subject).    Interestingly, business teachers were also least likely to spend more than an hour having students give presentations or lead discussions.   In fact, only 1% of business teachers report this activity as compared to 21% of vocational teachers.  Given the importance of working in teams and making presentations in business settings, this finding is striking.

Table 14: Frequency of group work activities by subject and level taught

For some academic subjects, differences between middle school and high school level teachers were notable. It seems clear that in science, group work is more prevalent in middle schools while in social studies it is more prevalent at the high school level. (See Table 14.)  Not only do high school social studies teachers do more group work, but they also report less seatwork than middle school social studies teachers. (Only 19% reported more than an hour of seatwork compared to 48% of middle school social studies teachers.)

In a second survey question about the frequency of group work (mentioned briefly earlier), teachers were asked about how often they have students work in groups in order to come up with a joint solution or approach to a problem. At the secondary level, 44% of academic teachers reported that they had students work in problem-solving teams on a weekly basis.  This activity is most commonly found among math teachers, particularly middle-school math teachers, 59% of which report group problem-solving at least weekly. Science teachers are the next most likely to have students work in this type of group, but this also occurs more often in middle school classes.  Most elementary teachers also reported using this teaching strategy, but, as with other frequency measures, it is difficult to compare the answers of the elementary teachers because most of them taught self-contained (whole-day) classes and thus had more time to implement a variety of teaching methods.  (See Appendix Table A-5.)

The last question about group work in the TLC survey concerns whether, as a means of introducing a new unit to students, the teacher divided students into small groups to discuss topics related to the new unit. Overall, just over one-third (37%) said they did so on the current lesson being taught. Introducing a unit this way, which is probably more time-consuming than other approaches, tended to happen more at the middle school level and varied by subject. For example, middle school science teachers used group discussions as unit starters more than high school science teachers did.  However, in contrast, high school English teachers used groups in this way more often than middle school English teachers did. These school-level differences within subjects parallel what was found for the other group work items in Table 14.

Relationship of Group Work to Cognitive Challenge, by Subject and Level Taught

So far, we have described the extent and variety of group work that occurs in the different subjects.  However, a more central question for understanding teacher pedagogy is to learn to what extent small group work is associated with cognitively challenging tasks rather than being reflective of "mere activity." To do this, we compare scores on our Cognitive Challenge Index of teachers who make substantial use of group work (i.e., teachers who are above the 50^th percentile for their subject-matter in overall group activity) and Cognitive Challenge scores of teachers in the same subject who use small group strategies less often.

The findings in Table 15 clearly support the idea that teachers whose students do group work are much more oriented towards providing instruction that is cognitively challenging than are teachers who use group work strategies less often, if at all.  Across all subjects, the effect size between above-average group-work-using teachers and below-average users on Cognitive Challenge is +0.9, or close to a full standard deviation. Within each subject and school level (every one except secondary foreign language), teachers who use group work more than the typical teacher in that subject average at least 0.7 standard deviations higher on Cognitive Challenge than teachers who are less active in using student groups. (Analogous correlation coefficients to these effect size measures are presented in Appendix Table A-6.) 

Among the three specific indicators of group work, the highest effect size was found for the item distinguishing between teachers who had students solve problems in small groups on a weekly basis versus those who used that method less often or not at all (E.S.=+0.9). [14] In fact, for a majority of subject-level categories, it was this activity of group problem-solving that had the highest effect size on Cognitive Challenge. Group problem-solving is particularly associated with high Cognitive Challenge among social studies teachers (both middle and high school), middle school math teachers, and elementary teachers. The effect sizes for all four of those groups was greater than 1.0 standard deviation units.

The strategy of using group discussions as part of unit introductions is also associated with substantially higher Cognitive Challenge.  Like the first item discussed, this relationship was true for teachers of every subject and level studied except for foreign language teachers. The use of group discussions to introduce a unit seems particularly likely to be motivated by a constructivist theory of learning, which argues as its central thesis that prior student understandings should be the starting point for instruction. This approach was particularly associated with high Cognitive Challenge in computer classes (E.S. = 1.1).

By comparison, reporting an hour or more of group activity in the last five was somewhat weaker than the other two items in terms of predicting Cognitive Challenge (E.S.=+0.5). In some subjects, (e.g., vocational-education, high school social studies and math, and middle school science), teachers may routinely be able to spend an hour or more having students work in groups, without engaging groups in reflective writing activity, posing probing questions, or having them engage in problem-solving. Even so, for other subject-level combinations—middle school math and high school English, in particular—the teachers who were more likely than average to assign group work to their selected class over the previous five class meetings were also the teachers whose practice involved more cognitively challenging activities.

Table 15:  Effect Size of Group Work on Cognitive Challenge

Student Projects and Hands-on Activities

Student projects constitute a fairly wide range of teacher-directed learning activities. What are called "projects" by different teachers vary in their duration; in the number and types of different tasks involved; in how much variation between students in project content that a teacher permits; and in the degree of independence, creativity, and initiative encouraged.  Nevertheless, compared to other common types of classroom assignments, projects tend to be much longer in duration, involve a wider variety of tasks, permit greater variation among students in the work that they do, and provide for greater independence, creativity, and initiative than other school assignments. 

Overall, about one-half of the teachers in our study (48%) report that students in their selected class are involved each month in projects of a week or longer in duration. Elementary teachers are more likely to report having their students do projects than are secondary teachers of academic subjects (56% vs. 39%).  That difference is likely to be in large part due to the fact that most elementary teachers have the same students in class for many more hours than secondary teachers do.  The difference in project use between elementary teachers with self-contained classes and other elementary teachers is nearly as great (60% vs. 47%) as the difference between all elementary teachers and secondary teachers of academic subjects.

Among secondary teachers, projects are a more central part of teaching in the applied subjects than in academic subjects.  Projects lasting a week or longer were most common among fine arts and computer teachers (92% and 74%, respectively), but also occurred in a majority of vocational education and business education classes. However, they were also part of the teaching practice of a majority of English teachers, both at the middle school level (60%) and high school level (64%; see Table 16). From a different perspective, given the larger numbers of teachers in academic subjects, it is still the case that more than half of all student project activity in secondary schools takes place in English, science and social studies classes. ( See Table A-7. )

Table 16: FREQUENCY OF PROJECT ACTIVITY
BY SUBJECT AND LEVEL TAUGHT

 previous page                    next page