Independent Evaluation of EnTeam Implementation

at Roosevelt 9^th-Grade Center, St. Louis Public Schools

Corey Drake, Ph.D.

College of Education -- University of Missouri - St. Louis

June 2005

 

Context

In the winter and spring of 2004, EnTeam professional development was offered to the teachers at Roosevelt Ninth-Grade Center in St. Louis, MO. Teachers participated in a group EnTeam workshop and then were offered individualized support in implementing EnTeam activities in their classrooms and integrating those activities with the on-going academic curriculum. Participation in this individualized professional development was not mandatory.

 

Methodology

Pre- and post-surveys were collected from teachers measuring teachers' beliefs and practices regarding cooperative learning and its use in their classrooms. Individual interviews were also conducted with two school administrators and three classroom teachers who had participated in the group and individual EnTeam activities.

 

In the sections that follow, survey results are presented, followed by interview findings and questions to pursue in further research.

 

Results

Survey Results

Twenty-five teachers responded to the pre-survey and 20 teachers responded to the post-survey. Survey responses were not matched by individuals (surveys were anonymous); pre- and post-mean responses were compared.

 

Teachers' Prior Experiences with Cooperative Learning. At the beginning of the EnTeam intervention, teachers reported a moderate level of experience and practice teaching with cooperative learning strategies. Only one teacher (out of 25) reported having little or no experience teaching with cooperative learning prior to participation in EnTeam. Teachers were evenly split between reports of those experiences as predominantly positive and reports of those experiences as mixed.

 

In reporting on their own learning with cooperative learning strategies, teachers agreed to a high degree (4.22/5) that, "interaction with others stimulate(s) your learning." Teachers agreed to a moderate-high degree that, "trial and error help(s) you learn" (3.96/5), that, "cooperative learning helped you learn" (3.68/5), and that, "you use models or mentors for learning" (3.6/5). Finally, teachers reported to a moderate degree (3/5) that, "learning is an independent activity."

 

Finally, in self-reports of their own teaching practices, teachers moderate use (3.38, where 1=rarely and 5=frequently) of cooperative learning experiences in their lesson design. On average, they reported spending 48% of classroom time on cooperative learning, though there was significant range in these responses from 5% to 80%.

 

Therefore, according to teachers' self-reports, almost all teachers had some prior experience with cooperative learning. On average, teachers reported moderate to high levels of agreement with the importance in their own learning experiences of factors central to EnTeam - interaction with others, trial and error, and the non-independence of learning.

 

Teachers' Use of Cooperative Learning After Participation in EnTeam. In the post-intervention survey, teachers reported some change (2.65, where 1=not at all and 5=a great deal) in their definitions of "cooperation" or "cooperative learning." Responses were fairly evenly spread across the range from 1-4 (only one response was a "5"), with a median response of 3.

 

Teachers' responses about their changing definitions of the meanings of cooperation and cooperative learning were closely, but not perfectly, related to their self-reports of their use of EnTeam activities in the 3 months between the pre- and post-surveys. Teachers' responses about their use of EnTeam activities also varied widely, with 3 teachers reporting using EnTeam activities several times, 5 teachers using EnTeam activities a few times, and 11 teachers using EnTeam activities rarely or not at all.

 

Overall reports of teachers' recent classroom practices provided some evidence of increases in teachers' design of lessons incorporating cooperative experiences (3.7 vs. 3.38) and in the percentage of classroom time devoted to cooperative learning activities (59.5% vs. 48%).  Interestingly, those teachers who reported using EnTeam activities a few times reported very high overall levels of use of cooperative learning in their lesson design and as a percentage of classroom time. At the same time, half of the teachers who reported using EnTeam activities "not at all" still reported using cooperative learning activities in their classrooms 80 percent of the time, while 2 of the 3 teachers who used EnTeam activities several times only reported using cooperative learning activities in 50% of their classroom time. These results suggest that teachers were making clear distinctions between EnTeam activities and cooperative learning activities more generally and that the two practices were not necessarily correlated.

 

Teachers' Perceptions of the Factors Limiting Use of Cooperative Learning. Between the pre- and post-intervention surveys, perceptions of barriers to using cooperative learning decreased in 4 out of the 5 categories (Table 1). The perception of cooperative learning activities as being more time-consuming remained relatively constant. In other words, teachers in the post-survey were 1) less likely to view cooperative learning as play instead of serious education, 2) less likely to report that individuals were not held accountable in cooperative learning, 3) less likely to feel frustrated in creating cooperative learning activities, and 4) less likely to feel that pressure from state tests limits the use of cooperative learning activities. This finding about the decrease in perceived barriers is likely to be related to the reported increases (discussed above) in teachers' uses of cooperative learning activities and/or of EnTeam activities.

	Pre-Intervention	Post-Intervention
Play, Not Serious (5 = CL is serious)	3.6	4.1
Individuals Not Accountable (5 = Held Accountable)	2.9	3.4
Frustrating to Create Activities (5 = Very Frustrating)	3.0	2.5
More Time-Consuming (5 = More Time-Consuming)	3.2	3.3
State Tests (5 = High Degree of Pressure)	3.3	2.9

Table 1. Pre- and Post-Intervention Perceptions of Factors Limiting Use of Cooperative Learning

 

Interview Results

Two key points emerged from our interviews with administrators and teachers. First, each interviewed participant approached the EnTeam activities with different goals or purposes for their participation - for instance, helping teachers learn about cooperative learning, improving character education, improving education in content areas and in special education. Interestingly, despite this diversity of goals and purposes, each participant also thought that EnTeam served their own purpose well, with the exception of one teacher who didn't think it worked in their content area, but did work in character education. Other teachers did think it worked in their content areas, though they acknowledged it was difficult to link EnTeam activities to the content in a non-simplistic way.

 

Second, all interviewees raised the issue of the significant number of teachers who did not participate in the EnTeam activities beyond the initial group introduction. When asked why more teachers did not take advantage of EnTeam activities, three points came up repeatedly:

• Teachers are resistant to change or do not see the need for change.
• Teachers needed to have more experience doing the EnTeam activities themselves.
• It is difficult for teachers to consider using EnTeam given the heavy demands of content coverage.

Again, these were only conjectures from participants about why other teachers did not participate more fully and they must be understood as such. However, implicit in their comments is a need for teachers to better understand and experience ways in which EnTeam activities can be linked to or support content-based activities. Some of the interviewees provided good examples of how individual teachers managed to do this.

 

Overall, interviewees were quite positive about their experiences with EnTeam, though all offered ideas and comments about how to get more teachers involved and help teachers incorporate EnTeam more into daily classroom activities and routines.

 

More detailed comments from teacher and administrator interviews are found in Appendix A.

 

Questions for Future Research and Evaluations

All of the results reported above are based on a small sample and represent, in effect, a pilot version of a full EnTeam evaluation. Nonetheless, the results raise several interesting and significant questions that should be incorporated in future evaluations. First, how do teachers perceive the relationship between EnTeam and cooperative learning more generally and how can EnTeam professional development be targeted to those teachers who are a) unfamiliar with or inexperienced in the use of cooperative learning, b) already using cooperative learning in some form and do not perceive a need for additional cooperative learning activities (e.g., EnTeam), and c) using cooperative activities, but would like to incorporate EnTeam into those activities? This question is related to the more general question of how EnTeam can better present its activities as ones that can be integral to content-based instruction, rather than as activities that are supplemental, or additions, to content-based instruction. These are important issues in terms of the scalability and sustainability of EnTeam and should be addressed in future evaluations both with teachers who are full participants in EnTeam and those who participate in more limited ways.

 

A second question to be addressed is the mechanism by which participation in EnTeam group and individual activities led to increases in the use of cooperative learning in the classroom and decreases in perceived barriers to the use of cooperative learning. These findings were intriguing in that the positive results were found even with teachers who incorporated individual EnTeam activities in their classrooms in only a very limited way. In other words, the positive effects of participation in EnTeam were more in terms of teachers' beliefs and practices about cooperative learning more generally and less about the incorporation of specific EnTeam activities into individual classrooms.  It will be important to investigate and better understand how participation in both individual and group EnTeam activities leads to changes in teachers' beliefs and practices not only about EnTeam specifically, but also about cooperative learning more generally.

Appendix A - More Detailed Comments from Interviews

 

Overall Assessment of EnTeam Implementation:

• TW great to work with and have work with teachers. Doesn't respond immediately - Listens first and then offers ideas/interventions based on his theory and techniques.
• Character ed. program was starting to run out of fuel; teachers were struggling with what to do next.
• EnTeam provides a win-win situation for everyone and all groups. Very, very good at promoting cooperation and win-win situations.
  • Example: In a special ed. class doing an EnTeam activity, everyone, including students and observers, was involved. This is a sign of success - more than expected.

 

 

Relationship between EnTeam and Prior Ideas about Cooperative Learning:

• EnTeam activities were good beginning for teachers learning process of cooperative learning. Principal/school need to follow up with in-service next year.
• Education has to change - Can no longer be teachers standing up, lecturing, and handing down edicts.
• Benefit of cooperative learning for students is that kids are more involved in learning, but it is hard any time you change the whole structure of teaching.
• In the short-term, goal is for students to do better. Long-term goal is for teachers to be educated in paradigm of cooperative learning.
• Have used cooperative learning and peer tutoring before, but not as much as would like. These activities were different from EnTeam because they weren't always win-win. EnTeam activities are unique - students thinking to get an optimized situation.
• Before, thought of cooperative learning as group work. Now, cooperative learning with EnTeam is win-win.

 

Teachers and EnTeam:

• These kinds of activities are a whole lot of work for teachers. They'll try it, but it is a question of getting them to adapt and incorporate it into daily classroom practice. Not impossible, but an extended learning process for teachers. Principal plays an important role and has to give it value and reinforce it.
• Would be nice to have an innovative way to incorporate individual assignments and still meet objectives and then come together at end. We know that students need to talk to learn, but generally provide individual instruction and board work that keeps students on task. The environment is not as fun as we would like, but routine helps with discipline and students seem to act out more in novel environments.
• To use EnTeam more, need to know enough to do it on own, given pace of class and amount of material to be covered.
• One teacher did not try EnTeam immediately because had so much material to cover. Eventually accommodated lesson to EnTeam activity, though simpler content than what class is currently working on.
• Hard to think about doing cooperative learning with math, but helpful in character education. If someone could come up with a way to do it in math, would be open to trying it.

• Some teachers need to change their mindset re: teaching style. This is not going to be resolved in 1 or 2 semesters.
• Need more in-services where teachers actually participate in EnTeam activities. Teachers can then build on and expand on these activities.
• Would like teachers to take more advantage of EnTeam resources.
• Teachers who didn't take advantage possibly felt that they needed that kind of support/resource. Focused on their own thing and not open to intervention or new teaching techniques/styles.
• Those teachers/classes that did use EnTeam successfully took the theory/technique and expanded it to multiple lessons/different lesson types.

•  
  • Example: Special ed. and regular ed. classes teamed for activity. Formed groups of 4 or 5 and couldn't tell who was in regular education and who was in special education.
  • Example: Math class struggling with unfocused students who were not on-task. TW/EnTeam offered some intervention strategies that worked.

 

Students and EnTeam:

• Makes the environment enticing for kids.
• Teachers should always have at least 3 activities going on in the classroom and at least 1 should involve teaming or a project approach to make it more intriguing/interesting for students.
• Perspective of learning will be different.
• EnTeam presents a breath of fresh air and lets students know that math can be fun.
• EnTeam activities get students' focus off of themselves. They listen better and are more cooperative in the classroom. Helps them see that, with help, the can do things.
• Some students didn't follow rules. Some students didn't see connection between EnTeam activity and academic activity.

 

Ideas for Future Implementation:

• Use right from the beginning of the year.
• Students need help understanding how to behave in large groups and during assemblies.
• Have people (rappers, athletes, etc.) come in and talk to the students about character education - Though this is not really an EnTeam activity, these guests could present EnTeam activities to kids.
• Approach implementation next time with in-service for staff instead of going into classrooms. Although the in-service was done, it was not in enough depth. Having more detail about EnTeam was and participating in some EnTeam activities would help teachers accept it a little more.
• Diversity is a big issue and it is about more than race. For instance, different interests, turf issues, students coming from other areas. EnTeam could help teach students the difference between clubs and gangs and that (in clubs) students can be accepted in a positive manner and still be accepted.