Book Summary: The Data Teams Experience

The Data Teams Experience--A Guide for Effective Meetings (Angela Peery, The Leadership and Learning Center 2011)
     *And including information on how we plan to implement these teams at BHS

This is the "red book" that Suzanne purchased for each high school teacher in 2011 - 2012 which along with some other materials from The Leadership and Learning Center form the basis for our Transformation Plan strategy called Teacher Data Teams (TDTs).  Hopefully everyone has read this book by now in preparation for data teams, but in case not, here is a summary.  The book is an easy and fast read though, so you should try it as we start into data teams.

Chapter 1 Data Teams and Alphabet Soup
The Leadership and Learning Center (located in Colorado and founded by Douglas Reeves) created the term Data Teams.  It is about how data-driven decision-making can be done at the classroom teacher level.  The MDE is pushing this process as a school reform strategy, and it is included in our PLA/Transformation Plan and in our performance evaluation system.

Data Teams can take on many forms and at BHS we'll be using content area teams:  ELA, math, science, social studies, electives.  Support/Sp. Ed. teachers will be members of one of these five teams.  So, for example, the science team might have four members:  three science teachers and one support teacher.  In general what a team does is to examine work generated from common formative assessments.  In a school our size, our formative assessments will usually not be common (e.g., the Biology teacher will be bringing Biology class data to the meeting, the Physics teacher will be bringing Physics class data, and so on), but the data team can still examine the student work and data together because instructional strategies, students' strengths and weaknesses, how teachers and students interact--basically the nuts and bolts of running a successful class--are the same.  A data team might decide to try a particular reading comprehension strategy and that strategy can be applied in Biology, and Chemistry, and Physics.  A week later the team can look at students' results and try to figure out why the strategy seemed to work better in Chemistry class than in Biology.  What was different?  Did the teacher implement the strategy differently?  Is it worthwhile for the Biology class to try some of what was done in the Chemistry class?

What Do Data Teams Do?
"Data Teams hold collaborative, structured, scheduled meetings that focus on the effectiveness of teaching and learning" (p. 3).  Issues not related to teaching and learning are not in the purview of the data team and are discussed at another time.  There are six steps to the data team process:

1. Collect and chart/display data.
2. Analyze data and prioritize needs.
3. Set, review, and revise incremental SMART goals (specific, measurable, achievable, relevant, timely).
4. Select common instructional strategies to be employed to address the learning challenges discovered in Step 2.
5. Determine results indicators.
6. Monitor and evaluate results.

"When analyzing student work using the data team process, teams can determine which students are already proficient, which are close to proficient, and which are further away.  Sometimes teams design differentiated instruction for the various performance groups" (pp. 5 - 6).  This statement describes what we want from the data team process at BHS--we want to use it to advance our capabilities to differentiate instruction.

The chapter includes a section connecting data teams to RTI.  Support teachers may want to look at this to see how data teams will relate to IEPs and special education requirements.

Chapter 2 Getting Started--Focus, Structure, and Communication

How Are Data Teams Formed?

The Transformation Team has decided to go with the structure I described earlier:  content area teams.

What Roles Do People Play on Data Teams?

A team consists of a leader and members.  There are many ways to select a leader for a team, and the role of leader could even switch among members, but to get us started Suzanne has assigned each member of the Transformation Team to be a data team leader.  For 2012 - 2013 they are:

     Carole Bartig, Science

     David Brennan, Electives

     Candice Casey, ELA

     Kevin Lade*, Math

     Connie Portice-Brown, Social Studies

*Since I'm transferring to a position outside of the math department, I'll likely need to relinquish this role to another.

A Data Team Leader:  must be a good listener; an effective facilitator of dialogue; is not a pseudoadministrator; should meet with the principal at least monthly to discuss the content of meetings but not to critique one's peers; is not an evaluator; must sincerely believe that all students can achieve with appropriate support from adults; must be willing to challenge the views and assumptions of colleagues to strategize and innovate; should be well-informed about instructional strategies

Other roles:  recorder, data technician, data wall curator, timekeeper, focus monitor 

Responsibilities of these individuals at BHS:

• Leader:  create and email meeting agendas; forward the artifacts of team data to the district Data Coordinator; run the meetings; meet with other leaders and the principal monthly
• Recorder:  keep minutes (electronically right on the agenda form might be simplest); forward minutes to the leader, other team members, district Data Coordinator, and principal
• Timekeeper/Focus Monitor:  This person is key to keeping the discussion on track with the data team protocol--tightly focused on teaching and learning.
• Everybody:  organize your data prior to the meeting; be engaged participants; make meaningful contributions and commit to enacting selected strategies back in the classroom; listen respectfully; push the group's thinking forward; research instructional strategies to increase the team's repertoire
• Data Coordinator:  The PLA/Transformation Plan calls for a Data Coordinator to monitor and support many elements of the redesign plan.  With respect to data teams, the coordinator will act as the data wall curator, taking the artifacts of team data and creating displays that identify team goals and celebrate team successes.  Element 3 of the Transformation Plan (see that tab in the Livebinder) explains the public data wall and private data shared folder.  The coordinator will also provide logistical support to data team leaders and members.  The coordinator will not evaluate data teams; that is the responsibility of the principal.
• Principal:  monitor the progress and effectiveness of data teams; intercede and support struggling team members; evaluate teachers using the pertinent elements of the performance evaluation system--this implies that the principal must attend several team meetings during the school year

How Do Data Teams Determine a Focus to Begin Their Work?

There are several ways that a team could begin.  The Transformation Team has decided to initiate the process by having teams examine their pre-exam data.  This will take place during the first "regularly scheduled" before-school meeting.  All necessary preparation work (training, establishing team norms, setting the meeting day) will have already taken place on one of the beginning of the year PD days.  So, when the morning meetings begin, the teams can be off and running with the actual protocol, applying the templates specified in the team's agenda (note:  Steps 1 - 6 templates are available as downloadable documents in the Livebinder).

How Do Administrators Interact with Data Teams?

• Set the tone for data teams in the building
• Ensure that expectations are clear and that teams have the resources they need
• Provide dedicated time and tools to make data analysis easier (But our TIES gradebook doesn't have sophisticated tools so I think we'll be doing a lot of analysis "by hand" when it comes to classifying students into performance levels)
• Interact with teams and leaders frequently so that all teams are high performing
• Actively monitor teams by studying minutes, providing feedback, conducting frequent classroom visits to see if strategies are enacted during teaching, meeting with team leaders
• An administrator in attendance at a meeting doesn't lead the meeting, but is encouraged to participate like any other member

How Is the Work of Data Teams Communicated Within the School?

Minutes are forwarded to each member, the Data Coordinator, and the principal.  A data wall will be created to identify team goals and the instructional strategies members are employing, and to celebrate results when student achievement increases.  Individual teachers and students are not identifiable on this display.  A private folder (shared in Google Drive) is used for educators to monitor progress and make transparent what is happening in classrooms building-wide with respect to how effectively we're teaching students.  Teams are to retain and organize the artifacts of team data (i.e., the Steps 1-6 templates) electronically or in a 3-ringer binder.  Our redesign plan calls for illuminating what's going on in classrooms so that we can learn from each other.  We can't keep our best efforts secret behind the closed doors of our rooms; we need to share building-wide.

How Is the Work of Data Teams Communicated to Stakeholders Outside the School?

The book discusses "data fairs" which are not something that the Transformation Team has considered at this time.  But, that's not to say that won't change in the future as we refine our use of the protocol.  If we're making great gains, then we'll want to advertise that!  Also, because the school board is required to carefully monitor our progress, I would hazard a guess that the teams or Data Coordinator may be called upon to present information to the board.

Chapter 3 Step by Step--An Introduction to the Process Followed in Data Team Meetings

This chapter goes into greater detail about what teams actually do when they meet.  Because data teams are interested in monitoring students' growth, the data that we're concerned with will come from the formative assessments that we use at BHS for our learning goals.  It is essential that a preassessment of each LG be given so that when a selected strategy is used there is a baseline from which to determine growth (*note:  this means that if students are absent for the pre-assessment, then you must have them take it upon their return, perhaps during 7th period).

Now, here is where we must deviate from the "ideal" situation described in the book.  Because of the size of our district, it's not common for multiple teachers to have the same course and therefore be teaching the same learning goals at the same time, giving assessments at the same time.  Instead, teachers within a data team could be anywhere within their respective curricula.  Our data team meetings will necessarily be very flexible:  one social studies teacher might share the results of a preassessment about the causes of World War 2, another might be in the middle of a unit on the Middle East, the econ teacher might be about to give her last planned assessment about trade policies and domestic trade, and a support teacher might have some data on her caseload students' performance on their last assessments in various classes.  With all of this going on (and only 45 minutes once per week to conduct the meeting), the team can still identify a common instructional strategy to implement back in the classrooms; it's just that data may be trickling in at different times.  It is the Data Team Leader's job to aggregate this data in meaningful ways (e.g., by course, learning goal, and strategy).  The Transformation Team estimated about one hour per week for the Data Team Leaders to perform their duties--it is unknown whether compensation will be negotiated.

What Constitutes "Data" for a Data Team Meeting?

Data from formative assessments (the pretest and post-test) consists not only of the number and percentage of students who met a stated standard, and the number and percentage who fall into the performance levels for differentiation, but also information taken from students' responses.  Teams can analyze error patterns, conduct item analyses, and examine extended responses.  This is done with the intent to determine the effectiveness of instructional strategies, to figure out why one teacher seemed to be more effective than another (was it the content or something else?), and to inform decisions about future instruction.  Obviously, this sort of research into the cause data (a particular strategy) and effect data (students' performance) isn't gold-standard research, after all we're not attempting to compare experimental groups to control groups or to control for all of the other strategies that were used during instruction.  We're simply trying to bring a collaborative approach to the decision-making that goes into lesson planning and differentiation.

What Do Data Teams Do When They Meet?

Here are the steps again:

1. Members need to arrive at the meeting with any necessary data already disaggregated and charted by performance level.  An Excel spreadsheet is available for this (see Step 1 template in Livebinder).  Teachers are to provide numerical data and classify each student into a performance level.  I'm taking a total guess here, but I'd figure on 15-minutes per class of 30 students for you to order their papers from high to low score, access the template, type in the pertinent numbers and student names, email the data to your team leader, and print yourself a hardcopy to bring to the next meeting.  And, most important, don't stop there!  Use the results to go back into your weekly lesson plan form and revise what you'd planned.
2. The team analyzes the data and prioritizes needs.  It is important to go beyond labeling students into their "level."  Make inferences about their performances.  For example, let's say that in math class some students bombed a pretest on graphing linear equations.  So what?  That's not helpful to me.  Instead, I need to guess at the reasons why they couldn't perform the skill, because then I have something to target strategies at.  There is a template for Step 2 in the Livebinder, and while it could be filled out electronically, I think it may be easier to bring it to the meeting as a hardcopy and write on it with pen or pencil.
3. Set a SMART goal (or review an existing SMART goal in order to set a new goal or revise if not enough students reached proficiency).  There is a template for Step 3 in the Livebinder, which could be completed with pen/pencil, but at some point these SMART goal statements do need to get into your team's minutes and sent to the Data Coordinator.
4. Determine instructional strategies to be implemented in order to raise student achievement.  It's strategies (plural) because we are meshing data teams with differentiation.  The book cautions that inexperienced teams (that's us!) will struggle to do all of this in the limited time allotted to meetings, and it is crucial that a strategy be agreed upon and immediately implemented in classrooms.  So, until we get better at the process, teachers will probably have to do a lot of the differentiating during their regular planning periods.  There is a template for Step 4 in the Livebinder, which you may find easier to use as a hardcopy with pen/pencil.  More important than completing this template to perfection is for you to open up your weekly lesson plan form and adjust your differentiated strategies.  (As an example of all of this, imagine you're on the Electives Data Team and you all agree to use the Cornell note taking strategy in your respective classes in the hopes of raising student achievement from pre- to post-test on whatever LG you're studying.  Here in step 4 you'd plan how to make Cornell notes successful for students within these four performance levels:  maybe the top group needs 1-minute explanation and a sample, and they're off and running; while the bottom group needs a teacher think-aloud, groups of 3 people with assigned roles, and feedback/edits made by another group).  Am I explaining this so that it makes sense?  (1) Get data, (2) Make inferences from the data, (3) Set realistic goals for the whole class, (4) Pick a strategy and figure out how to differentiate it so that every student actually benefits from it.
5. Determine results indicators.  "These are the agreed-upon evidence that the strategy was effective.  They describe the teacher behaviors that will be seen as the strategies are implemented and the student actions that provide evidence of the impact of the strategies upon learning" (p. 24).

"Data Team members must leave this meeting knowing the SMART goal, what they will do to improve instruction, how they will monitor their own efforts, when and how the post-assessment will be administered, and when the next meeting will occur" (p. 24).

How Is Student Work Used in a Data Team Meeting?

Here again we will get better with experience.  Initially there probably won't be sufficient time to dig down into students' answers for every teacher who brings student work to the meeting.

Chapter 4 Introduction to the Crucial Sixth Step--Monitoring and Evaluation

"In the places where Data Teams have been most successful, principals, content-area supervisors, instructional coaches, and central office administrators regularly visit Data Teams during scheduled meeting times" (p. 34).  

It is possible that due to the particular locations within the curricula of the teachers in a data team that nobody has any new data to bring to a meeting.  Perhaps pretest and post-test data was discussed at the last meeting and everyone just happens to be somewhere in the middle of a unit.  With no new data to bring to the table, the team can't go through the five steps.  In this situation the team holds a monitoring meeting.  The team can discuss how the implementation of strategies is going, or might add or replace a strategy, or might examine student work, or might brainstorm strategies for differentiation.  Just keep the focus on teaching and learning.

Rubrics are provided to evaluate the effectiveness of data team collaboration (pp. 38 - 39).  The Transformation Team adapted these rubrics when developing elements of the performance evaluation that relate to teachers' participation in data teams.

pp. 40 - 41 identifies several norms that teams should adhere to, and recommends anonymous surveys be conducted periodically so that teams can internally monitor their progress.

Chapter 5 We're a Team, Now What?  The Initial Challenges Teams Face

"Once Data Teams are formed, there is often a period during which teachers adjust to working with each other in this new manner" (p. 45).  The book discusses a model of group development by Bruce Tuckman.  Apparently, we will begin in the forming stage, being busy with organization and logistics, getting better acquainted, being nice to each other, and doing little work of significance.  And apparently we all may get frustrated that the forming stage takes a while, so team leaders and members and principal, remember that this is going to occur!

Next we'll enter the storming stage, where we turn our attention to problems to be solved, namely students' learning challenges.  But, now that we've become comfortable presenting our own views, controversy and conflict will result.  Tuckman says some teams move through this stage quickly and others are stuck here forever.  Team leaders will get frustrated in this stage and support from the principal or instructional coach might help teams to develop better collaborative skills.

Next we'll hit the norming stage, where everybody takes responsibility and keeps their commitments.  It's all groovy.  And finally there's the performing stage, where we're like the gods of data teaming.

The chapter goes on to discuss the roles and challenges confronting team leaders and team members.

Chapter 6 Hitting Our Stride--What High-Performing Data Teams Do

This chapter tells some success stories from a few school districts.

Chapter 7 Under the Magnifying Glass--The Steps in More Detail

Each step is carefully detailed and three levels of performance are described.  The Appendix gives rubric charts that show these levels for each step so that teams can evaluate their progress at each step.  I'll tease everybody who's been reading this summary by telling you that this chapter is probably the most helpful one in the book, but I'm not going to tell much about it.  You'll have to read it for yourself.  

One area I do want to highlight though is the discussion of SMART goals because I've mentioned them a few times, but haven't explained what they are.  A SMART goal is about setting a short-term goal that can be reached in a brief instructional cycle (e.g., setting a goal for a LG).  A SMART goal has five parts:  specific, measurable, achievable, relevant, timely.  Here's an example from Algebra 1:

     The percentage of Algebra 1 students proficient and higher in "LG 1 Reason quantitatively

     and use units to solve problems" will increase from 15% to 60% as measured by a post-test, 

     given in one week.

Here the goal is obviously specific, measurable, and time-bound.  Where I had to make a serious decision was in determining where to take that 85% of students who failed my pretest.  Should I have set a goal to move from 15% proficiency to 100% or maybe 90%?  I think that would depend upon the students in my room.  For instance, what if my data showed that I only had enough students "who are likely to be proficient after instruction" and the students "who need targeted instruction or additional time" to bring my class proficient rate to 60% in a unit of reasonable duration?  Perhaps my fourth group "students who need extensive interventions" is very large.  I have a tough decision to make here.  If I set my goal too high, I won't make it; and if I set it too low, then I'm shortchanging students.  Data teams are not some magic bullet that will make all students proficient.  They are simply a mechanism for bringing more minds to bear on choosing instructional strategies while giving those minds some data to chew on to inform the decisions.

Chapter 8 Troubleshooting and Frequently Asked Questions

Appendix--Instructional Data Team Meeting Rubric