Book Summary: Transformative Assessment

Transformative Assessment (W. James Popham, ASCD 2008)

We’ve been using formative assessments at BHS for a few years now.  Are we using them properly?  We’ve based our system of assessing students’ progress and tracking their progress toward learning goals on Marzano’s books.  But, he’s not the only researcher out there.  What do others have to share, and are their ideas helpful to us?  Popham is widely considered to be “the expert” when it comes to assessment.  It will be interesting to learn what he has to say on the matter, plus, he writes in a snappy casual style, which is refreshing.

In the preface Popham makes a bold statement that classroom assessment “can fundamentally transform the way a teacher teaches” (p. vii).  Already I’m feeling bad because even though I’ve supposedly used formative assessments for a few years now, I don’t think my instruction has been fundamentally changed.  Thus, I don’t think I’ve really been using “formative” assessments even if that’s what we’ve been calling these periodic tasks for which we’ve entered scores in Pinnacle gradebook. 

Popham identifies four “levels” of formative assessment:

•  Level 1:  Teachers use formative assessment to collect evidence by which they can adjust their current and future instructional activities.
• Level 2:  Students use formative assessment evidence to adjust their own learning tactics.
•  Level 3:  There is a complete shift in the culture of a classroom.  Classroom assessment isn’t used to compare students based on grades, but as a means to generate evidence from which teachers and students can adjust what they’re doing.
• Level 4:  There is schoolwide adoption of one or more levels of formative assessment.

Where are we at BHS?  No doubt some teachers are more successfully incorporating the various levels, but I think we’re well short of Level 4. 

Chapter 1 Why, What, and Whether

Popham explains the history of the term “formative assessment” and follows up with his definition:  “Formative assessment is a planned process in which teachers or students use assessment-based evidence to adjust what they’re currently doing” (p. 6).

At BHS our redesign plan gives a lot of attention to this.  The building-wide lesson plan form, along with the data team process, constitute a planned process to adjust instruction based on in-class checks for understanding (assessments).  Our whole approach to lesson planning is changed.  No longer are teachers to plan out a unit or week’s worth of material, submit it to the principal, and then rigidly stick to that plan.  Instead, lesson planning is to be responsive to the needs of the students in the classroom as identified using regular checks for understanding.  The lesson planning form is to be completed and submitted (e.g., each Monday), but it is to be continuously updated throughout the week.  Plus, differentiated strategies are to be applied to students at different performance levels with respect to the learning goal(s) being studied. 

At BHS the redesign plan also calls for greater emphasis on students modifying academic behaviors based on assessment evidence.  Under our Marzano-influenced efforts teachers were to encourage students to track their own progress and create action plans, but, we didn’t seem to have much success with that intervention.  In fact, many teachers quickly abandoned having students even track their scores, let alone using class time to be responsive to their performance results.  In 2012 – 2013 this needs to change.  Teachers are to plan for opportunities to support students in taking personal (and perhaps cooperative) responsibility for their own learning.

The third crucial piece to the definition is the expectation that the formative evidence will be used to adjust what is currently being done.  For some teachers this will also be a change.  The data isn’t to be used to plan how you’d teach a topic differently next year, it’s to formulate how you’ll teach it in the next 30 minutes, or maybe more realistically, tomorrow.  For this reason, teachers may have to use completely different assessment tasks.  Timely feedback and opportunity for data analysis is essential.  A formative assessment cannot be a large test or project that will take the teacher several days to “score.”  This isn’t to say that you can’t use those tasks, but that they aren’t most appropriate for formative assessment. 

Chapter 2 Learning Progressions

“Formative assessment is all about decision making.  Those decisions, made by both teachers and students, invariably revolve around the following two-part question:  ‘Is an adjustment needed, and, if so, what should that adjustment be?’” (p. 23).  Popham says that it’s pretty much impossible to adjust everything in response to data, and so there needs to be a framework around which decisions can be made.  This framework is a learning progression (yes, the same thing we’ve created when following Marzano to make rubrics for our learning goals).  A learning progression is a sequenced set of subskills and bodies of enabling knowledge that, it is believed, students must master en route to mastering a more remote curricular aim.  Popham cautions:

• A learning progression isn’t unerringly accurate.  It's just what a teacher thinks is a sequence of steps to complete understanding of a concept or skill.
• A particular learning progression isn’t suitable for all students. 
• A learning progression isn’t necessarily better if it’s more complex.
• A learning progression tells teachers what and when to assess. 

1.      What should be assessed?  Answer:  The subskills and enabling knowledge identified in the progression.
2.      When to assess?  Answer:  Before proceeding to the next building block in the progression.

Marzano said all these same things, but we seemed to get caught up in worrying about gathering four scores for a learning goal instead of worrying about gathering the right evidence at the right time.  I’ve described elsewhere (see the Livebinder) how our approach to recording scores will change when we switch to TIES gradebook.  Briefly, before starting a unit or mini-unit over a LG give a pretest (this score is entered into TIES, but given 0 weight).  Use the results from this pretest to formulate your lesson plan for the unit (e.g., maybe you’ll need one or two weeks to adequately teach the LG) including making a start at differentiated strategies.  Check for understanding along the way (for what and when, see Popham’s advice).  These scores can also be recorded in the gradebook to identify learning progress (but they still carry 0 weight).  Continuously adjust lesson plans and differentiated strategies based on these checks.  Once all, or a majority of, students do well enough on a check for understanding that you predict success on a post-test, give a post-test.  In TIES, record a score that represents your best judgment of a student’s current understanding for the LG (and weight this 100%).  Later in the course if the LG comes up again (or a student shows improvement or maybe even goes backward) just change this “final” score.

There’s a cool overlap of Popham’s discussion of how to use learning progressions and formative assessments with the data team protocol we’ll be using in 2012 – 2013.  Data teams only meet once per week so teachers may need to use the protocol on their own when planning in order to remain timely.  A step in the protocol is to examine student data in order to make inferences.  Popham  suggests that the inferences we’re after are:  (1) has the student already mastered the building block?, (2) has the student partially mastered it?, (3) has the student not mastered it at all?  And, if there’s lack of mastery, to try to figure out why.  From there, plan next steps.

Popham spends the rest of the chapter explaining a procedure to develop a learning progression.  It is quite similar to the procedure that we used when following Marzano.  I’ll just list the steps:

1. Thoroughly understand the target curricular aim.
2. Identify all requisite precursory subskills and bodies of enabling knowledge.
3. Determine whether students’ status with respect to each preliminarily identified building block can be measured.
4. Arrange all building blocks in an instructionally defensible sequence.

Chapter 3 Teachers’ Instructional Adjustments

Here are four steps in Level 1 formative assessment:

1. Identify Adjustment Occasions.  *Note:  this is not referring to those on-the-spot responses teachers make such as giving one-on-one help on a math question or asking a follow-up question to a student.  Instead, these represent the “most significant choice-points associated with students’ movement toward mastery of the target curricular aim” (p. 53).  This is about teachers looking forward into their lesson plans to identify occasions to gather evidence in order to adjust instruction.  Fortuitously the building-wide lesson plan form anticipated this recommendation and has a field for teachers to identify their planned assessment occasions.
2. Select assessments.  There are lots of choices:  traditional, letter cards, questioning during discussion, whiteboard responses, traffic-signal, item sampling
3. Establish adjustment triggers.  This refers to teachers deciding ahead of time what will trigger a decision to adjust instruction.  Almost all instructional adjustments boil down to either increase instruction or decrease instruction, so teachers need a trigger for both.
4. Make instructional adjustments.

Chapter 4 Students’ Learning Tactic Adjustments

“Level 2 formative assessment consists of student-determined adjustments to their learning tactics, not teacher-dictated adjustments that students are supposed to make” (p. 72).  Teachers must play a supportive role, not a decision-making one.  But, teachers cannot only be permissive of students taking personal responsibility, they need to be strongly encouraging.  Teachers will need to teach students what a “learning tactic” is and why it’s important that students become personally responsible for their own adjustments.  This requires initial orientation and ongoing support.  One type of support is clarifying curricular expectations:  (1) the nature of the immediately upcoming curricular aim, (2) the evaluative criteria to be employed in judging students’ mastery, (3) the chief building blocks involved in mastery.  In all this, use language that the students will understand, explain assessment procedures, and share extreme responses (i.e., really strong vs. really weak student or mock responses).

The same four steps that apply in Level 1 formative assessment work here too. 

1. The teacher tells students when the assessments are (i.e., when there will be adjustment occasions).
2. The teacher will most likely have to prepare/direct assessments (because most students don’t have the initiative or capacity to do if for themselves).  Some student choice can be brought into play if there are optional assessments or maybe peer assessment.
3. Students also need to establish adjustment triggers.  The teacher’s role here isn’t to tell a student what trigger to use, but to monitor periodically and make suggestions when the student’s learning tactics aren’t working.
4. The teacher probably will have to suggest alternative learning tactics.  But, the teacher doesn’t make any decisions for the student.

Popham uses the rest of the chapter to discuss Level 3 formative assessment.

Chapter 5 Schoolwide Implementation

Chapter 6 What It Can’t Do

Much of this chapter is a rather high-level explanation of why using formative assessments in the classroom doesn’t necessarily correlate with increased achievement on standardized tests.  Popham goes through many of the arguments that most of us are familiar with (test bias in relation to SES, instructionally insensitive instruments (they’re designed to rank students, not compare quality of instruction), aptitude-linked questions, too many curricular targets, dysfunctional reporting of results (e.g., at the strand level instead of focused curricular aims).  Anyway, so after a whole book touting formative assessment, we’re left with his conclusion that we probably won’t see increases on standardized tests due to the very way these tests are designed and revised over time.  I guess our hope then is that properly utilized formative assessments will lead to more effective instruction and greater student success in classrooms, which in turn will lead to more engaged and motivated students willing to persevere and give greater effort during the MME.