Low-stakes student testing still has negative effects

It appears changes are coming to Alberta’s provincial achievement tests. Instead of assessing learning at the end of grades 3, 6 and 9, there is talk of testing at the beginning of each year (and perhaps every year) to identify each student’s strengths and weaknesses. Given the low cost of machine-marked tests, there is also discussion of retesting at the end of the year to see what gains have been made (cue ritualistic chanting of “accountability and transparency, uber alles!”).

On the surface, this sounds pretty reasonable. Yet it is worthwhile considering what kind of impact this approach would likely have on teachers and instruction. The government is saying that it has no interest in linking testing results to pay (so-called high-stakes testing).

That sounds levelheaded.

But my sense of Alberta’s Conservatives is that they will say virtually anything to get what they want and then conveniently forget their promises. And, even if Education Minister Jeff Johnson is earnest in his promise, governments have a hard time resisting the temptation to exploit the surveillance opportunities that come with any new technology.

But let’s say annual pre- and post-testing is on the up-and-up and teachers and parents get info on student performance at the beginning and end of the year. What effect is that going to have on instruction?

Even without explicit pay incentives, teachers are going feel pressured to ensure their students (as a group) get high scores. The pressure will come from parents who will use the tests as the desiderata for determining who the “good” teachers are and which schools are “good” (no doubt egged on by Fraser-Institute rankings of schools).

The pressure will also come from principals (and perhaps peers) who will use the test scores to inform their assessment of teacher performance. Even with no explicit consequences, hearing one’s boss or colleague say “boy, it looks like you had a tough year this year” is something no worker wants to hear. It means a loss of status and credibility and all the intangible benefits those things entail for workers. It also means fear of future consequence if one’s scores don’t improve.

In essence, the existence of a testing technology that makes workers “transparent” to their boss (google panopticon for a fuller discussion) and strips performance of context (which is the purpose of quantification) creates the negative effects most often associated with high-stakes testing. These effects are pretty well established:

• Teachers will teach to the test. Maybe emphasizing tested content is a good thing. My guess is that it will make for a less diverse curriculum that emphasizes easily testable material (e.g., calculation, definition, association) moreso than creative application of knowledge.
• Teachers will teach test taking. Taking a test is a learned skill. Good teachers will teach students how to game tests. Yeah, the government can control for that to some degree. But the bigger issue is that gaming tests a skill with limited (and frankly negative social) utility outside of the school system.
• Teachers will triage students: The biggest gains over the year will come from those students who enter the year in the middle of the pack. High scorers simply have few gains to make. And low-scorers require a lot of effort to see test gains (or have other challenges that make gains unlikely). So teachers will (quite rationally) spend most of their effort maximizing the gains for average students.

This list of behaviours isn’t meant to demonize teachers (who are generally hard working and lovely people). It is simply meant to identify the behaviours that this kind of testing rewards—even without explicit employment consequences.

Over time, this kind of testing will also serve a sorting function among teachers (“Griffindor!”). To the degree that teacher performance varies due to effort and other factors, some teachers will eventually amass records of better and worse performance. Teachers with better records will then be able to use these records to acquire “better” jobs (i.e., jobs at schools where there are fewer students who struggle) because principals will (if only informally and perhaps on the QT) use test scores as a selection criterion.

The effect of this sorting is that “good” schools will get the better teachers while “bad” school will get worse teachers, creating a vicious cycle. Obviously there are counterbalancing factors in hiring (e.g., some good teachers will relish a challenge, the flow of teachers is and can be constrained in many ways).

But why create a system that naturally produces bad classroom and system-wide effects? Why create a system where the public must rely upon teachers and principals to act contrary to their own interests to avoid those effects?

Now there may well be some value in testing. It might well inform teachers’ practice. Although how much slack the average teacher has in his or her workload to address individual weaknesses is a fair question to ask. It is also fair to ask whether teachers really need standardized tests to identify students’ strengths and weaknesses?

A second value of testing is that it will make it easier to hold teachers to account for student progress. There are lots of issue with this, the most obvious being that learning outcomes are not often or fully within the control of the teacher. And, of course, suddenly we’ve drifted towards high-stakes (for teachers) testing haven’t we?

I certainly appreciate the public’s appetite for better information about their children’s progress. But could that not be remedied via incremental change—such as more quantitative report cards and replacing the tedious and uninformative “student demonstrations of learning” with parent-teacher conferences?

-- Bob Barnetson