Testing terminology: Formative and Summative assessment

I've talked before about specialist terminology a few times in the past.

My view (in short) is that terminology should be meaningful to be useful, but far too often teachers forget to explain why a word means what it means.

For example, the terms "formative assessment" and "summative assessment" are very common. I first encountered them when studying with the OU. There was a brief explanation of how they affected us as students and they moved on. When I later started studying teaching, I was given more formal explanations, and it struck me as interesting that there was actually less clarity about the meaning of the terms on the teacher training courses than in general student populations.

But I don't recall any of my teacher trainers ever pointing us towards the words themselves, because actually in this case I believe the simplest explanation for the concept comes from explaining the words. (If the explanations below aren't clear, please feel free to mention it in the comments.)

Formative Assessment

The term "formative" comes from the verb "to form" – i.e. to shape or to develop. Our "formative years" are the years of our youth that shape us, and make us who we are.
Formative assessment is any kind of test or assessment that is designed to give the student feedback or assistance in developing their skills.

Summative Assessment

The first syllable here – "sum" – says it all. It's about adding up and calculating scores.
Any assessed piece of work which is used as part (or all!) of a final course grade is a summative assessment.

A false dichotomy

As well as a lack of clarity in the description of both terms, people are often confused because formative and summative assessment are often presented as though they are mutually exclusive categories. However, if we look at the actual definitions, we find that while some assessments fall into one category or the other, many fall into both.

For example, when I took my first degree, the typical structure of assessment was made up of three elements:

1. Weekly homework that would be marked in class. Our results would be a reference for ourselves as what to work on during our study out of class, and the tutors would specifically address as many of our problems as they could in the class. No marks would be recorded.
2. Monthly assignments that would be marked and returned with relatively detailed feedback as to our strengths and weaknesses. Some of the same material might be covered in future assignments or in the final exam, so it was a good idea to read the feedback very carefully. Marks would be kept and would make up 50% of our final mark.
3. An exam. No feedback would be given, just a numerical score. This was 50% of our final mark.

Looking at them, number 1 is clearly formative (feedback, no grade), and number 3 is clearly summative (grade, no feedback), but number 2 has elements of both. Indeed, number 2 is both formative and summative, and most continuous assessment fulfills both roles.

Conventional use of the terms

In my experience as a student in higher education, the terms are typically used in a much narrower sense.

First of all, the terms are only ever used with students to describe assignments done in our own time for submission. Exams are not described as "summative assessment", even though they technically are. The term "formative assessment" is typically used to describe a sort of "practice assignment" handed in early on in the course to ensure the students understand what's expected of them.

That's all well and good for labelling things in the student handbook, but the teacher needs to be working at a deeper level than that, and not thinking of an assessment as "a formative assessment" or "a summative assessment", but as an assessment that has formative and/or summative functions.

It might seem quite a subtle difference, but I personally think it's an important one.

After writing my post on gap-fills and cloze tests in language lessons, I went back to a blog post that I'd read some time ago, which was part of the Pools project that had introduced me to technology such as Hot Potatoes. I commented about how I had never seen the purpose of fill-in-the-blanks as a learning tool and his response was that it was a test.

Now, it has often been observed that repeated testing aids student retention, so most teachers would assume that anything that is a test is valid as a retention aid. Is this true?

As I said in my earlier post, the cloze test and the gap-fill rely on having a sound internal model of the language under test.  Does doing a cloze or gap-fill help build that knowledge?

In the previous post, I said that doing these tests early appeals to conscious knowledge.  Many of the most successful students will look at a fill-in-the-blanks exercise and reason through it.  If you ask them how they did it, they'll say things like "that's a noun and that's an article, so the thing between them must be an adjective".  As the conscious strategy proves so successful, the learner will continue to apply it and will perhaps never develop the gestalt.  And because any attempt to use gestalt at this time will appeal to the student's first language, the student who approaches the test in the intended way will be penalised.

In effect, the student learns how to pass the test, rather than learning the specific competencies that the test was originally designed to measure.

How much damage does this really do?  In my opinion, a lot.  As students go through their academic career, they will be expected to do more complicated things, and they will be expected to do them quicker.  But there is only so fast that we can conscious churn through these rules.  Sooner or later, we can only succeed by gestalt, but how can we encourage that?  The mechanics of testing militate against going by your gut, because this leaves you with a worse mark.

I'm not saying that we shouldn't use testing as a learning mechanism early on, but that we should look again at what constitutes appropriate testing - what tests the student can carry out in a way that supports, rather than hinders, learning.

Testing, testing, 1, 2, 3...
This of course does not only hold for testing with fill-in-the-blanks.  It has been said by a number of academics, with figures to back them up, that testing aids recall.  I'd like to present an argument against that.

"Are you mad?" I hear you cry.  Yes, but that's beside the point.

My point isn't that their figures are wrong, but that the word "test" is something of a red-herring.  What is it we do when we test a student's knowledge?  We check that we can recall accurately what they have been taught.

If we discuss this as "testing", we have in our mind a goal -- scores, marks, grades.  We don't want to focus on that goal when we use testing as a learning aid -- we need to focus on the process.

The process involved in testing is accurate recall and application of learned information.  A task can be designed to require recall and application without being strictly a test.  If the figures say that testing is a successful classroom strategy, might it not be because they are comparing "tests" against other tasks that do not require any recall or genuine sentence construction skills?  I'd like to demostrate how some of the most common exercises fail to rely on these skills.

The problem with drills 
Form drills, pattern drills, substitution drills; these repetitive exercises fail to teach because the core language we expect the student to learn from the task is never subject to recall.  The teacher says it and the student repeats it, changing only a vocabulary item of very small grammatical features.  We cannot learn to recall something without doing it.

The problem with communicative tasks
Anyone who has been at either end of a classroom recently is likely to have come across communicative exercises based on an idea like the "knowledge gap".  Each student has partial information or a partial picture, and the students have to talk to each other in the target language to get the information from each other.  However, being understood by your classmates is different from being understood by a native speaker, and accuracy not only slows down the task but potentially renders you incomprehensible to a classmate of lower ability.


In general, though, regardless of the exact nature of the task, the class starts by presenting or otherwise providing the students with the language that they are going to use.  If we attempt to do this by eliciting the information from the class, we may get one or two to recall it, but most will not -- instead they will end up holding the patterns in working memory, often meaninglessly and mechanically, and parroting the phrases during the class.

Conclusion

We all want to receive or provide the best education possible.  Empirically we know that "testing" aids this, but that statement is an oversimplification of the real situation.  There is nothing magical about "a test" that makes it more effective than "an exercise".  We must examine what the core activity is in what we consider a test to be, and we must find ways to incorporate that into the day-to-day teaching process.

I contend that the distinction implied between "teaching" and "testing" is artificial -- teaching we consider to mean presenting information and going through some kind of repetitive "training" regimen, whereas testing is an unsupported check of recall.

The idea that "testing aids retention" is therefore an obstacle to good practice, because it prevents us from looking at the nature of the tests to identify what really happens.  I believe the real point is that recall practice aids retention.  If so, our task design must always be built around developing recall, or the student will never be able to spontaneously produce language.