Testing terminology: Formative and Summative assessment

I've talked before about specialist terminology a few times in the past.

My view (in short) is that terminology should be meaningful to be useful, but far too often teachers forget to explain why a word means what it means.

For example, the terms "formative assessment" and "summative assessment" are very common. I first encountered them when studying with the OU. There was a brief explanation of how they affected us as students and they moved on. When I later started studying teaching, I was given more formal explanations, and it struck me as interesting that there was actually less clarity about the meaning of the terms on the teacher training courses than in general student populations.

But I don't recall any of my teacher trainers ever pointing us towards the words themselves, because actually in this case I believe the simplest explanation for the concept comes from explaining the words. (If the explanations below aren't clear, please feel free to mention it in the comments.)

Formative Assessment

The term "formative" comes from the verb "to form" – i.e. to shape or to develop. Our "formative years" are the years of our youth that shape us, and make us who we are.
Formative assessment is any kind of test or assessment that is designed to give the student feedback or assistance in developing their skills.

Summative Assessment

The first syllable here – "sum" – says it all. It's about adding up and calculating scores.
Any assessed piece of work which is used as part (or all!) of a final course grade is a summative assessment.

A false dichotomy

As well as a lack of clarity in the description of both terms, people are often confused because formative and summative assessment are often presented as though they are mutually exclusive categories. However, if we look at the actual definitions, we find that while some assessments fall into one category or the other, many fall into both.

For example, when I took my first degree, the typical structure of assessment was made up of three elements:

1. Weekly homework that would be marked in class. Our results would be a reference for ourselves as what to work on during our study out of class, and the tutors would specifically address as many of our problems as they could in the class. No marks would be recorded.
2. Monthly assignments that would be marked and returned with relatively detailed feedback as to our strengths and weaknesses. Some of the same material might be covered in future assignments or in the final exam, so it was a good idea to read the feedback very carefully. Marks would be kept and would make up 50% of our final mark.
3. An exam. No feedback would be given, just a numerical score. This was 50% of our final mark.

Looking at them, number 1 is clearly formative (feedback, no grade), and number 3 is clearly summative (grade, no feedback), but number 2 has elements of both. Indeed, number 2 is both formative and summative, and most continuous assessment fulfills both roles.

Conventional use of the terms

In my experience as a student in higher education, the terms are typically used in a much narrower sense.

First of all, the terms are only ever used with students to describe assignments done in our own time for submission. Exams are not described as "summative assessment", even though they technically are. The term "formative assessment" is typically used to describe a sort of "practice assignment" handed in early on in the course to ensure the students understand what's expected of them.

That's all well and good for labelling things in the student handbook, but the teacher needs to be working at a deeper level than that, and not thinking of an assessment as "a formative assessment" or "a summative assessment", but as an assessment that has formative and/or summative functions.

It might seem quite a subtle difference, but I personally think it's an important one.

Possessives and terminology

A couple of years ago, I wrote a post describing my objections to the traditional definitions of possessive adjectives and possessive pronouns. At the time, I still favoured calling what is called the possessive pronoun a "possessive adjective" (eg. "it is mine"), and calling the possessive adjective a "possessive pronoun" (eg "my car"0 for English... in theory.

However, practicality is another thing entirely, because there is nothing more confusing than using the same words as someone else to mean entirely the opposite thing, so I have never used the terms that way for students. In fact, I actively avoid using either term if I can possibly avoid it, as it doesn't seem helpful.

So recently I've been working on trying to find ways to better categorise grammar, and I've settled on what actually seems like a reasonable compromise.

For the possessive "adjective" of traditional grammar, I'm going with the alternative from the previous post -- the possessive determiner. It aligns with this, that, a and the, so it naturally falls into the class of determiners. This doesn't mean it isn't a pronominal form -- it actually means that forms like "John's" have to be considered determiners themselves... which is entirely logical, as "John's car" is "the car that belongs to John", but "John's" has replaced the definite article; hence "John's" must be a determiner anyway.

For the possessive "pronoun", I'm going with the possessive predicate. I decided on this when I was thinking about it as a predicative adjective in sentences like "It is mine," or "that's yours." Of course, that's not the only situation it occurs in, which is something I overlooked a little when blogging from the unbearable heat of Corsica in the summer. There is no predicate in "I'll give you mine," or "She didn't have hers, so she took yours." But at least it gets away from the counter-intuitive implication that the other form is not a pronoun.

I will continue to think about this, but if I do ever come up with a better term, I can just do a search-and-replace on what I've already got without any problems....

Horses for courses...

Perhaps my previous pontifications on the word “course” were unconvincing, but I think the word “course” can account for a lot of the problems in modern education.

Last year, I took a year of full-time study. My last exam was on a Friday, and the next day I found myself on a busy train home. I ended up sitting at a table with a couple of guys, and it turned out one of them was a retired tutor from a technical college. In Scotland, “college” has two meanings –either part of a higher education university, or more commonly a further education institute focusing on vocational studies.

 

He was working with trades – very practical, hands-on stuff like welding. He was already working there when the system of “modules” came in. A module, like the word implies, is a self-contained unit of learning, and it's a concept that is all too familiar in all areas of education. Well, this old guy described it as a disaster. The material he was working with, he said, just couldn't be broken down that way; everything was interconnected.

 

But that wasn't unique to his field. Less than a fortnight before, I was in a meeting with a couple of the staff from the course I'd taken discussing my feedback, and this was precisely my complaint there – modularisation.

 

I was studying the second year of a four-year degree scheme, and my initial concern was that I would be lacking some of the prerequisite knowledge from the first year modules. In particular, I was concerned that the module on short-form fiction would rely on literary analysis skills and terminology from the first year. But there was none of that. It was a true module – it was self-contained. No prerequisite knowledge. But that meant that (in my opinion) it sorely lacked depth, and at the end of it, I did not feel that I was ready for degree-level literary analysis courses, as would be expected of me if I continued to the third year. So in turn I concluded that the courses on offer could not be up to the standard I would expect of a third year course.

 

Of course, I do have unforgivingly high standards, in no small part down to the excellent quality of education I received for my first degree (at Edinburgh University).

But still, I felt that my dissatisfaction was caused not because the university I was studying in was not as prestigious as Edinburgh, and not because the teaching staff weren't up to scratch. No, the teaching staff were all well-educated and highly motivated, and had the benefit of an exceptionally good teacher:student ratio.

The biggest problem was simply this word “module”, this idea that skills and knowledge can be compartmentalised and packaged neatly, and this institution had fully implemented a completely modularised syllabus.

Which brings me back to the word “course”, and in particular the three ways the University of Edinburgh used it.

 

In my first year, I did three “courses”: Computer Science, Maths and Artificial Intelligence.

Each of these was composed of two “half-courses”, which would be the equivalent of a “module”. But the very fact that it was a “half” course made it clear that it was not a stand-alone “unit” of education, it was part of an ongoing learning path that extended through the year.

 

The third use of “course” was in the term “degree course”. My “degree course” was Computer Science, and everything I was taught in first and second year was a predetermined path that prepared me for the advanced study of third and fourth year. I had a much freer choice of courses in third year, and it would be tempting to call them “modules” here, but even as we made our third year choices we were all thinking about our final year, as each individual course opened up certain possibilities in the fourth year: our third year choice restricted the path – the course – our fourth year would follow.

 

So while it is true that not every university can be Edinburgh University, it's still useful to look at what Edinburgh does and see if it's worth copying.

 

Some universities pride themselves on having a wide choice of modules on offer. At Edinburgh, I had no choice whatsoever in first or second year. The university identified what I needed to know to prepare for my chosen degree, and they made sure I learned it well. (OK, there was stuff taught that I never really “needed”, but that's by-the-by.) And because we all learned the same things, when it came to third year, the lecturers all knew how much we knew, and they could build on it.

 

This took us further and deeper into every subject than we could have otherwise gone.

 

That's why the word “course” is so important. That's why we've got to know what it means, and what it doesn't mean.  The loss of the word "course" is travelling had in hand with the loss of a controlled progression in learning.

 

The Connectivist school of thought blames traditional schooling for students' poor ability to deal with content at levels of abstraction, whereas I believe it is the modulisation of the syllabus, a fairly recent development, that has caused this problem.  If each semester we start on a new separate module, one that stands alone and refuses to take as given the learning outcomes of specific prerequisite courses, then we are never, as students, taught to integrate our knowledge, and we are never given the opportunity to apply that knowledge at a higher level of abstraction.

 

This is not a small problem, specific to certain institutions – it's a growing problem, and it is becoming written into more and more of the documents that define our education systems.  Under the agreements that created the Bologna Process, students should now be free to move between institutions all over Europe, and in my English classes, I have several students who have moved from other French universities and even one who has transferred in from an Eastern European university to complete her studies in France.

 

The end result is a constant shallowing of our knowledge, and a sad, slow end to a once-great education system...

Everyone understands grammar.

I've often heard it said that grammar is difficult or that normal people don't understand grammar.  Some teachers believe that grammar would be useful in the language classroom, but only if the students came in understanding grammar in the first place -- they don't want to waste time teaching theoretical grammar before they can start teaching the target language.
And these are good arguments but for one small detail:
everyone understands grammar.
It has been observed that in cloze tests (passages of texts with blanked out words) native speakers will have a notion of the word-class of the correct answer before they know what the specific word is.
Take these three simple examples:

1. I like ____.
2. I want a ____ one.
3. Don't ____ me.

It's impossible to know what the original word was that the author intended, but at the same time, it's pretty obvious that number 1 is a noun, number 2 an adjective and 3 a verb.  If you were to put these sentences in front of any literate English speaker, they could give you any number of possible words for each example that fit the categories.

So they have an internal concept of word-classes, even if they don't have an explicit awareness of it.  This means that there's very little work for the teacher to do -- all you've got to do is point out what they already know.
Michel Thomas tried to do this a different way, and it's a way that tends to draw a lot of flak.  On his recorded courses, you can hear him say that he doesn't like the traditional definition of "person, place or thing", and then he gives a couple of examples of abstract nouns, which he says aren't really things.  He then goes on to introduce his prefered rule -- a noun is anything you can use "the" before -- and a couple of examples, using both abstract and concrete nouns.

It is quite trivial to prove that his rule is wrong -- "the white house"; adjective after "the"; wrong.  "John"; most proper nouns don't take "the"; wrong.  And yet for all it's inaccuracy, this "rule" seems to work better than most.  Why?  Because Thomas isn't trying to teach a new concept -- he is merely trying to evoke the concept that the learner already has.  In fact, he is doing exactly the opposite of defining, and I would argue that he is doing so on purpose -- if you already know about the concept, thinking too much will override your instinctual understanding, so thinking should be avoided.

What Thomas did was quite subtle.  He took the traditional (correct) rule, but said he didn't like it, freeing the student from feeling inadequate for not understanding it.  He then presented the case that causes consistent real problems for students, the abstract noun.  All he basically said in informational terms was "these are nouns too", but he worded it in such a way as to say "it's not your fault if these don't make sense in the old rule", swatting away any confusion and guilt or inadequacy.  His final comment, about "the", is valueless out of context -- it is factually incorrect, but he uses it to reinforce the concepts already clarified by the old rule and the abstract examples.  He ties them into one bundle.  The student never writes down the rule, the student never memorises the rule, in fact, regardless of what he says, he has not given a rule.  All he does is evoke the learner's internal concept of nouns and label it with the word noun.  I'm pretty confident that the average student coming out of a face-to-face class with Thomas would have forgotten he'd even said it, but they'd be able to label nouns pretty acccurately if asked to.

It took Thomas a few minutes to teach "noun", "verb" and "adjective" that way, less time than it takes me to explain what he did.

Any concept that exists in the student's native language has the potential to be taught in a similar way.

The secret is not to take too long, and not to get hung up on the technicalities.  All you need to do is evoke the concept and stick a label on it.

And if you think about it, that's what we do as teachers every day -- it's called "vocabulary".

In order to teach, you must understand

I was talking to several of my fellow students before Christmas about the courses on offer at SMO, Scotland's only college offering degrees through the medium of Gaelic, and in particular the lost opportunity to teach languages efficiently to a completely bilingual student body -- an opportunity that no other higher education institution in the country has.

The only foreign language course currently offered is a single 15-point module in Irish, and this isn't offered until 3rd year.  Now one person said that it had been raised before, and it had been asked why Welsh never found its way onto the syllabus.  Apparently the college feels it isn't similar enough to Gaelic to be worth doing.  OK, there aren't many cognates between the two languages and those that there are aren't exactly transparent, but similarity can go much deeper than mere transparency.  In fact, from an academic point of view, it's the less visible similarities that inform most.

I've been trying to improve my Welsh recently, and while doing so I've been looking for academically interesting issues, and one I found was in the adjectives.

In Gaelic, the most common adjective ending is -ACH.  In Welsh, adjectives usually end in a vowel followed by G.  Different?  Not as much as you'd think.  CH in Gaelic is a C modified by a process called "lenition" in all the literature.  Lenition is the weakening of a consonant.  There's a process of lenition in Welsh too, although it's called the "soft mutation" in Welsh learning literature.  Guess what C becomes under soft mutation...?  Yup, it's G.  So even though the end result looks markedly different, on a process level, it's identical.

Teaching in this way (in either direction)  may make the new language seem a little less arbitrary.  But more than that, it also takes a single case that covers many examples and encourages an intuitive sense of equivalences across the languages, because it teaches you to accept unconsciously that Welsh G is often Gaelic CH and vice-versa. Thinking in terms of lenition as a process may also help students cope with examples that are lenited in one language and not the other.  It opens up a framework of possibilities, and if we are open to more possibilities, we're more likely to understand new and unknown language.

But in order to do this, the teacher must understand the material at a far deeper level than they're intending to teach.  You cannot simply spend your career one lesson ahead of your students in the textbook, because you have to know more about the subject than the textbook teaches.

The teacher needs to know a fair bit about the history and science that explains the development the language, even if he isn't going to talk about history or linguistic processes in class.

To give another example, it helps to know that the O->UE and E->IE changes in Spanish (eg poder->puedo, tener->tiene) are to do with Latin long and short vowels.  Now I'm not sure, but I believe the words that undergo this change had short vowels in Latin, and the ones that never change had long vowels in Latin, but it might be the other way round.  I don't need to know for sure, because I'm not going to teach this in class.  But I understand the mechanism, which at the very least means I won't waste time looking for a non-existent pattern in the Spanish. ( Or more correctly, I've stopped looking for a non-existant pattern in the Spanish, because until I was told about Latin, everyone told me "it's irregular" and I didn't believe it could be.)

Also, while many books will present this as a feature of irregular verbs, I now know that it's no such thing -- in reality it's a fairly regular and productive phonetic feature.  If you look at morphemes that can occur in multiple positions this same change occurs, and it happens for nouns, adjectives and adverbs as well as verbs.

The nouns "puerta" and "portal" share a root, and the latter retains the monophthong O simply because it's not in the stressed syllable.
An accountant (contable) works with accounts (cuentas) and again we see the change in action.

If we teach the vowel change as an irregular verb feature, the student won't necessarily be able to make the link between puerto and portal, or contable and cuenta -- the student won't have an understanding that imitates the intuition of the native speaker.  One morpheme will end up being considered as two, and the learner will find it more difficult to learn vocabulary or to devine the meaning of new vocabulary on first encounter.

Knowing the background allows us to identify the important distinctions that we need to present to our students, and that's a very good thing.

As a teacher, I am aware that always-O and unstressed-O-stressed-UE are different phonemes which share certain allophones.  This knowledge isn't explicitly required by my students (I'm a language teacher, not a linguistics professor) but it certainly must be known by them implicitly if they are to have a natural understanding of the language.

How can we do this? 

The obvious place to start would be with a couple of common verbs.  Poder and tener are very frequent, very useful, and a traditional place to start.

We have to make it clear what phoneme the student is using, and the O and E phonemes are never unambiguous in an unstressed position, so when we teach new vocabulary, we should pick a word that places the O or E in the stressed position -- so we teach "cuenta" before "contable".  We make comment on the fact that cuent/contable share a morpheme (but we don't need the word "morpheme", because everyone's happy with the word "root") and that therefore it follows the same rule as verbs -- it's diphthongised in the tonic syllable and a monophthong elsewhere (although we can follow Michel Thomas's lead and replace the technical talk with the idea that the vowels "break under stress").

But plainly and simply, the student must have an implicit understanding of what's happening, or everything becomes arbitrary and meaningless, hence difficult to learn.

Choose your terminology carefully

People often get confused with all these fancy words in language learning and teaching.  I discussed my basic views on terminology before, but I want to look at one of the big problems that poor thought over terminology can cause: false dichotomies caused by false opposites.

For example, in the use of reading or listening, we often talk about intensive and extensive input.

• Extensive input means reading or listening to a lot.
• Intensive input is reading or listening closely and carefully, perhaps going back over material to get a lot of detail.

These things are not opposites, and on a conscious, intellectual level, most people recognise this.  Of course, they're not completely compatible either, because it's hard to read a lot if you're reading it slowly and carefully.

But once you stop thinking very carefully, most people start talking about the two things as though they were truly opposite.  Why?  Because whoever chose the terminology wanted two names that looked like a set, but inadvertantly made them look like opposites.

(Of course, this is just an extreme example of the problem of counterintuitive grammatical terminology, such as "regular" in the earlier article).

Use of liguistic terminology

(I've been busy this week and didn't have time to finish the promised article on phonology, so here's something that's been sitting in my drafts folder for a while.  It's quite relevant now as I've been using a fair bit of jargon of late.)
I've taken a bit of flak on a number of forums for my use of linguistics jargon (particularly when I get it wrong!), so I want to clarify something here: I use jargon to describe what concepts are to be taught (and sometimes how to teach them), but I do not advocate use of jargon itself with beginners, unless they are students of linguistics anyway.

The international standardisation on Latin terminology is quite useful in that I'm now able to discuss linguistics in several different languages.  It really impresses people that I can teach them grammar in their own language, but it's little more than a parlour trick.  A few regular sound changes and the appropriate suffix and your subjunctive is subjonctif or subjuntivo.  It doesn't generally get any harder than the Italians and Germans who call in a congiuntivo and Konjunktiv respectively.

Having studied a lot of grammar, I'm not only comfortable with plain conjunctions, but also with coordinating vs subordinating conjunctions so the terminology is useful to me.  (Subordinating conjunctions, see?)

The labels we give language aren't always meaning to the new learner, so don't really help.  But in the original Greek and Latin, they were designed specifically to help.  Take, for example, Latin's dative case.  "Dative" is a Latin adjective (oh look, adjective, another meaningless term!) derived from the word for "giving", and describes one of the fundamental uses of the case: indirect object as recipient or beneficiary.

"Accusative", on the other hand, comes from Greek, where it originally could mean either "for something caused" or "for the accused" (at least according to wikipedia).  The Romans picked one translation, and really chose the less useful one, "for the accused", which misleads people even to this day.  The accusitive is most commonly used for the direct object, and most "speaking" words use an indirect object for the person you're speaking to, yet the word "accusative" seems to suggest it's to do with speaking.

But not all languages use the Latin system.  Basque is a highly inflected language, and their cases are simply named by inflecting the word "who?"  Basque can do this very neatly due to it's nature, and while it isn't as neat in English, you could still name cases similarly.  Have a look at these and tell me that they are descriptive mnemonic labels:

who-did | who-done-to | where-to | where-from | who-to | who-from

True, such a description could become quite long or complex for certain languages, but it's still better than trying to remember things by such meaningless terms as alative and ablative (two words which are very easily confused -- they fit all three of my categories of confusion: similar form, similar usage, and frequent co-occurrence.

The same goes for sounds.  If you want to talk about a "bilabial unvoiced aspirated plosive", just say P.  If you trying to get a student to pronounce a "bilabial unvoiced unaspirated plosive", you just need to get the student to pronounce an "unaspirated P".  However, you don't need the word "unaspirated", but you do need the concept of aspiration.  Call it the "puffiness" of a sound, call it the "breathiness"... what you call it isn't important as long as you teach the difference.  If you want, you can even call it "aspiration", but there's no point introducing the term until after the student is relatively comfortable with the concept.  In fact, it is probably counter-productive to introduce the term "aspiration" too early, because it means something completely unrelated in colloquial English (related to goals and ambition).  Hearing a word automatically evokes its meaning, so the old meaning will interfere with learning the new concept.

So if I use terminology in this blog, it isn't my personal seal of approval on its use -- it's a concession to its current use in expert circles.  I don't think it's of practical use for beginners.

A concession to reality

On the other hand, if your students are going to be going out into the big wide world without you and are going to be relying on reference books to continue, then yes, they're going to need the terminology.  So teach it.  But look again at what I wrote about "aspiration", because again you really need to teach the concept before the word.  A word is a label for a meaningful "thing", whether a physical item, a phenomenon or just an abstract concept.  How are we supposed to learn words if we don't yet know what that "thing" is?  A word learned without meaning goes against the whole idea of language.  It's a disordered state, and once the student is in a disordered state, the teacher has lost control.

A massive change of opinion

Isn't it interesting how quickly you can change your own opinion by reasoning something through?  In the course of writing this post my own view of linguistic terminology has gone from "vehemently against" to "neutral-to-slightly-for".  I was always against it as I felt it was meaningless to the learner, but in talking about teaching the term after the concept, I realised that taught that way, it isn't meaningless at all.  I'd still prefer a more intuitive terminology, but maybe the old stuff isn't as big a problem as I thought....