Guardians of Grammar

I came across a link before Christmas that I found quite interesting.  It was a Guardian guest article by an applied linguistics lecturer, Dr Catherine Walter.  The headline was bold and clear: "Time to stop avoiding grammar rules", so of course I was interested, being firmly in the pro-grammar camp myself.  The subhead went for the jugular: "The evidence is now in: the explicit teaching of grammar rules leads to better learning"

Excellent, I thought, everything I've ever said has been vindicated.  Does this mean the tide will now turn in the teaching world?  Sadly not.

The article was something of a disappointment.  Despite its mention of meta-analyses, it provided precious little evidence and instead went down the road of discussion views and hypotheses... views and hypotheses which I personally agree with, but do nothing to convince those currently holding the opposite view.

That said, I still find it difficult to see how this sort of logic fails to convince people:

most English language learning takes place in countries where English is not the predominant language: a foreign language situation. Much of the thinking leading to strictures against grammar teaching has taken place in countries where English is the predominant language: a second language situation. The enormous difference in exposure to the target language makes arguments based on exposure or emergence much less plausible in the foreign language situation.

ie the leading lights of the ESOL/EFL world are working in an atypical environment (and of course Walter one of them) -- their students have plenty of opportunities for practice outside of class.  Why are so many teachers so quick to accept the pronouncements of people whose teaching environment and student base is so radically different from theirs?

(This argument alone goes beyond the article's boundaries of exposure vs training and drills into the fundamental identity crisis of the Communicative Approach: in a class with a shared native language, isn't English inherently a barrier to communication, not a facilitator of it?  Surely the communicative imperative is broken if the language gets in the way?)

She also talks about "chunking" -- a central pillar of the lexical approach.  She says:

But the best estimate is that there are hundreds of thousands of chunks in English; learning enough of these to have an appropriate chunk to hand in a given situation is not a quick or trivial job. With much less time and effort, learners can acquire grammar for putting together comprehensible phrases and sentences that can serve them on the long journey towards more native-like proficiency.

...which is perfectly correct.  If a chunk is composed of regular grammatical features, then the chunk can be understood as a construction until the learner has seen it enough times to identify it as a chunk.  Learning some of the underlying grammar rules of a language makes the learner capable of dealing with a lot larger a subset of the language than learning the same number of chunks.

But Walter doesn't go far enough, because she doesn't point out the inconsistency of those who espouse both "learn by induction" and "learn by chunks".  Learning by induction is supposed to be the "natural" way -- it's how babies do it.  But it is readily demonstrable that babies learn grammar before they learn chunks -- they cannot repeat a large chunk until and unless they have internalised the component language.  OK, you will find babies and older children who have incorrectly generalised two frequently co-occurring words as units, but even if you consider that chunking (the alternative interpretation is that they've mistakenly identified the two words as being single ones), the vast majority of early language appears to be unchunked.

If the justifications and theoretical underpinnings of the approach are inconsistent, why have any faith in it?

Just what is applied Linguistics?
A lot of teachers will be likely to reject Walter's views because she's an academic, a linguist.  Sadly, the term "applied linguistics" is somewhat opaque, because applied linguistics could involve so many things -- from forensic linguistics ("was this confession written by the accused?") right through to producing better instruction manuals for washing machines.  But no, applied linguistics is used almost exclusively to mean one thing and one thing only: language teaching.  Most "applied linguists" are genuine, honest-to-goodness teachers, but ones that do research as well (many universities make a lot of money by running summer EFL courses out of their Applied Linguistics departments).

Applied linguistics as a field is so far from ivory towers that one author named his textbook "An Introduction to Applied Linguistics: from Practice to Theory" (Alan Davies, Edinburgh University Press), in order to highlight that all language teaching theory starts in the classroom, and all theory is an attempt to describe and understand successful teaching.

So Walter isn't an outsider -- she isn't just someone who doesn't understand the reality of teaching, which is one of the criticisms most often aimed at academics who attempt to give advice on how to teach.  It didn't pop up in the comments thread explicitly, but there was a rumbling...  One commenter, Espoolainen, noted the lack of "chalkface examples".

The collective noun for "anecdotes" is "an internet"
We all know that the plural of "anecdote" is not "data", but yet anecdote is what the internet thrives on.  We don't want figures, we don't want proof, we want one or two stories with real protagonists.  It doesn't matter to many of the commenters on the article that the uncontrolled variables in a single example make it meaningless -- that's what people want to hear, but they don't call it "anecdote", they call it "chalkface examples" and kid themselves on that it's not really anecdote.  (On the other hand, I'd bet good money that if Walter had given any genuine examples, 101 commenters would have jumped on her for using anecdotes in her article.)

One of the other commenters linked to a paper by the National Research and Development Centre for adult literacy and numeracy(NRDC) which was proclaimed by the very next commenter as "more useful than the original article".

The report is full of "chalkface" examples.  One chapter is called "take 40 teachers: ESOL teachers' working lives", and the next "take 40 classrooms: teaching and learning strategies in the classrooms observed", followed by "Telling cases: ten classroom case studies".

That's a plurality of anecdotes, isn't it?

Meta-analyses and literature reviews
The problem with arguing any case in terms of figures, evidence or science, is that for every published paper you can cite to support your case, your opponent will be able to cite one that states exactly the opposite.

Good academic practice relies on a thorough and complete overview of all available data, which is nigh-on impossible for any one person to do, and a humungous waste of time for everyone to do.  This is where meta-analyses and literature reviews come in.  Someone (more often a group of people) sits down and sifts through all the published papers they can get their hands on and try to work out what they all mean when taken together.

This means eliminating studies with unreliable methodologies, checking whether the conclusions were truly warranted and balancing the volume of evidence on both sides.

This is the sort of material that Walter claims backs her up, and when pressed she quoted the following sources:
Norris, J. M. & L. Ortega. 2000. Effectiveness of L2 instruction: a research synthesis and quantitative meta-analysis. Language Learning 50/3: 417-528.
Gass, S. & L. Selinker. 2008. Second Language Acquisition: an Introductory Course (Third Edition). New York: Routledge/Taylor.
Spada, N. & Y. Tomita. 2010. Interactions between type of instruction and type of
language feature: a meta-analysis. Language Learning 60/2: 1-46.
Spada, N. & P. M. Lightbown. 2008. Form-focused instruction: isolated or integrated?
TESOL Quarterly 42: 181-207

Now let's look at the NRDC study.  In the introduction they say:

The five NRDC Effective Practice Studies explore teaching and learning in reading, writing, numeracy, ESOL and ICT, and they set out to answer two questions:

1. How can teaching, learning and assessing literacy, numeracy, ESOL and ICT be improved?
2. Which factors contribute to successful learning?

Even before NRDC was set up it was apparent from reviews of the field (Brooks et al, 2001; Kruidenier, 2002) that there was little reliable research-based evidence to answer these questions. Various NRDC reviews showed that progress in amassing such evidence, though welcome where it was occurring, was slow (Coben et al, 2003; Barton and Pitt, 2003; Torgerson et al, 2003, 2004, 2005). Four preliminary studies on reading, writing, ESOL and ICT were undertaken between 2002 and 2004 (Besser et al, 2004; Kelly et al, 2004; Roberts et al, 2004; Mellar et al, 2004). However, we recognised the urgent need to build on these in order greatly to increase the research base for the practice of teaching these subjects.

So what we've got is a group saying that they were essentially starting from scratch; that they were incapable of performing a meta-analysis with the existing research.  That makes this a single, solitary study, with 500 students (not a lot if you're building a national strategy on it).  And there's absolutely no mention of Norris and Ortega's meta-analysis, either to rule it in or rule it out of their work.  There are two possible interpretations of its absence: either they didn't know about it or they didn't like its conclusions.  If it's the latter, in my book that's just dishonesty.  If you disagree with something, don't ignore it: explain why you are disregarding it, so that readers can judge for themselves.

The NRDC report is pretty much what Walter complained of in the article:

Each approach has been defended with carefully structured arguments, and some approaches have been embraced enthusiastically by ministries of education around the world.

Arguments, not evidence, define policy.  That's what she's saying, and it's all too often true -- just Google the term "policy-based evidence"!

Taking off the blinkers of English
Dr Walter came so close to making an important point when she said:

There is a problem with English: it is a morphologically light language. It doesn't have many different verb endings, and its nouns only inflect for plural. If the language under discussion were Polish, with its three noun genders and seven cases, the idea that teaching grammar rules wasn't necessary would probably not even occur.

Why did she not expand on this?  I came across a masters thesis on the net a while ago on the teaching of Finnish and it took as a given that conscious grammar study was required, because decades of figures showed conclusively that immigrants really never quite "got" the language without a bit of explanation.

The problem with Finnish is often claimed to be its complexity, but the author of the thesis was more interested in the difference between Finnish and the immigrants' native languages.  Just look at the very first sentence of the abstract:

Research has shown that explicit instruction with a focus on forms is needed in learning a very different language.

Difference: that's what we should be looking at.

Consider that one of France's top-selling language products is Assimil, a course that makes a big deal out of learning by assimilation.  We'll leave aside the fact that there's still a fair bit of explicit grammar explanation in the books and instead look at the situation in terms of differences.  French is anything but a language isolate, being part of a broad continuum of Romance languages.  When you consider that Italian and Spanish are two of the most popular languages for study in France, it's easy to argue that Assimil got its reputation by teaching languages with a very low "difference" from the learner's native tongue.  Even German, the other big neighbour, isn't too different from French in many respects.  (French may not mark case in nouns, but its pronoun system still makes a distinction between subject, direct object, indirect object and possessive that maps closely enough to German's nominative, accusative, dative and genitive to give the learner a head-start.)

ESOL has justified itself generally by dismissing any learner errors as unimportant, but it can be readily seen that the most common errors are caused by fundamental differences between languages, and the most obvious example is the use of articles, because most learners have some kind of problem with them.  Speakers of article-less languages either leave them out or add them in an arbitrary and meaningless way (NB: arbitrary, not random: most are consistent in their misuse) and speakers of Romance languages tend not to be able to chose between an indefinite article and the number one.


Even if English isn't very complex, it is still different in several fundamental ways from any given language, and if that's a problem for Finnish, it must be a problem for us too...


...and it is, because these problems never go away.  There's no magic tipping point where the Spanish speaker has had enough exposure to English to tell the difference between "a car" and "one car", or where a Polish speaker suddenly can distinguish between "cars" and "the cars".  It just doesn't happen.

Walter's cardinal sin
But Walter's biggest mistake was trying to make it sound as though her conclusions were based on some piece of brand-new research.  It grabs the attention, but it undermined her argument when she got called on it.

Why not simply point out that the evidence has been available for years to those who were willing to listen?

I know it's a hard argument to sell, but it's the only intellectually honest one.  Giving an explanation of why orthodoxy is so slow to change won't necessarily have an immediate effect, but in the long term it should open peoples critical faculties.

Why orthodoxy is so slow to change
The human lifecycle is a fairly predictable thing.  We are born. We go to school.  If we do well at school, we'll probably end up in university.  After university, we walk out into the world looking to do things differently and better than those who came before us.  But nobody listens to us and we end up as the least important employee in our companies.

Over the next decade or two, the genuinely gifted start to rise to prominence in their careers.  It's only now that they can start to impose "their way" on the world, but their information is over a decade out of date by this point.  Their view of the mainstream is in reality a river that ran its course a long time past.

This leads them to reject new research that supports the old orthodoxy that they have already "proven" wrong in their heads.

Would any of us in our 30s or 40s pass our university exams if we went back now?  Probably not -- our information is hopelessly dated (unless you're a mathematician, perhaps).  So who are we to tell others in our field what to do?

Don't they remember what it was like to go into that first job and have everything you knew about your field rubbished by people who just didn't understand the new stuff?  Why do we visit this same humiliation upon the next generation?