Pattern identification and language learning

A few weeks ago I read an article reporting on a scientific study which had found that students who are good at abstract pattern matching tasks perform better in Hebrew language lessons. (Unfortunately I don't have access to the original journal paper.)

Now, we have two ways of interpreting this outcome: the fatalistic and the optimistic.

The fatalistic interpretation

A fatalist will say that it is proof positive of the existence of talent, and that those who do not have this talent are doomed to failure at learning languages.

The optimistic interpretation

The optimistic interpretation is to say that the successful learners are succeeding despite the teaching, and that this study is, by shedding new light on what the successful learners actually do, showing what teachers should be doing if they want to be successful.

"Rules" are a bit out of fashion in language teaching.  I've always said that this is because people have been teaching incorrect rules, rather than that rules are inherently unsuited to language teaching.  A linguistic "rule" is (or should be, at least) nothing more than an observed pattern emerging from real usage.  If successful learners are those who can identify patterns, then we must assume that after identifying these patterns, they learn them.

So I would suggest that the logical conclusion is that we should be teaching these patterns to students, rather than relying on them identifying them.

An example pattern

When I was learning French (my first foreign language) at high school, I noticed the distinction between the "long conjugations" and the "short conjugations" of the present tense (the long conjugations are the nous and vous forms, and the short conjugations are the rest), and I noticed that in irregular verbs, these forms were almost always regular.  (Is there any verb other than être that has irregular long conjugations?)

So while my classmates were attempting to memorise the irregular verb tables by rote, I was saving myself time and effort by only memorising the conjugations that didn't arise out of bog standard, regular conjugation.

Now it could be that the advocates of the discovery method are right, and that part of my success was down to the fact that I worked this out for myself, but I doubt it.  And even if that's true, is it fair to trade off the success of the majority against the success of a lucky few?  It is obvious that being told this would have reduced the effort required by my classmates to learn their irregular verbs.

Teaching patterns vs teaching rules

The problem with most of the "rules" traditionally presented in grammar books is that they are more strictly ranked and regimented than they are in real life.  Real patterns in real language can't be so neatly packaged by tense.

Take one of the patterns in Spanish taught by Michel Thomas:
The third person plural conjugation of a verb is the third person singular conjugation plus N, except in the preterite.

Now that's not how he taught it, but that's the concise description.  When he taught it, he simply used it in the present, and then got the students to apply it in the other tenses.

One of the reasons some people don't like this pattern is that "except" bit, but this really isn't a problem, because the pattern also holds for the second person singular: third singular +S, except (again) in the preterite.

It's regular, it's predictable.  Even the "exception" is regular in that it's an exception for both 2S and 3P.

This is the sort of pattern that I suspect all those successful learners are finding, and this is the sort of pattern that made Thomas such an effective teacher.

So let's find those patterns, and let's teach them.

The obligatory car analogy...

When you want to explain something complex to a non-expert, there's no tool more useful than a good analogy. Sadly, there's are few tool more open to abuse than the humble analogy, and in a great many cases, the subject of this abuse is the humble automobile. There is a rule of thumb on the internet that says you should never trust a car analogy.

The danger in analogy is that it comes to what appears to be a logical conclusion, even when the analogy is false, but thankfully we've had the conceptual tools to analyse logic since at least Ancient Greece. Heck, even the words “logic” and “analogy” come from the Ancient Greek language.

Regular readers will know that I'm not a fan of the “learning/acquisition” distinction, or the school of thought that says that rules don't matter, and that the only way to “acquire” is through exposure.  Well, recently I was reminded of that particular school-of-thought's own pet car analogy, and I would like to dismantle it here.

Grammar, they tell us, is unimportant. Do we need to know how a language works in order to speak it? Well, they say, consider a car: do you need to know how the engine works in order to drive it?

The reasoning seems persuasive to those who are predisposed to listen, but as with all analogies, the problem lies in the equivalence of the analogised items.

Is “how a language works” analogous to “how the engine works”? Certainly not – it is analogous to “how the car works”. Some commentators would suggest that the engine is how the car works – I would like to argue against this.

To a driver, a car is not the engine. From the very beginning, the goal of the engineer has been to abstract away features that the driver shouldn't have to think about and turn the engine into something of a “black box” – you read the instrumentation, manipulate the controls, and then the car responds in a consistent and predictable way based on what you tell it to do. The driver does not need to know what “RPM” means to recognise when they're over-revving the engine – revolutions-per-minute, cylinder cycles... irrelevant – but the driver does have to be told that over-revving is a bad thing, and has to learn the “rules” of reading the needle and listening to engine noise to avoid doing it.

The acquisition crowd are not, I hope, suggesting that you could put someone in a car with no knowledge of the steering system, gearbox, speed controls and indicator and just let them get on with it. The end result of this would at best nothing, at worst a seriously damaged car. OK, so you're not going to destroy someone's brain by throwing them into a language at the deep end, but if they can't even start the language's “car”, they're never going to get any useful feedback at all.

So we have three elements in the target of our analogy:

• The car as a whole
• The car's control system
• The car's engine

The question is, is the grammar the “control system” or the “engine”. Quite simply... urm... possibly maybe both....

Grammar as Control Sytem

Most of the grammar of a language is unambiguously “control system”, as the speaker must directly manipulate it in order to make himself understood.

Consider the spark-plugs in a diesel engine. Wait... a diesel engine doesn't have any spark-plugs. But this doesn't matter – this makes practically no difference to the driver. The accelerator works the same as the accelerator in a petrol engine with its spark plugs, and pressing it down harder makes the wheels spin faster. “The car”, as an entity, is operated identically – as far as the driver is concerned, it “works the same way”.

But let's look at a grammatical distinction, and for the sake of the argument I'll take the use of articles. English has them, Polish doesn't. If articles were like spark-plugs, that would mean that the article is entirely irrelevant to the manipulation of the language, but this is patently false. If you don't correctly manipulate the article, your sentence is wrong.

So a great many grammar rules are undeniably part of the control system.

Grammar as engine

Grammar as a whole has been a very expansive and extensive field of study – in fact, I'm led to believe that grammar originally meant the description of a whole language. Grammar today usually means “everything except vocabulary, pronunciation and spelling”, so a lot of stuff gets caught up in it which may be considered “engine”. Historical changes, derivational morphology (the etymology of the word presuppose is of very little use to the average learner of English) and distinctions like that between reflexive and impersonal/pronominal pronouns in the Romance languages.

But to use these few examples as a reason to throw out all conscious description of grammar is hugely short-sighted.

My car analogy

So that's their car analogy disproven, and I'd like to replace it with one of my own.

To ask someone to learn a language without grammar is like putting someone in the driving seat of a car without drumming the words “mirror, signal, manoeuvre” into their heads, and without telling them never to cross their hands at the wheel.

On the other hand, a lot of grammar-heavy teaching is like teaching someone to drive by carrying out the exact same manoeuvre 20 times in superficially different (but functionally identical) locations, then moving on and doing the same thing with a different manoeuvre.

This is not what any good driving teacher does.

A driving teacher takes the beginner to a safe, simple environment (eg an empty car park) and teaches the basic rules of operating the vehicle. The learner won't even be allowed to start the engine until they've started building the habit of checking all mirrors. Then they will learn to start and stop. A bit of controlled speed, then a bit of steering. The complexity increases steadily, and the instructor chooses increasingly complex environments so that the learner has to apply and combine the rules in ever more sophisticated ways. Rules are introduced gradually, as required, and then applied and manipulated as the situation demands. The teacher initially picks routes that don't require turning across traffic, then picks a route with a safe across-traffic turn, then adds in crossroads, traffic lights, roundabouts, filter lanes etc one by one. But these features are never treated as discrete items to learn individually – they are elements of one continuous whole that must be practised in the context of that whole.

This is what a good driving teacher does, and this is what a good language teacher does. Listen to one of Michel Thomas's courses¹ and you'll see that's exactly what he does: an increasingly complicated linguistic environment, and no language point ever treated in isolation beyond its basic introduction.  That's proper teaching, and it's all built on grammar and rules.

1I mean a course that he himself planned and delivered, not one of the courses released after his death.

Backflipping the classroom – nothing new under the sun.

Before it's sudden closure, the Fundamentals of Online Education reintroduced me to two terms I'd previously encountered in passing, but never really thought too much about (I probably wasn't actively teaching at the time, so didn't really have much of a framework of reference to evaluate them against): the flipped classroom and backwards design.

The flipped classroom is a fairly simple idea, and its theoretical merits should be immediately obvious. I believe it arose in higher education, so let's look at it in that context. Every year, a lecturer delivers the same lectures (more or less) to a room of students. Lectures are not generally highly participatory, particularly early on in degree schemes (during my 1^st and 2^nd year in Edinburgh University, we were well into the 3 figures even in my smallest lecture group). But teaching time is a precious resource, and very limited. Why are we wasting the time of some of the most intelligent people in the world by having them say the same thing year in, year out, rather than freeing them up to get extra time with the students, dealing with problems? And why do we, as students, end up doing most of our practice exercises at home, where there's no-one to help us when we go wrong or get confused?

So the goal of the flipped classroom is to overturn the orthodoxy. Let's make the lectures available as video for study beforehand, and then when students come into class, the teacher's dedicated to what they individually need.

There are several reasons that this might not be such a good thing in practice for many subjects, but that's not what I'm interested in today. No, today, I just want to show that this is not a new idea.

What I've read about the flipped classroom seems to be coming more out of science faculties than arts, which is not surprising to anyone who had friends that studied literature at uni. Us science students used to mock the arts students for their light workload, because they had fewer classes on their timetable than us, but we saw that backbreaking pile of books they were carrying and thought “there but for the grace of God go I.” A literature student may have to read a long, heavy novel every week, and they have to read it before class. Their timetable is as empty as it is because they have very few lectures, and instead have more seminars where they discuss what they've read.

The same is true for a lot of arts degrees. You may be expected to read a major treatise by one of the great thinkers before going to a Philosophy class, and if you're studying classics, you might even be expected to read it in the original!  So it is wrong to suggest it's a new idea, simply because we now attempt to apply it to science classes.

Does it matter that it's not a new idea? In and of itself, no. However, in practical terms, if you don't acknowledge that someone is already doing it, you deny yourself the opportunity to go and ask the experts how it should be done!

Anyone who wants to “flip” their classroom should instead by asking how science can be made more like the arts. They should be asking arts lecturers what works and what doesn't; what can be passed to the student to do beforehand, and what has to be kept for the classroom. They should be auditing arts courses and experiencing for themselves the phenomenon they wish to replicate.

The other idea was backwards design.

Backwards design is the idea of starting by setting out what you want the students to know at the end of the course; then by deciding how you will verify how they have learned it; and finally you work out how to teach it.

For this to be presented as new or in anyway unusual is pretty hard to swallow, because people do this all the time, it's the absolute norm in schooling. A national committee writes a national curriculum. The exam board plans an exam format to test the criteria set out in the curriculum. Finally, the teachers and textbook writers are given the curriculum and sample exams and write their lessons.

Now, the traditional line is that teachers should be teaching to the syllabus, not the exam, but in reality, most teachers know that the exam is the primary goal for the students and they do indeed “teach to test”.

I said presenting it as something new was hard to swallow, but in fact I actually found it more frightening than anything. Were there people who weren't actually doing this?!?

...and then I realised: there are, and as a language teacher, I'm one of them.

It's been a source of frustration to me ever since I got into languages almost a decade ago that language teaching seems to have institutionally rejected the notion of a “syllabus”. There is no list of what a student should know at any level. We're asked to “learn/teach the language” rather than “learning/teaching the test”.

It's a laudable goal, but it leaves the learner or teacher, and particularly the self-teacher, in a rather bewildering forest of choices. Where to start? What next? Can I afford the time to cover this language point properly, or do I need to make do with an incomplete understanding and move on to something else?

For a long time I was convinced, though, that Cambridge (for example) had to have some kind of syllabus internally; a list of words, expressions and language points that examiners are allowed to include at every level, but now I'm beginning to wonder. Do they give their examiners the same advice they give us, the mere teachers that only have to prepare the students to sit and pass an exam based on unpredictable language: to use their “judgement” to pick something “appropriate to their level”?

Because to be blunt, institutions like Cambridge are completely failing in their goals. A responsible teacher will always “teach the test”, and if you don't give us the language we need to teach, then we have no choice but to devote more of our time to exam techniques, and we end up spending less time teaching language.

So I'm very much in favour of the goals of backwards design, but I'm worried that by naming it and treating it as something new and different, it will come up against resistance to “change”, even though it is not, in fact, real “change” – it's a defence of the longest standing traditions in education against a combination of flawed teaching ideologies and sloppy practice.

So these philosophies have created two obstacles for themselves by pretending to be new: they discount all the existing evidence, and they turn off people who might otherwise be convinced by the past experience of their colleagues.

What do we learn when we learn by doing?

The University of Georgia MOOC on online education was starting to look very interesting before it suddenly folded. On one hand, it covered a lot of interesting theoretical pedagogy. On the other, the practical pedagogy of the course itself seemed sub-optimal. As I only got one week in before they closed up, so I can't really say all that much about it.

One academic the course has introduced me to is Roger C. Schank. Roger is an AI lecturer who later specialising in learning. Schank's big idea is that of learning by doing. It's a simple and compelling idea – he claims traditional classrooms don't work because they are far too theoretical and divorced from any real “need” to learn.

There is certainly a lot of truth in this. A book of drills (whether it be arithmetic, grammar, or rote historical facts) does little to demonstrate either why the information is important or the contexts to which it is relevant.

The title of this post is lifted straight from a report Schank wrote in 1995 for his university. It's a huge piece of writing – almost thirty thousand words long – and to be perfectly honest with you, I didn't read it to the end. But why would I? It's called a “technical report”, but in truth it's little more than an oversized opinion piece. There's no technical data: he does not appeal to research, he does not appeal to figures, he just makes unsupported statements. What is most telling is that there are only 5 citations in his bibliography, and four of these are to himself.

As he argues without evidence, I feel perfectly entitled to dismantle his argument without citations. Besides, he's supposed to be an academic, and I'm just a blogger!

First up, Schank fails to address in the first 25% of his essay (which is approximately what I read) the biggest concern that has always been raised against the idea of whole-subject-learning, learn-by-doing or whatever label you chose to attach to the idea: lack of breadth. (A quick skim suggests that that it's not addressed further down, so if it is, it's clearly not given the prominence it deserves. Besides, as it is the single most important concern of most critics, you need to address it early or you lose our attention.) A single task will only lead to a single solution, with some limited exploration of alternative strategies. It teaches the students how to cope with a situation where they lack knowledge, rather than minimising those situations by providing them with knowledge.

There's been research into various aspects of this. I've seen reports claiming to prove schoolchildren given a whole-subject education have a much narrower set of knowledge. This should be pretty obvious, I would have thought... so maybe I'm just displaying confirmation bias and reading the stuff that supports my view.

Of course, there was the study that showed that doctors trained by case-study rather than theory performed better in diagnosing patients, but as I recall it, this was tempered by the fact that their diagnoses took a lot of time and money, because they tended to follow a systematic approach of testing for increasingly less common problems or symptoms. A doctor trained on a theory-based course was more likely to formulate a reasonable first hypothesis and start the testing process somewhere in the middle. The conclusions we can take from this are mixed. You can claim that the traditionally-trained doctor is better at diagnosing on the grounds that he can do it with less testing; or you can claim that only the end result matters, and the case-study-trained doctor is better. You can argue that minimising mistakes is the ultimate goal, or you can argue that the time taken in avoiding mistakes is too great in that it delays treatment for other patients.

So, anyway... Schank does nothing to convince me that it is possible to cover the breadth of a traditional course in learn-by-doing, but there is a video on Schank's site of a course he designed at Carnegie-Mellon about ten years ago, and it alludes to what I think is the only real argument about it. One of the senior course staff and one or two of the students talk about the idea of forgetting everything that's been taught in a traditional course. If challenged, would that be the basis of Schank's response? The logic certainly appeals: if you're not really learning anything in a traditional academic course, the breadth of what is covered is irrelevant.

But is anything ever truly forgotten? I've recently gone back into coding after a long hiatus – even when I worked in IT, I never had any serious programming to do. But when I come up against a problem, I find myself thinking back to mostly-forgotten bits of theory from my CS degree days, and looking them up on the net. Tree-traversal algorithms, curried functions, delayed evaluation... But if I had never encountered these ideas before, how would I even know to look for them?

This is not a mere theoretical problem. I'm not usually one to complain about “ivory tower academics”, but goddamn it, Schank's brought this on himself. And I quote:

"One of the places where real life learning takes place is in the workplace, "on the job." The reason for this seems simple enough. Humans are natural learners. They learn from everything they do. When they watch television, they learn about the day's events. When they take a trip, they learn about how to get where they are going and what it is like to be there. This constant learning also takes place as one works. If you want an employee to learn his job, then, it stands to reason that the best way is to simply let him do his job. Motivation is not a problem in such situations since employees know that if they don't learn to do their job well, they won't keep it for long.

Most employees are interested in learning to their jobs better. One reason for this is, of course, potential monetary rewards. But the real reason is much deeper than that. If you do something often enough, you get better at it -- simple and obvious. When people really care about what they are doing, they may even learn how to do their jobs better than anyone had hoped. They themselves wonder how to improve their own performance. They innovate. Since mistakes are often quite jarring to someone who cares about what they are doing, people naturally work hard to avoid them. No one likes to fail. It is basic to human nature to try to do better and this means attempting to explain one's failures well enough so that they can be remedied. This self-correcting behavior can only take place when one has been made aware of one's mistakes and when one cares enough to improve. If an employee understands and believes that an error has been made, he will work hard to correct it, and will want to be trained to do better, if proper rewards are in place for a job well done. "

Many graduates, particularly CS grads, will immediately be able to identify with me when I say this isn't true. When you leave university, you typically know how to do things correctly, but once you get into the real world, “correctly” is too time-consuming, too expensive. (Except in safety-critical roles, such as avionics or military systems.) In the short term, this is OK – pragmatically, that's the way it's got to be.

In the longer term, though, things start to go haywire. We get so habituated to our way of doing things that we soon learn to identify it as the “right way” of doing things. Someone comes along with a new way, a better idea (probably a recent grad) and we dismiss the idea as wishful thinking.

In computers more than any other field, this problem is easily apparent. I remember suggesting a very simple change to a database system and being told “you can't do that with computers”. A) You can do pretty much anything with computers. B) I was talking about something that is built into the database software we were using!!! Yes, a standard feature of the software, and the team's top “expert” didn't know about it; and because he was the expert and didn't know about it, it was as though it didn't exist.

More generally, the problem becomes visible when a programmer switches languages. A C programmer who learns Python normally ends up writing his Python code using the features that most resemble those of C. The unique features of Python are designed to overcome specific problems and difficulties with a lower-level language such as C, but to the expert C coder, these aren't really “problems”, because he long ago rationalised them away and learned to cope with them. He doesn't realise there is a “problem”, so he doesn't have any reason to go looking for a solution. Even within languages, some people always do things the “hard way” because they've simply never thought to look for an “easy way”.

So Schank is being hopelessly naïve. The key feature of expertise is automaticity – experts have internalised enough that they don't have to think about what they're doing. They close their minds to learning, because there's more value in doing a job suboptimally but quickly than in doing it optimally but slowly. People need to develop breadth before they become experts – before they become “set in their ways”.

Now to answer the question that started this article, what do we learn when we learn-by-doing?

We learn to be adequate, not brilliant. We learn to get by, not to excel. We learn to stop thinking. We learn, paradoxically, to stop learning.