Online education's elephant in the room.

It's funny how things come together to give you a better understanding of your own mind. A couple of weeks ago I got caught up the internet debate on mass-participation online education started by an American stats professor critiquing Udacity'sIntroduction to Statistics by Sebastian Thrun.  Then the other day I started debating online education again, this time triggered by the Technology Review article The Crisis in Higher Education linked and debated on Slashdot. One thing I didn't mention in the first debate, but did in the second, was something that has been bugging me for a very long time, and it's really only thanks to the recent debates I've been having with Owen Richardson on DI that I was finally able to articulate it.

These massive courses claim the potential to be better than anything that's come before, thanks to the availability of masses of automatically-collected feedback that will be used to improve them. This, theoretically, means the fastest pace of change in the history of education.

But is that really the case in practical terms?

Right now, I'm at the steepest part of the learning curve with respect to the courses I'm delivering at the university. I can't write more than one full lesson plan at a time, as in each new lesson I receive crucial feedback on what my students are capable of. So I'm constantly revising my material.

My father, during his career as a Chemistry teacher, delivered the same course year after year to classes of no more than 20 pupils at a time. Every time he taught a lesson, though, he was looking for improvements and refinements based on the reaction of the class. If someone made a mistake, he'd try to change the teaching to remove the possibility of someone in the next class making the same mistake.

So in the case of a conscientious teacher, material is revised for every 20 students taking the course.

 

Sebastian Thrun's first sitting of the Artificial Intelligence course had 160,000 pupils. OK, only 14% completed the course, but 22,400 students is still an incredibly high number. That's 1120 iterations of a class for me or my Dad. We're talking about numerous lifetimes of teaching. For a course taught once a year, it's equivalent to going back to the first millenium AD, not only before the computer, but before algebra, cartesian geometry and even the adoption of the Hindi-Arabic number system in Europe.  So we're talking about "A.D. DCCCXCII", not "892 AD".

A millenium's worth of teaching, with no improvement – I think that qualifies as the slowest rate of change in education ever, rather than the fastest.

Worse than that, while Thrun complains that his contemporaries are simply throwing existing courses onto the net without making them truly match the new paradigm, these are at least courses that have a fair amount of real-world testing behind them.  By contrast, his attempt at completely new means that he has giving a course to over 20,000 students without having tested it even once (as far as I can see). That's... worrying.

So what's the source of the problem?

The problem as I see it has two root causes: the medium and (as always) money.

The medium.

The current trend to massive online courses is a development of MIT's OpenCoursware initiative. Essentially, MIT videoed a bunch of lectures and stuck them online with various course notes, exercise sheets and textbook references. I know a few people who got a lot out of one or two courses, but often the quality was bitty, with incomplete materials (due to copyright or logistical reasons) and little motivation to complete.

The early pioneers of the current wave saw a major part of the problem as being in the one-hour lecture format, and revised it to a “micro-lecture” format, delivering short pieces to camera, interspersed with frequent concept-checking and small tasks.

But however small the lecture, it is still fundamentally the same thing, with a live human writing examples on some kind of board, and any revision means the human going back to the board and writing it out again, and giving the explanations again. The presented material cannot be manipulated automatically, so the potential for rapid revision and correction is reduced.

Money.

Revising a course manually takes time, and time is money. Squeezing several lifetimes' worth of improvements into a rapid development cycle isn't a part-time job – it's probably more than a full-time job, yet in the brave new world of online education, this is nobody's day job. Most of the course designers are still teaching and researching, and Thrun himself is still doing research while working at one of the world's biggest tech companies and trying to start up a new company.

No-one's yet really worked out the way to cash in on these developments, so no-one's investing properly.

Here in the UK, online education (on a smaller scale) is already on the increase, but mostly as a cost-cutting measure. That's fine as a long term goal, but in the short-term there is a need for massive investment in order to get things right.

What are we left with?

Not a lot, frankly. Data-mining requires a widely-varying dataset, in order to allow the computer to detect patterns that are too subtle or on too large a scale for a human to be able to pick up independently. But the data collected on these online programmes is pretty much one-dimensional. There are no variables explored in the teaching – there is one course, so the feedback can say if something is difficult or easy (based on number of correct answers and time taken to answer) – it can't tell us why, and it can't tell us what would be better. That means that the feedback from 22,400 students is less valuable to a good teacher than one question from an average student during an average class. That's.... worrying.

So much for the revolution.

So what's the solution?

If there's two parts to the problem, there must be two parts to the solution.

Medium

The Open University has, over the years, moved away from lectures to producing TV quality documentaries that use the best practices of documentary TV to present material in a way that genuinely enlightens the viewer.

 

As a documentary isn't a single continuous lecturer, it would theoretically be possible to have a computer modify and re-edit a documentary to make it easier to understand.

 

On the most basic level, a difficult concept might be made easier by inserting an extra second of thinking time at a certain point in the video -- an algorithm would be able to test this dynamically.  Conversely, the algorithm might find that reducing the pause is more effective, and do so dynamically (we assume then that the concept is easy and that extra time allows the student to become distracted).

 

Then there's the slides and virtual whiteboards used in the videos themselves -- produced in real-time as the presenter speaks.  This splits the presenter's attention, often resulting in rushed, unclear writing, or pauses and hesitations in speech.  Revising the visuals means redoing the whole video.

 

Why doesn't the computer build the visuals to the presenters specification, but with the ability to modify them to optimise to student feedback?

 

Eventually, we would get to the point where a course definition is a series over voice-over fragments and descriptions of intended visuals, and the computer decides what to put where.

 

But the reason that'll never happen in the current model is reason 2:

Money

Where there is a genuine incentive to drive down the cost of education, there on-line education will find its most fertile ground. When you look at the tuition fees in places like Stanford, Harvard and MIT, you'll see that these aren't the schools with the biggest incentive to make online education work.

Instead, we need to look to Europe, and in particular the countries with significant public funding for higher education. Universities funded by the public purse are under intense pressure to cut costs – it's the only way to balance the books in a shrinking economy.

However, the universities alone can't make this happen, as the current pressure is for cost savings NOW, and so they're producing online programmes with insufficient research and the quality of education is suffering for it.

Governments are sacrificing students to the God of Market Forces, when they should instead be planning intelligently. Instead of cutting funding to force universities to be more economical, they should be investing to make universities more economical. Give universities money now in order to produce high-quality programmes that will reduce costs for years to come.

 

But It Will Not Be Cheap – quite the opposite.  The creation of a genuinely high-quality online course is phenomenally expensive in terms of up-front costs, while being ridiculously cheap in the long term.

The current clientele of Udacity, edX and Coursera will no doubt feel cheated that I'm talking about education for the classic “student” rather than the free “everyman” approach of Coursera et al, but there's no need to. Established, well-researched, properly tested and adequately trialled online courses may take a while to perfect, but once they exist, their running costs will be so low that they will surely be made widely available. And while they're being developed, they're going to need a constant source of beta testers, and that's going to mean people who're doing it for personal interest, not for grades – ie you. The end result will still be open education, but it will be better.

Sometimes fate brings things together in a way that helps illustrate something you've been trying to explain.  But it's not normally fate -- there's usually an intrinsic link.  This time, is was the How-To-Learn-Any-Language forum, which I've been looking in on recently as a result of getting increased traffic from there to this blog.

There was a link there to an interesting project on Kickstarter -- Endangered Alphabets II.  Now I'm pretty broadly in favour of that project, although there's a slight irony in bending the scripts to his chosen medium of wood carving; after all, the diversity of scripts in India is mostly down to the availability of local materials.  Most Indian scripts evolved out of the Devanagari, changing over the years.  Where they wrote on banana leaves, the script was curved (as straight lines would split the leaf).  Others used ink on animal hide, others a clay tablet and stylus, etc.  As a travelling exhibition, perhaps the natural setting would be better, but there's no denying that what he's produced so far is quite beautiful, and the educational side of the project is very much worthwhile.

But any time I look at a project on Kickstarter, I always have a nose around for what else is on offer, and I particularly tend to look at languages.

There's not a lot on the go at the moment, but one project that's seeking funding is called The Simplest Teach Yourself Spanish Textbook, aiming to raise $8,800 (US) to retypeset, illustrate and generally prettify a little textbook that a teacher has produced and uses with her own students.  She tells us in her video that

I use it with all my students, and they tell me that thanks to the simple explanations, things finally make sense.

There's two problems here:

1. "things finally make sense" -- this implies that we're not talking about true beginners.  That a book is good enough for a false beginner does not imply that it is adequate for a genuine ab initio new learner.  I can't stress this enough -- "I learned more in 5 days with X than in 5 years of traditional classes is a claim that's really hard to prove".
2. "I use it with all my students" -- therefore we really have no proof of the effectiveness of the book, but merely claimed testimony of Kristen's abilities as a teacher.

IE. there has been no genuine testing of this book.  I was having a discussion on the comments thread on my post Michel Thomas vs Direct Instruction, and I suggested that it's pretty difficult to get people to buy into the full DI philosophy, so the most important point to get across is the importance of trialling the material.  In the case of Kirsten's book, the important thing would be to get it tested with other teachers, not her.  Are the explanations really that clear in the book, or is it something she says when presenting the material in class.  After all, if the explanations were really that clear, why would any teacher be needed?

Furthermore, she wants to spend the money on professional layout and illustrating to make it "even more engaging".  But I ask: what is the pedagogical value of the illustrations?  I imagine the usual response would be to make it more attractive, therefore engaging.  The pedagogical value, I said.  And pseudoscientifically, the answer is "to lower the affective filter".  I say "pseudoscientifically", because it is a claimed based more on belief than evidence.

There has been a constant drive to redesign and reorder books to match the expectations of the learner.  This has led to a culture of boxouts, callouts, side bars and other fancy panels.  Essentially, many textbook writers started to try to mimic the layout of magazines, "because that's what people read".  However, any good magazine designer will tell you that the function of magazines is fundamentally difficult from the function of a textbook.  No-one reads a magazine front-to-back -- they flick through and read fragments of articles before choosing what to read.

This is precisely what you do not want when studying a language.  I've used magazine-layout books with school age kids, and it's very difficult to keep their eyes on the right part of the page.  This book also made several errors in trying to select material that would be "engaging" to students, with exercises that the kids themselves derided as pointless (and justifiably so -- there was a picture of a pair of twins, and the students were to discuss which one of them was called Such-and-such, and which one was called Something-or-other, and justify their choice) and songs that they didn't like (a poor-quality Michael Stipe soundalike singing "Everybody Hurts" painfully off-key was once dismissed as "emo" by a metal-loving teenager).  The pedagogical value of these exercises was limited and the failure to engage student attention would have been self-evident if they trialled it under real-life conditions.

Going back to Kristen's textbook (Spanish is Your Amigo), there is no mention of any revisions due to student feedback since her first Kickstarter project got the early version.  I'm sure there will have been things that needed extra work or explanation, but she's presumably simply answered them adequately in class such that it's not a problem... for her students.  But for anyone else using the book, it will be.  The project does not mention a teacher's manual, which might point out common student problems or suggestions for activities that tie in with the course content.  (It's a poor substitute for making a course that anticipates these problems itself, but it would be a start.)

And that was what I was trying to get across in my comment the other day.  You cannot change someone's view of the fundamentals of teaching overnight, but if you can convince someone to adopt a rigorous testing methodology, you've made progress.

The learning experience for the writer doesn't end when they start doing proper testing, because of course brute-force trial-and-error doesn't work, and the sooner people start testing, the sooner they'll realise they need to think more carefully about planning and design.  And then they'll start looking for information on effective methodologies.

Laziness

Last night I sat in on an Italian conversation group.  My Italian is really rusty, and I'm rubbish with irregular verbs.  Part of me was silently screaming inside about it being "good enough", and I'm now worried I've hit a point where it's going to be difficult to motivate myself to study.  I can understand a heck of a lot on TV and radio, and I can make myself understood, so why should I put the effort in to do it right?

And yet, if I'm stepping into someone else's learning experience, I have a duty not to spoil it for them, don't I?  If I'm throwing out weird Italiañol, it's hardly going to help them learn.

I suppose I now understand a bit better why all the foreigners I met in Edinburgh stopped improving, and if I can genuinely motivate myself to learn Italian properly, maybe I'll work out how to help all those people too.

But one thing's for certain -- effort's not the answer.  If it feels like hard work, I'm not going to be able to force myself to do it.  Well, maybe I can, but what's the value if I do?  I learn Italian, but I learn nothing about how to help those who just can't see the point in making the effort.

Michel Thomas vs Direct Instruction

So I popped in to check for new comments the other day and there were none, but there was a spike in reader numbers.  Checking the stats, I spotted that most were views of the post Talent Schmalent, so I had a look at the traffic sources and found it was coming from a post on the How-To-Learn-Any-Language.com forum about Michel Thomas and Engelmann's Direct Instruction.  I'd exchanged a few emails with Owen Richardson after he commented on my blog, but at the time I was finishing a job at the Ionad Chaluim Chille Ìle, on the beautiful island of Islay, and preparing to travel a thousand miles across the continent to the equally beautiful island of Corsica, where I'm now teaching.

It's a shame that he sent me links to the scans of the preface to the book (also in the thread), because it was actually massively off-putting.  The whole thing is just too passionately enthusiastic, with the air of having been written by a cult disciple rather than a rational intellectual.  So I read it, and didn't read anything more.

Fortunately, when I saw the thread on HTLAL, I decided this time to read the online module on Direct Instruction from Athabasca University.  It's a third party item, so it's pretty neutral.  It doesn't therefore tell us how great and wonderful it is, but dispassionately tells us that it was empirically more successful than other methods in independently monitored trials.  And it even makes clear that this was a study of complete methods, so wasn't tracking independent variables.  So the conclusion of the study was that Direct Instruction was the best of the tested methods, not "the best method ever".  Having read that, I was more open to reading further on the subject.

However, it did remind me why I hadn't looked into Direct Instruction further after reading about it in Jonathan Solity's book on the Michel Thomas Method -- because it's really fundamentally rather different.  Solity's book diverged far too far from the actual Thomas method when he started talking about exemplars and non-exemplars in the DI manner.  The example he gave was of "over" vs "not over" with a ball or a table.  Thomas rarely, if ever, used examplars and non-exemplars.

To use examplars and non-exemplars in a language course would be far too abstract, and you would be back to learning about the language rather than learning the language.

Many professional Spanish teachers, on first hearing the MT Spanish course, would probably be horrified that he taught "es" as "it is", with no explanation of the difference between "ser" (to be for permanent characteristics; I am Scottish) and "estar"(to be for temporary conditions; I am tired).  He taught purely by exemplar, and not by non-exemplar.

How would we teach "ser" and "not ser"?  Would it be like this?

• I am a teacher.  Ser or not ser?
• I am tired.  Ser or not ser?
• I am hungry.  Ser or not ser?

Because Direct Instruction calls for the widest possible variation in exemplars and non-exemplars, I cannot see any way of doing it while actually manipulating the language itself, as this would involve introducing far too much new information in one go.

Solity's justification for claiming Thomas's taught by examplar and non-exemplar was weak, because after introducing one thing, several hours later he would teach a different (but related) concept by exemplar, and then contrast the two.  That isn't a non-exemplar, as we are never asked to define it by what it's not, only by what it is.  What Thomas does here is far better defined as "integrative reconciliation", a term defined by the late David Ausubel, a pivotal figure in the development educational psychology and cognitive science after the behaviorist* years.  (I've written a bit about him before on several different occasions.)

Ausubel talked about what he call "reception learning", where the information was given, as opposed the better-known "discovery learning" proposed by one of his contemporaries, Jerome Bruner.  He argued that given information was not necessarily rote, and that discovered information was not necessarily meaningful, and I would personally agree with that.

Two of the key items in meaningful learning, he suggested, were "progressive differentiation" (the studying of a concept initially at a simple level then increasingly breaking down the concept into more and more complex subdivisions) and "integrative reconciliation", by which he means constantly comparing and contrasting new concepts to previously-learned ones to remove any ambiguity or confusion.

So when Thomas eventually does compare ser and estar, he's reconciling two potentially conflicting pieces of information with each other -- ie "es is he is" and "está is he is".  That's integrative reconciliation, not exemplars and non-exemplars.

Perhaps language teaching could be done better if it follow the principles of DI, but I can't see how.  After all, Thomas's teaching-by-exemplar-only works extremely well, because the prompts he uses are (mostly) individually unambiguous.  Notice that he doesn't constantly ask what "it is" is in Spanish (that's not unambiguous) but keeps asking for "it's possible", "it's improbable" etc.  The exemplars and exercises don't seem to provide much opportunity for overgeneralisation (as long as you complete the course, that is!), so he doesn't actually need to use any non-exemplars.

So rather than DI being able to improve on Thomas's techniques, I'd really say it's more likely that Thomas's techniques can be used to improve on DI.

That said, I've found a lot that I like in DI.  In particular, Engelmann wrote an interesting polemic against the guys that dismiss him out of hand, called Socrates on reading mastery (another of Owen's links), where he has an imagined debate between the philosopher and an educational guru who refuses to see the value in DI.  While he does seem a bit bitter at times, he demolishes the complaints against him fairly resoundingly.  In truth, a lot of education isn't methodology, but ideology.  We convince ourselves that something is best without any evidence, and then we dismiss empirical evidence on the grounds that our unproven principles aren't followed.  You'll hear the same thing in criticism of Thomas -- "it can't work because you have to translate," "there's too much English," "you can't 'learn' a language, you have to 'assimilate' it," etc etc ad nauseum.

And Engelmann is an ardent supporter of "basic skills"/"bottom-up" teaching, which is something I think is only logical.  Starting from large-scale problem solving increases task complexity significantly.  In algebra or in science, you have to control for one variable at a time, and in bottom-up teaching you control for one variable at a time.  But when you are trying to manage multiple variables, you need a lot more information before you have full control of even one variable, and in the meantime, you risk drawing false conclusions and making overgeneralisations about the data/formula/language features.

This is a point that Engelmann makes in the Socrates story.  He points out that an evil person could make a bad course that follows Rosenthal's principles of what makes a good course, and Rosenthal agrees.  He then extends that to it's logical conclusion:

If it is possible to design a failed program on purpose, isn’t it possible for some program designers to create a failed program because of bad judgement...?

This is a point that I often try to make myself, and it's something that a lot of people find hard to accept.  Just because something works for one person doesn't mean it's good -- the goal in all teaching is to eliminate the possibility of misinterpretation.  This is true on the level of materials themselves just as much as on the level of guidelines for producing materials.

Another interesting difference between Thomas and DI is that DI relies on chorusing, but Thomas instead makes everyone take the time to think about it, but then he lets only one student answer.  Would chorusing be possible in an MT-style course?  Would it be desirable?  My gut reaction is "no", because it would take some of the life out of the language.  Rationally, I could back that up by pointing out that the responses from the students are often slightly halting, so they're not going to be able to do it simultaneously.

Which leads to a question: are Thomas's students on the CD struggling because he went too quickly?  Does DI say we should slow it down?  Maybe.  Is absolute mastery of these at full speed required before moving on?  Maybe.  But regardless of how well we know Thomas's material, it's only an introduction to the language, and there's a lot still to be learned after, so you've still got a fair learning path to get fully up-to-speed before you're going to be able to really use the language anyway, so there's going to be plenty of opportunity for ongoing practice as you continue.  Also, it may not really be desirable to have complete mastery of the grammar with virtually zero vocabulary.

But that's all conjecture.

But the most troubling conclusion that Engelmann reaches is that a script is better than an independent teacher.  Troubling, because it's likely true.  When I did my CELTA course, it's amazing how many of my questions about methodology and task selection were answered with "Use your judgement as a teacher."  Well, sorry, but I couldn't have that judgement until somebody taught me how to be a teacher, and that is why I was on a teacher training course in the first place.  Teacher judgement does indeed open up the possibility of making errors of judgement.

And I have to ask myself whether there is any point in me teaching Spanish MT-style, or if I should just tell everyone to buy the CDs, which already exist and probably teach the language a bit more effectively than I do.  (Which is just one of the reasons I'm not currently teaching Spanish to English speakers, but English to French speakers.)

That isn't to say that Engelmann's scripts, or Thomas's recordings, are universally optimal -- I sincerely doubt they are (and, in fact, I know that Thomas's are not) -- just that they are better than most teaching, and applying judgement risks introducing errors of judgement.  Perhaps many teachers would do better teaching from a script initially and trying to internalise the logic and the process of teaching before ever being forced to operate independently.

But I don't believe DI is the be-all-and-end-all of education, and I don't believe the MT method can be improved by the blind application of DI principles.

The MT method is not well-enough understood, and I think not even MT himself knew what it was -- there are pretty fundamental differences between some of his courses, and despite "telling" his method to two people, there is no document that adequately describes it, and the courses claiming to follow his principles have surprisingly little in common with his teaching.  DI may give us an extra frame of reference within which to view and discuss Thomas's teaching, but no more than that.


*Yes, I distance myself so far from behaviorism that I even spell it in US English. ;-)

Initial observations about Corsican

Before I came here, I did a bit of digging about on the internet, trying to find out about the Corsican language.  There wasn't hellish much info in English, and even in French it was pretty sketchy.  Now I'm here, though, and I've picked up a book of Corsican grammar (in Corsican, just for the hell of it) so I can start to puzzle it out for myself.

My first reaction from some of the free resources online was to describe Corsican as "Italian with a Catalan accent", because of the vowel reduction in unstressed syllables.  However, vowel reduction in Corsican is not as extreme as that in Catalan.

One thing that wasn't described too well in the sources I'd viewed online was consonant changes, and in reality it isn't particularly difficult.  I'd been trying to puzzle through single and double consonants, and my initial reaction was to treat them as in Italian, where a double consonant is lengthened (a process called "gemination".  But the first pronunciation guides talked about consonant changes -- a T being pronounced as D for example.

As it turns out, in the south the model is more like Italian -- T is /t/ and TT is /tt/ -- and in the north these changes occur, but even these aren't alien to someone familiar with other Romance languages.  It's a process called "lenition" (the weakening of consonants) and it happens frequently in many peninsular Spanish dialects (the archetypical example being the word "Madrid" in a Madrid accent) as well as in almost all other dialects in the letter V.  A consonant is weakened with it occurs "intervocalically" (ie between two vowels).  The strong form occurs when bounded on one side by another consonant.  And if there's no other consonant, just doubling the same consonant gets round the problem (which is roughly analogous with the R/RR distinction in Spanish and several other languages).

So in the north of Corsica "atta" would be pronounced /'ata/, "ata" would be pronounced /'ada/ (as would "adda") and "ada" would be pronounced /'aða/ -- which is the same soft D as in Madrid (/ma'drið/).

So Corsican is just another point on the spectrum of Romance languages.  The interesting part is how the north and south have such different ways of rendering the same consonants, but that in so many respects the northern and southern dialects are extremely similar and consistent.  Before starting on Corsican, I would have assumed that such a large difference in pronunciation would have broken mutual comprehensibility and that the dialects would have diverged to the point of being considered completely distinct languages, but that doesn't seem to have happened.

More thoughts on possessives

One thing I avoided commenting on last time was the question of agreement.  Possessives in the Romance languages agree with the gender and number of the possessed item, but that doesn't happen in English.  OK, so English adjectives don't inflect for gender or number anyway.  So I was curious, and Googled "German possessives" -- sure enough, German possessive "adjectives" decline for agreement with the noun... and there is no distinction between his, her and its, even though each of these has its own pronoun.  The English possessives have taken on a fundamentally different grammatical nature than even German... so what does Google say about Dutch?  It turns out that it has a his/her/its distinction too.  Which is interesting... so what about Danish?  His, her and its, and only the first personal and reflexive possessives decline to agree with the object of possession.

This would suggest that there is a fundamental difference in the internal logic of the language goes at least as far back as the viking Danelaw, and possibly right back to before the Anglo-Saxon invasion, back to the split between the High German tongues and the Low Germanic and North Germanic languages.

One way or another, the English possessive hasn't been a true adjective in many, many lifetimes... if ever.