Too many cooks spoil the net....

When I first took up English teaching in 2007, the internet was an incredibly useful resource.  If I was stuck for a lesson idea, a quick Google search or a glance at one of my favourite sites would give me the inspiration and ideas I needed to build a useful lesson.

But now, the TEFL world and his dog are all posting their ideas on the internet.  A search that would have brought up a handful of useful links in 2007 now brings up a load of low quality worksheets and seemingly aimless tasks.  The act of searching for material is now arguably more time-consuming than just sitting down and writing your own material from the ground up, leading to the wonderful paradox that publishing more information leads to less reuse of material.  And this potentially snowballs, as all those teachers who're making their own material "because they can't find good stuff" start to publish their stuff too, further exacerbating the problem.

Now this is not to say that the authors of these materials aren't good teachers, but it is clear that the worksheets and activities don't fully encapsulate the spirit and methods of their class.  The warm-ups, the support activities, even the teacher's personality have a transformative effect on the material presented, and to take a list of a dozen questions and divorce them from that context robs them of their meaning and effectiveness.

People often compare the sharing of documents to the open-source software movement (well, it's getting less and less common now that document sharing is getting more and more common) but I've always considered that a falacious comparison.  Open-source is about exposing the underlying logic to the wider world, and allowing them to improve that logic; but a document is merely the conclusions reached by the author, not the logic he followed to make those decisions.

A very well designed learning exercise or test will follow a very strict process to ensure a sufficiently wide coverage of concepts and minimise the influence of luck on obtaining the correct answer, but as soon as you change one question in a set, you can end up breaking the balance of concepts tested and leave out something important.

If we're honest with ourselves, though, most of us will admit that we don't properly balance our tasks.  There will be times when we hand out a worksheet and realise that we've missed an important case, and I know that as soon as I started marking the last grammar test I set, I started spotting gaps where concepts weren't tested while other concepts were tested multiple times.

Which is, of course, why we look to other people to provide us with exercises and tests -- it is far more efficient to have one person spend all the time making a meticulously balanced question set and then have hundreds of teachers reuse those questions.

That's why books are such a great idea in theory -- it's just a shame that in practice a great many language teaching books don't live up to the promise; which is where the internet was supposed to help.  Unfortunately, the online material I've found to date isn't of great help.  There's two major camps: the let's-dump-our-worksheets-and-move-on crowd and the oh-look-what-I-can-do-with-technology crowd.

The first lot is basically what I've already talked about -- problem sets with little or no guidance on how to build a coherent lesson around them.

The second lot is people who have learned how to use some flashy little piece of software, but more often than not they find themselves being controlled by the software, rather than being in control of it.  This leads to a proliferation of pairing exercises (question halves and question-with-answer) and ordering exercises (sentence order or line-by-line conversation) because that's what the author knows how to do with the software.  It's a further weakening of pedagogy.

The last example I saw of this was for revision of conditional sentences (if...).  There was an introductory page that described the four conditional types, and then a selection of revision exercises, but each exercise focused on one type of sentence only, so the learner never needed to choose which type of conditional to use, just remember how to form the given type.  These sort of structurally-focused exercises are usually only recommended when initially teaching the form, with student scaffolding being reduced continually until they are able to make free, independent choices.  But as I said, this was allegedly a revision page... yet they were doing exercises designed for introducing the structure, because that was the type of exercise the author knew how to create.  And it looked nice, too.

So, yes, the internet is slowly becoming more of a problem, not a solution.

A way forward...?

This problem isn't really anything new -- it has been the perennial problem of the internet.  In the early days, the web was a collection of articles written mostly by academics, so it was high quality, low volume.  As more and more people started posting stuff online, the volume went up, and the average quality went down.

"No problem," the academics told us, "the network will self-organise, and the cream will float to the top."

The mechanism by which this self-organisation took place was intelligent linking.  A trusted source recommends other trusted sources, and surfers navigated that way.  It worked -- that's how I found a lot of information online at the turn of the century.

Then came Google, whose algorithm worked on the same principles -- links acted as recommendations, and the value of a link was related to the linking site's rating.  It was very effective.

But this network of links just doesn't exist in the language resource world.  Most resources are what we would technically consider "leaf" nodes in the network -- they are end-points that don't lead anywhere else.  Even when they do, it's normally only to other materials on the same site -- there are very few teachers' resource sites that aren't dedicated to keeping you on their site and their site alone.  Those that do link to other sites are (at least in my experience) pretty unfussy about what they link to, listing far too many resources and in effect simply echoing the results of a Google search in a different format.

This means that Google has very little to go on when trying to rate resources for language teachers, and this leads to a paradox from Google's point of view: Google has become so popular as the way to find resources that people are no longer building up the web of links that Google relies on.  This isn't true in all fields, where forum posts have started to replace traditional websites as the source of recommendations.

However, most fields have a certain sense of simultaneity -- TV programmes, for example, are broadcast at a fixed time, and their importance fades.  Most spheres are subject to such fashions, so people will be talking about the same thing at the same time.  But language points don't come into or go out of fashion.  Every teacher in the world teaches them... but not at the same time.  Although there are forums related to the topic, it's not a topic that is really suited to the medium.

So there's not enough information out there to let Google separate the wheat from the chaff.  It's a mess.

What's needed is for teachers who find genuinely useful material to start cataloguing it selectively, publishing a useful collection of links to a small number of resources that cover the major language points that most teachers need.  Sites that favour quantity over quality.  And we need to start sharing the links to those sites.  And we need to start using those sites, rather than Google.  If you know of any such sites, feel free to add links in the comments section!

Collaborative materials

More than that, though, if we genuinely want to share our materials, we need to make sure that they can be updated and improved upon.  Millions of man-hours are wasted by producing multiple flawed worksheets, when we could make minor modifications to each other's and produce something of lasting value.

I doubt I'm the only teacher who alters the free materials I've downloaded from the internet, but like all the others, I keep my modifications to myself because the author's given me permission to use the material, but not to redistribute it.  This is a shame, because some of the best designed materials I've come across have been from non-natives, and fixing one or two little non-native errors would make them into something valuable... but I refuse to use or recommend anything with even one non-native error in it.

But just permission to republish isn't enough, because that wouldn't stop the proliferation of materials -- it would worsen it.  A dozen different sites with slightly different versions of the same worksheet... that would be a nightmare.  We have to look at the software world again and look at how they control their edits, updates and revision; how they resolve differences of opinion... or not (projects often "fork" into two versions when people can't agree on a single way to progress, and quite often these forks are merged together a few years down the line).

Or...

We could stick to the books and materials we've produced ourselves.  Your choice.

Online education's elephant in the room.

It's funny how things come together to give you a better understanding of your own mind. A couple of weeks ago I got caught up the internet debate on mass-participation online education started by an American stats professor critiquing Udacity'sIntroduction to Statistics by Sebastian Thrun.  Then the other day I started debating online education again, this time triggered by the Technology Review article The Crisis in Higher Education linked and debated on Slashdot. One thing I didn't mention in the first debate, but did in the second, was something that has been bugging me for a very long time, and it's really only thanks to the recent debates I've been having with Owen Richardson on DI that I was finally able to articulate it.

These massive courses claim the potential to be better than anything that's come before, thanks to the availability of masses of automatically-collected feedback that will be used to improve them. This, theoretically, means the fastest pace of change in the history of education.

But is that really the case in practical terms?

Right now, I'm at the steepest part of the learning curve with respect to the courses I'm delivering at the university. I can't write more than one full lesson plan at a time, as in each new lesson I receive crucial feedback on what my students are capable of. So I'm constantly revising my material.

My father, during his career as a Chemistry teacher, delivered the same course year after year to classes of no more than 20 pupils at a time. Every time he taught a lesson, though, he was looking for improvements and refinements based on the reaction of the class. If someone made a mistake, he'd try to change the teaching to remove the possibility of someone in the next class making the same mistake.

So in the case of a conscientious teacher, material is revised for every 20 students taking the course.

 

Sebastian Thrun's first sitting of the Artificial Intelligence course had 160,000 pupils. OK, only 14% completed the course, but 22,400 students is still an incredibly high number. That's 1120 iterations of a class for me or my Dad. We're talking about numerous lifetimes of teaching. For a course taught once a year, it's equivalent to going back to the first millenium AD, not only before the computer, but before algebra, cartesian geometry and even the adoption of the Hindi-Arabic number system in Europe.  So we're talking about "A.D. DCCCXCII", not "892 AD".

A millenium's worth of teaching, with no improvement – I think that qualifies as the slowest rate of change in education ever, rather than the fastest.

Worse than that, while Thrun complains that his contemporaries are simply throwing existing courses onto the net without making them truly match the new paradigm, these are at least courses that have a fair amount of real-world testing behind them.  By contrast, his attempt at completely new means that he has giving a course to over 20,000 students without having tested it even once (as far as I can see). That's... worrying.

So what's the source of the problem?

The problem as I see it has two root causes: the medium and (as always) money.

The medium.

The current trend to massive online courses is a development of MIT's OpenCoursware initiative. Essentially, MIT videoed a bunch of lectures and stuck them online with various course notes, exercise sheets and textbook references. I know a few people who got a lot out of one or two courses, but often the quality was bitty, with incomplete materials (due to copyright or logistical reasons) and little motivation to complete.

The early pioneers of the current wave saw a major part of the problem as being in the one-hour lecture format, and revised it to a “micro-lecture” format, delivering short pieces to camera, interspersed with frequent concept-checking and small tasks.

But however small the lecture, it is still fundamentally the same thing, with a live human writing examples on some kind of board, and any revision means the human going back to the board and writing it out again, and giving the explanations again. The presented material cannot be manipulated automatically, so the potential for rapid revision and correction is reduced.

Money.

Revising a course manually takes time, and time is money. Squeezing several lifetimes' worth of improvements into a rapid development cycle isn't a part-time job – it's probably more than a full-time job, yet in the brave new world of online education, this is nobody's day job. Most of the course designers are still teaching and researching, and Thrun himself is still doing research while working at one of the world's biggest tech companies and trying to start up a new company.

No-one's yet really worked out the way to cash in on these developments, so no-one's investing properly.

Here in the UK, online education (on a smaller scale) is already on the increase, but mostly as a cost-cutting measure. That's fine as a long term goal, but in the short-term there is a need for massive investment in order to get things right.

What are we left with?

Not a lot, frankly. Data-mining requires a widely-varying dataset, in order to allow the computer to detect patterns that are too subtle or on too large a scale for a human to be able to pick up independently. But the data collected on these online programmes is pretty much one-dimensional. There are no variables explored in the teaching – there is one course, so the feedback can say if something is difficult or easy (based on number of correct answers and time taken to answer) – it can't tell us why, and it can't tell us what would be better. That means that the feedback from 22,400 students is less valuable to a good teacher than one question from an average student during an average class. That's.... worrying.

So much for the revolution.

So what's the solution?

If there's two parts to the problem, there must be two parts to the solution.

Medium

The Open University has, over the years, moved away from lectures to producing TV quality documentaries that use the best practices of documentary TV to present material in a way that genuinely enlightens the viewer.

 

As a documentary isn't a single continuous lecturer, it would theoretically be possible to have a computer modify and re-edit a documentary to make it easier to understand.

 

On the most basic level, a difficult concept might be made easier by inserting an extra second of thinking time at a certain point in the video -- an algorithm would be able to test this dynamically.  Conversely, the algorithm might find that reducing the pause is more effective, and do so dynamically (we assume then that the concept is easy and that extra time allows the student to become distracted).

 

Then there's the slides and virtual whiteboards used in the videos themselves -- produced in real-time as the presenter speaks.  This splits the presenter's attention, often resulting in rushed, unclear writing, or pauses and hesitations in speech.  Revising the visuals means redoing the whole video.

 

Why doesn't the computer build the visuals to the presenters specification, but with the ability to modify them to optimise to student feedback?

 

Eventually, we would get to the point where a course definition is a series over voice-over fragments and descriptions of intended visuals, and the computer decides what to put where.

 

But the reason that'll never happen in the current model is reason 2:

Money

Where there is a genuine incentive to drive down the cost of education, there on-line education will find its most fertile ground. When you look at the tuition fees in places like Stanford, Harvard and MIT, you'll see that these aren't the schools with the biggest incentive to make online education work.

Instead, we need to look to Europe, and in particular the countries with significant public funding for higher education. Universities funded by the public purse are under intense pressure to cut costs – it's the only way to balance the books in a shrinking economy.

However, the universities alone can't make this happen, as the current pressure is for cost savings NOW, and so they're producing online programmes with insufficient research and the quality of education is suffering for it.

Governments are sacrificing students to the God of Market Forces, when they should instead be planning intelligently. Instead of cutting funding to force universities to be more economical, they should be investing to make universities more economical. Give universities money now in order to produce high-quality programmes that will reduce costs for years to come.

 

But It Will Not Be Cheap – quite the opposite.  The creation of a genuinely high-quality online course is phenomenally expensive in terms of up-front costs, while being ridiculously cheap in the long term.

The current clientele of Udacity, edX and Coursera will no doubt feel cheated that I'm talking about education for the classic “student” rather than the free “everyman” approach of Coursera et al, but there's no need to. Established, well-researched, properly tested and adequately trialled online courses may take a while to perfect, but once they exist, their running costs will be so low that they will surely be made widely available. And while they're being developed, they're going to need a constant source of beta testers, and that's going to mean people who're doing it for personal interest, not for grades – ie you. The end result will still be open education, but it will be better.

The weirdness of learning something you already "know"...

I've started looking at starting on Corsican a few times, but it's always seemed really difficult.  Why?  Because all the beginners' stuff looks too obvious to me to be worth looking at.  If it's not like Italian, it's like Spanish or French.  Everything I'd found was phrase-based, and I could understand most of them with no effort.  My brain saw no reason for effort.

That's something I find quite interesting, because I've always argued that extrinsic motivation (motivation from outside the material/course) is not enough -- all teaching has to be intrinsically motivating.

Well, today I came across a slightly better site (for French speakers).  The lessons there aren't brilliant, as they're just information and the exercises on the site aren't related to the lessons.

But the information's there, so I can juggle it about to suit myself.

Maybe it's time to dive back into the Python NLTK and write myself a wee bit of self-teaching software....

I love learning languages... but I hate language learning

Ok, so yesterday I had my last exam of the semester, so I decided to take a break from Gaelic and start working on my Welsh.  I never really did much study before, but trying to catch as much as I could by watching the Welsh-language soap opera Pobol Y Cwm regularly has helped some of it stick (but not all, by a long shot).

So I went to the college library, and started reading Asterix ym myddin Cesar, the Welsh translation of Asterix the Legionary.  Oooooh... it's tough going.

So rather than attempt to struggle through it in the library with a dictionary, I decided to check it out and take it back to my room to go over it seriously with a grammar book.  I was the first person ever to do so -- which isn't surprising given that there isn't even a Welsh course here...

So I took my copy of Teach Yourself Welsh Grammar off my bookshelf, and started reading... then stopped.  You see, while I love learning languages, the vast majority of language learning material is excruciatingly bad.  I know that this book isn't a language course, but it is aimed at learners.  So when the first chapter after the pronunciation guide starts by individually listing 31 different circumstances in which the soft mutation occurs, it immediately loses its audience.  There's no structure -- just a list.  In several of these circumstances, LL ard RH are immune to mutation.  Did they group these together?  They're numbers 1, 5, 6, 18 and 28.  There's no implication that these are in any way related, meaning the learner risks trying to learn 5 exceptions instead of one group.

I'm trying to extract enough information to teach myself, but I'm overwhelmed by information -- I have to try to read and understand it all in order to identify the patterns and salient points.  It's tiring, frustrating, and to a great extent insulting.

Yes, insulting.  Because at one level, the mere existence of the book is a claim by the author that this is good enough for the learner.  And if the book is good enough for the learner, then it must be me that is the problem.

I'm lucky -- I feel insulted.  Many, many people genuinely believe that they're at fault -- that they're "stupid" or "not good at languages".  And they think that I'm good at languages.  Well believe me, I'm not.  Even despite spending countless hours in this sort of book, I still can't make head nor tail of some of them.  If anything I'm worse at languages than the average, and I've only got where I am today because I refuse to believe I'm incapable.

The hardest part for me in learning any new language is getting started, because in general there's just too much information thrown at you in an unstructured and poorly thought out way.

So for those of you starting out and discouraged by your materials, remember: you're not the only one.