29 September 2012

Online education's elephant in the room.

It's funny how things come together to give you a better understanding of your own mind. A couple of weeks ago I got caught up the internet debate on mass-participation online education started by an American stats professor critiquing Udacity'sIntroduction to Statistics by Sebastian Thrun.  Then the other day I started debating online education again, this time triggered by the Technology Review article The Crisis in Higher Education linked and debated on Slashdot. One thing I didn't mention in the first debate, but did in the second, was something that has been bugging me for a very long time, and it's really only thanks to the recent debates I've been having with Owen Richardson on DI that I was finally able to articulate it.

These massive courses claim the potential to be better than anything that's come before, thanks to the availability of masses of automatically-collected feedback that will be used to improve them. This, theoretically, means the fastest pace of change in the history of education.
But is that really the case in practical terms?
Right now, I'm at the steepest part of the learning curve with respect to the courses I'm delivering at the university. I can't write more than one full lesson plan at a time, as in each new lesson I receive crucial feedback on what my students are capable of. So I'm constantly revising my material.
My father, during his career as a Chemistry teacher, delivered the same course year after year to classes of no more than 20 pupils at a time. Every time he taught a lesson, though, he was looking for improvements and refinements based on the reaction of the class. If someone made a mistake, he'd try to change the teaching to remove the possibility of someone in the next class making the same mistake.

So in the case of a conscientious teacher, material is revised for every 20 students taking the course.
Sebastian Thrun's first sitting of the Artificial Intelligence course had 160,000 pupils. OK, only 14% completed the course, but 22,400 students is still an incredibly high number. That's 1120 iterations of a class for me or my Dad. We're talking about numerous lifetimes of teaching. For a course taught once a year, it's equivalent to going back to the first millenium AD, not only before the computer, but before algebra, cartesian geometry and even the adoption of the Hindi-Arabic number system in Europe.  So we're talking about "A.D. DCCCXCII", not "892 AD".
A millenium's worth of teaching, with no improvement – I think that qualifies as the slowest rate of change in education ever, rather than the fastest.

Worse than that, while Thrun complains that his contemporaries are simply throwing existing courses onto the net without making them truly match the new paradigm, these are at least courses that have a fair amount of real-world testing behind them.  By contrast, his attempt at completely new means that he has giving a course to over 20,000 students without having tested it even once (as far as I can see). That's... worrying.
So what's the source of the problem?
The problem as I see it has two root causes: the medium and (as always) money.
The medium.

The current trend to massive online courses is a development of MIT's OpenCoursware initiative. Essentially, MIT videoed a bunch of lectures and stuck them online with various course notes, exercise sheets and textbook references. I know a few people who got a lot out of one or two courses, but often the quality was bitty, with incomplete materials (due to copyright or logistical reasons) and little motivation to complete.

The early pioneers of the current wave saw a major part of the problem as being in the one-hour lecture format, and revised it to a “micro-lecture” format, delivering short pieces to camera, interspersed with frequent concept-checking and small tasks.
But however small the lecture, it is still fundamentally the same thing, with a live human writing examples on some kind of board, and any revision means the human going back to the board and writing it out again, and giving the explanations again. The presented material cannot be manipulated automatically, so the potential for rapid revision and correction is reduced.

Revising a course manually takes time, and time is money. Squeezing several lifetimes' worth of improvements into a rapid development cycle isn't a part-time job – it's probably more than a full-time job, yet in the brave new world of online education, this is nobody's day job. Most of the course designers are still teaching and researching, and Thrun himself is still doing research while working at one of the world's biggest tech companies and trying to start up a new company.
No-one's yet really worked out the way to cash in on these developments, so no-one's investing properly.

Here in the UK, online education (on a smaller scale) is already on the increase, but mostly as a cost-cutting measure. That's fine as a long term goal, but in the short-term there is a need for massive investment in order to get things right.

What are we left with?
Not a lot, frankly. Data-mining requires a widely-varying dataset, in order to allow the computer to detect patterns that are too subtle or on too large a scale for a human to be able to pick up independently. But the data collected on these online programmes is pretty much one-dimensional. There are no variables explored in the teaching – there is one course, so the feedback can say if something is difficult or easy (based on number of correct answers and time taken to answer) – it can't tell us why, and it can't tell us what would be better. That means that the feedback from 22,400 students is less valuable to a good teacher than one question from an average student during an average class. That's.... worrying.

So much for the revolution.

So what's the solution?

If there's two parts to the problem, there must be two parts to the solution.

The Open University has, over the years, moved away from lectures to producing TV quality documentaries that use the best practices of documentary TV to present material in a way that genuinely enlightens the viewer.
As a documentary isn't a single continuous lecturer, it would theoretically be possible to have a computer modify and re-edit a documentary to make it easier to understand.
On the most basic level, a difficult concept might be made easier by inserting an extra second of thinking time at a certain point in the video -- an algorithm would be able to test this dynamically.  Conversely, the algorithm might find that reducing the pause is more effective, and do so dynamically (we assume then that the concept is easy and that extra time allows the student to become distracted).
Then there's the slides and virtual whiteboards used in the videos themselves -- produced in real-time as the presenter speaks.  This splits the presenter's attention, often resulting in rushed, unclear writing, or pauses and hesitations in speech.  Revising the visuals means redoing the whole video.
Why doesn't the computer build the visuals to the presenters specification, but with the ability to modify them to optimise to student feedback?
Eventually, we would get to the point where a course definition is a series over voice-over fragments and descriptions of intended visuals, and the computer decides what to put where.
But the reason that'll never happen in the current model is reason 2:


Where there is a genuine incentive to drive down the cost of education, there on-line education will find its most fertile ground. When you look at the tuition fees in places like Stanford, Harvard and MIT, you'll see that these aren't the schools with the biggest incentive to make online education work.
Instead, we need to look to Europe, and in particular the countries with significant public funding for higher education. Universities funded by the public purse are under intense pressure to cut costs – it's the only way to balance the books in a shrinking economy.
However, the universities alone can't make this happen, as the current pressure is for cost savings NOW, and so they're producing online programmes with insufficient research and the quality of education is suffering for it.

Governments are sacrificing students to the God of Market Forces, when they should instead be planning intelligently. Instead of cutting funding to force universities to be more economical, they should be investing to make universities more economical. Give universities money now in order to produce high-quality programmes that will reduce costs for years to come.
But It Will Not Be Cheap – quite the opposite.  The creation of a genuinely high-quality online course is phenomenally expensive in terms of up-front costs, while being ridiculously cheap in the long term.

The current clientele of Udacity, edX and Coursera will no doubt feel cheated that I'm talking about education for the classic “student” rather than the free “everyman” approach of Coursera et al, but there's no need to. Established, well-researched, properly tested and adequately trialled online courses may take a while to perfect, but once they exist, their running costs will be so low that they will surely be made widely available. And while they're being developed, they're going to need a constant source of beta testers, and that's going to mean people who're doing it for personal interest, not for grades – ie you. The end result will still be open education, but it will be better.


Nìall Beag said...

...and not long after writing that, I hear that the governor of California is kicking off an initiative to fund permissive-license* free digital textbooks.

That's not the same thing as online education, but there's no reason the same content couldn't be applied in both media.

* The article calls them open-source, but a book is more object code, not source code. There's a lot of logic in the design that isn't explicitly apparent in the end product.

Richard said...
This comment has been removed by a blog administrator.
Richard said...

Sorry about that which was submitted prior...

An excellent essay! As we appear that we are traveling/changing faster than a locomotion, I think your intuitive/counter-intuitive thoughts are in order. What is, may not soon be, but what will be, is anyone's guess? In addition, I thought you had an excellent point that a lecture is a lecture regardless of the length; and the use of documentaries as a substitute is, in my opinion, not a bad idea. However, what do I know...I still like to listen to good, old-fashion, lectures AND documentaries. Maybe we, as educators, are trying too hard at coming up with spoon-feeding methodology? Hmmm...just a thought.

Nìall Beag said...

Thanks (I've just tidied away your earlier post).

I'm certainly not against spoon-feeding -- I'm a "basic skills" guy at heart, anyway.

But no, all good teachers predigest the material to the point where it is very easily understood and assimilated. More than a decade after abandoning undergraduate AI for pure Computer Science, I found myself watching Andrew Ng's Machine Learning course and understanding the basics of learning algorithm optimisation far better than ever before, because every spoonful was the perfect size (a real Goldilocks situation).

It says far less about the medium than it does about Ng as a teacher.

The problem is when we allow the spoon to lead the teaching process -- a bit of a tail wagging the dog. (I'm a walking cliche today, and I usually avoid cliches like the plague.)

The big danger for online education is that the technology may simply cause us to ask the wrong questions.

When I was working in IT, we had access to an e-learning library, and really the testing was crude and simplistic, and mind-numbingly dull. My experiences of university-level e-learning with the Open University, the University of the Highlands and Islands, Coursera, edX and Udacity is nowhere near the depths that Thomson/Netg plumbed, but it does feel like formative questioning is being dumbed down anyway.

Muvaffak GOZAYDIN said...

For 17 years I have been involved with online education.
I always claimed that initial development of an online course is so expensive but when we share it by millions then coast per person is nill.
First time in my life I hear your comments same as my views .
In Turkey we have developed online courses for K12 in the last 15 years . Normal development expense for a 9th grade Physics course ( lasting 35 weeks , 3 session per week ) was $ 1 million even in Turkey . Now it is shared by 17 million K12 children for years . Cost is nill .
I do not think that no American college spent ever that much for a course .
Even MIT did not spend that much since they had much many material ready to be presented any how .
Now MITx will be shared by millions and they will continue to make money. Even a mere $ 10 fee is more than enough when they reach millions .

Muvaffak GOZAYDIN said...

My motto :
1.- ONLINE Material to be presented must be wortwhile to be presented , that is from top schools of the world , such as MIT and Harvard, Stanford, Yale

2.- Initial investment is around $ 1 million , therefore millions should follow it

3.- To attract millions online should be global. Only USA does not constitute so large audience

4.- To attract millions globally it must come from the top school.
( Coming back to number 1, so no harm )

5.- It must have a small fee
for self financing .

6.- Even a small fee makes a huge income
Nothing is free , any project should be able to cover cost+ small profit

7.- Most important in HE is 18-22 years olds . They need HE more than anybody else . So Small fee perfect online programs are needed for 18-22 years olds ( 20 million of them only in USA ).

8.- Program should provide a degree, for the beginning a certificate may be OK in order not to create many opponents

Thanks once more Niall.

Muvaffak GOZAYDIN said...

Dear Niall

How can I see Thomson/Netg

Thanks billion

Muvaffak GOZAYDIN said...

Dear Niall
Coming to California
( I am from Caltech and Stanford for 8 years )
why in the world people have limited visions even high administrators .

etextbook is a fantastic technology .

But why don't you say

etextbooks prices are 10 % of the retail prices for every college in the USA not only in Cal .

That covers all expenses plus publishers make a huge profit .

Random Review said...

@ Muvaffak GOZAYDIN "7.- Most important in HE is 18-22 years olds. They need HE more than anybody else . So Small fee perfect online programs are needed for 18-22 years olds ( 20 million of them only in USA ). "

As a 35 year old stuck in a dead end job I really do take strong exception to that. Or should I just give up and spend the next 3-4 decades of my life washing other people's dishes because I'm not young anymore and the young people somehow need it more?

@ Cainntear: lot's of really good posts lately (I wouldn't normally post a comment just to say that, but since I'm commenting anyway...).

Nìall Beag said...

@ Random Review,
Thanks. Being back in the classroom has kind of got me thinking properly again (as has Owen's material on DI).

Anyway, I don't think you need to be offended by Muvaffak's post, because looking at it in terms of raw numbers, HE is mostly populated by school-leavers. Distance education has historically aimed itself more in your direction.

Most of my Open University classmates were people who were studying their way out of low-paid jobs or women who had given up their studies to have children.

But the OU is now marketing itself to school-leavers, on the grounds that excessive fees are putting youngsters off full-time study. This doesn't mean the nature of the courses is changing, and they will still be available to you.

A focus on online/flexible HE for 18-22 year olds wouldn't mean that the same quality of education wouldn't be available to you -- quite the opposite. It would mean more investment, and a higher quality of education available to you.

Owen Richardson said...

It just occurred to me that I should add this.

The core problem with online education (or any attempt to improve education with IT technology/any technology period)...

Well, imagine you have an architect, or an electronics person ("electronicsician"? :P ), who didn't know how to design a functional building or circuit on paper.

Is giving them a computer going to help?

And in those hypothetical cases of ignorant architects or circuiteers, Computer Assisted Design tools have already been created!

For the actually case of educational engineers, they have yet to learn the skills to be good engineers period, regardless of their design and development tools!

Writing CAD programs for developing educational courses would have to come after that.

And yes, I realize the original subject of this post was more about using technology to present education than to develop it, but in the case of optimizing online education they are properly two highly connected sides of the same thing (what with the "automated feedback" possibilities and whatnot).

Nìall Beag said...

The Thomson/netG courses are a commercial offering that my former employer subscribed to. I think it may have been bought by Skillsoft... I'm not sure who does it all now, and I imagine it would be very difficult and/or expensive to get access individually, as it is really targeted at businesses as a bulk staff training package.

Nìall Beag said...

@ Muvaffak GOZAYDIN

The problem with charging is that you either:
* reinstate a barrier to re-entry (poor countries not allowed);
* create a two-tier system (poor countries can learn, but not get certificates); or
* you create a perfectly fair system that the world will call unfair.

See my post on the matter.