This morning, I checked my email like I always do, and Coursera were plugging their latest "specialization" -- one for so-called cloud computing.
Coursera specialisations were originally launched as a single certificate for a series of "signature track" (ie "paid for") courses, but there's always the free option alongside it.
So I was very surprised when I clicked on the link for more information about the specialisation, then clicked through to the course, and it was only offering the $49 paid-for version. Now I did go back later and track down the free version of the course by searching the course catalogue, but the notable thing was that you can't get to the free version by navigating from the information about the specialisation.
It's there -- it is -- but by making click-through impossible, they're actively trying to push people into the paid versions. This suggests that the business model isn't working, and it's not really much of a surprise -- there's no such thing as a free lunch, and the only free cheese is in the mousetrap.
Some of the universities seemed to be using the free courses as an advert for there accredited courses, but it's a very large and expensive way to advertise -- teaching thousands in order to get half-a-dozen extra seats filled on your masters programme -- and so really the only way to get money is to get more of the students to pay.
Is it worth it for the student?
Cloud Computing costs £150, and going by their time estimates, that's between 120 and 190 hours of work. The academic credit system here in Scotland says that ten hours of work is one "credit point", and there are 120 credits in a year. Timewise, the Cloud Computing specialisation is then roughly equivalent to a 15-point or 20-point course -- ie. a single "module" in a degree course. A 15-point module costs £227.50, and a 20-point module costs just over £300, so £150 for this seems like a pretty good deal. Of course, this is only the cost to students resident in Scotland to begin with, and it is controlled by law to stay artificially low -- in England, the basic rate would be £750 for a 15-point course or £1000 for a 20-point one, but many universities "top-up" their fees by half again: £1125 and £1500 respectively. And English universities are still cheaper than many of their American counterparts.
So the Coursera specialization could be half the price of a university equivalent, or a tenth, or even less, depending on where you live. Sounds like a good deal, right?
Sadly, though, the certificates are worthless -- almost all the institutions
offering courses through Coursera (and EdX, and FutureLearn) are
allowed to accredit their own courses for university credit, but they choose not to. If they
accredited a £30 course as university-level study, they'd be competing
against themselves, and they'd kill the market for their established
distance courses, and perhaps even their on-campus courses.
If they can run a course for £150, is there any justification for their usual high prices? Well... yes. Coursera is on a freemium model (free for basic use, pay for "premium" services), but in reality everything on Coursera is still the "free" part of the freemium. The online-only courses are not viable for universities for a number of reasons, so it's the fully-accredited courses run by the universities themselves that make it possible for the universities to offer the cheap courses "on the side", using repurposed versions of their existing course materials.
Technology and knowledge sharing can and should be used to reduce the cost of education. When I studied languages with the Open University, I looked at the cost of the course I was taking, vs equivalent unaccredited alternatives -- I could have bought equivalent books and spent more time with a one-on-one teacher than I did in group tutorials, and still only spent half of the money I did with the OU. If I hadn't wanted to get the degree, it would have made no sense at all to continue with them, but I want to teach in schools, so I need the degree.
So yes, there is undoubtedly unnecessary expense in education and there's a lot of "fat" that could be trimmed away, but the Coursera model won't do it, and for now it remains something of a distraction -- a shiny object that draws our attention away from the real problems and solutions.
Showing posts with label MOOC. Show all posts
Showing posts with label MOOC. Show all posts
30 January 2015
05 October 2014
What's it like to lose language?
I recently started an online course about children's play with FutureLearn. One of the optional readings for week 1 was an interview with a practicing psychologist who had a stroke [journalofplay.com], triggering aphasia from which she has never fully recovered.
There's a lot of intriguing ideas in there, although it's a very long article and it wasn't interesting enough for me to get to the very end, but I figure a few of my regulars would enjoy it anyway.
There's a lot of intriguing ideas in there, although it's a very long article and it wasn't interesting enough for me to get to the very end, but I figure a few of my regulars would enjoy it anyway.
11 June 2013
Link drop: to myself!
A couple of weeks ago, I got invited to write an article for a multiauthor blog entitled MOOC News and Reviews, all about these newfangled online free course thingummijigs. Well, it seemed like a good opportunity to continue writing about these things without constantly boring my language-orientated readers here with it.
My first article has just gone online, in which I discuss the benefits to the learner of having a good old-fashioned whinge once in a while, and the barriers that online discussion places to the student who feels confused or dissatisfied.
My first article has just gone online, in which I discuss the benefits to the learner of having a good old-fashioned whinge once in a while, and the barriers that online discussion places to the student who feels confused or dissatisfied.
04 May 2013
Coursera offering free teacher training!
I've just been nosing around on Coursera looking for interesting courses to take. I'd read recently that they'd signed up several new course providers, including the first of their providers that aren't accredited universities.
My first reaction was to doubt the value of non-university courses, but one of these suppliers has brought with them something that was lacking in the previous material: course progression. Some of the universities have been joining Coursera just because it's the in thing, and others have been using it as an advert for their distance education programmes. But it's never in a university's interest to offer an entire programme for free.
Enter the Commonwealth Education Trust, a charity whose mission is to provide teacher training at primary and secondary level to improve children's education in developing countries in the Commonwealth.
Their whole goal is to provide complete teacher training for free, so teaming up with Coursera reduces their costs and extends their reach and their 8 module teacher training programme is a win for everyone involved.
Their main target is at practicing teachers who haven't had any formal training, and I'm intending to follow it as a supplement to my CELTA certificate, which I always felt was slightly insufficient as teacher training. The CET programme is estimated at between 180 and 280 hours in total, covering 46 weeks of activities spread over about 16 months (the first sitting of module 1 starts this August, and the first sitting of the final module starts next November). In total, that's actually comparable to the amount of time you're expected to spend on a 4 week intensive CELTA course, so I suppose I'm hoping there's a difference due to the quality of content, and the fact that this is general teaching with no specific language focus (I've always felt that language teaching suffers due to a belief that "language is different", so the lessons from general teaching are sometimes ignored). Also, the pacing of the course should theoretically help long-term retention: my CELTA felt heavily "crammed", with no proper consolidation of learning.
On top of this, the Trust are also offering some kind of certification for people who complete all 8 modules:
I'll be taking it this year (or at the very least "starting it" -- I've got a poor record with free online courses, not having completed a single one yet), so I'll let you know how I get on. There's a second sitting starting next January.
My first reaction was to doubt the value of non-university courses, but one of these suppliers has brought with them something that was lacking in the previous material: course progression. Some of the universities have been joining Coursera just because it's the in thing, and others have been using it as an advert for their distance education programmes. But it's never in a university's interest to offer an entire programme for free.
Enter the Commonwealth Education Trust, a charity whose mission is to provide teacher training at primary and secondary level to improve children's education in developing countries in the Commonwealth.
Their whole goal is to provide complete teacher training for free, so teaming up with Coursera reduces their costs and extends their reach and their 8 module teacher training programme is a win for everyone involved.
Their main target is at practicing teachers who haven't had any formal training, and I'm intending to follow it as a supplement to my CELTA certificate, which I always felt was slightly insufficient as teacher training. The CET programme is estimated at between 180 and 280 hours in total, covering 46 weeks of activities spread over about 16 months (the first sitting of module 1 starts this August, and the first sitting of the final module starts next November). In total, that's actually comparable to the amount of time you're expected to spend on a 4 week intensive CELTA course, so I suppose I'm hoping there's a difference due to the quality of content, and the fact that this is general teaching with no specific language focus (I've always felt that language teaching suffers due to a belief that "language is different", so the lessons from general teaching are sometimes ignored). Also, the pacing of the course should theoretically help long-term retention: my CELTA felt heavily "crammed", with no proper consolidation of learning.
On top of this, the Trust are also offering some kind of certification for people who complete all 8 modules:
On the satisfactory completion of each course you will receive a statement of accomplishment related to the course. On the completion of all the courses you may contact the Commonwealth Education Trust to request a statement of accomplishment related to the overall program.I'm not aware of whether the Trust is part of any recognised accreditation scheme, but it's certainly likely to be looked on favourably if you're applying for voluntary teaching work in a Commonwealth country.
I'll be taking it this year (or at the very least "starting it" -- I've got a poor record with free online courses, not having completed a single one yet), so I'll let you know how I get on. There's a second sitting starting next January.
07 April 2013
Adaptive learning systems: nothing to be afraid of.
I was meandering through various blogs last week, following various links to material that I found idly interesting. On Stephen Downes's website, I saw his link to a blog post by David Wiley (a name getting frequent mention in the OU's H817 MOOC*). The post discusses the dangers of adaptive learning systems -- systems that track your learning and teach your stuff.
(* Actually, I'm starting to get a little stir-crazy reading a lot of data-free opinion pieces from the same four names: Siemens, Cormier, Downes and Weller.)
Wiley's criticism is that you don't own any of the material you access, and he accuses the adaptive learning companies of exploiting our willingness to pay for services while expecting content for free.
There are several problems here:
In languages, you get course books, learners books, textbooks, workbooks etc... a whole bewildering array of paper that leads you through your learning, but when you've finished, you've got practically no need for any of it, and it sits gathering dust on your shelf as you can't bring yourself to get rid of stuff that cost so much to collect, but in the end, all you ever use is a dictionary (probably online) and a single reference grammar book. That reference grammar book was no use to you when you started out, of course, as the examples contained far too many unfamiliar words, and the ordering of the book made it impossible to really understand anything new.
The same in computers, where a learner's book would hold your hand through the various concepts required to learn a new technology, but looking back on it later, the learner's book was never any good for looking up basic concepts (which were drawn out over entire chapters) and didn't contain enough information on the advanced concepts that would be of some use to you by this stage. So that book goes to the second-hand shop and you buy a desk-reference or bookmark a webpage.
And yet publishers continue to attempt to sell books to suit both markets, invariably falling between two stools in the process.
A learner doesn't need access to a teaching text after the class is over, and is free to go out and buy a reference book instead.
Because an adaptive learning system is not an attempt to replace the textbook -- it's an attempt to replace the teacher. A computer, in theory, is capable of producing a more individualised learning path for each student than a teacher, thanks to the computer's essentially limitless perfect memory. I cannot remember every single difficulty each of my students has, but a computer can.
Most sharing at the local level is facilitated by having a shared syllabus -- if the lesson is for 2nd years, you're safe to do it with any 2nd year class, and unsafe to do it with any 3rd year class. But one way or another, there has to be some way to ensure that students get tasks they've never seen before, because while a good story is no worse for being told a second time, a good lesson is destroyed by being taught a second time.
In the classroom, you can always rely on "we already did that" to let you know, and then you improvise something else, but could you do that in an adaptive learning system? I don't really think you can. You can't just accept any old feedback from the student (natural language processing systems aren't that sophisticated yet) and a big red button marked "already done it" would be very off-putting to the student and would make the software look really unprofessional.
No, the software has to know what you've done and what you haven't, and that means keeping control of when and how the learner accesses the material.
In essence, though, I think that Downes and Wiley are objecting on ideological grounds rather than practical, pedagogical ones. They have aligned themselves with a rather dubious view of learner-centred education where the learner makes all the choices, apparently empowering and enabling them. Adaptive learning systems take the diametrically opposite view: that by taking the decisions away from the learner and instead presenting whatever the learner most needs or is best ready for at any given moment, the learner attains a much more complete and well learned education.
And anyway, all the evidence is on the side of the adaptive systems guys, because connectivism and the like breaks away from the proven techniques of staggered repetition, planned progression and learning-by-testing which are the very foundations of adaptive learning, and replaces them with an almost entirely unstructured meander through materials effectively chosen by a known non-expert (the learner) with no real "testing" of concepts.
And yet this type of vague handwavery is presented in the absence of discussion about the many known effective techniques in a course that is supposed to be part of a masters-level module. I am appalled.
(* Actually, I'm starting to get a little stir-crazy reading a lot of data-free opinion pieces from the same four names: Siemens, Cormier, Downes and Weller.)
Wiley's criticism is that you don't own any of the material you access, and he accuses the adaptive learning companies of exploiting our willingness to pay for services while expecting content for free.
Adaptive learning systems exploit this willingness by deeply intermingling content and services so that you cannot access one with using the other.But how is this any different from any other "teaching" experience? Wiley seeks to equate adaptive learning systems with textbooks, but is he right to do so?
There are several problems here:
- The difference between a teaching text and a reference book.
- Course as "teaching" vs course as "material".
- The need to ensure that your material is new to the student.
The difference between a teaching text and a reference book
Perhaps I'm lucky in that the subjects I have studied make a big distinction between these two categories.In languages, you get course books, learners books, textbooks, workbooks etc... a whole bewildering array of paper that leads you through your learning, but when you've finished, you've got practically no need for any of it, and it sits gathering dust on your shelf as you can't bring yourself to get rid of stuff that cost so much to collect, but in the end, all you ever use is a dictionary (probably online) and a single reference grammar book. That reference grammar book was no use to you when you started out, of course, as the examples contained far too many unfamiliar words, and the ordering of the book made it impossible to really understand anything new.
The same in computers, where a learner's book would hold your hand through the various concepts required to learn a new technology, but looking back on it later, the learner's book was never any good for looking up basic concepts (which were drawn out over entire chapters) and didn't contain enough information on the advanced concepts that would be of some use to you by this stage. So that book goes to the second-hand shop and you buy a desk-reference or bookmark a webpage.
And yet publishers continue to attempt to sell books to suit both markets, invariably falling between two stools in the process.
A learner doesn't need access to a teaching text after the class is over, and is free to go out and buy a reference book instead.
Course as "teaching" vs course as "material".
As a teacher, I give out lots of material in class, but during the course of the lesson, the material is quickly "consumed" -- worksheets are filled in, and by the end of the day, the student has not accrued any additional work-at-home material over and above any specific homework I may set. Do Wiley and Downes object to this? Am I cheating my students if my material is not infinitely reusable?Because an adaptive learning system is not an attempt to replace the textbook -- it's an attempt to replace the teacher. A computer, in theory, is capable of producing a more individualised learning path for each student than a teacher, thanks to the computer's essentially limitless perfect memory. I cannot remember every single difficulty each of my students has, but a computer can.
The need to ensure that your material is new to the student.
Even where there is a developed culture of sharing between teachers, ever teacher holds some things back for themselves. Why? It's the "old standby" -- that exercise or activity that can be adapted to various levels to provide an emergency lesson when the projector breaks or the new books haven't arrived. If you don't share that lesson, then you're safe to use it with any and every class, but if you share it with your colleagues, there's a very high chance that sooner or later you'll have a class say "we did that with Mr So-and-so", and your stumped.Most sharing at the local level is facilitated by having a shared syllabus -- if the lesson is for 2nd years, you're safe to do it with any 2nd year class, and unsafe to do it with any 3rd year class. But one way or another, there has to be some way to ensure that students get tasks they've never seen before, because while a good story is no worse for being told a second time, a good lesson is destroyed by being taught a second time.
In the classroom, you can always rely on "we already did that" to let you know, and then you improvise something else, but could you do that in an adaptive learning system? I don't really think you can. You can't just accept any old feedback from the student (natural language processing systems aren't that sophisticated yet) and a big red button marked "already done it" would be very off-putting to the student and would make the software look really unprofessional.
No, the software has to know what you've done and what you haven't, and that means keeping control of when and how the learner accesses the material.
In essence, though, I think that Downes and Wiley are objecting on ideological grounds rather than practical, pedagogical ones. They have aligned themselves with a rather dubious view of learner-centred education where the learner makes all the choices, apparently empowering and enabling them. Adaptive learning systems take the diametrically opposite view: that by taking the decisions away from the learner and instead presenting whatever the learner most needs or is best ready for at any given moment, the learner attains a much more complete and well learned education.
And anyway, all the evidence is on the side of the adaptive systems guys, because connectivism and the like breaks away from the proven techniques of staggered repetition, planned progression and learning-by-testing which are the very foundations of adaptive learning, and replaces them with an almost entirely unstructured meander through materials effectively chosen by a known non-expert (the learner) with no real "testing" of concepts.
And yet this type of vague handwavery is presented in the absence of discussion about the many known effective techniques in a course that is supposed to be part of a masters-level module. I am appalled.
04 April 2013
More Maths for MOOCs!
The guys behind connectivist MOOCs seem to be against the teaching atomic, well-defined concepts, which is all well and good, but I think those same concepts might inform their theories a bit better.
This morning, I received a "welcome to week 4" message from the organiser of the OU MOOC Open Education. He comes across as a really nice guy on both email and video, which makes it a lot more difficult to criticise, but one thing he said really caught my attention as indicative on the problems with the "informal education" model that the connectivist ideologues* profess. (* I refuse to call them theorists until they provide a more substantial scientific backing for their standpoint -- until then, it's just ideology.)
So, the quote:
The best examples, though, come in the realm of birthdays.
If you take the birthdays (excluding year) of the population of a country (eg the UK) and plot a graph, you'll get a smooth curve peaking in the summer months and reaching its lowest in the winter months. Now if you take a single city in that country (eg London), you'll find a curve that is of almost indistinguishable shape, just with different numbers. Take a single district of the city, and the curve will be a similar shape, but it will start to get "noisy" (your line will be jagged). Decrease to a single street (make it a large one) and the pattern will be barely recognisable, although you'll probably spot it because you've just been looking at the curve on a similar scale. Now zoom down to the level of a single house... the pattern is gone, because there aren't enough people.
In physics, this is the difference between life on the quantum scale and the macro scale. Everything we touch and see is a collection of tiny units of matter or energy, and each of those units acts as an independent, unpredictable agent, but there are so many of these units that they appear to us to function as a continuous scale, describing a probability distribution like in the example of birthdays. Why should I care whether any individual photon hits my eye if the faintest visible star in the night sky delivers 1700 photons per second. The computer screen I'm staring at now is probably bombarding me with billions as we speak. The individual is irrelevant.
Again, let's look at birthdays.
Imagine we've got 365 people in a room, and for convenience we'll pretend leap years don't exist (and also imagine that there aren't any seasonal variations in births and deaths).
What is the average number of people born on any given day?
Easy: one.
And what is the probability that there's someone born on every day of the year?
This one isn't immediately obvious, but if you know your stats, it's easy to figure out.
First we select one person at random.
He has a birthday that we haven't seen yet, so he gets a probability of 1, whatever his birthday is OK.
Now we have 364 people, and 364 target days out of 365.
Select person 2 -- the chance he has a birthday we haven't seen yet is 364/365.
Person 3's chance of having a birthday we haven't seen is 363/365... still high.
...
but person 363's chance is 3/365, person 364's chance is 2/365 and person 365's chance is 1/365.
To get the final probability of 1-person-per-day-of-the-year, we need to multiply these:
1 x 364/365 x 363/365 x ... x 3/365 x 2/365 x 1/365
Mathematically, thats
365! / (365^365)
or
364! / (365^364)
It's so astronomically tiny that OpenCalc refuses to calculate the answer, and the Windows Calculator app tells me it's "1.455 e-157" -- for those of you who don't know scientific notation, that "e minus" number is the number of zeros, so fully expanded, that would be:
0.000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 145 5
...unless I've miscounted my zeros, but you can see clearly that the chances of actually getting everyone with different birthdays is pretty close to zero. It ain't gonna happen.
A better statistician than me would be able to predict with some accuracy the number of dates with multiple birthdays, the number of dates without birthdays etc, but not which ones they would be.
The reason we have these gaps is that we two relatively high numbers and one relatively low number. That gives us a predicatable probability distribution with predictable gaps.
Now, Martin Weller tries to look at and comment on 3 random blog posts when he connects. Now let's imagine that this was a policy that everyone adhered to without exception (either above or below).
OK, so let's imagine our MOOC has 365 students (so that we can continue using the same numbers as above) and that for each post we put up we comment on 3. What's the chances that every blog post gets at least one comment?
Well this is pushing my memory a bit, cos I haven't done any serious stats since 1998, and it's all this n-choose-r stuff. Oooh.... In fact, I can't remember how to do it, but it's still going to be a very small probability indeed.
In order to get a reasonable chance of everybody getting at least one comment, you need to get rid of small numbers entirely, and require high volumes of feedback for each user. But even though we're talking unfeasibly high volumes here, it still doesn't guarantee anything.
Now in theory, their approach is perfect, because even if they don't achieve a 100% completion rate/0% dropout rate, you should be able to get something back. However, their execution was flawed in a way which convinces me that Andrew Ng didn't write the algorithm!
You see, the peer reviews appear essentially to be dealt out like a hand of cards -- when the submission deadline is reached, all submissions are assigned out immediately. Each submitter gets dealt 5 different assignments, each assignment is dealt out to 5 different reviewers. It doesn't matter when you log in to do the review -- the distribution of assignments is already a fait accompli.
How did I come to this conclusion? Well, after the very first assignment in the Berklee songwriting course, I saw a post on the course forums from someone who had received no feedback on his assignment. I immediately checked mine: a full set of 5 responses.
Even though they had distributed the assignments for peer review in a less random fashion, they did nothing to account for the dropout rate, even though the dropout rate is reportedly predictably similar across all MOOCs -- and not only similar, but very high in the first week or two. So statistically speaking, gaps were entirely predictable.
Informal course organisation will never work, no matter how "massive" the scale, because the laws of statistics don't work like people expect them to -- they don't guarantee that everyone gets something -- they guarantee that someone gets nothing.
This morning, I received a "welcome to week 4" message from the organiser of the OU MOOC Open Education. He comes across as a really nice guy on both email and video, which makes it a lot more difficult to criticise, but one thing he said really caught my attention as indicative on the problems with the "informal education" model that the connectivist ideologues* profess. (* I refuse to call them theorists until they provide a more substantial scientific backing for their standpoint -- until then, it's just ideology.)
So, the quote:
"As always try to find time to connect with others. One way of doing this I've found useful is to set aside a small amount of time (15 minutes say) and just read 3 random posts from the course blog h817open.net and leave comments on the original posts."Don't get me wrong, I commend him on this. Many MOOC organisers take a step back and stay well clear of student contributions, for fear of getting caught up in a time sink. No, my problem is the word "random" coupled with the number "3".
The cult of random
There is a scientific truism about random: when the numbers are big enough, random stuff acts predictably. You can predict the buying patterns of a social group of thousands well enough to say that "of this 10000 people, 8000 will have a car", or the like.The best examples, though, come in the realm of birthdays.
If you take the birthdays (excluding year) of the population of a country (eg the UK) and plot a graph, you'll get a smooth curve peaking in the summer months and reaching its lowest in the winter months. Now if you take a single city in that country (eg London), you'll find a curve that is of almost indistinguishable shape, just with different numbers. Take a single district of the city, and the curve will be a similar shape, but it will start to get "noisy" (your line will be jagged). Decrease to a single street (make it a large one) and the pattern will be barely recognisable, although you'll probably spot it because you've just been looking at the curve on a similar scale. Now zoom down to the level of a single house... the pattern is gone, because there aren't enough people.
In physics, this is the difference between life on the quantum scale and the macro scale. Everything we touch and see is a collection of tiny units of matter or energy, and each of those units acts as an independent, unpredictable agent, but there are so many of these units that they appear to us to function as a continuous scale, describing a probability distribution like in the example of birthdays. Why should I care whether any individual photon hits my eye if the faintest visible star in the night sky delivers 1700 photons per second. The computer screen I'm staring at now is probably bombarding me with billions as we speak. The individual is irrelevant.
But massive means massive numbers, right?
I know what you're thinking -- with MOOC participants typically numbering in the thousands, these macro-scale probabitimajigs should probably cut in and start giving us predictability. Well yes, they do, but not in the way you might expect, because MOOCs deal with both big numbers and small numbers.Again, let's look at birthdays.
Imagine we've got 365 people in a room, and for convenience we'll pretend leap years don't exist (and also imagine that there aren't any seasonal variations in births and deaths).
What is the average number of people born on any given day?
Easy: one.
And what is the probability that there's someone born on every day of the year?
This one isn't immediately obvious, but if you know your stats, it's easy to figure out.
First we select one person at random.
He has a birthday that we haven't seen yet, so he gets a probability of 1, whatever his birthday is OK.
Now we have 364 people, and 364 target days out of 365.
Select person 2 -- the chance he has a birthday we haven't seen yet is 364/365.
Person 3's chance of having a birthday we haven't seen is 363/365... still high.
...
but person 363's chance is 3/365, person 364's chance is 2/365 and person 365's chance is 1/365.
To get the final probability of 1-person-per-day-of-the-year, we need to multiply these:
1 x 364/365 x 363/365 x ... x 3/365 x 2/365 x 1/365
Mathematically, thats
365! / (365^365)
or
364! / (365^364)
It's so astronomically tiny that OpenCalc refuses to calculate the answer, and the Windows Calculator app tells me it's "1.455 e-157" -- for those of you who don't know scientific notation, that "e minus" number is the number of zeros, so fully expanded, that would be:
0.000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 145 5
...unless I've miscounted my zeros, but you can see clearly that the chances of actually getting everyone with different birthdays is pretty close to zero. It ain't gonna happen.
A better statistician than me would be able to predict with some accuracy the number of dates with multiple birthdays, the number of dates without birthdays etc, but not which ones they would be.
The reason we have these gaps is that we two relatively high numbers and one relatively low number. That gives us a predicatable probability distribution with predictable gaps.
Now, Martin Weller tries to look at and comment on 3 random blog posts when he connects. Now let's imagine that this was a policy that everyone adhered to without exception (either above or below).
OK, so let's imagine our MOOC has 365 students (so that we can continue using the same numbers as above) and that for each post we put up we comment on 3. What's the chances that every blog post gets at least one comment?
Well this is pushing my memory a bit, cos I haven't done any serious stats since 1998, and it's all this n-choose-r stuff. Oooh.... In fact, I can't remember how to do it, but it's still going to be a very small probability indeed.
In order to get a reasonable chance of everybody getting at least one comment, you need to get rid of small numbers entirely, and require high volumes of feedback for each user. But even though we're talking unfeasibly high volumes here, it still doesn't guarantee anything.
Random doesn't work!
You cannot, therefore, organise a "course" relying entirely on informal networking and probability effects -- there must be some active guidance. This is why Coursera (founded by experts in this sort of applied statistics) uses a more formalised type of peer interaction.The Coursera model
Coursera's model is to make peer interaction into peer review, where students have to grade and comment on 5 classmates' work in order to get a grade for their own work. The computer doesn't do this completely randomly, though, and assigns review tasks with a view to getting 5 reviews for each and every assignment submitted.Now in theory, their approach is perfect, because even if they don't achieve a 100% completion rate/0% dropout rate, you should be able to get something back. However, their execution was flawed in a way which convinces me that Andrew Ng didn't write the algorithm!
You see, the peer reviews appear essentially to be dealt out like a hand of cards -- when the submission deadline is reached, all submissions are assigned out immediately. Each submitter gets dealt 5 different assignments, each assignment is dealt out to 5 different reviewers. It doesn't matter when you log in to do the review -- the distribution of assignments is already a fait accompli.
How did I come to this conclusion? Well, after the very first assignment in the Berklee songwriting course, I saw a post on the course forums from someone who had received no feedback on his assignment. I immediately checked mine: a full set of 5 responses.
Even though they had distributed the assignments for peer review in a less random fashion, they did nothing to account for the dropout rate, even though the dropout rate is reportedly predictably similar across all MOOCs -- and not only similar, but very high in the first week or two. So statistically speaking, gaps were entirely predictable.
What Coursera should have done....
The problem was this "dealing out" in advance. If they'd done the assignment redistribution on a just-in-time basis. When a reviewer starts a review, the system should assign one with the minimum number of reviews. No-one should receive a second review until everybody has received one, and I definitely shouldn't have received 5 when some others were reporting only getting 3, 2 or even none."Average" is always a bad measure
As a society, we've got blasé about our statistics, and we often fall back on averages, and by that I mean "mean". But if the average number of reviews per assignment is 4.8 of a targeted 5, that's still not success and it's still not if it doesn't mean that everyone got either 4 or 5.Informal course organisation will never work, no matter how "massive" the scale, because the laws of statistics don't work like people expect them to -- they don't guarantee that everyone gets something -- they guarantee that someone gets nothing.
27 March 2013
Suitability of MOOCs - H817 Activity 12
The OU free MOOC Open Education set the following question as activity 12:
And for now, I'll restrict myself to the type of MOOC proposed by Cormier, Siemens etc, the "connectivist" MOOC.
So I'll answer "yes" and "yes" and "no" and "no".
One of the bits of material supporting this activity was a video interview with the aforementioned Mr.s Cormier and Siemens.
What really jumped out at me was that little after a minute into it, George Siemens basically says that the system emerged from how they were running online conferences. Sound familiar? Well, a few weeks ago I came to the conclusion that the MOOC had far more in common with a conference than a "course".
So it's utterly trivial to ask whether the MOOC has a place in any given field: if there are conferences in that field, a conference-type MOOC can work.
So that's "yes" and "yes". Now onto "no" and "no".
I'll start with a quote from Isaac Asimov that I picked up from somewhere in the last week while working through blog posts on MOOCs:
But in the MOOC setting, it's particularly salient. The whole idea of connectivism is to learn from each other... but we're not experts. Everything I've read or heard from Cormier or Siemens to date seems to mention but quickly gloss over the fact that their MOOCs have focused on educational technology, a field with many informed practitioners, but no confirmed experts. In fact, on of the papers mentioned in the disastrous Fundamentals of Online Education Coursera module described online education as being "at the buzzword stage", a thin euphemism for the fact that it's all opinion and no "knowledge". And that's the space that conferences have always occupied: the point where we're sitting on the boundaries of the state of the art, where informed practitioners of roughly equal knowledge try to contemplate and push those boundaries.
But when there is an expert, why should we rely on the knowledge of peers, who may in fact turn out to be wrong?
Nowhere can this be more clear-cut than in the computer field (or at least "the discrete mathematics field", of which CS is a subset).
At the level of programming, there can be no subjective discussion about the best way of carrying out a given operation, because the methods can be empirically measured. We can measure execution time, we can measure memory constraints, we can measure accuracy of results. We get a definite right and wrong answer. Yes, we can devise collaborative experiments where we pool our resources and share our data to find out what those right and wrong answers are, and in computer science courses we often do, but that serves not to teach the answer, but to teach the process of evaluating the efficiency of an algorithm or piece of software.
We do not generate more knowledge of how the computer works by discussing, only of how we work with it.
So there's my first "no", but this is not really specific to computer science, because in any undergraduate field, you teach/learn mostly the stable, established knowledge of the field. Very little in an undergraduate syllabus is really open to much subjectivity in terms of knowledge, and in arts degrees, the subjectivity is restricted pretty much to the application of established knowledge.
Everyone discussing MOOCs at the moment seems to be talking about "HE" (higher education -- ie. universities) and not acknowledging that fundamental split between undergrad and postgrad.
So I've stated that no undergrad stuff can follow a connectivist approach, is it still worth saying anything about language specifically?
I think so.
Because language learning, more than any other field of education, can be scuppered by overenthusiastic learners -- the biggest obstacle in any language course is the presence of learners: how can I learn a language by hanging around with a bunch of people who don't speak the language? And yet, for most of us these courses are vital if we are ever to learn a language.
And I myself have benefited greatly from informal networks of learners offering mutual support, so why not a MOOC? Because the informal networks I have benefited from are of vastly different levels, so there's always been someone with some level of "expertise" above you. But once you formalise into a "course", you're suddenly encouraging a group without that differentiation; a group of roughly equivalent level. An overly confident error pushed by one participant can become part of the group's ideolect -- a mythological rule that through the application of collective ignorance crowds out the genuine rule. Without sufficient expert oversight, how is this ever to be corrected?
A language MOOC would most likely be of far less use than either traditional classes or existing informal methods....
Before we examine MOOCs in more detail, briefly consider if the MOOC approach could be adopted in your own area of education or training. Post your thoughts in your blog and then read and comment on your peers’ postings.Now, just which field should I address? Computer science or language learning? How about both?
And for now, I'll restrict myself to the type of MOOC proposed by Cormier, Siemens etc, the "connectivist" MOOC.
So I'll answer "yes" and "yes" and "no" and "no".
One of the bits of material supporting this activity was a video interview with the aforementioned Mr.s Cormier and Siemens.
What really jumped out at me was that little after a minute into it, George Siemens basically says that the system emerged from how they were running online conferences. Sound familiar? Well, a few weeks ago I came to the conclusion that the MOOC had far more in common with a conference than a "course".
So it's utterly trivial to ask whether the MOOC has a place in any given field: if there are conferences in that field, a conference-type MOOC can work.
So that's "yes" and "yes". Now onto "no" and "no".
I'll start with a quote from Isaac Asimov that I picked up from somewhere in the last week while working through blog posts on MOOCs:
“Anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that 'my ignorance is just as good as your knowledge.'”This could have been written for Web 2.0. (No further explanation needed.)
But in the MOOC setting, it's particularly salient. The whole idea of connectivism is to learn from each other... but we're not experts. Everything I've read or heard from Cormier or Siemens to date seems to mention but quickly gloss over the fact that their MOOCs have focused on educational technology, a field with many informed practitioners, but no confirmed experts. In fact, on of the papers mentioned in the disastrous Fundamentals of Online Education Coursera module described online education as being "at the buzzword stage", a thin euphemism for the fact that it's all opinion and no "knowledge". And that's the space that conferences have always occupied: the point where we're sitting on the boundaries of the state of the art, where informed practitioners of roughly equal knowledge try to contemplate and push those boundaries.
But when there is an expert, why should we rely on the knowledge of peers, who may in fact turn out to be wrong?
Nowhere can this be more clear-cut than in the computer field (or at least "the discrete mathematics field", of which CS is a subset).
At the level of programming, there can be no subjective discussion about the best way of carrying out a given operation, because the methods can be empirically measured. We can measure execution time, we can measure memory constraints, we can measure accuracy of results. We get a definite right and wrong answer. Yes, we can devise collaborative experiments where we pool our resources and share our data to find out what those right and wrong answers are, and in computer science courses we often do, but that serves not to teach the answer, but to teach the process of evaluating the efficiency of an algorithm or piece of software.
We do not generate more knowledge of how the computer works by discussing, only of how we work with it.
So there's my first "no", but this is not really specific to computer science, because in any undergraduate field, you teach/learn mostly the stable, established knowledge of the field. Very little in an undergraduate syllabus is really open to much subjectivity in terms of knowledge, and in arts degrees, the subjectivity is restricted pretty much to the application of established knowledge.
Everyone discussing MOOCs at the moment seems to be talking about "HE" (higher education -- ie. universities) and not acknowledging that fundamental split between undergrad and postgrad.
So I've stated that no undergrad stuff can follow a connectivist approach, is it still worth saying anything about language specifically?
I think so.
Because language learning, more than any other field of education, can be scuppered by overenthusiastic learners -- the biggest obstacle in any language course is the presence of learners: how can I learn a language by hanging around with a bunch of people who don't speak the language? And yet, for most of us these courses are vital if we are ever to learn a language.
And I myself have benefited greatly from informal networks of learners offering mutual support, so why not a MOOC? Because the informal networks I have benefited from are of vastly different levels, so there's always been someone with some level of "expertise" above you. But once you formalise into a "course", you're suddenly encouraging a group without that differentiation; a group of roughly equivalent level. An overly confident error pushed by one participant can become part of the group's ideolect -- a mythological rule that through the application of collective ignorance crowds out the genuine rule. Without sufficient expert oversight, how is this ever to be corrected?
A language MOOC would most likely be of far less use than either traditional classes or existing informal methods....
# I talk to the MOOCs, but they don't listen to me...
When I was studying languages with the OU, I found it very difficult to motivate myself to do most of the task. These tasks I would happily do in a classroom, but on my own, I couldn't be bothered.
What was the difference? Two things: one, in the classroom, if you don't do the task, you just sit there waiting for the others to finish -- you don't actually get the time back; two, normally your work will be examined by someone else -- either the teacher or a classmate, and if someone reads or hears your work, it has a purpose. Even if only half of your classwork is ever read or heard, it at least provides some kind of motivation.
But when the book I was reading on my told me to write 200 words on my opinion of the treatment of minority languages in Spain, I knew I could be doing something else with my time, and I couldn't be bothered sitting down and writing something no-one else would ever read.
MOOCs, they would have us believe, address this, by making sure you have peers available at all times to read and comment. Sadly, there's no guarantees, with some postings to group forums getting lots of views and/or comments, and some getting none at all. The act of writing becomes an act of uncertainty -- it's like talking to the darkness without knowing whether or not there's actually anyone there.
I don't know about you, but this doesn't really motivate me to write much. The latest task description:
Looking at the writing style of many of my peers, I'm not the only one with these doubts. More than a few of the blog posts barely classify as prose, instead being little more than the writer's personal lecture notes.
This creates something of a death-spiral. Because some of the blog posts don't lend themselves to reading, people don't read them, and don't comment on them. This discourages them from viewing the blog aggregator, which means they don't see and don't comment on the genuinely readable posts, leading authors to become despondent about the lack of views, leading them to write without the expectation of gaining a readership, which leads to them not putting the effort in to make their posts readable, so people don't read them....
And yet, when we eventually tire of this and give up, the guys behind the MOOC don't view it as a pedagogical failure -- they shrug their shoulders and talk about learning choice and learner independence, and say that by leaving the course we're exercising those characteristics they want most to instill in us.
But I don't take courses to learn learner independence. I take courses to get expert guidance to aid me in the acquisition of new domain knowledge, because while I can operate adequately as an independent learner, expert guidance gets me there quicker.
What was the difference? Two things: one, in the classroom, if you don't do the task, you just sit there waiting for the others to finish -- you don't actually get the time back; two, normally your work will be examined by someone else -- either the teacher or a classmate, and if someone reads or hears your work, it has a purpose. Even if only half of your classwork is ever read or heard, it at least provides some kind of motivation.
But when the book I was reading on my told me to write 200 words on my opinion of the treatment of minority languages in Spain, I knew I could be doing something else with my time, and I couldn't be bothered sitting down and writing something no-one else would ever read.
MOOCs, they would have us believe, address this, by making sure you have peers available at all times to read and comment. Sadly, there's no guarantees, with some postings to group forums getting lots of views and/or comments, and some getting none at all. The act of writing becomes an act of uncertainty -- it's like talking to the darkness without knowing whether or not there's actually anyone there.
I don't know about you, but this doesn't really motivate me to write much. The latest task description:
Before we examine MOOCs in more detail, briefly consider if the MOOC approach could be adopted in your own area of education or training. Post your thoughts in your blog and then read and comment on your peers’ postings.Well... who am I writing it to? Who's going to read it? Is anyone actually going to see it before it rolls off the bottom of the monolithic, uncategorised course blog aggregator?
Looking at the writing style of many of my peers, I'm not the only one with these doubts. More than a few of the blog posts barely classify as prose, instead being little more than the writer's personal lecture notes.
This creates something of a death-spiral. Because some of the blog posts don't lend themselves to reading, people don't read them, and don't comment on them. This discourages them from viewing the blog aggregator, which means they don't see and don't comment on the genuinely readable posts, leading authors to become despondent about the lack of views, leading them to write without the expectation of gaining a readership, which leads to them not putting the effort in to make their posts readable, so people don't read them....
And yet, when we eventually tire of this and give up, the guys behind the MOOC don't view it as a pedagogical failure -- they shrug their shoulders and talk about learning choice and learner independence, and say that by leaving the course we're exercising those characteristics they want most to instill in us.
But I don't take courses to learn learner independence. I take courses to get expert guidance to aid me in the acquisition of new domain knowledge, because while I can operate adequately as an independent learner, expert guidance gets me there quicker.
24 March 2013
Evaluating Open Education Resources (H817)
I'm getting rapidly disillusioned with the Open University's MOOC/non-MOOC Open Education. After kicking off with a course "reading" that was a 77 slide PowerPoint file with no speaker notes, in week 2 they set a long reading from a decade ago, on a topic called "Learning Objects". Now, it's not the length of the post in itself that bothers me, and the age is not a problem as this notion was a significant stepping stone to the open education systems of today... what winds me up is that after the link to the article, there was a little button marked "reveal" comment. After the link. So you would assume, wouldn't you, that it was to be read after reading the article... which is what I did. Here is the content of the hidden comment in full:
This week's activities then follow on with one of the most spectacularly vague tasks ever, and judging by the stuff coming up on the course blog aggregator, I'm not the only person who thinks so. Our task is to look at several repositories of "open education resources" (OERs) and evaluate the suitability of the material presented for assembling a course on "digital skills".
I'm presuming that they've chosen the task title "digital skills" to allow it to be an open task, but they've taken the original MOOC philosophy to its erroneous ultimate conclusion. The philosophy of MOOCs (as embodied in change.ca) is the idea of learner independence, and the notion that learners work better when they can choose what to work towards, but yet unrestricted choice has been shown to be absolutely crippling, because with open choice comes indecision. (If you're interested in this idea, check out Barry Schwartz's TED talk The Paradox of Choice.)
Consider also that many of the great artists imposed limits on themselves, such as Pablo Picasso's famous "blue period" (not that I personally rate Picasso's work much), in order to stimulate extra creativity.
But here I am with an excruciatingly vague task description, and there's nothing in the task to force me to narrow down and focus on a particular aspect of the large potential space of meaning before I am expected to wade through gigabytes of texts and videos looking for things that are specifically relevant or useful.
And the course to date hasn't given us any real guidance on how to evaluate the usefulness and applicability of the material anyway. And we're back to this idea that there's no rules, and that individual creativity and "engagement" with material will show us the way, throwing out all the hard-learned lessons in pedagogy, instructional design and other closely related fields.
It is far easier to do a complex task by following a defined process than to try to intuit the process by attempting to complete the task. Early guidance can develop good patterns of activity that are internalised over time and become automatic.
Note: Downes goes into detail on many aspects that are not necessary for this course. You do not need to read the article in detail – your aim is to gain an understanding of what learning objects were and why they were seen as important.... and you'll see why I was unhappy. It's utterly sloppy design to leave you reading the whole thing before telling you not to!
This week's activities then follow on with one of the most spectacularly vague tasks ever, and judging by the stuff coming up on the course blog aggregator, I'm not the only person who thinks so. Our task is to look at several repositories of "open education resources" (OERs) and evaluate the suitability of the material presented for assembling a course on "digital skills".
I'm presuming that they've chosen the task title "digital skills" to allow it to be an open task, but they've taken the original MOOC philosophy to its erroneous ultimate conclusion. The philosophy of MOOCs (as embodied in change.ca) is the idea of learner independence, and the notion that learners work better when they can choose what to work towards, but yet unrestricted choice has been shown to be absolutely crippling, because with open choice comes indecision. (If you're interested in this idea, check out Barry Schwartz's TED talk The Paradox of Choice.)
Consider also that many of the great artists imposed limits on themselves, such as Pablo Picasso's famous "blue period" (not that I personally rate Picasso's work much), in order to stimulate extra creativity.
But here I am with an excruciatingly vague task description, and there's nothing in the task to force me to narrow down and focus on a particular aspect of the large potential space of meaning before I am expected to wade through gigabytes of texts and videos looking for things that are specifically relevant or useful.
And the course to date hasn't given us any real guidance on how to evaluate the usefulness and applicability of the material anyway. And we're back to this idea that there's no rules, and that individual creativity and "engagement" with material will show us the way, throwing out all the hard-learned lessons in pedagogy, instructional design and other closely related fields.
It is far easier to do a complex task by following a defined process than to try to intuit the process by attempting to complete the task. Early guidance can develop good patterns of activity that are internalised over time and become automatic.
21 February 2013
What do we learn when we learn by doing?
The University of Georgia MOOC on
online education was starting to look very interesting before it
suddenly folded. On one hand, it covered a lot of interesting
theoretical pedagogy. On the other, the practical pedagogy of the
course itself seemed sub-optimal. As I only got one week in before
they closed up, so I can't really say all that much about it.
First up, Schank fails to address in the first 25% of his essay (which is approximately what I read) the biggest concern that has always been raised against the idea of whole-subject-learning, learn-by-doing or whatever label you chose to attach to the idea: lack of breadth. (A quick skim suggests that that it's not addressed further down, so if it is, it's clearly not given the prominence it deserves. Besides, as it is the single most important concern of most critics, you need to address it early or you lose our attention.) A single task will only lead to a single solution, with some limited exploration of alternative strategies. It teaches the students how to cope with a situation where they lack knowledge, rather than minimising those situations by providing them with knowledge.
Many graduates, particularly CS grads, will immediately be able to identify with me when I say this isn't true. When you leave university, you typically know how to do things correctly, but once you get into the real world, “correctly” is too time-consuming, too expensive. (Except in safety-critical roles, such as avionics or military systems.) In the short term, this is OK – pragmatically, that's the way it's got to be.
In the longer term, though, things start to go haywire. We get so habituated to our way of doing things that we soon learn to identify it as the “right way” of doing things. Someone comes along with a new way, a better idea (probably a recent grad) and we dismiss the idea as wishful thinking.
In computers more than any other field, this problem is easily apparent. I remember suggesting a very simple change to a database system and being told “you can't do that with computers”. A) You can do pretty much anything with computers. B) I was talking about something that is built into the database software we were using!!! Yes, a standard feature of the software, and the team's top “expert” didn't know about it; and because he was the expert and didn't know about it, it was as though it didn't exist.
Now to answer the question that started this article, what do we learn when we learn-by-doing?
One academic the course has introduced
me to is Roger C. Schank. Roger is an AI lecturer who later
specialising in learning. Schank's big idea is that of learning
by doing. It's a simple and
compelling idea – he claims traditional classrooms don't work
because they are far too theoretical and divorced from any real
“need” to learn.
There is certainly
a lot of truth in this. A book of drills (whether it be arithmetic,
grammar, or rote historical facts) does little to demonstrate either
why the information is important or the contexts to which it is
relevant.
The
title of this post is lifted straight from a
report Schank wrote in 1995 for his university. It's a huge
piece of writing – almost thirty thousand words long – and to be
perfectly honest with you, I didn't read it to the end. But why
would I? It's called a “technical report”, but in truth it's
little more than an oversized opinion piece. There's no technical
data: he does not appeal to research, he does not appeal to figures,
he just makes unsupported statements. What is most telling is that
there are only 5 citations in his bibliography, and four of these are
to himself.
As he argues
without evidence, I feel perfectly entitled to dismantle his argument
without citations. Besides, he's supposed to be an academic, and I'm
just a blogger!
First up, Schank fails to address in the first 25% of his essay (which is approximately what I read) the biggest concern that has always been raised against the idea of whole-subject-learning, learn-by-doing or whatever label you chose to attach to the idea: lack of breadth. (A quick skim suggests that that it's not addressed further down, so if it is, it's clearly not given the prominence it deserves. Besides, as it is the single most important concern of most critics, you need to address it early or you lose our attention.) A single task will only lead to a single solution, with some limited exploration of alternative strategies. It teaches the students how to cope with a situation where they lack knowledge, rather than minimising those situations by providing them with knowledge.
There's been
research into various aspects of this. I've seen reports claiming to
prove schoolchildren given a whole-subject education have a much
narrower set of knowledge. This should be pretty obvious, I would
have thought... so maybe I'm just displaying confirmation bias and
reading the stuff that supports my view.
Of course, there
was the study that showed that doctors trained by case-study rather
than theory performed better in diagnosing patients, but as I recall
it, this was tempered by the fact that their diagnoses took a lot of
time and money, because they tended to follow a systematic approach
of testing for increasingly less common problems or symptoms. A
doctor trained on a theory-based course was more likely to formulate
a reasonable first hypothesis and start the testing process somewhere
in the middle. The conclusions we can take from this are mixed. You
can claim that the traditionally-trained doctor is better at
diagnosing on the grounds that he can do it with less testing; or you
can claim that only the end result matters, and the
case-study-trained doctor is better. You can argue that minimising
mistakes is the ultimate goal, or you can argue that the time taken
in avoiding mistakes is too great in that it delays treatment for
other patients.
So,
anyway... Schank does nothing to convince me that it is possible to
cover the breadth of a traditional course in learn-by-doing, but
there is a video on Schank's
site of a course he designed at Carnegie-Mellon about ten years
ago, and it alludes to what I think is the only real argument about
it. One of the senior course staff and one or two of the students
talk about the idea of forgetting everything that's been taught in a
traditional course. If challenged, would that be the basis of
Schank's response? The logic certainly appeals: if you're not really
learning anything in a traditional academic course, the breadth of
what is covered is irrelevant.
But is anything
ever truly forgotten? I've recently gone back into coding after a
long hiatus – even when I worked in IT, I never had any serious
programming to do. But when I come up against a problem, I find
myself thinking back to mostly-forgotten bits of theory from my CS
degree days, and looking them up on the net. Tree-traversal
algorithms, curried functions, delayed evaluation... But if I had
never encountered these ideas before, how would I even know to look
for them?
This is not a mere
theoretical problem. I'm not usually one to complain about “ivory
tower academics”, but goddamn it, Schank's brought this on himself.
And I quote:
"One of the places where real life learning takes place is in the
workplace, "on the job." The reason for this seems simple
enough. Humans are natural learners. They learn from everything they
do. When they watch television, they learn about the day's events.
When they take a trip, they learn about how to get where they are
going and what it is like to be there. This constant learning also
takes place as one works. If you want an employee to learn his job,
then, it stands to reason that the best way is to simply let him do
his job. Motivation is not a problem in such situations since
employees know that if they don't learn to do their job well, they
won't keep it for long.
Most employees are interested in
learning to their jobs better. One reason for this is, of course,
potential monetary rewards. But the real reason is much deeper than
that. If you do something often enough, you get better at it --
simple and obvious. When people really care about what they are
doing, they may even learn how to do their jobs better than anyone
had hoped. They themselves wonder how to improve their own
performance. They innovate. Since mistakes are often quite jarring to
someone who cares about what they are doing, people naturally work
hard to avoid them. No one likes to fail. It is basic to human nature
to try to do better and this means attempting to explain one's
failures well enough so that they can be remedied. This
self-correcting behavior can only take place when one has been made
aware of one's mistakes and when one cares enough to improve. If an
employee understands and believes that an error has been made, he
will work hard to correct it, and will want to be trained to do
better, if proper rewards are in place for a job well done. "
Many graduates, particularly CS grads, will immediately be able to identify with me when I say this isn't true. When you leave university, you typically know how to do things correctly, but once you get into the real world, “correctly” is too time-consuming, too expensive. (Except in safety-critical roles, such as avionics or military systems.) In the short term, this is OK – pragmatically, that's the way it's got to be.
In the longer term, though, things start to go haywire. We get so habituated to our way of doing things that we soon learn to identify it as the “right way” of doing things. Someone comes along with a new way, a better idea (probably a recent grad) and we dismiss the idea as wishful thinking.
In computers more than any other field, this problem is easily apparent. I remember suggesting a very simple change to a database system and being told “you can't do that with computers”. A) You can do pretty much anything with computers. B) I was talking about something that is built into the database software we were using!!! Yes, a standard feature of the software, and the team's top “expert” didn't know about it; and because he was the expert and didn't know about it, it was as though it didn't exist.
More generally,
the problem becomes visible when a programmer switches languages. A
C programmer who learns Python normally ends up writing his Python
code using the features that most resemble those of C. The unique
features of Python are designed to overcome specific problems and
difficulties with a lower-level language such as C, but to the expert
C coder, these aren't really “problems”, because he long ago
rationalised them away and learned to cope with them. He doesn't
realise there is a “problem”, so he doesn't have any reason to go
looking for a solution. Even within languages, some people always do
things the “hard way” because they've simply never thought to
look for an “easy way”.
So Schank is being
hopelessly naïve. The key feature of expertise is automaticity –
experts have internalised enough that they don't have to think about
what they're doing. They close their minds to learning, because
there's more value in doing a job suboptimally but quickly than in
doing it optimally but slowly. People need to develop breadth before
they become experts – before they become “set in their ways”.
Now to answer the question that started this article, what do we learn when we learn-by-doing?
We learn to be
adequate, not brilliant. We learn to get by, not to excel. We learn
to stop thinking. We learn, paradoxically, to stop learning.
15 February 2013
If a MOOC isn't a course...?
After writing
the post on the word “course”, I started pontificating on what a
constructionist MOOC is if it's not a “course”.
And it is, “most importantly”:
Cue much philosophising on my part. Is a course an “event”? It doesn't feel like one. Is that because most courses are suboptimal, and therefore unexciting? I could accept that as a valid argument, but I personally have enjoyed many courses that didn't feel like “events” to me, so I was unconvinced.
It's an event “where people can get together and work and talk about it [a topic] in a structured way.” It certainly doesn't sound like a course.
And so it was that one morning earlier this week, I found myself lying in bed, hoping to get another half-hour nap in, and this image in my head:
Online conferences have been tried before – live video streaming of a series of scheduled speakers and open “seminars” on text chat – but they've never proved popular.
This, I would argue, is because they attempted to replicate the conference format lock, stock and barrel. But the conference format is a compromise – most importantly, conferences are squeezed into a short space of time because of logistics. You couldn't spread a conference over a couple of months, because people would have to travel back and forth, and it would get very expensive very quickly. This means that the seminar sessions are short and limited – you can't research your response to a talk in the toilet break between the speech itself and the seminar on the topic.
What we have in the Cormier MOOC model is the conference with those time restrictions stripped away. The talks are available wherever you are, without travel, so you can take your time to discuss and research your reactions to them. You're no longer forced to choose between two equally interesting talks just because the scheduler put them on at the same time.
The MOOC guys came up with a great idea, but not only didn't give a clear picture of what it is, what it does, or what it replaces; they claim it's something it's not, it does something it doesn't, and replaces something that it is completely different from.
So I started
rewatching Cormier's video What is a MOOC.
Cormier says:
After all, as he
says, “it has facilitators, course materials, it has a start and an
end date, it has participants”
OK so far...
It's also:
…good so far....
And it is, “most importantly”:
Cue much philosophising on my part. Is a course an “event”? It doesn't feel like one. Is that because most courses are suboptimal, and therefore unexciting? I could accept that as a valid argument, but I personally have enjoyed many courses that didn't feel like “events” to me, so I was unconvinced.
It's an event “where people can get together and work and talk about it [a topic] in a structured way.” It certainly doesn't sound like a course.
And so it was that one morning earlier this week, I found myself lying in bed, hoping to get another half-hour nap in, and this image in my head:
I could also
picture the structure of the change.mooc.ca course, which appears to
consist of one guest speaker per week, and free discussion.
An event...
...where you
choose what material to engage with...
...comprised of a
series of guest speakers...
...where there are
no assignments...
...and you network
with like-minded people...
Eureka! A cMOOC
is a conference.
Online conferences have been tried before – live video streaming of a series of scheduled speakers and open “seminars” on text chat – but they've never proved popular.
This, I would argue, is because they attempted to replicate the conference format lock, stock and barrel. But the conference format is a compromise – most importantly, conferences are squeezed into a short space of time because of logistics. You couldn't spread a conference over a couple of months, because people would have to travel back and forth, and it would get very expensive very quickly. This means that the seminar sessions are short and limited – you can't research your response to a talk in the toilet break between the speech itself and the seminar on the topic.
What we have in the Cormier MOOC model is the conference with those time restrictions stripped away. The talks are available wherever you are, without travel, so you can take your time to discuss and research your reactions to them. You're no longer forced to choose between two equally interesting talks just because the scheduler put them on at the same time.
So the MOOC as a “distributed conference” is far better in many
respects than a physical conference. It's a good thing, it just isn't a course. And if conferences are now easier and cheaper to run, we may be able to replace some courses with conferences, but they're still not courses.
Do I feel stupid for rubbishing MOOCs previously, now that I've
realised this?
No. My biggest problem developing as a teacher is that people keep
suggesting techniques, methods, tools and strategies that “work
well” in the classroom, but they never tell me when or why to apply
them.
The MOOC guys came up with a great idea, but not only didn't give a clear picture of what it is, what it does, or what it replaces; they claim it's something it's not, it does something it doesn't, and replaces something that it is completely different from.
13 February 2013
Putting the cart before the course.
After getting in a discussion with Debbie Morrison on her blog Online Learning Insights, I came to
realise that we were misunderstanding each other over the vagueness
of the definition of what a MOOC really is, and I was going to write
a post about this, when I realised that in order to do that, I would
first have to consider what the meaning of the word “course” is.
This train of thought had actually been
idling in the station for a while, ever since I watched a video of atalk given by Roger Schank to staff at the World Bank. Something
Schank said didn't ring true – he described the word “course”
as imply a race, winners and losers. I wasn't happy with this
interpretation, but it wasn't until I got into the discussion with
Debbie that I realised why.
A course is not a race, although many
races take place on courses. No, a course is a route, a path. A
river finds its “course” to the sea.
Many in Western education look at the
Eastern tradition with some sense of awe. We are told that in the
East a teacher isn't a “teacher”, but “one who has walked the
path”, and a student isn't a “student” but a seeker of knowledge. But the question is: do the Easterners know this? The origin of our word
“course” shows that our system is built on the same philosophy,
but we're not aware of this. Maybe they are blind to the meaning of
their words, just as we are blind to the meaning of ours.
Now where does that leave us on the
meaning of MOOC?
Well, in a rather pedantic sense, the
MOOC as proposed by the people who coined the term is not a “course”
at all, because there is no set path whatsoever, which is in fact part of the
point of connectivist learning theory – the learning experience
(for wont of a better term) is driven and steered by the students,
with each student finding their own path through the information
presented. Cormier says in his video introduction to the idea of a
MOOC that there is “no right way to do the course, no single path.”
If course is synonymous with path, this is a paradox.
Is it useful to define this as “not a course”? You may not agree with me, but I think it's not only useful, but perhaps even essential.
Connectivism is just the latest combatant in an on-going ideological war between whole-subject and basic skills teaching. Why do I call this an ideological war, and not a pedagogical one? Because neither side really has much of an argument or evidence behind them.
In the real world, basic skills and whole-subject teaching are two ends of a spectrum of teaching styles, and most teachers use different parts of the spectrum at different times.
In general, education follows a progression that starts with the aim of teaching certain well-defined basic skills, then we start to use them in more complex environments. As we progress through our education, we build levels of abstraction over those basic skills.
The question is whether we can learn those basic skills and abstract skills and basic skills at the same time. I personally believe that we can, but that it is less efficient. If you put the cart before the horse, the horse can push it, but it would be more efficient pulling it behind.
But perhaps the reason that the MOOC as proposed by Dave Cormier doesn't really fit the term “course” is because what it aims to replace isn't really a “course” either. Cormier mentions “lifelong learning”, but “lifelong learning” is a term that is in itself ambiguous.
There are two sides to lifelong learning – there's what we'd traditionally call “adult education”, which can be stereotyped as evening classes offering high school or university-level classes to adults who dropped out of the education system at a young age; then there's continuing professional development (CPD) for people who are qualified and working in a degree-educated field. (Well, not two sides, per se, as there's a whole spectrum in between, but never mind.)
I really think that what Cormier did
was create something that was far more orientated towards that
highest level of abstraction: the qualified, experienced practitioner
who already had a well-developed framework for understanding the
material presented, and as I said in a previous post, at that stage
most professional development takes place in seminars, not in strict
“courses”.
The whole concept of informal learning is now firmly entrenched in the Scottish teaching system. Teachers are set targets of CPD points to acquire through the year. Many of these are given through traditional in-service training days and seminars, but teachers are expected to top these up with other things through personal initiative. There is a large catalogue of activities that qualify as optional points, even down to watching a television documentary on a subject related to your teaching field.
That informal online learning is
effective for people for whom informal offline learning is already
known to be effective should not be a surprise
One of the biggest influences on my thinking about education was Michel Thomas. Thomas studied psychology, and he wanted to study the learning process. He reportedly chose to teach languages for a very simple reason: languages provide the best opportunity to work with a student with zero starting knowledge of the subject being taught. Eliminating the variable of prior knowledge made reaching conclusions about the effectiveness of teaching easier, he reasoned.
The style of teaching he developed is
demonstrated in the courses he recorded before his death for Hodder.
(He recorded courses in Spanish, French, German and Italian. The
other languages released in his name are very different indeed.)
They are all examples of very tight control by the teacher, offering
a well-defined, clearly sign-posted learning path. As the courses he
produced are live recordings with genuine students, you can observe
him diverge from his preferred path as a reaction to things said by
the student (for example, when one student makes a mistake
conjugating a verb in Spanish and accidentally says an imperative,
Thomas is forced to introduce the imperative early, because he
doesn't want to tell the student that he's wrong), so there's a fair
degree of flexibility there, but we can see that there is a definite
“course” there.
The amount that the students on the recordings pick up even in just the first 2 hours is quite extraordinary – I've never seen anyone else achieve similar results
But he's working with absolute beginners, and beginners need a path to follow, they need a course. It would be a mistake, I believe, to try to use the term “course” to describe an undirected, pathless learning experience, as this leads to the conclusion that such a learning experience is a replacement for a genuine guided course, when I really think it's only something that can really be done once the student has finished with courses.
08 February 2013
The misunderstandings in MOOCs
I had another article lined up and ready to go, and then I started reading about MOOCs on-line, and in particular a lot of the reaction to the total collapse of Coursera's MOOC Fundamentals of on-line education (#foemooc), one of the biggest and most revealing ironies in history.
One article quoted a law professor as follows:
We are currently experiencing a similar bubble in Web 2.0. So-called investors* have ploughed money into companies like Facebook and Instagram, only to see stock prices suffer when the market suddenly realises that they have no business plan.
[* The modern stockmarket makes a mockery of the term "investor". "Investing" is supposed to mean putting money in so that a company can grow, but most high-profile IPOs, including Facebook and the like, are simply a change of ownership. The money is for the previous owner, not the business.]
Why do we want to replicate that model at a cost to our valued educational institutions?
I've commented on that side of things before, noting Udacity's move toward corporate sponsorship: MOOCs as adverts for a given vendor.
This MOOC-as-sponsorship model might even work to some extent for the universities too -- the University of Edinburgh's Coursera MOOC E-Learning and Digital Cultures (#edcmooc) is based on a module taken from one of their masters programmes, and it's possible that the increased exposure it gives the course among potential students will get back the cost of putting the MOOC version together and on-line. The risk, though, is that if too much material is available for free, the adverts will basically kill the market for the product.
One article in particular caught my attention. Over at the blog online learning insights, blogger and instructional designer Debbie Morrison wrote a post about three takeaways from the collapse of the Coursera Fundamentals of Online Education course.
Her first "takeaway" is, in my ever so humble opinion, completely and utterly wrong. Totally. Completely. Utterly. Let's take a look:
1) The instructional model is shifting to be student-centric, away from an institution or instructor-focused model.In a massive, open and online course with thousands of students, the instructor must relinquish control of the student learning process. The instructor-focused model is counter intuitive to the idea of a MOOC; in the MOOC model the student directs and drives his or her learning. The pedagogy used for traditional courses is not applicable to a course on a massive scale. With the Web as the classroom platform, students learn by making connections with various ‘nodes’ of content [not all provided by the instructor] on the Web, they aggregate content, and create knowledge that is assessed not by the instructor, but by peers or self. This pedagogy builds upon the constructivist theory, and more recently a theory developed by Downes and Siemens, the connectivist learning model.
Wrong... how so?
Debbie Morrison clearly knows a thing or two about building online courses; unlike me, she's actually written them. But when she claims that the a tight teacher-led course is "counter intuitive to the idea of a MOOC", she's projecting her views into a reality that is distinctly different. The origin of the MOOC is in the field of Computer Science and Artificial Intelligence. The original MOOCs were very much instructor-led, with very specific, well defined tasks for the student to follow.
Because the tasks were very tightly controlled, marking could be automated, and hundreds of thousands of students could be individually assessed by machine.
Morrison is mistakenly applying the standards for her own niche, the small-to-medium online course, to a very different animal.
Morrison's courses are comparable to a seminar-based course in more traditional education. Every generation has accepted that seminar-based courses are better than lecture-based ones, but that they're more work for the teacher because the uncontrolled variables lead to a massive space of potential outcomes and directions for the class.
Online education for small classes is a good realisation of the seminar-based course, because the online medium removes some of the time pressures on the instructor in terms of organisation, logistics and delivery; time which can be devoted instead to tailored group and individual feedback.
But when you've got one instructor and one teaching assistant for a class of 41,000 students (foe) or 1 instructor for a class of 150,000 (claimed by Ed Tech Magazine for a Udacity course), then that's out the window. A teacher cannot be a "guide" or a "facilitator" to a group that big. A teacher cannot even have a concept of the individual students as human beings (we're well beyond Dunbar's Number here).
A massive course cannot be student-centred because there are simply too many students.
And let's go back to the seminar-style class for a minute. Seminar-based classes are far more common in taught postgrad courses than in undergrad courses, and while you might encounter them in undergrad courses, you'll only see them in degree-year modules, or in very specialised, low intake degree schemes. As I said before, seminar based courses are more work, but more specifically, they take a lot more time to grade. An open-ended course leads to an open-ended assessment. If you have a class of 400 students, marking essays is an endless task, and it is very difficult to mark them all fairly and equally, but if you give them a structured exam, marking is quick, easy and completely objective.
But more than that, the standard system provides a gradual shift in abstraction. Perhaps this is a happy accident rather than by design, but as we progress through our degree scheme, we should be building up a toolkit of useful skills and concepts that can later be applied elsewhere. As that toolkit expands, we are slowly given greater freedom to apply and think.
My opinion is, and always had been, that the MOOC must be seen as a vehicle for "basic skills" courses. Many within the educational establishment reject basic skills teaching in favour of "whole topic" teaching, but I personally believe that they've done so on the wrong grounds.
If you were to tell me that basic skills teaching is categorically bad, I would disagree with you. Basic skills are prerequisite to advanced skills.
But if instead you were to tell me that teaching basic skills properly and exhaustively is good in theory, but unfortunately not practical given class sizes and constraints on classroom time, I think I would be able to accept your point.
And this is where online can really flourish: we can take the basic skills load out of the classroom. Have the students work on the basic skills individually, using constrained tasks that a computer can assess. These types of tasks can be completed much quicker without all the usual classroom faff of handing out sheets, explaining the task, checking up on students, going over the answers, addressing individual errors with the whole group etc. The volume of work that is assessed or otherwise receives feedback can be increased, as the teacher's time is no longer a limiting factor.
Then the students can go into the class with the basic skills they need to address larger, more abstract, more challenging tasks. Lessons can be more rewarding for both the student and the teacher.
Debbie Morrison proposes turning a course with a 5-figure class roll into a student-centred, open-ended seminar course, but that is precisely what the FOE Mooc attempted to do, and it was a total train wreck. Debbie's "takeaway" from the incident isn't asking us to learn from their mistakes, but to repeat them.
One article quoted a law professor as follows:
“Part of what Coursera’s gotten right is that it makes more sense to build your user base first and then figure out later how to monetize it, than to worry too much at the beginning about how to monetize it,” said Edward Rock, a law professor serving as the University of Pennsylvania’s senior adviser on open course initiatives.Is he mad, or just plain ignorant? It's this sort of thinking that leads people to compare the MOOC boom to the internet bubble at the turn of the century. After all, that bubble was based on companies building massive user bases with no idea of how to monetarise them.
We are currently experiencing a similar bubble in Web 2.0. So-called investors* have ploughed money into companies like Facebook and Instagram, only to see stock prices suffer when the market suddenly realises that they have no business plan.
[* The modern stockmarket makes a mockery of the term "investor". "Investing" is supposed to mean putting money in so that a company can grow, but most high-profile IPOs, including Facebook and the like, are simply a change of ownership. The money is for the previous owner, not the business.]
Why do we want to replicate that model at a cost to our valued educational institutions?
I've commented on that side of things before, noting Udacity's move toward corporate sponsorship: MOOCs as adverts for a given vendor.
This MOOC-as-sponsorship model might even work to some extent for the universities too -- the University of Edinburgh's Coursera MOOC E-Learning and Digital Cultures (#edcmooc) is based on a module taken from one of their masters programmes, and it's possible that the increased exposure it gives the course among potential students will get back the cost of putting the MOOC version together and on-line. The risk, though, is that if too much material is available for free, the adverts will basically kill the market for the product.
One article in particular caught my attention. Over at the blog online learning insights, blogger and instructional designer Debbie Morrison wrote a post about three takeaways from the collapse of the Coursera Fundamentals of Online Education course.
Her first "takeaway" is, in my ever so humble opinion, completely and utterly wrong. Totally. Completely. Utterly. Let's take a look:
1) The instructional model is shifting to be student-centric, away from an institution or instructor-focused model.In a massive, open and online course with thousands of students, the instructor must relinquish control of the student learning process. The instructor-focused model is counter intuitive to the idea of a MOOC; in the MOOC model the student directs and drives his or her learning. The pedagogy used for traditional courses is not applicable to a course on a massive scale. With the Web as the classroom platform, students learn by making connections with various ‘nodes’ of content [not all provided by the instructor] on the Web, they aggregate content, and create knowledge that is assessed not by the instructor, but by peers or self. This pedagogy builds upon the constructivist theory, and more recently a theory developed by Downes and Siemens, the connectivist learning model.
Wrong... how so?
Debbie Morrison clearly knows a thing or two about building online courses; unlike me, she's actually written them. But when she claims that the a tight teacher-led course is "counter intuitive to the idea of a MOOC", she's projecting her views into a reality that is distinctly different. The origin of the MOOC is in the field of Computer Science and Artificial Intelligence. The original MOOCs were very much instructor-led, with very specific, well defined tasks for the student to follow.
Because the tasks were very tightly controlled, marking could be automated, and hundreds of thousands of students could be individually assessed by machine.
Morrison is mistakenly applying the standards for her own niche, the small-to-medium online course, to a very different animal.
Morrison's courses are comparable to a seminar-based course in more traditional education. Every generation has accepted that seminar-based courses are better than lecture-based ones, but that they're more work for the teacher because the uncontrolled variables lead to a massive space of potential outcomes and directions for the class.
Online education for small classes is a good realisation of the seminar-based course, because the online medium removes some of the time pressures on the instructor in terms of organisation, logistics and delivery; time which can be devoted instead to tailored group and individual feedback.
But when you've got one instructor and one teaching assistant for a class of 41,000 students (foe) or 1 instructor for a class of 150,000 (claimed by Ed Tech Magazine for a Udacity course), then that's out the window. A teacher cannot be a "guide" or a "facilitator" to a group that big. A teacher cannot even have a concept of the individual students as human beings (we're well beyond Dunbar's Number here).
A massive course cannot be student-centred because there are simply too many students.
And let's go back to the seminar-style class for a minute. Seminar-based classes are far more common in taught postgrad courses than in undergrad courses, and while you might encounter them in undergrad courses, you'll only see them in degree-year modules, or in very specialised, low intake degree schemes. As I said before, seminar based courses are more work, but more specifically, they take a lot more time to grade. An open-ended course leads to an open-ended assessment. If you have a class of 400 students, marking essays is an endless task, and it is very difficult to mark them all fairly and equally, but if you give them a structured exam, marking is quick, easy and completely objective.
But more than that, the standard system provides a gradual shift in abstraction. Perhaps this is a happy accident rather than by design, but as we progress through our degree scheme, we should be building up a toolkit of useful skills and concepts that can later be applied elsewhere. As that toolkit expands, we are slowly given greater freedom to apply and think.
My opinion is, and always had been, that the MOOC must be seen as a vehicle for "basic skills" courses. Many within the educational establishment reject basic skills teaching in favour of "whole topic" teaching, but I personally believe that they've done so on the wrong grounds.
If you were to tell me that basic skills teaching is categorically bad, I would disagree with you. Basic skills are prerequisite to advanced skills.
But if instead you were to tell me that teaching basic skills properly and exhaustively is good in theory, but unfortunately not practical given class sizes and constraints on classroom time, I think I would be able to accept your point.
And this is where online can really flourish: we can take the basic skills load out of the classroom. Have the students work on the basic skills individually, using constrained tasks that a computer can assess. These types of tasks can be completed much quicker without all the usual classroom faff of handing out sheets, explaining the task, checking up on students, going over the answers, addressing individual errors with the whole group etc. The volume of work that is assessed or otherwise receives feedback can be increased, as the teacher's time is no longer a limiting factor.
Then the students can go into the class with the basic skills they need to address larger, more abstract, more challenging tasks. Lessons can be more rewarding for both the student and the teacher.
Debbie Morrison proposes turning a course with a 5-figure class roll into a student-centred, open-ended seminar course, but that is precisely what the FOE Mooc attempted to do, and it was a total train wreck. Debbie's "takeaway" from the incident isn't asking us to learn from their mistakes, but to repeat them.
03 February 2013
MOOCs? Bah humbug!
Yesterday I got an email from the Georgia Tech online education Coursera MOOC explaining why it was important that people signed up for groups. I felt it was slightly patronising to start off with, but made particularly so because I received it a while after I'd clicked "Assign Me A Group". The reason I hadn't signed up for any groups was simple: the course opened on Monday and the group signup had crashed before I had a chance to look at it, and I didn't have the time to look at it again until Saturday, because I had a very hectic working week (28 and a half student contact hours, plus planning, marking and reporting). So I'd been quite annoyed to log in on Saturday morning hoping to do my week's MOOC work and find that I couldn't do my week's homework because I wasn't in a group.
The course has now been postponed indefinitely, presumably because there were too many people in certain groups and too many people without groups. Overall, their approach seems massively shortsighted -- the original problem with group signups was that they'd done it with a Google Docs spreadsheet with wide open permissions, meaning it could all too easily get mucked up. A spreadsheet can't handle that sort of concurrency anyway.
Then there was the links to the videos that had been uploaded at way too high a resolution and took too long to download.
Overall, for a course specifically teaching online education, the organisers appear to have had a spectacularly poor grasp of the technological side of things.
I signed up for another MOOC on online education, again with Coursera, but from the University of Edinburgh: Elearning and Digital Cultures. This also opened on Monday, and I didn't get a chance to look at it until Saturday, when I saw that the instructions for the week included watching a bunch of short films before tuning into a live webcast with the tutor on Friday. They didn't give enough notice, and it really misses the point of global online education if you have to watch at a certain time, even though the segment is not interactive.
So I'm getting pretty cynical about MOOCs and the woolly thinking and poor planning surrounding them. So thank heavens for Edinburgh University's Introduction to Philosophy, featuring a scruffy looking bloke in a poor-fitting jumper talking to a camera. It's finally something I feel I can relate to. (And to think I only signed up because I saw the Edinburgh logo, and there was no other Edinburgh courses at the time!)
The course has now been postponed indefinitely, presumably because there were too many people in certain groups and too many people without groups. Overall, their approach seems massively shortsighted -- the original problem with group signups was that they'd done it with a Google Docs spreadsheet with wide open permissions, meaning it could all too easily get mucked up. A spreadsheet can't handle that sort of concurrency anyway.
Then there was the links to the videos that had been uploaded at way too high a resolution and took too long to download.
Overall, for a course specifically teaching online education, the organisers appear to have had a spectacularly poor grasp of the technological side of things.
I signed up for another MOOC on online education, again with Coursera, but from the University of Edinburgh: Elearning and Digital Cultures. This also opened on Monday, and I didn't get a chance to look at it until Saturday, when I saw that the instructions for the week included watching a bunch of short films before tuning into a live webcast with the tutor on Friday. They didn't give enough notice, and it really misses the point of global online education if you have to watch at a certain time, even though the segment is not interactive.
So I'm getting pretty cynical about MOOCs and the woolly thinking and poor planning surrounding them. So thank heavens for Edinburgh University's Introduction to Philosophy, featuring a scruffy looking bloke in a poor-fitting jumper talking to a camera. It's finally something I feel I can relate to. (And to think I only signed up because I saw the Edinburgh logo, and there was no other Edinburgh courses at the time!)
02 February 2013
Peer Instruction
Well that was serendipitous. Just yesterday I linked to an old post of mine called The Myth Of Groupwork. My argument against groupwork was that students don't know how to teach each other, and that even if they did, the nature of the task doesn't present teaching as the goal, so instead of working together to learn, the students work together to fill in the gaps on the sheet. A "task-focused" approach or, as I compared it to in that post, a "pub-quiz" approach to a question sheet.
But today I sat down to watch some videos from a Coursera MOOC on online education, provided by Georgia Tech (Georgia the US state, not Georgia the country). Now I'm not too impressed with the course in a lot of respects so far, but it is full of very good information that I will definitely learn from. (A fuller review may come later, when I've made a bigger dent in my workload here.)
I've just paused a video midflow to write this because the course instructor has just started talking about an idea called Peer Instruction. Apparently it was "discovered" in 1990 by a guy who "discovered" that lecturers "didn't work". Peer-led learning is nothing new really, and everybody knows that lectures alone don't work. But leaving that aside...
Eric Mazur, the man credited in the video with this "discovery" did make a useful observation, even if others probably did before him. Students who had learned a new concept in class successfully were often better able to explain it to a peer (that had been in the same lesson but hadn't understood) than the teacher, because the student has just gone through the process of reasoning out the problem.
There is one intrinsic flaw in this reasoning: a good teacher should be capable of giving a better explanation than a non-expert peer; Mazur's discovery was in effect nothing more than discovering he wasn't a particularly good teacher. Which is a pretty good explanation, for why this teaching idea was "discovered" by one of the leading lights in optical physics (>groan<... sorry) rather than a pedagogy or education professional.
This is OK... that's how universities work, and that's why learner independence is so important in a traditional university: lecturers are subject experts, not education experts.
But it's when I compare this idea of Peer Instruction to my observations of groupwork that things start to get interesting, because the incident refered to in my previous post wasn't something that we'd just learned. It was a grammar class with a mixture of natives, long-term learners and recent learners, so re-evaluating it from the perspective of Peer Instruction, an essential element was missing: the people who understood the concepts we were being tested on had already known them a long time before the class started, so they didn't have the recent experience of having recently worked out the answer that successful peer instruction is based on.
One of my philosophies (and a frequent undercurrent in my posts here) is that it's safer to assume a teaching technique is bad than good until you understand how it works and when, where and why it's appropriate. Most groupwork is justified by the overly simplistic notion of "learning from your peers", but the idea of "learning from peers who have recently learned the concept" is massively more useful.
Now that I better understand the why and when of groupwork, I'm far less negative about it, but that doesn't mean I'll suddenly take it up wholeheartedly. In my situation, this is entirely academic: my classes are at such a mixed level that I the central idea of peer instruction fails: the students who understand the concept generally understood the concept (at least in part) before the lesson -- they do not have the recent experience of working out how the language point works....
But today I sat down to watch some videos from a Coursera MOOC on online education, provided by Georgia Tech (Georgia the US state, not Georgia the country). Now I'm not too impressed with the course in a lot of respects so far, but it is full of very good information that I will definitely learn from. (A fuller review may come later, when I've made a bigger dent in my workload here.)
I've just paused a video midflow to write this because the course instructor has just started talking about an idea called Peer Instruction. Apparently it was "discovered" in 1990 by a guy who "discovered" that lecturers "didn't work". Peer-led learning is nothing new really, and everybody knows that lectures alone don't work. But leaving that aside...
Eric Mazur, the man credited in the video with this "discovery" did make a useful observation, even if others probably did before him. Students who had learned a new concept in class successfully were often better able to explain it to a peer (that had been in the same lesson but hadn't understood) than the teacher, because the student has just gone through the process of reasoning out the problem.
There is one intrinsic flaw in this reasoning: a good teacher should be capable of giving a better explanation than a non-expert peer; Mazur's discovery was in effect nothing more than discovering he wasn't a particularly good teacher. Which is a pretty good explanation, for why this teaching idea was "discovered" by one of the leading lights in optical physics (>groan<... sorry) rather than a pedagogy or education professional.
This is OK... that's how universities work, and that's why learner independence is so important in a traditional university: lecturers are subject experts, not education experts.
But it's when I compare this idea of Peer Instruction to my observations of groupwork that things start to get interesting, because the incident refered to in my previous post wasn't something that we'd just learned. It was a grammar class with a mixture of natives, long-term learners and recent learners, so re-evaluating it from the perspective of Peer Instruction, an essential element was missing: the people who understood the concepts we were being tested on had already known them a long time before the class started, so they didn't have the recent experience of having recently worked out the answer that successful peer instruction is based on.
One of my philosophies (and a frequent undercurrent in my posts here) is that it's safer to assume a teaching technique is bad than good until you understand how it works and when, where and why it's appropriate. Most groupwork is justified by the overly simplistic notion of "learning from your peers", but the idea of "learning from peers who have recently learned the concept" is massively more useful.
Now that I better understand the why and when of groupwork, I'm far less negative about it, but that doesn't mean I'll suddenly take it up wholeheartedly. In my situation, this is entirely academic: my classes are at such a mixed level that I the central idea of peer instruction fails: the students who understand the concept generally understood the concept (at least in part) before the lesson -- they do not have the recent experience of working out how the language point works....
23 October 2012
Udacity review: Web development (C253)
OK, so this isn't strictly about
language, but I've been following Udacity's course on web app
development as I've been working on designing a language learning app
for a while now, and I'm really not too hot on web technologies at
the moment (and where would you put a language learning app other
than on the web these days?).
I've written about MOOCs before, and
shortly after writing that post I read a very detailed review of Udacity's Introduction to Statistics at the AngryMathblog.
A lot of commenters suggested that
the problems identified were unique to the particular course, but
with it was with those criticisms in the back of my head that I've
spent several hours over the last couple of weeks rattling through
this course, and I have to say that I have very similar concerns to
Delta over at AngryMath.
To summarise, Delta picked out a “top
10” of problems:
- Lack of planning
- Sloppy writing
- Quiz regime
- Population and sample
- Normal curve calculations
- Central Limit Theorem not explained
- Bipolar difficulty
- Final exam certification
- Hucksterism
- Lack of updates?
Everything there matches to my own
observations with the web development course, except the final exam
(which I haven't reached yet – I'm on unit 6 of 7) and
the stats-specific items (4,5,6) – although there are problems with
Steve Huffman's course that are analogous to these.
1. Lack of planning
It is not uncommon
to hear Huffman change his mind halfway through a unit, or
even after giving a quiz. Mostly, this is because he uncovers
another quirk in the Google AppEngine or one of the Python code
libraries that affects the outcome. OK, we can forgive the guy for
not being an expert on a relatively new technology, but why didn't he
take a couple of hours to check all these things before starting
filming?
In
video 4.38 he even says "One final password thing. I know I
promised the last one was the final one but..." Now he really
should have known he was going to say that when he filmed the
previous segment, and if he really wanted to change it, he could have
gone back and reshot part of the earlier section in order to edit it
out (or even just redubbed the section in question).
If he can't plan
an hour or two ahead, it throws his whole schedule into doubt.
2. Sloppy writing
Huffman makes several spelling errors during the course on some pretty fundamental computing terms, talking about “algorithims” (ouch) or a database being “comitted” (yuck). After having “protol buffers” on screen for half a minute, he spots it and corrects it to “protocol buffers” (5.16).
His handwriting becomes progressively
more crooked, moving across the screen at an angle, and he
consistently and clearly writes his quote marks as open and close
quotes on the whiteboard, even though most computers make no
distinction (and Python, along with most languages, definitely
doesn't).
This is core stuff he's dealing with, and he's failing to be precise.
3. Quiz regime
The quizzes seem just as forced as Delta found in the stats course, with annoying simple ones, then difficult ones that require you to remember an exact command that you've seen once, to ones that suffer from a rather odd sense of humour. I was not familiar with the “hunter2” meme, and the constant reference to that value forced me to go and look it up. Not particularly interesting. As an inside joke, using it as the default password example was sufficient – giving it as an incorrect option to several multiple-choice quizzes was unnecessary and distracting.
But the other thing that I really
noticed about the quizzes in this course is more serious: they just
didn't feel like an integral part of the lesson. Most of them
started with a dedicated video, rather than just being asked at the
end of a video. This inserted a little pause as the next video
loaded. You'd sit there waiting as Huffman unnecessarily read
out the answers (I can read, as you may have noticed). That wasn't
the worst of it, though. Huffman insisted on constantly telling
you you were about to have a quiz. Why? Isn't it enough to ask the
question?
Worse, this kills one of the clearest
pedagogical rules: don't overwrite useful information in working
memory – take full advantage of the "echo effect". I found myself
lost on several occasions, because after giving me new information,
Huffman would wipe the “echo” from my working memory by
telling me “I think it's time for a quick quiz”. There'd then be
a pause while the next video loaded, where the only thing repeating
in my head was the fact that there was going to be a quiz – the
information I needed to actually complete the quiz was gone. I skipped the quiz and went straight to the answer, because I didn't know it, and there was no scaffolding or structured guidance in the question.
And then, of course, whether I got the
answer right or wrong (or didn't even try), Huffman decides to
explain why all the answers are right or wrong anyway. No attempt
was made to focus on my specific misunderstandings, and when you're
giving a course to thousands of people, wouldn't it make sense to
take a little extra time and include a few extra video snippets to
match the different answer combinations to the quizzes? A couple of
hours of your time to save 10-20 minutes each for thousands of people
is a good trade-off (and what you might consider being a “good
citizen”, Huffman, as your own course proposes we all should
be).
4. Population and sample / ACID
Delta complains that Thrun's course doesn't present a clear distinction between two fundamental statistical concepts – I would say that Huffman's course similarly fails when it touches on databases. It's not as serious a problem, as this isn't a database course, but if you're going to teach something, for pity's sake, teach it right. ACID stands from Atomicity, Consistency, Isolation and Durability. Huffman's explanation in unit 3 fails to fully define consistency, leaving it difficult to see the difference between atomicity and consistency. The confusion is compounded by the fact that the whole definition of ACID relies on the idea of a database “transaction”, which Huffman readily admits to not having talked about before. (So I could actually add this into “poor planning” above if I wanted to.)
5. Normal curve calculations /
multiple frameworks and libraries
There's not necessarily anything as
fundamental as this missing from this course as the normal curve, but
the end result of something “magical” happening (ie powerful,
important, and not understood) is present. By jumping about from
framework to framework and library to library, Huffman keeps
introducing stuff that we, as learners, just aren't going to
understand. To me, that decreases my confidence: I like to
understand (which is why I'm taking the course).
6. Central Limit Theorem not
explained
No real equivalent, I suppose.
7. Bipolar difficulty
The difficulty problem in Thrun's stats
course is slightly different from the problem here. Thrun asked
questions that he didn't expect the student to know the answer to
(oddly), but here Huffman expects you to know the answer...
except that he has a very odd set of assumed prior knowledge.
For example, he starts with the
assumption that you have never encountered HTML before, but HTML is
extremely well known now, even among non-geeks. But then he assumes
you know Python. Python is a fairly popular programming language at
the moment, but really – not everyone
knows it. I'm also willing to wager a fair chunk of cash that most
Python scripters are very comfortable indeed with HTML, but that the converse is not true.
Now, I might be
doing him a disservice – his assumption no doubt comes because
Udacity's own Computer Science 101 course teaches Python, but
then again the course prerequisites don't mention either Udacity CS101 or specifically Python:
What do I need
to know?
--------------------------------------------------------------------------------
A moderate
amount of programming and computer science experience is necessary
for this course.
See? No mention of Python. Now I've
got a degree in Computer Science, so I've got what I thought was a
“moderate amount” of experience. But as soon as he asked a
code-based question, I was stuck. Not only did I not know the
appropriate syntax, but often I had no idea of the type of structure
required.
You see, Python is a very
sophisticated, very high-level language that does lots of clever
things that a lot of the lower-level languages don't. It has very
useful and flexible tools for manipulating strings and data-sets, and
even allows you to build “dictionaries” of key/value pairs. A
great many of the tasks presented in the course were easy if you were
familiar with the structures. If you weren't, you wouldn't A) know
how to write the code or B) know where to look for the answer, or what it would be called. OK,
so the answer to B is “the course forums,” I suppose, but that's
hardly adequate, surely? Audience participation is great and all,
but shouldn't good teaching prevent these blockages, these obstacles
to the learner?
8. Final exam certification
As I said, I haven't got that far yet. I suspect retaking will be less of an issue as a lot more of the material will be practical, and you can't expect to pass a coding exam by trial and error.
9. Hucksterism
Huffman doesn't seem to be as
evangelistic as Thrun, but he still does talk a bit too positively
after some of the quizzes (despite not knowing whether I got the
answer right or wrong), and he does say from time to time that now we
“know how to” do something. Are you sure? I've followed a
fairly tightly defined process – take away the scaffolding, and
could I repeat it? That's not guaranteed.
10. Lack of updates?
The grating
positivity does seem to die down during the course, so there's some
evidence of responding to feedback, but the course first went out
months ago, and despite presumably thousands of completed courses,
there's no evidence of them going back to attempt to fix any problems
in the earlier videos. As I stated in my previous post on MOOCs: any
conscientious teacher reconsiders his material after any class, which
means an update for every 20-30 students – this course has had a
lot more students than that, so where are the updates.
My own evaluation
So the above was
recreating Delta's complaints, with the specific purpose of defending
him/her against those who claim that the AngryMath article was unfair
as it focused on a sample size of one. But I'd also like to post my
views in their own terms.
Because
to me, the big problem wasn't one that appeared in Delta's top 10; it
was that the course is not what I would consider a university-level
course. Or at least, not a complete
university-level course. What I have experienced so far feels a
little too blinkered and focused on one project. I don't remember
any course at any of the three universities I've studied at where the
teaching was driving so clearly towards one single end-of-course task. Each of
the end-of-unit programming tasks brings you closer to that final
task, and there feels like there's a lack of breadth. As I went
through my programming tasks as a student at Edinburgh, we were dealing with
incrementally increasing code complexity, but on an exponentially
increasing problem base – no more than two homework tasks would be
as closely linked as all the tasks here. In essence, what we're
doing is more like a “class project” than a full “class”. Most courses in Edinburgh would change the programming tasks substantially from year to year (certain courses excepted – my hardware design and compiler classes were fairly specialised), but Udacity simply cannot do this as the tasks are fundamental to the syllabus structure.
And Huffman, we're told, “teaches from experience” – which basically translated to "he is not a teacher," in layman's terms. He does an admirable job for someone who hasn't
been trained in pedagogy, but really, seriously, would it kill them
to get an actual teacher to teach the course? Huffman's
awkwardness and uncertainty about the format is the reason he keeps
killing the echo effect – he hasn't developed the instinct to know
how much time and space we need to process an idea. At times, he
gives a reasonably broad view of the topic, but at others, he just
splurges onto the page what is needed for the task at hand. There's
no progressive differentiation of concepts, and he doesn't use any
advance organisers to help the learner understand new concepts.
Case
in point: introducing ROT13/the Caesar cypher without once
demonstrating or even describing a codewheel – a video demonstration of the code wheel is easy, cheap and clear. His demonstration with lines on the virtual whiteboard was not clear. Even if you don't use a codewheel, you can always use the parallel alphabets method:
So, yeah, I can see that Thrun really genuinely believes that the
educational establishment doesn't “get it” when it comes to new
education, but he's throwing the baby out with the bath water if that
means getting rid of educationalists altogether.
Teaching vs
training
But Udacity isn't completely abandoning academia – oh no; it's
recreating its mistakes. A recent post on the Udacity blog repeats
that hoary old complaint that education simply doesn't adapt fastenough to newtechnologies.
In Udacity's own words:
Technologies change quickly. While savvy companies are quick to adapt to these changes, universities are sometimes slower to react. This discrepancy can lead to a growing gap between the skills graduates have and the skills employers need. So how do you figure out exactly what skills employers are looking for? Our thinking: work with industry leaders to teach those skills!
It's the old “academic” vs “vocational” debate once again,
and just as many universities are sacrificing their academic
credentials by providing more and more courses that are mere
“training courses” for a given technology, that's what Udacity is
becoming. Forthcoming courses from Udacity are pretty specific:.
- Mobile Applications Development with Android
- Applications Development with Windows 8
- Data Visualization with Mathematica
Thrun keeps talking himself up as an alternative to
university, but he's starting to repaint his site as something that's
more an alternative to a Sams/Teach Yourself/for Dummies coursebook.
Because as they say:
We are working with leading academic researchers and collaborating with Google, NVIDIA, Microsoft, Autodesk, Cadence, and Wolfram to teach knowledge and skills that students will be able to put to use immediately, either in personal projects or as an employee at one of the many companies where these skills are sought after.
That's not what university is about. So Thrun doesn't like
university. Fine. But plenty of us do. Stop criticising
universities for being universities. If you want to be a vendor-specific bootcamp, knock yourself out, but please don't criticise universities for teaching us how to think instead of leading us through the nose on writing a Sharepoint site.
The UK used to have a strong distinction between vocational
institutions (known as “colleges of further education”) and
academic institutions (universities, higher education). It's a
useful distinction, and we should have both – it's not an
“either/or” question.
On the other hand, I suppose Thrun's worked out the answer to how to fund MOOCs: sell out to big business. I hope they're paying you well enough.
Subscribe to:
Posts (Atom)