30 January 2015

The slow death of the MOOC continues

This morning, I checked my email like I always do, and Coursera were plugging their latest "specialization" -- one for so-called cloud computing.

Coursera specialisations were originally launched as a single certificate for a series of "signature track" (ie "paid for") courses, but there's always the free option alongside it.

So I was very surprised when I clicked on the link for more information about the specialisation, then clicked through to the course, and it was only offering the $49 paid-for version. Now I did go back later and track down the free version of the course by searching the course catalogue, but the notable thing was that you can't get to the free version by navigating from the information about the specialisation.

It's there -- it is -- but by making click-through impossible, they're actively trying to push people into the paid versions. This suggests that the business model isn't working, and it's not really much of a surprise -- there's no such thing as a free lunch, and the only free cheese is in the mousetrap.

Some of the universities seemed to be using the free courses as an advert for there accredited courses, but it's a very large and expensive way to advertise -- teaching thousands in order to get half-a-dozen extra seats filled on your masters programme -- and so really the only way to get money is to get more of the students to pay.

Is it worth it for the student?

Cloud Computing costs £150, and going by their time estimates, that's between 120 and 190 hours of work. The academic credit system here in Scotland says that ten hours of work is one "credit point", and there are 120 credits in a year. Timewise, the Cloud Computing specialisation is then roughly equivalent to a 15-point or 20-point course -- ie. a single "module" in a degree course. A 15-point module costs £227.50, and a 20-point module costs just over £300, so £150 for this seems like a pretty good deal. Of course, this is only the cost to students resident in Scotland to begin with, and it is controlled by law to stay artificially low -- in England, the basic rate would be £750 for a 15-point course or £1000 for a 20-point one, but many universities "top-up" their fees by half again: £1125 and £1500 respectively. And English universities are still cheaper than many of their American counterparts.

So the Coursera specialization could be half the price of a university equivalent, or a tenth, or even less, depending on where you live. Sounds like a good deal, right?

Sadly, though, the certificates are worthless -- almost all the institutions offering courses through Coursera (and EdX, and FutureLearn) are allowed to accredit their own courses for university credit, but they choose not to. If they accredited a £30 course as university-level study, they'd be competing against themselves, and they'd kill the market for their established distance courses, and perhaps even their on-campus courses.
If they can run a course for £150, is there any justification for their usual high prices? Well... yes. Coursera is on a freemium model (free for basic use, pay for "premium" services), but in reality everything on Coursera is still the "free" part of the freemium. The online-only courses are not viable for universities for a number of reasons, so it's the fully-accredited courses run by the universities themselves that make it possible for the universities to offer the cheap courses "on the side", using repurposed versions of their existing course materials.

Technology and knowledge sharing can and should be used to reduce the cost of education. When I studied languages with the Open University, I looked at the cost of the course I was taking, vs equivalent unaccredited alternatives -- I could have bought equivalent books and spent more time with a one-on-one teacher than I did in group tutorials, and still only spent half of the money I did with the OU. If I hadn't wanted to get the degree, it would have made no sense at all to continue with them, but I want to teach in schools, so I need the degree.

So yes, there is undoubtedly unnecessary expense in education and there's a lot of "fat" that could be trimmed away, but the Coursera model won't do it, and for now it remains something of a distraction -- a shiny object that draws our attention away from the real problems and solutions.

23 January 2015

The militant wing of immersion and the examiner's dilemma.

So, last time I was talking about a discussion I had with a communicative approach hardliner. A couple of days later, I had a new student ask for exam prep classes, so I got out my exam prep materials and had a quick look over them to remind myself of the specifics of the Cambridge Advanced exam, and I very quickly remembered something else from Sunday's conversation.

One of his big bugbears about the Scottish education system was that the foreign language exams all have their instructions in English. This, of course, is a natural consequence in the belief in immersion above all else -- if language must be immersive, then native-language task instructions clearly break the immersion, and therefore burst the language "bubble".

But here's the thing: when I prepare students for an exam, I explain the language task to them and then practise it over and over. By the time my students reach the exam, they don't need to read the instructions. Now the exams I prepare people for are international exams, so economies of scale dictate that the exam questions stay in English. My students go into the exam, don't need to spend time reading and understanding the question and can instead focus on carrying out the actual task that is set for them.

But there are people who don't do a lot of preparation for exams, and will go in and need to read the task. Sometimes they misunderstand the task, which means they lose marks. A hardliner would say this is fair enough, because if they don't understand English, they shouldn't pass an English exam. That would be all well and good if anyone really understood the question first time round, but students who prepare are not being tested on understanding the nature of the task, so this is inherently asymmetrical.

Indeed, most adherents to a target-language-only method are also likely to believe in the "four skills" model of language (which I don't agree with, but that's not the point here), which is fundamentally incompatible with target-language-only exam instructions.

How so? Well, if you believe that language is composed of reading, writing, speaking and listening, then it follows that you should test the four components individually. However, if you put task instructions in the target language, then every exercise includes a reading component, and you cannot objectively measure the students' levels in the other four skills.

It's a dilemma I have heard discussed even at university level, and it's very much a living debate, so nobody really should be putting forward their views as though they are objectively correct, because as with everything, we can all agree that a line has to be drawn somewhere, but we all have different views on where.

I personally feel that with a student cohort with a shared native language, native-language task instructions are the fairest way to ensure that students are being tested on the skills that we claim to be testing.

But what about listening tasks? Should we be asking the comprehension questions in the native language too, in order to ensure that we are genuinely assessing their listening comprehension? I kind of think we should, but at the same time, it doesn't feel right. But I have personally done exam past papers with students where they have clearly understood the meaning of the recording, but didn't understand the synonym used in the answer. How can you lose a mark in a listening comprehension test for failing to understand a piece of written language?

But of course, that argument does start to extend to the reading comprehension test too, because you can understand the set passage perfectly, but again have problems with the question. Here it is a reading comprehension problem leading to a lost reading mark, but there is still a fundamental question to answer about whether you should be setting an exam where you cannot determine the cause of the students' errors.

When you think about it, though, the problem in both previous paragraphs (although only one example of the various types of errors that students might make) is not really one of listening or reading anyway -- it's a vocabulary problem; vocabulary, which we do not consider worthy of the title "skill".

Some examiners have tacitly recognised this, and stopped trying to explicitly score the "four skills" individually, such as the Open University, whose degree-level exams have only spoken and written assessment, with the written part incorporating listening and reading as source material for an essay writing task. It's a holistic approach that accepts that trying to identify why a student isn't good enough isn't really an issue -- either they are or they aren't. I was perfectly happy with the approach as a student, and I would be happy with it as a teacher.

Language is certainly too complicated for us to ever hope to devise a truly fair and balance way to assess student attainment, but the current orthodoxy has tied itself in something of a knot trying to reconcile two competing goals. So are we offering immersion, or are we assessing the skills?

18 January 2015

Shooting my mouth off...

A student of mine invited me round to his flat for Sunday lunch -- I big traditional paella, which was absolutely delicious. Now this isn't twitpic, so I'm not going to bore you with a photograph hashtagged #nomnomnom -- no, I'm more interested in a discussion I had.

Regular readers know I can be more than a little opinionated at times, and I'm not afraid to disagree with people, so when I met another teacher shortly after I arrived, the conversation quickly turned heated.

First, he asked where I taught. I said I was teaching privately because I don't like the way things are done in schools. He asked what I meant, and I explained that I don't like mixed native-language groups, because the problems a Spanish person has with English are completely different from those a Polish person has (a stereotypical TEFL class in Edinburgh is composed of one Polish person, one Italian, and then a whole pile of Spanish people). Targeting lessons at resolving student problems is then really difficult.

He had a bone to pick with that "mixed-ability is real life, there are no heterogeneous groups in the real world."

Mixed ability is real-life, true, and you will never have a truly heterogeneous class group, true. However, this argument doesn't hold up to logical analysis -- a simple reductio ad absurdum (note: this is not a strawman) is enough to cut it down: we do not mix absolute beginners and advanced students, therefore we all draw a line somewhere on the scale between heterogeneous and completely mixed; every teacher sees that line as being somewhere different, and "no such thing as heterogeneous" is no more a justification for his chosen line than it is for mine.

The next thing he said was quite interesting, and certainly bears reflecting upon. He suggested that my desire to teach students in more uniform groups was not respecting their individual needs. It's an interesting viewpoint. He felt that teachers who propose heterogeneous groups in order to reduce individual differences so that they could give one lesson and not worry about addressing individual needs. This may be true of some teachers, but it is not true of me. Personally, I find that in a heterogeneous group, I can predict individual needs better, because actually most Spanish speakers I've met have exactly the same problems... which means they are not "individual problems" at all. If I eliminate all the group problems early, then I can really deal with the genuinely individual problems as they come up.

He wasn't convinced... far from it. Now he objected that I was talking about "accuracy" when... (wait for it!)... "communication is the important thing." Oh dear -- my least favourite meaningless statement. I had used an example of a particular error that a lot of Spanish people make: even if they normally remember to put the adjective before the noun (eg "a pretty girl"), when they qualify the adjective, it tends to migrate to after the noun (eg "*a girl very pretty"). He (quite correctly) responded by saying that this type of error does not interfere with communication. However, just as with mixed-ability groups, there must be a line somewhere, and inductive logic allows us to generate incomprehensible:
  1. the US colloquial term "purdy" is readily understood to mean "pretty"
  2. foreigners who have difficulty pronouncing /ʌ/ will usually be understood if they instead use /æ/
  3. => It therefore follows that saying "pardy" instead of "pretty" will be understood.
Except, of course, it doesn't. Language has evolved to contain a lot of redundancy, and one or two steps of difference is acceptable, but the effect of errors on communication is cumulative, and there's a critical mass where the information finally gets lost (as well explained by Claude Shannon's information theory, in particular the Noisy Channel Theorem).

Let's continue the induction.
  1. "a girl very pretty" is understood to mean "a very pretty girl"
  2. => "a girl very pardy" is understood to mean "a very pretty girl"
  3. Oh, and swapping T and D isn't normally a problem either, eg "breat and budder" instead of "bread and butter"
  4. => "a very party girl"
Three errors, and the meaning is gone. But is this because it's just a phrase and not a sentence? Let's add in another "insignificant" error that's common in the English of Spanish speakers and get ourselves a sentence to look at:
  1. dropping a subject pronoun that can be inferred from the context, though incorrect in English, does not hinder comprehension. eg "Last night, met a very pretty girl" instead of "Last night, I met a very pretty girl."
  2. => "Last night, met a girl very party"
Four errors, and not a lot of meaning left. You might just get it, but it's going to be an effort to understand.
  1. Superfluous "the" added to "last night" doesn't interfere with comprehension. (In many sentences, this is true.)
  2. => "The last night met a girl very party"
And of course Polish and Chinese people have a tendency that we can add in here
  1. Dropping of "a" or "the" doesn't usually interfere with comprehension -- eg "give me apple" can easily be understood as either "give me an apple", "give me the apple" or even "give me apple" from the context
  2. => "The last night met girl very party"
Oh yes, and I've still got some Ts I could make into Ds
  1. The lazd nighd med girl very party
Notice how the first two Ds do nothing to interfere with understanding, as you would still recognise "last night", but it doesn't make sense to accept it in this situation as that sets up a habit that will also affect words and phrases that aren't ambiguous.

The third D now makes things more difficult. Is that "met" or "made"? Or maybe we're talking about a "med girl", ie. a student doctor.

Can we agree that the lazd nighd med girl very party is not comprehensible? I hope so.

But let's rewind and look at all the individual sentences we can make with one single error:
  • Last night, I met a very purdy girl
  • Last night, I met a girl very pretty
  • Last night, met a very pretty girl
  • Last night, I met a very preddy girl
  • Lazd nighd, I med a very pretty girl
  • The last night, I met a very pretty girl
None of these are going to cause a native speaker too much trouble, although the last one may be ambiguous depending on the context, but when we add all these errors together, the result is incomprehensible.

The boundaries of comprehensibility are difficult to judge, especially for a teacher, whose own ability to understand non-native language is much better developed than that of an average native speaker, even if only due to the difference in the amount of contact time they have with non-natives.

if comprehensibility can be measured, it can only be done in terms of the number and severity of errors... ie. comprehensibility can only be gauged by measuring accuracy.

And if comprehensibility and accuracy are so tightly connected, you cannot declare that one is more or less important than the other.

But I digress.

The conversation moved on, and the other teacher lamented Scotland's lack of attention to and alignment with the CEFR. He was there when it started, he told me, back in 2001. Now it's been a while since I've mentioned the CEFR on this blog, but when I did, I gave a fairly strong opinion -- an opinion that I readily repeated to my new acquaintance... who then clarified that when he said "he was there" he didn't just mean he was teaching -- he was involved in setting it up. Ah, right.

As it turns out, the guy isn't actually an active teacher any more either... he's now employed in his country's diplomatic service and is in Scotland to liaise locally on the teaching of foreign languages in Scotland's schools and universities.

They say you should pick your battles carefully, and in this case I didn't -- this was a debate that could not be won. To argue against Communicative Language Teaching with someone whose entire career and professional identity was built on the championing of CLT is not going to get you anywhere.

It's not just that I couldn't change his mind, though, but also that he couldn't change mine. It is very rare that I come out of a debate on language teaching without at the very least questioning myself, but that typically occurs when a question of personal style or perspective comes in. When you reconcile the personal views with the formal views, you can start to see why people believe what they believe, even if you don't share that belief. Both parties get to reanalyse themselves against someone else's frame-of-reference, and try to analyse the other in terms of their own. It offers new perspectives.

But if someone is truly, deeply invested in the orthodoxy, all you will hear is the dogma. There will be nothing new -- you will have heard it before from teacher trainers, colleagues and boss. You will have read it in several books and magazine/web articles.

Ah well, never mind.

16 January 2015

Duolingo: Web 2.0, free labour and the power of ignorance

Last time I wrote anything here, I had decided I was going to get some German under my belt. So I've tried out a couple of things on the net, and I've spent a lot of time on Duolingo, which in many ways is a very good resource, but is frustrating in the way it keeps generating nonsensical phrases and fragments.

Well, it turns out they recently added an interesting clause to their user agreement:
Temporary Restrictions on Users from the European Union
Users within the European Union are not presently allowed to submit materials for translation or translated materials to Duolingo. While these users can continue to use the educational services offered through the Website, they will not be involved in the translation of any documents. If you submit a request for translation or translated materials to Duolingo, you thereby warrant and represent that you are not currently within the European Union, did not translate the document within the European Union, and will not be within the European Union when your translation request has been finalized.
So what's going on here then?

I have always felt that most dot-com organisations run on a model that breaches workers' rights laws. In most countries, a for-profit organisation is not allowed to solicit or accept free labour, and yet a great many commercial internet sites rely on free labour for their profit.

When YouTube first launched, all advertising revenue was kept by the site -- uploaders made no money. YouTube argued that the uploaders weren't working for the site, so didn't need paid... and yet, without the uploaders, there would be no site. YouTube changed their business model later on to grant uploaders a share of the advertising take. The reason they did this was so that they could get on board professional media (including music videos) and then also to stop the higher quality amateurs from migrating to sites that were willing to split the profits. Market forces worked in the interests of the little guy... this time.

But what about Facebook's big translation push at the time of the public share offering? The public sale brought in enough money to translate the site into all the world's major languages several times over, and yet they did not pay a single translator, instead "crowdsourcing" the translation. It would be one thing if they had opened translation to any and all languages, but they chose the languages and were only interested in the "big" languages that would draw plenty of users and make Facebook more money.

If it was small languages, I could understand: you don't want to pay for a translation to eg Irish when all the users will happily use the English version -- it doesn't make you any money. But when you're translating into Spanish, one of the world's most widely spoken native languages, you'll make your money back many times over even if you pay for one of the world's best translators.

Facebook clearly thought that if so many other websites had got away with free labour, they would too, but they inadvertently brought the issue to far more public attention than they expected.

You see, translators have real power in Europe. With such a linguistically diverse base, the institutions of the European Union are full of translators, which makes them one of the most powerful lobby groups you can imagine. Seriously, there is no-one who "has the ear" of a Brussels bureaucrat than the person who's talking in that ear throughout the meetings.

Now I don't recall ever hearing of any sanctions being made against Facebook for this, but the groundwork was set and Duolingo walked right into the problem, because more than any other site, their business model is built on unpaid labour... and crucially unpaid translation. Duolingo seeks to generate income by having learners translate documents for paying clients as part of their "immersion" in the language. Already, Duolingo is translating articles for Buzzfeed and CNN. Their justification is that the translators are getting something in return -- the teaching. I can see where they're coming from, to a point, but that's the same justification people try to make for internships as a source of unpaid labour.

So somewhere along the line, Duolingo has been warned off and put up these "temporary restrictions"... but didn't tell anyone about it. It's there, right at the end of the Ts&Cs, but they didn't actively notify users, and there is no notice on the translation page to warn you that you might be about to do something potentially illegal.

But it gets worse, because they don't only leave you access to the section that is illegal, but they actively encourage you to use it. I've been using it a lot recently, and after most exercises, it tells me to try the translation.

Now, if you're sitting at a computer in the UK and try to access BBC Worldwide clips on YouTube, you won't get anything. Why? BBC Worldwide content is licensed for use outside the UK, and YouTube knows where you are. The same thing happens on plenty of sites.

Duolingo makes no attempt to block based on location, but there is no technical reason that they shouldn't. I cannot imagine that a company their size would not be tracking user locations anyway, in order to optimise their marketing strategies and their technology. They must know. Furthermore, there is even a section in the profile (optional, admittedly) for you to tell them where you live.

It's a pretty stupid course of action, if you ask me. With geolocation being such a simple and standard admin task (although admittedly not 100% accurate), failure to attempt to identify and block EU-based users could be argued to be negligent. That negligence is surely made worse by the fact that they are leading their users not only to arguably (not tested in court) break the law, but also to indisputably break their own license agreement. And all the while their negligence allows them to continue selling translations to commercial clients.

It's a dangerous path, and it could lead to a very messy end....