Authentic: long v short pt 5

Today was a wee bit frustrating.  I spent a solid chunk of time trying to get the ntlk data module installed, and with it the file english.pickle that would have allowed me to do part-of-speech (POS) tagging.  This would have made it almost trivially easy to eliminate the proper nouns and get a genuine look at the real "words" that are of interest to the learner.

Ah well, it looks like it wasn't meant to be.

So I started working towards custom code to eliminate the proper nouns manually, something which would be handy in the future anyway.  The first step was to identify some candidates for further inspection, and seeing as I'm working with English, that's pretty easy: if it's not all in lower case, there's something funny about it.  I wrote the code to identify all the tokens (words) that contained capitals.  Yes, at this point I could have checked whether it was the start of the sentence or not, but that wouldn't have really helped, because proper nouns occur at the start of sentences too, so i'd still need to check.

When I generated my set of candidates, though, it was a little long.  For The 39 Steps, I was looking at 919 tokens to check manually, and that's a fairly short book.  As I'm doing this for fun, it seemed like checking that many would be a little bit boring, particularly in longer books.  (I later checked the candidate set for the 3 books in total, and it turned out to be over 3000 words, which is more than my time's worth.)

My first quick test then was to have a look at the difference in figures.  Eliminating every single item with any capitals in it drops the type:token ratio in The 39 Steps from 14.48% to 13.14% -- that's almost a a 10% drop (it's 1.35 percentage points, but it's 9.27 percent).  Before properly addressing the proper nouns, I wanted to see how big a difference this crude adjustment makes to the figures.  It seemed just a little too high to realistically be led by proper nouns alone.  But can that be?  I mean, how many words are likely to occur only at the start of sentences?

So on I went, hoping that the data I could generate at this stage would start to shed some light on this figure.

The first graph I produced showed me the running type:token ratios and introduction rates for both the full token set, and the token set with non-lowercase words eliminated:

The two pairs of lines follow each other pretty closely, getting closer together as they progress.  But in order to start getting a clear idea of what was going last time, I had to go to another level of abstraction and measure some useful differences.  So here is the difference between the running ratios for all words and lowercase only, and the corresponding difference in introduction rates:

Now you'd be forgiven for thinking that the difference is diminishing here -- I was fooled into thinking the same thing, but then I realised I was dealing with numbers here rather than proper stats, and I redid the analysis but with a difference in percentage:

The overall running type:token ratio does indeed decrease, but it halves (20% down to 10%) then stabilises.  The introduction rate, on the other hand, is all over the place -- there's no identifiable trend at all.  Even subsampling my data didn't give any clear and understandable trends (and since I'm using a desktop office package for my analysis it's a bit of faff to do the resampling automatically -- it's just further proof that I need to get myself familiar with the statistical analysis tools for Python (eg numpy), but my head's full with the NLTK stuff for now, so I'll leave the improved statistical stuff for
another time).  Here's the same graphs, but with 2000 word samples instead of 500 word samples:

So not promising, really.  Still no stable, identifiable trends.

Books as a series
But I had all the infrastructure in place now, so I figured I might as well rerun the analysis on the 3 books as a single body and see what came out.  Let's just go straight to the relative difference between the lines for all words and eliminating all words not entirely in lower case:

Oooh... now where did I leave those figures on where the individual books started...?  44625 and 152034, and there's a notable period of high difference (20-30%) from about 45000 words, and that massive spike you seem on the graph -- which is actually a 63.64% difference -- occurs from 152000-152500.

Bingo: we've got decent support for Thrissel's suggestion that a lot of proper nouns are introduced early on in... at least some novels.

Not the sort of information I was originally looking for, but actually quite interesting.  It's kind of turning the project in a slightly different direction than I had planned.  I'll just have to go with the flow.

What I did wrong today
One of the minor irritations of the day was when I started writing up my results, and after having done the coding, data generation and analysis, I realised a fairly simple refinement I could have made.  It was a real *palmface* moment: I could have simply taken my first list of candidate proper nouns and eliminated any candidates that also appeared completely in lower case.  Having done that, I would have been left with a much shorter list of candidates, and it may well have been worth my time manually checking the results.

>sigh<

But of course, that's as much the point of the exercise as anything: to work through the process and the problems and to start thinking about what can be done better.

It also occurs to me now that I also managed to eliminate every single occurrence of the word I from the books!  Quite a fundamental error, even if it only made a minute difference to the final ratios.

Perhaps I'm being a little too "hacky" in all this.  I'll have to pick up my game a bit soon....

Authentics: long v short - pt 4

Well, I had a nice weekend and visited some friends in Edinburgh for a wedding.  The weather's too good to spend too much time inside, so I'll just write up a few more tests then go and enjoy the sunshine.

Multiple books in a series
Today's figures come from two of the books I've already mentioned -- John Buchan's The 39 Steps and Greenmantle, and the next book in the series: Mr Standfast.

Again, the graphs at different sample sizes show different parts of the dataset more clearly than others:

The graph as 1000 words is too unstable to clearly identify the end of the the first book, and the start of the second book is only identifiable because the introduction rate is greater than the running ratio for the first time in any of my tests.

The 2500 and 5000 word samples give us a clear spike for the end of the first book, but the end of the second book is obscured slightly by noise, and becomes clearer again in the 10000 and 25000 word sample sizes, although the end of the first book is completely lost by the time we reach the 25000 word sample graph.

Having done all that, I went back and verified the peaks matched the word counts -- The 39 Steps is 44625 words long, and Greenmantle is 107409 words long, so ends with the 152034th word.  The peaks on the graphs all occur shorlty after 45000 and 150000.

It was the first graph, from the 1000 word sample set, that piqued my curiosity.  Having spotted the two lines intersecting at the start of the third book, I decided to check the difference between the running type:token ratio and the introduction rate, and I graphed that.  At 1000 word samples, there was still too much noise:

However, given that I already knew what I was looking for, I could tell that it showed useful trends, and even just moving up to 2500 word samples made the trends pretty clear:

Going forward, I need to compare the difference between books in a series, books by the same author (but not in a series) and unrelated books, and I believe that the difference between the running type:token ratio and the introduction rate may be the best metric to use in comparing the three classes.

Problems with today's results
I can't rule out that the spikes in new language at the start of each book aren't heavily influenced by the volume of proper nouns, as Thrissel suggested, so I'm probably going to have to make an attempt at finding a quick way of identifying them.  The best way of doing this would be to write a script that identifies all words with initial caps in the original stream, then asks me if these are proper nouns or not.

By treating the three books as one continuous text in the analysis, it looks like I've inadvertently smoothed out the spike somewhat at the start of each book.  In future I should make sure individual samples are taken from one book at a time so that the distinction is preserved.

Authentic materials for learners: long form or short form? (pt 2)

Well, as I was saying in part I, I've always claimed novels are easier for the learner than short stories, and I was wanting to back up my claims with some figures.  So for my initial investigation I fired up my copy of the free TextSTAT package and away I went.

I was talking about the type:token ratio last time, and that seemed as good a place as any to start.  I managed to skip the logical first step, which would have been to compare a novel and a short story, but I'll have to come back to that later.

What I started with was an Italian novel, but I got figures that were too high to be useful.  One of the problem with languages such as Italian is that they write some of their clitics in the same written word as the main word (EG "to know (someone)" -> conoscere; "to know me" -> conoscermi), increasing the type:token ratio significantly.  You've also got the problem that it has verb conjugations and it drops subject pronouns in most situations.  Overall Italian (and Spanish and Catalan, among others) would be a bad choice for a demonstration language.  Today, I'm using English as it's a very isolating language -- the only common inflections are past-tense-ed, second-person-present-s and plural-s.  This makes it easy to get a reasonably accurate measure of the lexical variety without any clever parsing.  I will most likely use French at some point too, because while it is not as straightforward as English in that sense (it's got a lot of verb conjugation going on), it doesn't have the same clitics problem as Italian, and the French don't drop their pronouns.
Today's findings: 1 - running ratios
I decided to look at how the type:token ratio changes as a text proceeds.  I wanted to measure this chapter by chapter, counting the types and tokens in chapter 1, then loading the second chapter into the concordance and checking the type:token ratio for chapters 1 & 2 combined, then 1, 2 & 3 etc.  I realised, however that it would be more efficient to load all chapters into memory at the same time and work down from the other end: all chapters, then close the last chapter and take the figures again, then close the second last chapter and take the figures again.
In the end, I got a nice little graph (using LibreOffice) that showed a marked tendency to decreasing type:token ratio as the books progressed:

The x-axis shows the chapter number, the y-axis shows the type:token ratio (remember, this is the type:token ratio for the entire book up to and including the numbered chapter).  Notice how the type:token ratio halves by around the 6th or 7th chapter.

So by one measure, the longer the novel is, the easier it would appear to be.
Today's findings: 2 - introduction rates
I figured I could go a bit deeper into this without generating any new data.  What I wanted to look at now was how much new material was introduced in each chapter -- ie. a ratio of new types to tokens. It's easy enough to do -- I could obtain the number of new types in any given chapter by deleting the running total at the previous chapter from the running total at the current chapter.
The graph I got was even more interesting than the last:

While the running ratio halves after 6 or 7 chapters, the introduction rate halves after only 2-4!  It certainly looks like each chapter will on average be easier than the last.
One curious feature is the large uptick at the end of the children's novel Laddie (green).  This illustrates one quirk that the learner should always bear in mind: kids books are often actually more complicated linguistically than adults' books, as the author on some level seeks to educate or improve the person reading.  The author of this book seems to have kept the language consistently simple through most of the book, but realising he was coming to the end, crammed in as much complexity as possible.

Another curious feature is that the figures claim no new vocabulary is introduced in the fourth chapter of The 39 Steps (yellow).  While this is theoretically possible, its more likely that it's ...ahem... experimenter error, which a quick look at the actual figures verifies: chapters 3 and 4 are listed in my output as being exactly the same length, which is more than a little unlikely.  It looks like I loaded the same chapter twice...
Further analysis
Notice that in both graphs, the figures are the same at chapter one.  This is to be expected, as every type encountered in the first chapter is encountered for the first time in the book (by definition).

So what happens if we stick the running ratio of type:token against the introduction rate of new types?

This:

So while the overall type:token ratio continues to fall notably from the 10th to the 20th chapter, suggesting decreasing difficulty, the introduction rate gets fairly erratic by around the 10th chapter (despite still tending downwards), so perhaps there is a limit after which it is not safe to assume that each chapter is a difficult as the last.

Perhaps the measure of efficiency is related to the difference between the running ratio and the introduction rate, and once that gap starts to narrow, there is no advantage?

Problems with today's findings
This was a first exploratory experiment, so I didn't conduct it with a whole lot of rigour.  Here are the main factors affecting todays results:

1. I didn't eliminate common words -- it is impossible to see from the figures I have how many of the types introduced at any stages are ones we would expect learners to know already and how many will be genuinely new to them.
2. When examining Pride and Prejudice and The 39 Steps, I hadn't told the concordancer to ignore case, so anything appearing at the start of a sentence and in the middle would be counted as two types -- eg that and That.  (It was the first time I'd used TextSTAT and I hadn't realised it defaulted to case-sensitive -- I won't make that mistake again.)
3. The length of chapters varies significantly from book to book and even from chapter to chapter within books, so the lines are not to scale with each other, and each individual line is not in a continuous scale with itself.  The graphs, though presented in a line, are arguably not true line graphs, as they occur from samples arbitrarily dispersed.

Accounting for these problems in the future

1. There are plenty of frequency lists on the net, so I'll be able to eliminate common words without any real difficulty.
2. The case sensitivity issue, now that I'm aware of it, will not be a problem.
3. When I ran the initial data, I was using TextSTAT as my installation of Python and NLTK was playing up (I had too many different versions of Python installed, and some of the shared libraries were conflicting).  I've now got Python to load NLTK without problems, so I can do almost any query I want.  Future queries will be sampled regularly after a specific number of words.

Experiments to carry out
At some point I'm going to want to go back and compare short stories with novels, but for now I'm going to head a little further down the path I'm on.

My first task is to work out a decent sampling interval: ever 1000 words? 5000? 10,000? 50,000?  I'll run a few trials and see what my gut reaction is -- that should be the next post.  (It might even prove that the chapter is the logical division anyway -- after all, it divides subjects, which would indicate different semantic domains...)
I also want to look at what happens when we look at sequels after each other.  Those of you familiar with John Buchan will notice that I've included such a pair as individual novels here -- The 39 Steps and Greenmantle.  I might include initial findings from this next time, as they'll determine my next step.
After this I'll either move on to looking at more pairs of original book + sequel (to look for a generalisable pattern), looking at longer serieses of books (to see if they get continually easier) or comparing book-and-sequel to two different books from the same author (to see if any perceived benefits from reading a book and its sequel are just coincidence and really only because of the author).

Caveat emptor
Remember, though, that this little study is never going to be scientifically rigorous, as I don't really currently have the time to deal with the volume of data required to make it truly representative.  However, it's nice to think how big a job this would have been before computers made this sort of research accessible to the hobbyist.  Many thanks to the guys who wrote the various tools I'm using -- your work is genuinely appreciated.

Authentic materials for learners: long form or short form? (part I)

Anyone who gets sufficiently far on in learning a language is going to want to start reading, watching or listening to materials intended for natives.  This is what a lot of language teachers refer to as "authentic materials".  (Now, an alarming amount of so-called authentic materials in the classroom are actually very heavily doctored, but that's not the sort of authentic materials I'm going to talk about today.)

I have often claimed that it is better for the learner to use longer materials than short materials (eg when discussing films vs TV serieses in a previous post).  This wasn't an idea I came up with myself, but advice I'd been given when I was at high school, although I can't remember who first suggested it to me.

Anyway, I was told at the time that the first one or two hundred pages of a novel contain most of the language (in terms of grammar, vocabulary and turn-of-phrase) that will occur in the entire book.  It therefore follows that the first two hundred pages of any work are the most difficult, and therefore the longer the book is, the easier the ending will be, because you won't be confused by the language.  This also means that the book is acting as active revision, and that by the end of the book, you will have learnt most of the major vocabulary in it.

A 50 page short story would intuitively sound easier to read than a novel, but this isn't really the case, because you're dealing with something that is going to be littered with new words on every single page.

And what about a piece of flash fiction?  Realistically, we're not going to expect much repetition at all.  Compare with the short extracts of authentic works printed in many classroom language textbooks -- none of the "content words" that are specific to the story are likely to be repeated at all, so they will be looked up by the reader, then promptly forgotten about.

Although I was told this about reading books (as opposed to short stories), I believe this holds for any form of literature, fiction or non-fiction, regardless of medium.

A half-hour documentary will be self-reinforcing in a way that a 4 minute news report on the same topic won't be (on TV or radio).  An 8 hour long TV series similarly will reinforce its language than an 80 minute feature film, or particulary a 10 minute short film.

Anyway, I've been repeating this advice for years, and I've always said that my experience backs it up. Well, in the little gap between finishing my Gaelic course and starting my next job, I was wanting to do a little work with corpus analysis software and it occurred to me that this would be a great little exercise to get me back into the swing of things, so I downloaded several resources: TextSTAT, a concordancer package written in Python at the Free University of Berlin; AntConc, a Linux/Mac/Windows concordancer by Lawrence Antony at Waseda University in Japan; and the Natural Language Toolkit for python, which will allow me to write more flexible, custom queries on my data.

Type:token ratio
One of the most basic statistical measures of diversity in a text is the so-called "type:token ratio".  The number of "tokens" in a text is the number of individual words, the number of "types" is the number of different word forms.

For example, the phrase "the cat chased the dog" has 5 tokens, but only 4 types, because the is only counted once when determining the number of types.

Or again, "I told the woman to tell the man to tell you" has 11 tokens (11 words in the sentence), but as the, to and tell occur twice each, there's only 8 types in the sentence.

The type:token ratio is exactly what you'd expect if you're at all familiar with statistics: the number of types divided by the number of tokens.  In the first example, we have 4:5 = 4/5 = 0.8 (or 80%) and in the second we have 8:11 = 0.727272... (roughly 73%).

Notice how the type:token ratio on the longer sentence is lower than that on the shorter sentence -- in this case it's a matter of my choice of words, but as a general rule, type:token ratio decreases with the length of text examined, which only goes to justify the advice of favouring long-form over short-form materials for the learner.

However, that's still to be proven in practice.

So over the next few weeks, I'll be experimenting with a bunch of public domain texts from Project Gutenberg.  I'll be trying to investigate the basic premise of whether long-form fiction is intrinsically easier than short-form, then investigating whether this extends to reading several books by the same author as opposed to books by different authors, and how much of a difference it makes whether these books are part of a series or individual stories.

The size of this study is going to be very small, as the main goal for me is simply to gain a better understanding of the technology and to reason through the process of designing logically sound research in a corpus, so the conclusions won't be scientific proof of anything, but it will hopefully be interesting (to me at least).

If you're aware of any research that covers the areas I'm looking at, please feel free to drop a reference in the comments, and if you have anything to add or suggest, I'm all ears.

Watching films as language study...

Well, I've been a little bit too technical and theoretical of late, so let's go for something more practical for a change.

A lot of people love the idea that you can learn a language just by watching films (such as Keith Lucas, discussed last week).  You can't.  Well, maybe there's one or two linguistic supermen out there who can, but for most of us, it won't work.

Can we get anything out of films?  Of course.
Can we get a lot out of films?  Hard to say.

First of all, if you're an absolute beginner, you're not going to understand anything watching the film without subtitles, and all proponents of target-language-only learning say that it's in understanding that we learn.

But unfortunately, once you start reading subtitles, you stop listening.  The brain, so I'm told, has only got one "language channel", and if you load it through the eyes, the brain tunes out the words hitting your ear so as not to mix up the two streams.  I'm sure you've tried talking to someone while reading or writing and found that you've written down a word from your conversation or suddenly said a word you've just read.

So once you start tuning out the sounds, your not going to learn much.

I first bought a DVD player in the January sales in 2005, with the express purpose of learning from foreign films.  My plan was quite typical: watch them with English subtitles, then later watch them again with the subtitles off.  Well, I never really did that -- I just kept buying and watching them with the subtitles on.  Not brilliant for my language skills, but now I've got one of the best DVD collections of anyone I know.  (Well, I know some people who have better collections, but at least mine's all originals!!)

However, after about a year, I started to notice little things at the start and end of sentences.  Little things like "you know...", "I see..." etc.  You know, little things that just seeped through before I started or after I finished reading the subtitles.  But I'm still only using one language channel.  People who can hold a conversation while reading a book aren't really doing two things at once, they're simply switching backwards and forwards between two tasks very rapidly, and this is what I started doing.  As a kid, I could never hold a conversation while reading, so it's not an innate talent on my part.  (My big sister always used to be able to do it.  I always assumed she was faking it or lying.)

Over the intervening years, I've been able to pick up more and more, but it seems to me that in a way I'm "primed" by the subtitles -- I'm anticipating how that would translate and what I hear is then matched against my expectations.

But really, the way to improve when you're good is to go without English subtitles (or whatever your native language is).  The first step to achieving that is to get material with target language subtitles.  The subtitles never match what is said on screen, so it's limited, but it does help you get tricky words.

Just now, I've been watching a French series Un Village Français. I tried watching it without the subtitles, but a few words slipped by me.  The first time I watched with subtitles on, I saw the word "scierie" and I realised it had to be "sawmill" ("scie" is "saw", and I knew the guy owns a sawmill from watching it before).  I'd watched two whole serieses without subtitles and never realised what this word was.  I hadn't even noticed that the word existed.  Two minutes with subtitles on, and I doubt I'll ever forget it.

But so far, so vague.
How did I start being able to listen while reading?  It's hard for me to say, as I wasn't really thinking about it at the time, but I believe it was when I started echoing my favourite actors to try and get the rhythm of the languages.  You can't do that without listening (obviously) and at first this got in the way of reading the subtitles and I ended up using the pause button a lot.  But having done that, it seems like my brain started realising that it had to listen and eventually I got there.

I only really noticed I was doing it when I went to see a French film and one of the characters was bemoaning the fact that kids today don't watch French cinema.  The subtitles talked about "rubbish from far away", the voice said "American crap".

But even after years, my "listening while reading" is still very limited.  It leaves me with a question I can't answer.  Do I get more out of watching with subtitles and hearing less of the speech or do I get more out of watching without subtitles and hearing more, even if I understand less?  It's impossible for me to measure this, and in the end the choice is made for me by circumstance, because if I have subtitles, I watch with subtitles.  If I don't, I watch without.

TV vs film for learners

But on a different tack, it's worth noting that watching serieses is far better for your language skills than watching films.  A film is relatively short, so there's little recycling of dialogue.  Each new film has potentially new accents and ways of speaking, but a 90 minute film finishes just as you're starting to get accustomed to the actors.

TV serieses, on the other had, offer several hours of dialogue written by the same scriptwriters, delivered by the same actors in the same accents, and covering the same topics.  The vocabulary and turn of phrase is repeated in throughout the length of the series, naturally reviewing and revising your learning. I've been following a particular series in Spanish for about two and a half years now, and I personally feel it has been immensely helpful to me.  Of course I've learnt a lot from other sources during the same time, but this has really aided my listening comprehension.

I already mentioned Un Village Français - I bought a two-series boxed set for around the cost of two full-priced feature films, and that's 10 hours of drama with 3 hours of historical documentary as bonus features for the price of 3 hours of film.  As I progressed through the series, I really did feel like I "tuned in" to the accents - there were things in the first few episodes that I should have understood (in terms of grammar and vocabulary, they were withing my boundaries) but that I didn't (because my ear wasn't picking up the detail of what the actors were saying).

So how do you make films and TV part of your learning strategy?

In the beginning, I don't think you really can.  At that stage, don't consider it "learning time", consider it "TV time".  Get used to the whole idea of subtitled foreign cinema with subtitles in your native language.  If you start to hear a word or two, great.  If you don't then it's no loss as this isn't "learning time".

I only really started getting serious with Spanish TV in the run up to my exams.  I'd studied a lot, I'd learnt a lot, but it still felt really disjoint.  I considered TV viewing as a type of revision -- I was hearing stuff I already knew, but used in many different ways.  I got used to the speed of natural speech in various accents, but I don't think I could have done that if I didn't already have a solid foundation in the grammar, because it reduced the amount of unknown material in the language.  In the end I picked up a couple of structural points too, and some good vocabulary, but mostly I mostly found that it took the language I knew in an academic context and made it more real and alive.

(And in the spirit of taking nobody's word for it and what I said in the follow-up, I'd like to point out that I can say definitively that I learned the Spanish construction "volver a hacer" from the Spanish series Águila Roja.  The fact that I can give a specific example suggests to me that I didn't learn much in this way.  Unfortunately, if you don't think about it, you can be misled into believing that remembering an example is proof of the effectiveness of a method.)

One thing I think would work well is to use a DVD player or computer video player that you can slow down.  I'd like to start watching foreign serieses with the first few episodes slowed by about 10 or 15% while I get used to the characters and their ways of speaking, then speed it up to normal speed for the rest of the series.  Unfortunately most of my serieses to date have been on-line without speed control, but I'll give it a go later in the year when I order in some French TV, and possibly Italian too.

Expository vs Naturalistic Language Examples

A couple of weeks ago, I was discussing authentic materials.  The main problem I identified was the lack of mutual reinforcement between individual texts (I hate that word, but I just can't find a suitable alternative...) meaning that very little language presented is retained.

So where did our modern love of "authentics" come from?

Authentic materials is actually one of the oldest tools in the language learner's toolbox.  Classical education has long focused on the reading of genuine Latin and Greek texts.  If you have a look at the Open University's course catalogue, you'll see that their classical language courses are called Reading Classical Greek and Reading Classical Latin, which is a pretty clear statement of the course goals.  The Greek course looks at a lot of literature in translation, but the Latin course is a perfect example of learning by authentic materials, as it looks at excerpts from Roman dramas and Cicero's speeches.

The use of authentic materials would even appear to go at the very least as far back as the heyday of the Roman Empire, where Greek was the fashionable language du jour.  Greek slaves were sold into rich Roman households where they would teach the children of the house to read and understand the works of writers such as Homer.

But despite two millenia as one of the most widely used tools in language learning, there are those who present the idea of using "real" language as a new and revolutionary idea.  In fact, many proponents of "real language" actively attack old ways of learning as ineffective and outdated.

But if we don't go straight for authentic material, what is there?

The very extreme opposite of authentic material is the stereotypical idea of trite sentences designed purely to demonstrate grammar points -- what I call expository language.

There are several classic examples of the absurdities that a purely expository approach leaves us with.

To the French person, the archetype is "My tailor is rich", which I'm told was the opening sentence of the original Assimil course.
In English, our traditional archetype is "La plume de ma tante" ("my aunt's pen", literally "the pen of my aunt") in such contrivances as "la plume de ma tante est sur le table".

Over a hundred years ago, people were already spending a lot of time attacking this approach.  The Danish language teacher Otto Jespersen wrote a book entitled How to Teach a Foreign Language (translated to English by Sophia Yhlen-Olsen Bertelsen) in which he put forth an argument for the so-called "direct" or "natural" method - ie that of teaching the language monolingually, by only speaking the target language.
"Disconnected words are but stones for bread;" he said, "one cannot say anything sensible with mere lists of words," and this is certainly true. "Indeed not even disconnected sentences ought to be used," he continued, "at all events, not in such a manner and to such an extent as in most books according to the old method," and while I wouldn't argue with this, we can see a little hint of what Decoo classes under the heading of "denigration of others" in his lecture On The Mortality of Language Learning Methods.

I'll reproduce some of Jespersen's examples, all taken from genuine courses of the time, for your benefit.
"My aunt is my mother's friend. My dear friend, you are speaking too rapidly. That is a good book. We are too old. This gentleman is quite sad. The boy has drowned many dogs."
Clearly there is no consistency or logic behind these, and it is hard to build up any sort of a bigger picture.

He then picks an example from a French book:
" Nous sommes a Paris, vous etes a Londres. Louise et Amelie, ou etes-vous? Nous avons trouvé la lettre sur la table. Avez-vous pris le livre ? Avons-nous eté a Berlin ? Amélie, vous etes triste. Louis, avez-vous vu Philippe? Sommes-nous a Londres ?"

And this is Jespersen's criticism of it:
"The speakers seem to have a strange sense of locality. First, they say that they themselves are in Paris, but the one (the ones?) that they are speaking with are in London (conversation by telephone?) ; then they cannot remember if they themselves have been in Berlin ; and at last they ask if they themselves are in London."

There is nothing in his criticism that really applies to any method, "old" or otherwise.  We are in fact looking at a criticism of choice of material.

I'd like to give a few examples that I think underline this point.

An Comunn Gaidhealach's Elementary Course of Gaelic was first published almost 100 years ago.  I picked up a reprint of the 1921 edition in a charity shop a couple of years back.  The first edition was written at the just after the high point of the "natural methods", and the revised edition was put together about 30 years after Jespersen's book, so it's quite likely that natural/direct thinking had an effect on both the original author and the author of the revised edition.  So let's have a look at some of the exercises in the book.

The first lesson has the following as a reading exercise (this is my translation of the original Gaelic)
The dog is at the door. The cat is on the floor. The swan is on the lake. The seal is on the rock. The man has a head. The cow and the bull are in the meadow.
There is a fort on the hill and there is a man in the fort. What is this? This is a hole. What is in the hole? There is a mouse in the hole. Where is the foal? The foal is in the stable. The boy is at the door with the cow....[etc]

This makes the mistake that Jespersen highlights of being disjointed and "jumping around" between subjects, but is certainly not as bad as his examples.  Jespersen's focus on the disjointedness misses the problems of the individual sentences. The author of the Gaelic book is trying to paint a picture, but he is writing expository text here -- his main goal is still to show the grammar, not to be natural.  Because of this, he ignores the problem of introducing new subjects with a definite article.  "The dog" and "the cat" are fine, because we are all acustomed to talking this way about family pets.  But "the swan" and "the seal" are more troublesome, as I'm likely to ask "which swan?"  The definite article assumes that we have a shared idea of a particular swan or seal.  We're more likely to say things like "there is a swan on the loch", as this doesn't assume any prior knowledge of the swan (I can now use the definite article, because I introduced the swan with "there is...").

The second paragraph is where this really starts to get troublesome, because we hit that old schoolboy motivation-killer: answer in sentences. "What is this? This is a hole." "Where is the foal? The foal is in the stable."  Point out to any teacher that natives don't answer in sentences and you'll get a simple and very logical answer: the reason for answering in sentences is to learn the grammar.  This is the very definition of expository language -- examples that exist purely to demonstrate a language point.

And here's where the "natural" and "direct" methods' justification starts to unravel.  When you're in a monolingual classroom, the simplest way to prompt a student to say something is by asking a question and demanding a fully formed response.  This means that your "natural" method is pretty much guaranteed to produce expository language and not naturalistic or authentic language.

"Answer in sentences" has pervaded language learning, and we see it not only in monolingual methods, but often the bilingual classroom will present new language with a native language explanation followed by monolingual practice.  Even methods using pure translation will often fall into this trap.  The original courses by Michel Thomas did not, but many of the courses written by others under the brand after his death do.  The Japanese course is a perfect example of expository language gone wrong.  The learner is asked to translate "do you want this?" and then "no, I want that."  Now there may not seem to be anything terribly wrong with this at first glance, but think about this: when I am talking to you, what is "this" to me is "that" to you.  This is even more problematic in Japanese, as it has a 3-way distinction equivalent to the Shakespearean "this" (near me), "that" (near you) and "yonder" (near neither of us).  The author is so fixated on the grammatical and lexical contrast between the two sentences that the physical logic of the dialogue is lost.  Again, the expository displaces the naturalistic, and the problem of meaningless and nonsensical language reappears.  Similar problems with here/there/yonder occur in almost all of the Pimsleur courses.  If you listen carefully, you'll often find yourself asking where the hotel is, only to be told it's "there", meaning where you are.

OK, so I have mostly given examples from bilingual courses or courses with explicit instruction.

One of the most vocal opponents of explicit instruction among the internet set is Stephen Kaufmann, Lingosteve on YouTube.  He is adamant that the only way to learn is by understanding bits of language.  He's put together a fairly sophisticated website dedicated to this idea, LingQ.  Kaufmann really hits that "denigration of others" that Decoo points out.  His whole argument is based on the same idea as Jespersen: he associates unnatural language with conscious methods.

But if we have a look at LingQ, will we find evidence of naturalistic or expository material?  Hmm....

Here's the first few lines of the first lesson in Portuguese (my translation):
"Welcome to LingQ.  My name is Mairo. What is your name? I live in Brazil. Where do you live? Do you want to learn Portuguese?..."

The conscious contrast between Mairo's personal information and his request for information from the learner is clearly expository.

And now an early Spanish lesson (again, my translation):
" Listen and repeat: What is your name? My name is Ana. What is his name? His name is Juan. What is her name? Her name is Maria. What age are you? I am 25 years old. What age is Juan? He is 22 years old. How old is Maria? She is 19 years old."
Here again we have clear expository goals: 1) question form vs statement form; 2) contrasting 1st, 2nd and 3rd person conjugations; 3) contrasting masculine and feminine pronouns in the 3rd person.

So even though we aren't going through any native-language instruction, we still get the problems that Jespersen was railing against.  The problem was not the medium of instruction, it was the material.

One form that is very widely used in both monolingual and translating courses is the dialogue.  Some of LingQ's texts are two-man podcasts.  Teach Yourself and Colloquial start each section with a dialog.  Assimil is based almost entirely on dialogues.  Dialogues often include the "answer in sentences" problem as described above, but not always.

The dialogue is said to give a natural context to the language, but sometimes this is assumed and the author ends up ignoring the naturalness of speech and produces a dialogue that is absurd almost to the point of meaninglessness, and becomes once more purely expository language.  This post was inspired by once such book: Beginner's Basque by Wim Jensen.  I can't say I was that hopeful when I picked it up -- it's by Hippocrene Books, who seem to specialise in cheap reprints -- but the first dialogue was worse than anything I have ever seen.  It comes with an English translation on the facing page, so I'll just use that (my comments are in italics.

Bernard: Good morning! I am Bernard. I am a boy. (Would anyone say this?  Certainly, the other person should be able to see that Bernard is a boy, so the effect is of someone with a learning disability.  Except that Bernard is not a boy.  The voice you here is of a man who would appear to be in his late twenties or early thirties.)
Johanna: Hello! I am Johanna. I am a girl. (Classic expository language -- using almost exactly the same structures with a word or two changed.  Again, the effect of learning difficulties comes through, and again, the voice actor is clearly an adult.)
Bernard: My name is Bernard. (Expository -- it restates known information needlessly, simply to demonstrate a different structure) I am Johanna's brother. (Woah there.  Who exactly is Bernard supposed to be talking to? I thought he was talking to Johanna, but there's no way he'd say this to her.)

Johanna: My name is Johanna. I am Bernard's sister. (Again we have an expository near-exact repetition, and again it really doesn't feel like Johanna's talking to Bernard.  Maybe they're introducing themselves to us?  Like a "piece to camera" in a video course?  It's not a particularly natural context though - it's what they call "breaking the fourth wall".)
Bernard: Johanna is a nice name. Your name is nice. (Nope, Bernard is clearly talking to Johanna.  But here again we have repeated information for contrast of structures, in this case attributive vs predicative adjectives.  Naturalisticness has been sacrificed again in favour of exposition.)
Johanna: Yes, it is nice, but Bernard is a nice name too. (And here we have a partial "answer in sentences" and more redundant echoing to demonstrate a particular form.)

Bernard: I am very glad. (??)
Johanna: See you!

This odd dynamic continues throughout the book.  The final dialogue in the book sees Johanna and Bernard discussing a family trip to the mountains.  From the dialogue, they clearly both know the plan, and take it in turns to say parts of it.  Who exactly are they presenting information to?  They are either saying things to each other they already know, or they're talking to you,

So really, dialogues are no kind of magic bullet.  Simply shifting your expository language into a dialogue does not automatically make it natural or meaningful.  Often it forces the author to be more consistent and coherent, but on the other hand, it can actually amplify the absurdity of some sentences by creating a clash between the expected behaviour in the context and the actual words of the participants.

But then we come to one of the most inexplicably popular figures in foreign language learning: Stephen Krashen.  Krashen was one of the big figures in the latest reincarnation of the direct/natural methods (and as Decoo says, in language, every method comes back again and again) and he was big on avoiding rules.  One of his justifications was getting people into "real" language "as soon as possible".  But as I said previously, supporters of authentic material allow it to be doctored and still call it authentic.  Krashen takes this self-deceit a fair bit further by that weaselly phrase "as soon as possible".  "As soon as possible" accepts that it's not possible right from the word go.  Have a quick look at a video of him in action, in a lecture he gave on his theories:

If you think about it, what did he start with?

He took a naturalistic piece of German and demonstrated that it wasn't an effective teaching strategy.  Then he presented a piece of very contrived expository language and called it "comprehensible input".  But it was not comprehensible.  Certain words and phrases were made very obvious, but you did not understand "what he said", but rather fragments of it.

So we go back to Jespersen's original argument -- that bilingual courses result in unnatural examples of the target language.  But monolingual courses are worse -- Krashen demonstrates quite aptly the opposite of his argument: that it is impossible to teach monolingually with natural language.  The one thing in favour of monolingual learning is that it does restrict the artificiality of the language -- the language must be unnatural to be understood, but it cannot be nonsensical or it will not be understood at all.

In that case, monolingual teaching is a bit of a crutch -- it gives us better results without having to fully address the problem.  But without these restrictions, and with a bit of brainpower, a bilingual course can do so much better.  It is extremely hard to elicit sentences like "do you know where it is?" and "I'm sorry, I didn't see you" in a monolingual classroom because of the non-specific function words, but these are extremely natural precisely because of those words; meanwhile they are actually very easy to prompt for by translation.  And once we're into function words, we move onto modality -- needs, desires etc.  These are very difficult to pick up from input, but in the Michel Thomas courses (the originals, not the potboilers produced posthumously), "wanting" appears 15 minutes into the course.  In Italian you'll be saying "I don't want to know", in German "What do you want to eat?" and in French "I would like to speak French". In the Spanish course it's actually held back until a full half hour into the course. *gasp*

Compare Krashen's demonstration with Thomas -- Krashen necessarily gives us easy words, because he relies on physical demonstration.  Thomas gives us words and structures that have vast conceptual meaning, but a very abstract, non-physical concept.  Krashen and his supporters would argue that because we are learning through translation, we are learning to translate.  Yet Krashen has never given any good demonstration of a reliable way to learn this very important functional language.  When it comes to grading authentics, it's the functional language that we generally need to remove to make it what he calls "comprehensible input", because it's inherently non-obvious.  If you want to get into native materials "as soon as possible", it's the non-obvious stuff that you need to teach/learn "as soon as possible".

So Jespersen is mostly wrong.  Yes, the worst examples of meaningless expository language could only occur in a bilingual course, but the cure is not to go monolingual, because only a bilingual translating course can employ genuinely natural language.

Word. New word. Different word.

One of the biggest problems facing many learners today is the problem of incidental vocabulary.  One of the prevailing themes in education is the preference for so-called "authentic materials".

"Authentics", to use a bit of teacherese, is just another word for "native materials".  Except you're allowed to doctor them a bit without making them less authentic.  Once you hit upper-intermediate, you will be subjected to more and more authentic materials.  Your lessons will be based around texts (that damn word again) drawn from newspapers, magazines and websites, or excerpted from novels or non-fiction books, and will be arranged in chapters based on themes like science and technology, arts, education and the like.

But there's suddenly a problem here.  Throwing together a heap of articles that are related in topic is all well and good if your goal is to study the topic, but if you are aiming to study the language, then the relationship between the articles is entirely superficial.  The structures that you are looking for just won't be there.

The simplest manifestation is in vocabulary.  Key vocabulary in that would be incidental for a native sends many a learner scuttling for his dictionary.  So a word appears once only in the article, and never again in the entire course -- that dictionary time is wasted as the learner does not learn the word.  In a textbook chapter on technology, you can switch from satellites to biometrics to textiles.  Each of these semantic domains has a vastly different stock of basic words.  The learner is left trudging through a bog of heavy, unknown words, and looking back later those are lost in a fog, never to be recalled.  A cursory look back may occur in a programmed "revision lesson", but it's a random sample of a fraction of the language covered, and usually limited to matching exercises (word+definition, sentence halves or question+answer) that tacitly admit that the student isn't capable of recalling the word without heavy prompting.

The same occurs with grammatical patterns, where a pattern may appear and be taught in one text, but then subsequent texts don't support, revise or otherwise consolidate that structure.  On the other hand, it's in the grammar that a lot of doctoring takes place, with complexity thrown out of window to make the text easier to understand.  But it is very difficult to program for increasing complexity when you're not writing the material yourself.

The end result is a lot of wasted time -- lots of material is presented to the learner, and little retained.  The goal of exposing the student to native materials slows down the learning process, and in effect means that students may leave the course less well equipped to deal with native materials than they otherwise would be.

But wait... there's more!
There is a secondary consequence of this.

The reliance on themed material favours the well educated, because they are more likely to have knowledge of the topic under discussion, and in many language pairs, the terminology will be very similar.  For example,  "biometrics" in French is "la biométrie".  This particular example came up in my last French course, and my classmates were confused by it, but as I already know a few things about biometrics, there was nothing in the articles themselves that challenged me.
The well educated and well read get better marks, and we justify this with by accepting that people who have done well in the past are "good students" and that those who haven't aren't.  This is only marginally better than the old fallacy of equating education and intelligence.
But in reality, whether the historically "good" students are genuinely better than those without a good history of attainment, that's a side issue.  The material presented favours those who really have the least need for favours.  In my book that's a bad thing.

It's particularly worrying to me that the course I was studying is with an institute of higher education that takes pride in the fact that it makes learning available to everyone, regardless of educational background.