R is for Research

“Where’s your evidence?!”  Given the vehemence with which some methodological matters are argued (especially on blogs), it’s surprising that this question doesn’t come up more often. Well, a good place to start looking for evidence might be The British Council Directory of UK ELT Research, compiled by Shelagh Rixon and Richard Smith,  the primary aim of which is “to disseminate and share information generally in the area of UK-based ELT research”. (You can access it via the TeachingEnglish website here).

It makes a fascinating read. I was particularly interested to find out what people have been researching at doctoral level. (Where else can you find this information, after all?)  These are just some of the topics that have recently been investigated, and whose findings I’d love to get my hands on:

• The lexis and grammar of English as a Lingua Franca
• The use of interactive whiteboard technology
• ‘Nativespeakerism’ and the status of non-native teachers
• Formulaic language and SLL
• A systemic view of emergent course design
• Collaborative learning via e-mail discussion
• Group influences on individual learner’s motivation

And this one, not least because it is research about the effects of research:

Andon, N. 2008. What roles do theory and research play in language teaching? A case study on the task-based approach in language teaching.

The researcher’s aim was “to examine the ways that language teachers make use of theory and research presented to them in the professional literature and on training courses”.

As both a writer of ‘professional literature’ and a teacher educator, this goes to the heart of what I do. I’m often accused (and probably guilty) of selecting research evidence to support my own point of view, and ignoring that which doesn’t;  or, worse, of not having any evidence at all. This is particularly the case with the Dogme ELT philosophy:  it’s not enough to wheel out a supportive bibliography in order to situate Dogme on a  firm theoretical base (as I did on Jeremy Harmer’s blog recently). Nor will anecdotal evidence do: the Dogme discussion list is strewn with feel-good accounts of  ‘successful’ materials-light, talk-driven classes. But people want concrete proof. They want research evidence.

Fair enough. But what kind of evidence would that be, and what’s the guarantee, anyway, that this evidence would satisfy the sceptics?

Let’s take Dogme: how could you provide convincing evidence that it works? Here are some possible lines of attack:

1. Measure the outcomes of teaching two matched groups, one taught with coursebooks, one taught without. Problems: too many variables (teacher, students, context factors…); what outcomes do you meaure (fluency? accuracy?) and how do you ensure your assessment criteria don’t automatically favour one approach over another? Also, it would probably need to be done over an extended period to produce significant findings.

2. Record and transcribe a sequences of ‘Dogme-style’ lessons, and track ‘emergent language’, i.e. language that learners have seemingly appropriated and then re-used subsequently, thereby showing that learning can take place without a pre-selected syllabus and solely through interaction. Problems: an enormous amount of work (all that transcription); you would also need to pre-test – but what would you be pre-testing for if the learnt language is not pre-selected? Also, without a control group, there’s no way of knowing if that same language would also have emerged in a more orthodox setting.

3. Ethnographic case study of a Dogme class over an extended period, using observations, interviews, questionnaires, etc, to gather a ‘thick description’ from the point of view of the participants. Problems: nothing to compare it with; too context specific, hence ungeneralisable; ‘Hawthorne effect’, i.e. subjects out-perform when they know they are being experimented on; attitudinal questionnaires are unreliable – subjects say what they think you want to hear.

4. Fine-grained, micro-analysis of classroom interactions in a Dogme class compared to a ‘traditional’ class, to demonstrate, for example, a greater quantity of and/or better quality of communicative, scaffolded, authentic, creative, etc language use in the former. Problems: again, assuming you could control the variables, the specificity of the findings is unlikely to satisfy the sceptics; also, because the findings are evaluated through the lens of a specific theoretical model – e.g. sociocultural learning theory – your conclusions depend on this theory being generally accepted – which it isn’t.

Have I missed anything out?

In the end, though, there’s probably nothing you can do to convince the doubters (let alone the cynics). Which makes one wonder: why do research at all?  One way of answering this question might be to re-assess what research is capable of achieving. Nunan (1992) distinguishes between two alternative conceptions of research: “The first view is that external truths exist ‘out there’ somewhere.  According to this view, the function of research is to uncover these truths.  The second view is that truth is a negotiable commodity contingent upon the historical context within which phenomena are observed and interpreted”  (p. xi-xii).   Researchers in the second tradition are interested less in proving a theory than in deepening their understanding of their own situated practices. This understanding may, in turn, influence the way these practices evolve.

But isn’t this a cop-out? Is there no way my research can be generalised to your context?

Reference:

Nunan, D. (1992). Research Methods in Language Learning. Cambridge: Cambridge University Press.