Tuesday, March 19, 2013

Is it a course? Is it a textbook? No, it's a MOOC!

Is it a Bird? is it a Plane? by Flicktone, on Flickr
Creative Commons Attribution-Noncommercial-No Derivative Works 2.0 Generic License  by Flicktone

It's hard enough trying to evaluate regular education because learning is not easily measured; if indeed we can even hope to do so. We can of course test to see what people have remembered or whether they can explain concepts in a clear and logical manner but how do we know what they have actually learnt? So if evaluating traditional education is tough how hard is it to evaluate a MOOC? A bit like nailing jelly to the wall as a colleague once said, though I can't remember when.

MOOC providers may believe that they are offering a university course and duly organise it as such but do the participants share that belief? If you sign up for a regular university course you make a commitment, invest money and time and expect a reward for your efforts. You buy into the course concept, accept the university's terms and step into line. Everyone understands what they're involved in and accepts the rules. When you sign up for a MOOC, however, you don't buy into the concept in the same way. You join the course for your own reasons, have your own objectives and preferences and are often uninterested in the certification that may or may not be available at the end. Many have no intention of completing the "course" at all, they're simply curious to see what the fuss is all about. When there is no consensus as to what the MOOC is, an evaluation becomes extremely difficult.

Stephen Downes' post, Evaluating a MOOC, claims that we can't evaluate MOOCs with the same criteria as traditional courses since they aren't actually courses. They are many things to many people.

"I think the best way to understand success in a MOOC is by analogy with, say a book, or a game, or a trip to the city.
The person taking the MOOC is like a person reading a book, playing a game, or taking a trip to the city. It is impossible to talk about 'the objective' of such an activity - some people want to learn something (and others something else), others are doing it for leisure (and others as part of their job), others to make friends (and others to get away from their friends for a while), etc."

A successful MOOC creates enough momentum to continue after the "course" is complete. Since the motivations of each participant are so different it is more relevant to evaluate the MOOC as an entity and look at what the community has achieved and how vibrant and self-sufficient that community has become. MOOCs will develop into new forms and new terminology will emerge to differentiate between them. Some will be traditional courses with a syllabus and learning outcomes and will expect students to accept the rules of engagement whereas others will be more fluid and become learning communities with learners coming and going, some staying for months and others simply dropping in for inspiration.

"MOOC success, in other words, is not individual success. We each have our own motivations for participating in a MOOC, and our own rewards, which may be more or less satisfied. But MOOC success emerges as a consequence of individual experiences. It is not a combination or a sum of those experiences - taking a poll won't tell us about them - but rather a result of how those experiences combined or meshed together."

The important point is that evaluation is only possible when we know what it is we are evaluating and that all participants are aware of what they are engaged in. Confused? I still am but I'm trying to work it all out.


  1. Unfortunately, this idea of people having different aims, while true, also obscures a far more important truth: the weaknesses that the courses exhibit.

    Note that I'm not saying that MOOCs in particular have weaknesses, but that all courses have weaknesses.

    The unique thing about MOOCs is that we've got no reliable way of separating "people who drop out because they're not serious about it" from "people who drop out because the course is too difficult/confusing/insert-negative-quality-here".

    Without that as feedback, the course designer is robbed of the motivation to improve the course.

    In the end, MOOCs may end up being worse than traditional teaching, but "proving" themselves better with impressively large figures that only represent a minority of students....

  2. It's hard to analyse the real significance of the MOOC phenomenon just now through the hype fog.
    In Downes' interpretation the MOOC is a network and should be judged as that and whether individuals stay or drop out is completely up to them. It all depends if the MOOC is trying to be a regular course or a sort of learning oasis. The xMOOCs are caught between trying to emulate mainstream university courses as well as trying to organise thousands of participants, many of whom don't want to be organised.
    It all depends on how we view a MOOC.Is it a course or is it a learning community?