edX 6.00x – the MOOC that failed to scale

I haven’t written very much recently about 6.00x, because other than the lectures posted for weeks 10 and 11 (which were excellent as usual) there’s been no measurable progress to report.

At the time of the second midterm exam the course team announced that it was going to drop one of the problem sets for this presentation, with the final two problem sets (9 and 10) due to be released on 12th December and 19th December respectively.

However, it’s now 18th December and no finger exercises for weeks 10 and 11 have been issued and nor is there any sign of problem set 9. Worse, there’s been no official communication from the course team about the absence of exercises and problem sets, with the last course-wide message posted covering the breakdown of scores from the second midterm exam. There have been a couple of staff responses published in answer to predominantly  polite and constructive questions posted on the course forum, with one reply from a staff member saying that the reason for the lack of announcements was due to the difficulty of posting such information on the edX platform! Further information provided in another answer suggested that despite the rapid increase in numbers of people working on edX as a whole, there was only one person working on publishing the problem sets for 6.00x and getting the automated graders to work.

This state of affairs is a massive and negative contrast to my experience of 6.002x run earlier on this year. Lecture materials and labs were consistently published around 2 weeks ahead of the schedule, allowing the type of learner that online courses are aimed at to plan ahead around family and work commitments. I can’t remember there being any significant problems or outages with the problem graders on 6.002x either.

Perhaps the reason for the current issues with 6.00x is that the concept of edX is simply failing to scale. By that, I don’t mean that the computing platform they’re using is unable to scale – quite the opposite, with around 7,700 students having tackled at least one question on the midterm 2 exam. Rather, this experience appears to suggest that the idea itself is not capable of scaling under the auspices of a single organisation trying to run multiple courses simultaneously, all of which were originally designed for traditional (rather than online) methods of delivery. It’s also been apparent that the presence of the originator of MITx and edX, Anant Agarwal, which was so obvious during the first run of 6.002x, has had no equivalent on 6.00x. From my perspective as a student it feels that the team behind 6.00x has struggled to deliver a smooth learning experience because the effort required in course conversion and leadership had been somewhat underestimated.

It’s all very frustrating as what has been a very interesting course has been soured by these issues. edX, despite all of the goodwill surrounding it has failed (so far) to deliver 6.00x to a standard that would persuade me to try another course from them in the near future – free or otherwise.

MITx 6.002x: post mortem 2

As promised, I’m now going to spend a little time reflecting on the experience of 6.002x as a whole.

1. Course content

As I said in my first post-mortem post, I was a little disappointed that there wasn’t more digital and more practical electronics in the course. There were a couple of weeks where I honestly thought I’d joined a maths course rather than an electronics one. However, having survived the whole experience I believe that the maths really was necessary to gain a proper insight into the subject. The content delivered was coherent and usually interesting (but please, the over use and misuse of  “fun” and “cool” is really annoying to English ears of my age) so I think it was my perception of the course before I took it that was to blame for my (very) slight disappointment of its scope.

2. Teaching method

6.002x attempted to re-create the classroom experience by presenting lectures in sequences. The vast majority of these consisted of voiced-over whiteboards (with what was sometimes a hilariously inaccurate scrolling transcript) consisting of a combination of legible information (pre-drawn) and when finished, almost illegible handwriting. If this approach is to be retained in future, the legibility has to be a key area of focus. There’s a big difference between the meaning of vi, vI, Vi and VI in large/small signal analysis and while it’s obvious when you’re listening to the soundtrack, it’s definitely not easy to unpick if you review the lecture slides afterwards. The transcript rarely got the differences correct too.

The lecture segments of each sequence were interspersed with tests and video demos. The demos were interesting, if  amateurishly shot and produced. I gave up doing most of the test problems after the first couple of weeks as I didn’t have enough time to devote to those as well as to understanding the material presented and completing the labs/homeworks. That observation is more of a feature of the way I like to learn rather than any problem per-se with the idea of tests or quizzes as a teaching method. I always used to largely ignore those kind of exercises in OU courses too. At least they were optional on 6.002x!

Sadly, I didn’t get much opportunity to watch the tutorial videos. The few that I did watch were excellent and had far clearer handwriting than the main lecture segments.

The recommended textbook for the course at a little over 1,000 pages was a monster! While incredibly comprehensive it was rather dull and worthy in tone – very unlike the personality of at least one of its authors! It really made me appreciate how good OU textbooks are.  Even the monster that was the DD303 Cognitive Psychology textbook was more digestible.  However – the textbook was made useful by the excellent signposting of the chapters in the lecture sequences. This factor alone made it rather more approachable and less daunting than it otherwise would have been.

However, my main criticism of the teaching approach was its slavish attempt to replicate the classroom and university experience. Frankly, it doesn’t translate at all well and becomes intensely irritating very quickly. The course team would do well to examine the lessons learned by the OU over the last 40+ years about how to present degree level material to distance learning students. I honestly don’t think that voiced-over handwritten lecture slides presented in real-time, with all the mistakes being made by the lecturer being corrected on the fly works as a teaching technique in this environment. It certainly didn’t enhance my learning experience.

3. The MITx virtual learning environment (VLE)

Beautifully minimalist, well organised and easy to navigate and find material in, it makes me wonder what on earth the OU and other HE institutions see in bloated monsters like Moodle. This VLE framework will be a real asset to edX in the future.

The discussion (question) forums have a couple of excellent features – such as the ability to tag posts (I found it incredibly useful and usually well done by the contributors) and the ability to easily identify staff contributions. However, the karma system (whereby students earn points off peers and can eventually become forum moderators) is ridiculous. There is rarely any correlation between a good forum moderator and valuable contributors. Indeed, during the second half of the course a number of  karma induced moderators appeared to positively relish their power and in some cases I believe it was abused – bullying behaviour is never acceptable in real life or online.

In future presentations I hope that at least two forums are set up – one for on-topic questions and the other for general chit-chat. I also think that it would be better to appoint student moderators and have them adhere to a published code of conduct – a bit like the OU(SA) forums ‘spirit of conference’ charter, rather than using a karma system that can be easily subverted.

I can’t remember if I’ve ranted about wikis in the past. The one created on the course was ok as far as they go, but they are useless as a learning tool. Psychological studies show that it’s far better to spend time actively making notes in a way that suits your learning style rather than trying to make them conform to some arbitrary ‘best practice’ format. You can always share them ‘as-is’ if you really want to afterwards – and there were a couple of students who did this. Their notes (alongside my own) were far more valuable to me than the sterile environment of the wiki.

The star of the VLE though was undoubtably the circuit sandbox. Even with its continually irritating little quirk of deleting a components when I was trying to input a value for them! Very, very good indeed and I hope that more use of it is made in future 6.002x presentations as a teaching tool.

4. Assessment

I liked the way that labs and homeworks were provided every week and contributed to the overall course grade. I even think that the ‘exam’ format used on 6.002x (more akin to an OU EMA rather than a traditional exam) is a reasonable way of proceeding with assessment in the future, rather than having to incur the costs of travelling to and taking exams in a testing centre – provided that something can be done to deal with plagiarism and cheating.

However, I really struggle to see how the totally automated ‘right/wrong’ assessment format used could be extended to other science or social science courses (such as psychology) that rely on the construction of well argued and evidenced essays – and I suspect it would never be able to transfer as an assessment format for humanities and arts courses.

5. Statistics

From the MITx 6.002x course information page:

6.002x had 154,763 registrants. Of these, 69,221 people looked at the first problem set, and 26,349 earned at least one point on it. 13,569 people looked at the midterm while it was still open, 10,547 people got at least one point on the midterm, and 9,318 people got a passing score on the midterm. 10,262 people looked at the final exam while it was still open, 8,240 people got at least one point on the final exam, and 5,800 people got a passing score on the final exam. Finally, after completing 14 weeks of study, 7,157 people have earned the first certificate awarded by MITx, proving that they successfully completed 6.002x.

6. Overall

I’ve had a great time taking this course (it’s released the inner geek in me) and I’ll certainly take future edX courses should they appeal to my interests. I didn’t spend as much time as I really should have done on the course – probably a maximum of 5 or 6 hours per week (with the exception of the midterm and final exams, both of which took me around 8 hours to complete). Leaving aside the nonsensical hype about Massive Open Online Courses (MOOCs) and the way that they will change education forever (they won’t, but they’re a valuable addition to the overall HE landscape) I wish the team at edX all the best for the future.

MITx 6.002x: post mortem 1

Now that the final exam has finished and the only thing that appears to be happening in the 6.002x forum is a lot of bickering about certificates, I thought I’d write a couple of posts to finish my journey off. This first post will simply look at how I did in the final exam. In the second post, I’ll reflect more generally on the experience and document what I thought was good / bad / indifferent about 6.002x as a whole.

The final exam had 10 questions, each with a number of different parts (1 mark per part), with 47 marks available in total. As I’d written in an earlier post, I needed to score 2/47 to ensure a pass and 34/47 to obtain an ‘A’ grade. In the end, I finished with 32/47 – a comfortable grade ‘B’ pass. Most of the questions I had difficulties with covered material from the first half of the course rather than the second half. I put that down to the second half of the course having become a little more practical in focus – in short, it contained the more interesting material!

Question by question:

1. Strain (5/5) – a nice simple resistive circuit problem to solve just to get into the swing of things.

2. Logic circuit (7/7) – another “gimme” as far as I was concerned. If I had a single criticism of the course content (and it’s not really the course’s fault, more my own expectations of it when I started) it’s that there wasn’t nearly enough digital in it. But at least there was a question on what little there was on the topic!

3. Switched capacitor (5/5) – straightforward stuff involving the calculation of a couple of different time constants. It took me two attempts to get all of the parts correct, as I hadn’t originally spotted that I’d need to re-calculate the time constant when the circuit was switched through the second capacitor … durr.

4. Bipolar Junction Transistor (0/9) – this is where I lost any chance of an ‘A’. It wasn’t really anything to do with a BJT – more a large signal analysis of a couple of voltage sources and a voltage controlled current source, followed by a small signal analysis. I got hopelessly lost and didn’t have the time to go back to first principles to sort it out. I still don’t think that it was a difficult or unfair question – I simply messed it up. Oh well.

5. Op amp with an RL filter (2/3) – straightforward, but I still manged to get the final part wrong as I’d missed out the RS resistor in the algebraic expression – I must learn to write more clearly.

6. Op amp FET (0/2) – I’ve no idea even now about how to solve this one! Hopefully the course team will publish a worked solution at a later date.

7. Trapping noise (3/4) – I have no idea what the part I missed out was asking for – otherwise, it wasn’t too bad a question.

8. Increasing Q (6/6) – no real difficulties with this one, apart from inexplicably multiplying one frequency by 2pi and not the other when working out the bandwidth on my first submission. I sorted that mistake out second time around.

9. Scope probe (4/4) – a repeat of one of the homework questions from a few weeks ago, just with slightly different values this time. Straightforward therefore.

10. Triode amplifier (0/2) – the course team weren’t joking when they said it was ” intended to stretch you beyond the material that we explicitly taught in this class” and “do not work on it until you have finished with the other problems.” More of a “WTF” moment than a “Aha” moment I think.

So 68% on the final exam; 86% overall. A very solid B and I’m pleased that I managed to stick with the course all of the way through.

MITx 6.002x final score: close, but not close enough

MITx 6002x - final score

The chart shows I gained a total of 86% overall – meaning I miss an ‘A’ by 1%. Rats!

As always, the questions I answered on the final exam seemed fairly straightforward and the ones I didn’t attempt seemed impossible! Even if I’d not made a silly mistake on one of the parts of the questions I did answer, I’d have still been short of the 87% mark by around 0.2% – and there definitely wasn’t another part question anywhere on the paper that I could have answered.

As the exam doesn’t formally close for some late starters until 1200 GMT tomorrow (11th June) I’ll leave my post-mortem until a later post. I’ll also reflect on the course as a whole. One of my original motivations for taking 6.002x in the first place was that I wanted to understand the strengths and weaknesses of the MITx approach as applied to teaching and having survived the experience over the last 14 weeks I think I’m in a much better place to write about that now.

All the best to anyone still grappling with the exam; congratulations to those who have passed and commiserations to those who have just missed out.

MITx 6.002x – almost time for the final exam

I’ve just completed the last homework (week 12) for 6.002x, so I’m running rather behind schedule as I haven’t even looked at the lectures for week 13 yet.  Normally, this wouldn’t be much of a problem, but as week 13 contains the final two sequences that could be tested on the final exam, I need to get a move on. Fortunately, in line with MIT practice, there’s no homework or lab for week 13. Equally fortunately, I’m very pleased that the Jubilee celebrations mean that I  have Monday and Tuesday off next week too!

All being well, these lucky breaks mean that I should get to the final exam in reasonable shape.

It’s going to be made available on June 6th from 2200 GMT (2300 BST) and will close at 1200 GMT (1300 BST) on June 11th. As with the midterm exam everyone has 24 hours from opening the paper to complete it, with 3 attempts per question permitted.

I’m currently 1.2% off the ‘C’ passing grade – which means I need to score just 3% on the final exam to gain my certificate. To obtain a ‘B’ I’d have to score 28%, with an ‘A’ available if I manage to achieve 72%.

Best wishes to everyone who’s going to attempt the 6.002x final exam next week. I wonder how many of the 120,000+ who originally registered for the course have made it this far?

MITx 6.002x week 10: (sine) waving but not yet drowning

This week’s lectures have been about the response of networks to a sinusoidal drive and how to analyse them. First of all, this involved using an incredibly difficult method based on solving differential equations (so difficult that the attempt terminates part way through after much baffling mathematics), a “sneaky” approach based on complex algebra and finally a “super sneaky” approach based on the impedance model.

This final method turns all of the steady state sinusoidal circuit analysis problems which seemed pretty difficult using the first two methods into problems which can be rather more simply solved by the application of Ohm’s Law, along, of course, with all of the usual circuit analysis techniques based on the node method, Thevenin, Kirchoff et al.

I think if we hadn’t been warned that the conclusion of the week was going to be relatively straightforward I might have been tempted to cut my losses and plough straight on into week 11 – but I’m glad that I didn’t. In the end, the lab and homework problems seemed to be fairly tractable once I’d thought about them properly – and been guided by the odd hint or seven from the discussion forum of course!

I now have the magic 59% mark showing up on my profile page – which, even if I complete the next two weeks homework and labs, I won’t be able to improve on until the final exam. Despite many pleas from students to the course team on the discussion forum, they still appear to be keeping silent about the form the final exam will take, when it will appear, how long we’ll have to complete it in and so on.

Not knowing when the final exam will appear is pretty frustrating, as one of the “joys” of being a part-time distance learner is that the rest of life tends to get in the way of study, in exactly the way it doesn’t when you’re full-time at a brick university.

One of the lessons therefore that the MITx/edX team ought to take from this first run of 6.002x is that certainty over the time windows for assessments at or very near the start of the course is essential. Without such certainty, it’s difficult to see how edX would ever get future students to pay for assessment, even if the delivery of course content remains free.

MITx 6.002x week 9 – the school of hard sums

This week’s theme on 6.002x was hard sums. Second order differential equations to be precise. However, for anyone still struggling to get through the lectures, don’t give up! It turns out that most of what we’re expected to do is find the characteristic equations of second order circuits – and that’s all. Certainly none of the lab work or homework required any difficult calculus.

Being exposed to all the maths was simply meant to be good for the soul apparently. I suppose that it does make sense to have at least a vague understanding of why circuit problems involving resistors, inductors and capacitors can be solved using just the characteristic equation of second order differentials, even if all the maths made the lecture sequences for week 9 pretty daunting at first.

Something else I learned this week was that unlike the rest of the world, electrical engineers write the square root of -1 as j, rather than i. Seeing j used in this way had me reaching for my favourite Internet search engine to make absolutely sure that I hadn’t missed something important since I took maths A level! I guess that it makes sense to do this so that it doesn’t get confused with the symbol for current. On that basis, it makes it even more of a shame that the subtle but important differences between VI, vI, vi and Vi when scrawled in the hand of Professor Agarwal are often lost. One conclusion that I’ve already drawn from this course is that it’s amazing that any electronic circuits ever get built correctly, given the huge potential there is for errors in notation.

Anyway, I’m hopeful that there will be no more hard sums on this course. On second thoughts, that may be somewhat of a vain hope …

MITx 6.002x week 8 – just about keeping up with the schedule

Having basked in the afterglow of a successful midterm exam for a little while, I returned to week 8 of the course on Sunday. The lecture sequences were on step, ramp and impulse inputs to RL and RC circuits, followed by a sequence on digital memory.

The first sequence of week 8 (S15) is a good example of a part of the course which really needs revising for online consumption before it’s presented again. The video lectures for this sequence were overly long and repetitive – making the whole experience boring (rather than “fun” or “amazing” as we’re always being told). As a result, it took substantially more than two hours just to watch the 27 separate videos (let alone understand their content). I really believe that the material could have been easily and better explained in half that number in 45-60 minutes. Slower is not always better – even if for some electronic circuits it may be!

It seems I wasn’t the only person that felt this way about this particular sequence as its provoked similar consternation in the course forum. There’s nothing new there – OU students were (and probably still are!) always good at voicing their feelings about pedagogic style in the module forums too.

However, what is particularly impressive on this occasion is the positive reaction of the 6.002x staff to the criticism of this particular sequence – well done, THuang! I’m not certain that other institutions would have reacted quite so well to student feedback of this type.

Anyway, on to the week 8 labs and homework. The lab was fine – once I’d realised that the value shown on the graphs didn’t relate to the end of the curve as I’d assumed – but the portion of the graph that was marked with a very faint dotted line! I wasted a couple of hours trying to figure out what I’d got wrong with my maths …

I found the homework questions for week 8 rather tricky on the whole. Part two was relatively straightforward, but part one was especially frustrating (though I got there in the end) and part three, on a memory circuit, was somewhere between the two in difficulty.

I got around to starting week 9 yesterday evening by attempting the lab – which I was able to complete without studying any of the material beforehand. Clearly, my intuition is being developed, just as Professor Agarwal promises!

The homework questions look rather more challenging though … time to get back to the video lectures. Fortunately, there are only 18 to watch in sequence 17!

MITx 6.002x – midterm complete

It’s Saturday afternoon(*) and I’ve just completed the 6.002x midterm exam. In total, there were five multi-part questions, covering some basic circuit analysis (Q1), Thevenin and Norton circuits (Q2), a common gate MOSFET amplifier (Q3), a logic circuit consisting of a couple of inverters and a NOR gate (Q4) and a diode circuit (Q5).

Q1 – which involved finding a couple of node voltages and the power drops over resistors in the circuit, Q2 and Q5 were straightforward, although I confess it took me three attempts to get the Norton current right for Q2 (even though I managed to get the equivalent Thevenin voltage right first time – go figure!). Q5 was a repeat of an earlier homework question, but with slightly different values used, so it was pretty simple.

Q4 was a little trickier. I manged the first few parts of it easily, but then managed to over complicate the calculations for the falling and rising time constants so ended up taking a second attempt to get those values correct. I spent ages (and all three attempts) trying to work out the maximum power that could be used by the circuit, assuming it obeyed the static discipline. Eventually, I realised that only two of the MOSFETs could be on at the same time (rather than the three I’d assumed originally) and managed to get the answer right.

The question on the common gate MOSFET amplifier was, for me, the toughest of the lot. On the first attempt, I found that I’d only managed to get one part correct. Furious scouring through my notes, the textbook and a couple of other sets of notes from the internet saw me get three out of five parts correct on my second attempt. My final attempt saw me get four out of five parts correct. I’m kicking myself over the part I got wrong – as it was one of the large signal parts of the question and I’ve now figured out what I managed to mess up. D’oh, as a popular cartoon character might exclaim.

So my overall course progress now looks like this:

MITx 6.002x progress - after the midterm exam (and part way through week 8!)
MITx 6.002x progress - after the midterm exam (and part way through week 8!)

I’m part way through week 8 at the time of writing (and had got part way through the lab for week 8 by the time I remembered to take a screen capture of my progress), so unless I manage to mess the remainder of the course up spectacularly, the only question that remains is what level of pass I’ll obtain. I’ve set my sights on an “A” now of course, but I’d be happy enough with a “B” too. There was an ominous warning on the course information page a few days ago that the algebra and maths was about to get difficult again in week 9 …

 

(*) In compliance with the MITx honour code, the publication of this post was deliberately delayed and scheduled to appear well after the 30th April 1200 GMT midterm exam deadline.

MITx 6.002x week 7 – no hard algebra or maths for once!

I wrote yesterday that I was concerned I needed to complete the week 7 materials, homework and lab this week, as well as tackling the midterm exam. However, encouraged by some of what I saw on the 6.002x forum yesterday evening, I dived straight into the material this morning and attempted the homework and lab this evening.

What a relief!

MIT must be feeling kind. Or perhaps the midterm exam really is going to be a stinker …