Bookies always win, but researchers don’t need to

One of the very first lessons that stuck when I started out on my psychology degree in 2007 was never to suggest in an essay that the results of a psychological experiment either proved or disproved something. This is because psychological researchers always express their results with reference to the probability that their findings might have been arrived at by chance. It’s common for researchers to reject this notion if the calculated probability of that having happened is less than 1 in 20. More precisely, researchers often state that their results are statistically significant if the probability of the null hypothesis being correct is less than 1 in 20 – the “magic” p < .05 .

This morning as I was sat on a train from Derby to London, studiously avoiding reading more of the personnel selection and assessment module for my masters, I came across an analysis of the odds at which horses have won flat races at over the last 8 years. In short, it concluded nothing surprising – the bookmaker always wins in the long run – but what did catch my eye was that 1,366 horses won races at odds of 20 to 1 or longer. In other words, these were examples where p < .05 (as calculated by the bookmakers) but the horse won anyway. Before anyone points it out in the comments, I do realise that I’m taking a statistical liberty or two by making this comparison …

Top Town Girl Top Town Girl

There are plausible explanations for this bias, psychological and otherwise, as the article in The Economist points out. Some of it may even come from our experiences as students. For example, I was constantly surprised at the large number of people on my undergraduate psychology degree who thought that a non-significant result in an experiment they were working on meant that they had somehow failed.

So now, when I read a research paper that reports a positive result, I’m always interested to see if other researchers have attempted to replicate the study or experiment and what their results and conclusions are. And while the presence or absence of negative results doesn’t necessarily tell the whole story, the absence of similar positive results always makes me rather wary.

This article was originally written for the University of Leicester Student Blogs, 19th December 2013.

One term down, five to go

My first few weeks as a distance learner at Leicester have flown by, so with one term down and five to go, I feel that ought to let you know how I’ve been getting on.

In hindsight, the first two weeks of the course were probably the most stressful. As well as having to find my way around an unfamiliar virtual learning environment (VLE), we had a two week deadline to submit a 1,000 word essay for our early assignment. As I’ve observed elsewhere, 1,000 word essays are far more difficult for me to write than 2,000 word essays.

However, perhaps the most confusing (all right, downright irritating) of all the new things to get to grips with was the requirement to submit assignments twice, once to Turnitin and once by email, using identical file names but without the references section and header page for the Turnitin submission. The process (for which there are no less than 10 pages of documentation in the course manual) feels like an accident waiting to happen. Even though I was incredibly careful working my way through it, I still made a small error that was commented on in my tutor feedback form. Grrr. Helping organisations to sort out broken business processes is part of my day job, so I’d be happy to sort this one out for the university for a small fee …

Last weekend I finished my second assignment for the Research Methods Module. Completing it has definitely helped to blow away the remaining academic cobwebs. While I was very pleased with my mark on the early assignment, this one (broken into four parts covering quantitative and qualitative research methods) has been a whole order of magnitude more difficult and time-consuming. Everyone in my family, including Archie the house-rabbit, are probably as relieved as I am that it’s been safely submitted.

Archie - Alpha Bunny So with the next assignment not due until 10th February, I bet you might think that I’ll be taking it easy over the Christmas break. Unfortunately not! As an experienced distance learner, I’ve always found it absolutely essential to use this time of year to get ahead of the schedule, so that the inevitable issues that crop up in my working and home life don’t totally derail the study effort. For me, it’s the Personnel Selection and Assessment Module up next. I’m also trying to put some serious thought into what I will be choosing to do for my dissertation in 2014/15.

On the whole though, it’s been so far so good. I’m enjoying the course and I’m getting what I expected to get out of it – and more. But if you would like more information about the joys and woes of distance learning, fellow Leicester student Chris has also made some excellent observations about the marketing hype vs the reality of distance learning, so I’d encourage you to read his thoughts too.

Onwards and upwards!

This article was originally written for the University of Leicester Student Blogs, 11th December 2013.

The Imposter

Progress with my MSc trundles on. I’m currently working on my second module assignment which to my great relief isn’t another essay. Instead, it consists of a number of short answer questions related to research methods .

Last weekend I managed to finish off around a quarter of the work and I’m hoping to make similar progress this weekend. Depending on how I feel when I’ve finished writing this, it may either involve me dusting off my copy of SPSS and crunching some numbers, or doing a piece of thematic analysis on an interview (and transcript) that we’ve been given. At the moment, I think that the stats questions seem slightly more appealing.

I’ve also made my debut on the university’s student blogging site this week. First posts are always incredibly difficult to write, but at least that’s one hurdle out of the way and I can concentrate on more interesting topics. (Yes, I know, not much hope of that is there …) For example, like why I changed the spelling of ‘realize’ to ‘realise’ on my first article a few hours after I’d hit the publish button – even though the OED tells me that I’m right and everyone else is wrong! Those of you who hounded me mercilessly until I made the change know who you are 🙂

Since I wrote my first post, I’ve had the result back for my early assignment. It’s worth all of 1/90th of the overall mark for the qualification, so in the big scheme of things it’s not particularly significant, but I’m pleased with my start.

I feel a little less like an imposter now.

OU taught masters degrees “not valued” and expect “no research”? What nonsense.

Tucked away on the OU platform website this week was an article written by Dr Petrina Stevens (the OU community representative on Senate). The full text of the article is available here and appears to be a brief summary of the key findings of the long-awaited postgraduate review, presented to Senate by Professor Sharon Ding.

I’m all too aware of the difficulties of getting the gist of what I hope was an incredibly detailed presentation and fiery debate into a brief article. However if what has been reported is an accurate reflection of the postgraduate review then I’m very disappointed indeed.

I have two main bones of contention with the article. First of all is the implication (presumably from the postgraduate review) that because some people don’t understand that Open University degrees are every bit as hard-won and rigorous in their academic standards as those awarded by other institutions, there must be something wrong with OU degrees per-se.

… the OU is fighting a perception by some people with no OU experience, that it offers degrees which have less value than those from more conventional and established universities. In order to be competitive and to attract students, the OU is changing the identity of its taught Masters degrees.

This kind of analysis is a classic example of mistaking a correlation (people who haven’t experienced the OU for themselves think its degrees are somehow not as good as other equivalents) with cause and effect (because people think this it means that the degrees really aren’t as good as those from elsewhere).

Even if, for the sake of argument, OU degrees were not every bit as good as those from elsewhere, I’m unconvinced that simply saying “everything is new and shiny now” is the right way to address such a perception.

The OU’s current marketing material doesn’t pander to these myths:

How highly are OU degrees rated?

Very. And not just academically. When potential employers see you have an OU qualification, it’s immediately obvious that you have other skills that candidates with non-OU qualifications might not – such as commitment and motivation. Students with the OU often have juggled many other commitments at the same time as studying and many employers have studied with the OU.

and:

We have more than 22,000 students studying at postgraduate level, many of them working towards a masters degree – e.g. an MA, MSc or MBA. We also offer PhDs and Research degrees (full or part time).

However, it’s the assertion that the current OU taught Masters programmes:

… by design these degrees omitted one element, and that was the chance for students to search out relevant and respected research (‘found’ material) to support their studies

which has annoyed me the most. This is a factually incorrect statement as well one which is potentially damaging to the reputation of the OU and of the students who have taken these qualifications.

Leaving the Masters degrees aside for a moment, my psychology BSc required independent research on both of the level 3 modules (DD303 and DD307). Even a couple of the level 2 modules (ED209 and DXR222) expected some of this too. It would therefore be very strange if at the Masters level all thought of independent research was simply abandoned!

And indeed, the (now withdrawn) taught Masters in psychological research methods certainly expected a significant level of independent research (I know, because I have a copy of the prospectus!) Even examining a few of the descriptions for currently available OU Masters programmes, it is obvious that self-guided research is almost invariably a significant component.

For example:

MA in English: “If you wish to develop your research and analytical skills and upgrade your qualifications the MA in English course is for you.”

MA in Childhood and Youth: “…you will choose either a small research project around a topic relevant to the module or a literature review. ”

Master of Laws: “Through your study you will have the opportunity to enhance your legal research skills and develop expertise in a chosen field.”

MSc in Engineering: “This MSc in Engineering course builds on previous study and offers you an opportunity to research an engineering topic in-depth.”

MSc in Technology Management: “… you can take Research project (T802), where the production of a dissertation will develop your academic research skills.”

I therefore really hope that this is a “rogue” article and that contrary to the impression it gives, the OU have been quietly investing their efforts in genuinely understanding how to best improve their taught Masters programmes.

The calm before the storm

… or perhaps I’m in the eye of the hurricane. There’s lots about to happen over the course of the next few weeks and months.

Changes at work mean that it looks as if I’m going to be busier than ever … but it’s nice to be wanted!

It’s about a week to go before the MITx circuits and electronics course starts. Despite that, there’s been no further communication from MIT about my registration or early access to materials … so I’m assuming that I’ll get some kind of email on the start day, which is Monday 5th March, and all will be revealed. Maybe.

I’ve finally got my act together and sent off the application forms for the Leicester University Occupational Psychology masters too. I’m hoping to hear something back from them in the next two or three weeks. Their online application process is quite slick and I’ve already had two notifications from them that they’ve received the references they needed from work and the OU. That’s really not bad going at all, as I only finally submitted my application on Sunday evening.

So things are getting busy and about to get busier.

And that’s without thinking about birthday parties, family holidays, an increasingly frail mother, graduation ceremonies (plural!), trying to get fit(ish), vegetable growing, trips to and from far-flung universities and time consumed by sundry DIY disasters. My efforts at sorting the light out above the shower over the course of the last couple of weeks deserves a post all of its own, but I’m trying to avoid thinking about it on the grounds that if I do, the light might just stay attached to the ceiling permanently. Gravity (and the light) have been winning so far.

Time to batten down the hatches. See you all on the other side of the storm. You’ll be able to spot me as I’ll be the one clinging to the life raft with a glass of red wine in my hand.

Lurking

When I graduated from Warwick University in 1985, I couldn’t wait to get away from the place.

I don’t think I even returned to the campus until the weekend of my 40th birthday – very nearly 20 years after graduating. It wasn’t that I didn’t enjoy my time there, but initially I couldn’t wait to get out into the world of work and then, after a bit of the shine of having money for the first time had worn off, I didn’t want to go back because I knew I might want to stay. Even the numerous times throughout the 1990s when I visited Sun Microsystems at their offices on the Science Park I made a point of not going onto campus!

I know how ridiculous that sounds. But what I’m about to tell you is equally ridiculous. Probably more ridiculous if I’m being honest.

Having graduated from the OU in December, I’m still lurking around.

Logging into StudentHome and reading the messages and wincing at the redesign; posting the occasional message in support of a social sciences “cafe” in the OUSA forums; reading people’s blogs; looking at the prospectus and even complaining to the OU that the information about postgraduate courses on one of the Open University’s advertising sites potentially breaks the advertising standards authority code of conduct as it’s misleading – you can’t, of course, currently start new postgraduate studies with them in either psychology or the social sciences(*).

So I need to stop lurking and get on with my application to Leicester University if I’m going to have a chance of studying again later on this year. I’ve already had the good news from my employer that they will part sponsor an MSc in much the same way they helped with my BSc, so I’ve no excuse not to get on with my application.

Now, what was the question on their application form that keeps making me smile? Oh yes:

Outline the changes you would make in your work and/or own time in order to accommodate the demands of a postgraduate distance-learning course

Been there, done that, got the scars.

(*) Update 27/02/2012 – the OU has finally updated their advertising site and have removed all references to social science and psychology masters courses.

News from the OU about postgraduate psychology provision

… has just arrived in my inbox from the Faculty of Social Sciences. It confirms what I’d been told by the Yorkshire region a few weeks ago, with the merest hint that something may be resolved in time for late 2012. The email is reproduced below, with my thoughts in italics.

Dear Tim Holyoake

I understand from PGSS colleagues in Region 07 that you have enquired about studying psychology at postgraduate level with the OU/Faculty of Social Sciences. I note from your academic record that you have recently achieved a first class honours degree in psychology with us. Warmest congratulations – this is a splendid achievement!

Thank you!

As you will be aware, the Faculty of Social Sciences has taken the difficult decision to withdraw its current postgraduate qualifications. This is due to the significant cuts in government funding of higher education institutions, as a consequence of which, it is no longer possible for the Faculty to sustain the number and diversity of its existing postgraduate modules and qualifications.

But the Browne review specifically excluded postgraduate study and funding from its remit. So this statement would seem to suggest a number of  concerns that the OU might have – such as the demand for postgraduate courses declining rapidly due to their potentially being far fewer graduates in future or the expectation that graduates will increasingly regard postgraduate study as unaffordable if they are worried about paying off loans incurred through undergraduate study. Either that or it’s simply a false statement – and frankly, I’m inclined to believe the OU on this rather than the politicians.

This ‘crossroads moment’ has afforded the Open University as a whole an opportunity to undertake a wide-ranging review of its postgraduate teaching and curricula.

I understand the need to review provision, but I think that most students were under the impression that this review was due to be completed in mid 2011, not sometime in 2012. The OU is about to lose a whole cohort – and possibly two or three cohorts – of suitably qualified graduates who would otherwise have gone on to postgraduate study with them. I certainly wouldn’t want to be one of the first students through a brand-new set of postgraduate modules!

As you can appreciate, all Universities need to refresh and renew their academic programmes in order to meet the challenges of an ever-changing world with ever-increasing demands for a more sophisticated and highly skilled workforce.

Indeed.

The University-wide review of postgraduate teaching is due to conclude in 2012. The Faculty is therefore not able to provide details of any future postgraduate programme at this time but we will publish information on our websites http://www.openuniversity.co.uk/pg-psychology and http://www.openuniversity.co.uk/pg-socsci when our plans are confirmed.

So no change from the situation of almost a year ago – except an apparent delay in the university-wide review of at least six months and probably longer?

I appreciate that this uncertainty makes planning further study with us somewhat tentative but would like to take this opportunity to wish you well with any future studies that you may undertake wherever this may be.

“Wherever” is the right word! It’s not only the OU which appears to be affected by the way in which HE provision has been mishandled by the current government. However, I’m still thinking that Leicester looks like a good option for me.

Kind regards

Hilary Canneaux

Senior Manager, Taught Postgraduate Studies

Faculty of Social Sciences

Thank you for taking the time to write to me Hilary. It’s not exactly the Christmas present I was hoping for, but at least it’s a clear confirmation of what I already understood to be the case.

So – one final thought – sign the petition if you haven’t done so already and are a UK resident. Please.

Will it be Leicester?

Having suffered a minor shock on Monday morning on hearing that Birkbeck are apparently reviewing the future of their masters qualifications in psychological sciences, I was relieved on Monday evening to find that the prospectus I had requested from Leicester University for their masters in Occupational Psychology had arrived. And very interesting it looks too. For a mere (gulp) £3,795 per year, a part-time distance learning masters could be mine in two years, starting in October 2012.

The modules on offer cover personnel selection and assessment, ergonomics, the psychology of organising, the psychology of occupational training and development, the individual at work, research methods and a dissertation. I wouldn’t necessarily choose to do a couple of the modules perhaps, but it seems to me that there would be sufficient wriggle room – particularly in the dissertation – to do something a little more exciting than study just psychometrics or undertake experiments of questionable value with teddy bears.

I was also reminded that the module on my current degree that I thought I’d get least out of (ED209 – child development) was rather good. I’ve even just about  recovered from the young lad on the ED209 DVD who continually insisted that the reason any object floated was “because it was shaped like a boat”.

Now, while I have absolutely no desire to end up in “personnel”  (after all, as Clint Eastwood as Dirty Harry once said, that’s for … well, he wasn’t very impressed when he was assigned to personnel after shooting lots of bad guys) there’s definitely enough in the Leicester course to be interesting and useful in my work too. Who knows, if I can convince our HR department that it would be useful to the business for me to do this course, then perhaps I can tease some sponsorship money out of them as well … (*)

Perhaps the biggest shock was their application form (Application form? I’m used to the OU – you just turn up, pay your money and you’re in!) They want to know all kinds of things about me, such as my previous qualifications, work experience, whether I can speak, read and write English and worse … why I want to do the course and if I already have any ideas for my dissertation! Oh, and they want references too. References! Anyone would think that they were going to be paying me, rather than the other way around…

In all honesty though, the form really isn’t that daunting … and I’m especially looking forward to answering the question “Outline the changes you would make in your work and/or own time in order to accommodate the demands of a postgraduate distance-learning course”. I’ll be able to (metaphorically) wrinkle my eyes up and with a furrowed brow, sniff the air and say something like:  “I was an OU student for five years … nothing could be better preparation for your course than that.”

But wait. I’m getting ahead of myself. It sounds like I’ve already made the decision to carry on – and I most definitely haven’t. There are still other courses to look at and other avenues to explore. There are also the views of my loved ones and their priorities and desires to take into consideration. I don’t want to be too selfish.

So – I’m making no decisions  until after Christmas at the earliest. It’s still quite novel having a life away from work that isn’t study … and it’s rather nice too. I’m not sure that I want to give it up again quite so soon – or ever.

Watch this space, as they say.

 

 

(*) Of course, I’m only joking about people who work in HR and personnel. I have first hand experience of how very dedicated they are where I work for example. But the “Dirty Harry” reference was just too tempting to resist. Sorry.

Call back in February – Birkbeck responds to my enquiry

I’ve just been speaking to a very helpful person at Birkbeck College regarding the enquiry I made last week about the masters courses run by their department of psychological sciences.

He suggested that I should call them back in February as there was a good chance that Birkbeck would also be cancelling its psychology masters courses for new students with effect from October 2012 – the next start date. When I asked why, he suggested that it was down to there being too much competition for students and also that their existing courses probably needed a complete redesign, as they weren’t working very well at the moment. It’s an honest, if slightly strange sales technique I suppose!

This is all beginning to get rather disheartening. Still, if there’s lots of competition for students, it must mean that there is something out there for me! It’s just finding it that seems to be rather tricky at the moment. Maybe I’ll have better luck with Leicester University after their prospectus arrives.

Working through the options

Having finally received a definitive “look elsewhere” email from the OU a couple of weeks ago, I’m slowly working my way through the options there are for distance learning masters courses in psychology and related subjects.

As October 2012 is the earliest entry date for all of the options I’ve investigated so far, I’ve plenty of time to consider things before I need to apply. So far, I’ve talked to Derby University, Birkbeck and I’ve just sent off for a Leicester University prospectus too (thanks for the prod in their direction Cecilia!)

As I’d be studying for fun personal development reasons, I’m in no hurry I suppose. October 2013, 2023 or 2033 would do … but I’d quite like to make a start before I forget everything that I’ve learned over the last five years! However, there is the small matter of wanting to do other things with my non-work time as well, which are all competing with the “study” addiction I appear to have.

As it looks as if I may need to help to fund another family member’s studies (and they’re more important, as Em would actually do something with hers instead of simply adding post nominals to a business card) that will probably enforce a bit of a pause too.

In the meantime, I’m ploughing on through Graham Richard’s book “Putting psychology in its place” – and a great read it is too. It’s one of those books that I’m sure I’d have enjoyed reading even before the OU degree, but the experience of having completed it makes it even more enjoyable. I’m finding it’s helping me to understand the relative contributions made to psychological knowledge by the particular research traditions that the degree covered – and also the topics it omitted. As such, it’s providing me with a useful framework to understand how much more there is to uncover and learn about in this fascinating discipline.