Off balance

I’m feeling a little off balance at the moment. Last Wednesday I was busy telling the ARIS and webMethods user groups that “numbers don’t speak for themselves”. I was talking about the creating business cases, but I believe the statement to be true more generally. Numbers only make sense if you can relate them to a specific context. Furthermore, the numbers used must report or measure something meaningful, otherwise there’s no point in collecting the data. (You can find my detailed explanations rants on both of theses topic here and here if you’re interested).

Anyway, this was me in action at the event. It looks a little as if I’m conducting an auction and that the chandelier is about to bring it all to a messy end.

ARIS user group meeting 1st March 2017I’d had an active week up to that point, and although I spent Thursday in the office, that day was busy too. Here’s my steps chart for the first part of the week …

Mon-Thu 27/2 - 2/3 steps… 43,611 in all. I should have been feeling great! Nicely (but not stupidly) over the 10,000 steps a day average we’re supposed to achieve, according to the NHS and others. But having wittered on about context, you should already know that I’m about to tell you what happened next.

Full week 27/2 - 5/3An average of under 1,700 steps a day for Friday to Sunday. Monday to Thursday wiped me out, so I’ve spent most of the time asleep or moping around on the sofa. I haven’t been eating (much) either.

I feel that given my opening salvo I should now provide some context to these numbers. After all, you could just assume that I’ve been really lazy for the last three days. I wish that was true!

My best case hypothesis is that I picked up a bug (or mild food poisoning) early last week. As I was rather ‘poorly’ on Thursday evening that explanation could make sense. My worst case hypothesis is that the lymphoma has started to put on a bit of a sprint. I’ve been feeling increasingly fatigued for some weeks now, with even the most sanguine of the consultants that I’ve been seeing starting to suggest that chemo might be needed ‘soon’. Having spent 2.5 years on watch and wait, I’m not sure if ‘soon’ means weeks or months or a year or more … sometimes I don’t want to know the numbers at all.

Anyway, the next few days should help me figure out which of the hypotheses is right. I’m starting to get a bit of energy back today, so I’m hopeful that the bug explanation proves to be the right context for last week’s steps chart.

Why surveys should always be piloted

This morning I completed an almost incomprehensible marketing survey. Here’s an example of one of the questions.

Marketing survey

Like all of the other questions in the survey, you have to answer it to proceed to the next question. There’s a fixed range of answers that can be selected, with nowhere for me to indicate that I didn’t understand the question. Most of the questions were like this, so my best guess is that YouGov’s client will end up with statistical noise and a sprinkling of confirmation bias.

My suspicion is that the survey wasn’t piloted before release with its target audience. If it had been, simple ambiguities (does 2030 mean half past eight tonight or is it something due to happen in 14 years?) would have been picked up, questions would have been rephrased to make them comprehensible to the lay-person and the ability to answer ‘don’t know/don’t understand’ would have been provided.

But even if such changes had been made, it’s doubtful that anything insightful will result from the survey. The client would have been far better to employ a qualitative research method to explore such hypothetical questions. A good first question would be to ask for a definition of a luxury brand, rather than making the assumption that the client, YouGov and the survey’s audience all share the same perspective. As it stands they’re likely to get some nice charts with average scores to a couple of decimal places, but little insight into what consumers really think.

The figures *don’t* speak for themselves

Inspired by the fourth Post40Bloggers writing prompt to discuss the topic: “In a world full of uncertainty, write about what you know for sure”, here’s why when anyone says to you that “the figures speak for themselves” your spider senses should start tingling and a red danger light start flashing in your brain. Metaphorically speaking of course. I do know for sure that we don’t have spider senses nor do we have a red danger light in our brain. But that’s a different article.

How many times have you heard someone say that “the figures speak for themselves” as a way of attempting to close down or win an argument? It happens all of the time. Some recent examples (courtesy of the might of my favourite interweb search engines) are:

People often appeal to figures because they appear to be objective, but all three of these examples are anything but a dispassionate telling of a single, objective truth. For example, Liam Byrne was attempting to use the figures from a recorded rise in the young unemployed by 20,000 in April 2013 to justify Labour’s jobs policy. In the meantime, youth unemployment (and unemployment in general) has fallen again – but I’m prepared to bet that he hasn’t changed his political stance. I doubt that he would even if youth unemployment fell to zero. So it would seem that these figures really weren’t speaking for themselves. Instead, this is an instance of someone taking a set of figures, which are undoubtedly subject to error and uncertainty, and then speaking for them. As are, I would humbly suggest, the other two examples.

I make this point endlessly at work to colleagues and clients alike. These days, rather than cutting code or managing other people for a living, I help people to determine whether or not a particular investment in software and services “makes sense”. Now, I’m a great believer that you must always start with the numbers and create a financial model of the possible risks and returns, but simply presenting a decision maker with a statement that “If you do x, it will realise £y” is never enough.

Instead, you have to use the figures to help your decision maker justify a particular investment in light of the organisation’s own aims and subjective beliefs about its future. For example, if a commercial organisation is single-mindedly focused on growth, then even the best financial case around saving costs won’t get the attention it deserves. The public sector is similar. At the moment, if an investment doesn’t fit the ‘all IT is a commodity’ or ‘open source is the answer to everything’ memes then the overall financial case is ignored, as positioning against either of these trends simply invites ridicule, justified or not.

So I hope I’ve convinced you of one of the things that I know for sure – figures don’t, and never will, speak for themselves.

 

Stop being a dick, Stuart Wheeler

Using the form of words favoured by comedian Adam Hills, Stuart Wheeler, the UKIP treasurer, needs to stop being a dick.

The latest drivel he’s reportedly spouting is that because women, in his opinion, are “nowhere near as good as men” at games like chess, bridge and poker it should somehow disqualify them from being properly represented in the ranks of business leaders.

Now, I’ve no idea how being good at games correlates with being an effective leader (nor where he gets the evidence to substantiate his claim – after all, Judit Polgar is a pretty decent chess player) but I do know that for most things, the differences found within a gender are far more significant than the differences found between genders.

As for leadership skills, the between gender differences are miniscule and vary between just favouring men or women on average depending on the measure used, as demonstrated, for example, in Janet Shilbey-Hyde’s 2005 comprehensive meta-study, The Gender Similarities Hypothesis.