Graphology: Mere wishful thinking?

A graphologist (handwriting analyst) was interviewed on BBC Breakfast this morning. Resisting the urge to immediately rant on Twitter about the pseudoscience of graphology, I headed upstairs to my study instead. Since then I’ve spent some time refreshing myself on the arguments for and against the art. My own interest is in its use at work, so I’m not that concerned whether Donald Trump’s handwriting indicates if he’s a narcissist or not (*).

Two claims are commonly made by graphologists. The first claim is that graphology can be used to accurately assess personality traits. The second claim is that graphology is an effective personnel selection method. Naturally, for a selection method to be effective, it should be predictive of eventual job performance.

These are extraordinary claims and therefore require extraordinary evidence, as Carl Sagan used to say. Unfortunately for people who use the services of graphologists, the evidence in the peer-reviewed personality and occupational psychology literature does not bear these claims out.

For the purposes of brevity, I’ve naturally been selective in the papers I’ve quoted from in the rest of this article. They are, however, broadly representative of the scientific consensus on graphology over the last 30 or 40 years.

Is graphology a good predictor of personality?

No. It isn’t.

Leaving aside the arguments about whether any instrument can (a) measure personality and (b) extrapolate job performance from those measurements (**), handwriting analysis is not a good predictor of personality.

The gold standard of personality assessment is widely regarded to be instruments that measure the ‘Big Five’ traits – Openness, Conscientiousness, Extraversion, Agreeableness and Neuroticism. If graphology could replicate the outcomes of a Big Five questionnaire, then it could claim to be predictive of personality.

However, studies conducted by Dazzi & Pedrabissi (2009) found that results from a Big Five questionnaire did not correlate with the assessment of graphologists. Furthermore, agreement between graphologists was poor. These results are in line with the earlier meta-analysis conducted by Neter & Ben-Shakhar (1989) which concluded that graphologists are worse than laypeople at predicting personality traits from handwriting.

Simply put, a BBC article quoting experts from 2005, says:

The British Psychological Society ranks graphology alongside astrology – giving them both “zero validity” in determining someone’s character. Dr Rowan Bayne, a psychologist who tested top graphologists against their claims, says the practice is “useless… absolutely hopeless”.

Is graphology a good predictor of job performance?

No. It isn’t.

Robertson & Smith’s (2001) review of personnel selection studies reports that the best predictor of eventual job performance is a candidate’s cognitive ability (intelligence) and integrity. Structured interviews also score well. Common elements of typical CV’s, for example years spent in education and years of job experience, score poorly, but still fare far better than graphology. Indeed, the only worse predictor of eventual job performance they report is age. Even something as ephemeral as personal popularity at work positively correlates to job performance (Garden, Hu, Zhan & Wei, 2018) at a level higher than graphology.

Accuracy of personnel selection methods
Accuracy of personnel selection methods – graphology has a lower accuracy than every method reported by Robertson & Smith (2001) except age (negative correlation).

Predicting future job performance during the selection process is hard. Even the best methods aren’t infallible. But graphology is not the answer.

Conclusion

Based on all I’ve read today, I find it impossible not to agree with this statement.

There is no doubt that when one carefully selects studies in terms of their methodological robustness, the evidence [for the efficacy of graphology] is overwhelmingly negative (Dazzi & Pedrabissi, 2009).

Graphology as practised today is mere wishful thinking.

Footnotes

(*) I did find a paper on narcissism and career success. It concludes that narcissism impacts success through increased occupational self-efficacy beliefs and career engagement – but has only a weak relationship to standard measures of career success, including job satisfaction and salary.

(**) See my earlier post – Does measuring personality make sense?

 

References

Dazzi, C. & Pedrabissi, L. (2009). Graphology and Personality: An Empirical Study on Validity of Handwriting Analysis. Psychological Reports, 105(3), 1255-1268.

Garden, R., Xu, H., Zhan, Y. & Wei, F. (2018). The Role of Workplace Popularity: Links to Employee Characteristics and Supervisor-Rated Outcomes. Journal of Leadership and Organizational Studies, 25(1), 19-29.

Neter, E. & Ben-Shakhar, G. (1989). The predictive validity of graphological inferences: a meta-analytic approach. Personality and Individual Differences, 10, 737-745.

Robertson, I.T. & Smith, M. (2001). Personnel Selection. Journal of Occupational and Organizational Psychology, 74, 441-472.

Personality at work

In my first post, I did say that I wasn’t here. Which is still true of course, but last Thursday (14th November) I did manage to make a fleeting visit to the university to listen to a guest lecture by Peter Saville, a celebrated occupational psychologist and entrepreneur.

At work, I’ve often found myself involved in the recruitment process and I’d like to think that I’m a better judge than most when it comes to identifying the right person for particular roles. After sitting through Peter’s talk, I’m probably going to be a little more cautious in future.

For example, 70% of all application forms and CVs contain serious errors of fact, because people are prone to exaggerating their qualifications and experience. References are problematic as some organisations refuse to produce them for fear of legal action – and who would ever voluntarily provide a referee to give less than glowing report? And as for interviews – well, apparently most interviewers make up their mind about a candidate in the first few minutes and then spend the rest of the time looking for evidence to back their initial opinion up.

It even seems that some organisations have used astrology during the selection process. Thankfully, I’ve never witnessed this, but in one study Peter mentioned 22% of the psychology students questioned thought that using a person’s astrological sign would be a reasonable characteristic to use for assessment. My mind boggled at the idea.

Having been instrumental in creating the OPQ and Wave personality inventories, most of the talk revolved around the challenges of creating questionnaires that are truly useful in occupational as opposed to clinical contexts. Issues with zombie questions, double negatives and idioms were all discussed and dispatched with style.

In summary, Peter said that questions used to assess personality at work need to be short, focussed on behaviour, self-referent, acceptable to candidates, managers and lawyers alike as well as having job relevance. I found myself nodding along in agreement. It was an excellent talk – well worth the journey down from Derby and the time off work to listen to.

However, that’s not to say that I’m entirely convinced by the personality measurement industry. After all, as Graham Richards points out in his book ‘Putting Psychology in its Place’, no-one can say for sure how ‘real’ the personality traits identified by these questionnaires are. They could simply be artefacts of the methods used to measure them or be hopelessly contaminated by the cultural context they are used in.

And if the personality traits measured during selection and assessment have no objective reality – or worse – traits that are of importance to job performance fail to be picked up because they aren’t socially or legally acceptable characteristics to measure, then perhaps organisations would do better to invest their efforts elsewhere.

But what do you think? Have any personality questionnaires that you’ve completed in the past been useful to you in understanding where your strengths might lie? I’d be very interested to know your thoughts.

This article was originally written for the University of Leicester Student Blogs, 20th November 2013.

 

Does measuring personality make sense?

This is a rather long post (nearly 1,800 words) and I’m fully aware that it will break (by some margin) the tl;dr(*) threshold of many people! But I’ve written it for me, primarily to remind me of some of the material that I suspect may come in useful if I ever do take up my deferred place on Leicester University’s occupational psychology MSc.

A few weeks ago, I’d put together half a blog post on the topic of personality but had left it unfinished, gathering dust in my WordPress “drafts” folder. What finally prompted me into finishing it was a blog post by Terence Eden called “Astrology for Businesses” which chimes with some of the concerns I have about the whole personality testing industry – and in particular the way in which such tests are used and misused by personnel departments.

Some background might be useful to start with. Simplifying enormously, two main ways of investigating personality have been used over the last century or so. The approach that most people are familiar with (particularly in a work context) is based on attempts to aggregate common features of individuals and tries to measure personality based on the differences that any one individual exhibits from the average. Researchers using this nomothetic approach are interested in establishing general laws or theories of personality that can be applied to everyone. An alternative approach to understanding personality is an idiographic one and is often seen used in clinical contexts. Here, the intention is not to psychometrically measure one individual against another, aggregating the results of thousands of individual tests to decide what is statistically ‘normal’, but to understand a person’s unique personality from the context of their lived experience, the behaviours they exhibit and their subjective feelings. However, it’s the nomothetic approach to personality measurement that I’m concerned with questioning in this post.

Within the nomothetic or individual differences approach to personality, numerous measurement systems have been proposed but can usually be categorised as either a “trait” or “type” theory of personality.

Trait theories of personality include Cattell’s 16 factor, Costa and McCrae’s 5 factor OCEAN and Eysenck’s 2 or 3 factor scales. All three are assessed using a battery of question items (personality inventory), designed to provide numerical scores for each factor making up an individual’s personality lies. For example Warm<–>Cool, Forthright<–>Shrewd and Group-Oriented<–>Self-Sufficient and 13 other bi-polar scales – are used by Cattell; Openness, Conscientiousness, Extraversion, Agreeableness and Neuroticism (OCEAN) are used by Costa and McCrae.

Type theories of personality also use questionnaires to assess personality. However, instead of representing people’s scores on a number of continuous dimensions, scores are used to assign people to personality types. One of the most commonly used in business settings is the Myers-Briggs Type Inventory (MBTI) which categorises individuals into 1 of 16 types. Other type theories include Friedman’s Type A/B and Keirsey’s 4 temperaments / 16 roles.

Now, if you accept that personality can be measured by means of questionnaires, the conventional debates about which instrument is the best tend to centre on discussions of statistical reliability (whether an individual gets the same result in a variety of situations and over a period of time) and validity (whether the questions used make sense, relate to the personality factor being measured and the extent to which it can be demonstrated that what is being claimed to be measured is actually being measured). But both of these debates start from the assumption that such scales and questionnaires really do measure personality and that from such measures, an individual’s behaviour can be predicted. And of course, it is their apparent predictive power which tempts businesses to use such instruments in helping with recruitment, promotion and role-assignment decisions.

However, in my view, there are a number of more fundamental criticisms that are rather more difficult to address by tweaks to existing trait and type theories.

Firstly, there is a fundamental assumption behind all of these theories that people behave consistently across a range of environments and settings. However, empirical studies that lend support to this notion of consistency tend to concentrate on looking at how people behave in similar settings on a number of different occasions. As such, it’s quite possible that although an individual may display behaviours associated with low extraversion in the workplace, they could still be the wildest of party goers in their social circle. In the late 1960s, Mischel used this type of evidence to question this assumption – and to argue that all such theories of personality are misconceived – a stance known as situationism. A 1928 study by Hartshorne and May which found inconsistencies in children’s cheating behaviour is one example used in support of this view. However, perhaps a reasonable counter to this argument might be that it is simply a different set of social norms that influences behaviour, rather than internal contradictions in personality. But if the environment and social setting has this large an impact on people’s behaviour, might it be far more appropriate to focus more effort on improving the working environment than on attempting to select the “right” type of person?

A second criticism of nomothetic personality theories is that they are universalist in nature. In other words, they expect that the same personality traits or types will be present in all human societies and cultures across time. Eysenck (the principal researcher behind a personality trait system which attempted to link personality directly to biological characteristics) viewed the quest to link individual differences with their biological underpinnings as being as important to the scientific understanding of personality as the periodic table is to the understanding of the chemical properties of the elements. However, there is no agreement between theorists as to how many different factors or types are required to fully describe a person’s personality, bringing into question claims of universality for such theories as well as the fixed nature of personality traits or types. It’s also interesting to note that the questions used on many personality inventories have changed over time. For example, the questionnaires used to assess people against the OCEAN traits have been revised several times, suggesting that at the very least there has been a ‘drift’ in what the factors mean.

Thirdly, the interests of the researcher (or the people funding them) as well as what are ‘socially acceptable’ personality traits to investigate also have an impact on these theories. For example few (if any) of the personality inventories commonly used in business settings attempt to measure how sadistic someone might be – yet there is reasonable evidence to suggest that such a trait is important indicator of success for certain kinds of employment!

As well as failing to measure aspects of personality that may exist, it is also possible that reification (measuring things that do not exist) may be occurring within the questionnaires used to measure personality traits. The best explanation I’ve seen of this is in Graham Richard’s book, “Putting Psychology in its Place” (the 3rd edition of which was published in 2010). In it he writes (page 285):

… not everything which can be measured necessarily exists. This may sound puzzling, but is actually not so self-contradictory as it seems. The argument is best made using a hypothetical example: were we living in the Middle Ages we might be very concerned about how devout people were. To measure this we devise a questionnaire containing such items as ‘I prefer reading a holy book to attending a tournament’, ‘A strange feeling of Grace sometimes descends upon me’, ‘I enjoy attending High Mass’ or to counterbalance the direction ‘I often find sermons boring’. (‘I have never been tempted by lust’ could serve as a lie item.) It is surely feasible that at the end of the day our ‘sanctity scale’ would appear to provide a handy way of measuring how holy people were. But no psychologist proposes that there is a measurable ‘sanctity’ dimension to personality, and not even the most devout psychologists have attempted to devise such a measure (though they have attempted to measure the levels and types of religious belief). Nor is this as far fetched as you might imagine; among the earliest pioneers of scientific measurement were the fourteenth century French scholars Jean Buridan and Nicolas D’Oresme whose efforts were spurred by the desire to quantify the amount of Grace in communion wafers.

Fourthly, the objectivity of the concepts used to describe personality are also suspect – which brings the scientific and universal claims of personality measurement into question. For example, the negatively construed ‘authoritarian’ personality type, defined immediately after the second world war when people were trying to understand the rise of Nazism, was defined as including people who were inflexible, closed-minded, intolerant of ambiguity and held obedience to authority in high regard. However, Jaensch, a German psychologist of the 1930s, had arrived at the definition of a similar trait, but evaluated it as being positive – ‘authoritarians’ were strong-willed, disciplined, had clear vision and so forth whereas ‘democratic’ types were weak-willed, dreamers, undisciplined and overly changeable. And if you’re still not convinced by the argument that the lack of objectivity and (historical) cultural concerns present in personality inventories tends to undermine their scientific claims, consider some of the questionnaire items from 1950 used to measure authoritarianism such as “Homosexuality is a particularly rotten form of delinquency and ought to be severely punished” and from Eysenck’s 1957 social attitude inventory which attempted to distinguish between right-wing and left-wing authoritarianism as part of the cold war effort – “It would be best to keep coloured people in their own districts and schools in order to prevent too much contact with whites”. Neither question could be successfully used today to measure ‘authoritarianism’ – even if they did perhaps have some cultural relevance in the 1950s in white America.

Now, of course, it may be possible that by investing ever more resources into research on nomothetic personality theories may eventually produce a system of measurement that is truly universal and scientifically sound. A “theory of everything” – similar to that which appears to be in reach of physicists in their field. But somehow, I doubt it. Instead, I’m much more in sympathy with the view expressed by Phillida Salmon – that psychometric assessment [of personality] is as brutal and as mutilating as the bed of Procrustes. But the power and wealth of the personality industry is such that I think we’re unlikely to see any change of emphasis within most businesses soon – to the likely detriment of both their employees and shareholders.

(*) tl;dr – too long, didn’t read

Bibliography:

Putting Psychology in its Place – Critical Historical Perspectives (3rd Edition) – Graham Richards, Routledge, 2010.

Critical Readings in Social Psychology – Darren Langdridge and Stephanie Taylor (editors), Open University Press, 2007.

Mapping Psychology – Dorothy Miell, Ann Phoenix and Kerry Thomas (editors), Open University Press, 2002.