Personality and Prediction – A Very, Very Slippery Slope
If you look around the web sites of personality assessment companies you will see a number of suggestions or direct statements that indicate that use of their assessment enables the user to predict performance.
At some level this claim of prediction simply enrages me because of the following:
- There is little if any concrete research about validity.
- It will almost certainly mean the context or situation the person finds themself in is irrelevant.
- It means you have to treat personality as an ‘etched in stone, thing.’
- And it might simply do damage to people and organizations.
I have really tried to understand where these claims may be coming from. After considerable reflection and investigating what I’ve come up with is that how these statements are defined is likely different than how I (and I imagine many others) would define them.
So when it comes right down to it, yes I’m still enraged. Let me lay out my thinking here and maybe someone can help me see another perspective, calm down and have a better night’s sleep!
First, I considered the combination of words that to me most clearly indicated what was being said on these web sites. I came up with:
By knowing the personality of someone by using our assessment future performance and success can be predicted.
Or to be even more concise:
Our personality assessment predicts performance.
So my first step was to consider assessment validity where such a claim is made. There should be research focusing on ‘predictive validity’ for the assessment. Predictive validity is one of the most difficult measures to determine since you have to correlate the assessments results, over time to some other criteria or variable and be able to prove that no other variables are affecting the correlation.
I have yet to see any web site making these claims of predictability talk about their predictive validity research. It’s simply doesn’t work.
So now I was somewhat annoyed and began to consider what prediction would mean to understanding the context in which that personality existed. If you are going to predict future performance in a broad way, such as ‘this person will be a good general manager’, then the context must be broad as well, such as any definition of good and any type of general manager. In a practical sense prediction from a broad perspective means context is not important.
If you are trying to predict performance in a narrow way then the context is more important and the definitions more specific, such as, ‘this person will drive short term goals forward as a general manager of small numbers of people in a start-up company’. It’s tough to even give an example here but I’m sure you get the drift; the assessment has to be so specific and sensitive to the context that you might as well just forget it and do an interview.
I have yet to see any web site making these claims of predictability say their assessments are specific enough to predict into specific contexts. They are selling assessments to anyone that will buy them with no consideration of the context in which prediction would be valid. Buyer beware I guess.
A quick aside about context. A little bit ago I was working with an executive of a bank and her performance review indicated she was performing really well and was a great fit for the role. Her boss changed and her next review indicated she was too aggressive and her performance needed to change. Hmmmm sounds like context mattered in this scenario!
So by now I’m losing sleep and began to consider what prediction might mean to how personality is understood. In order to predict into the future from a personality perspective you have to assume that personality is stable over time and over variable contexts. The assumption is that personality is primarily owned (often innately so) by the individual and is stable over time and place. Basically you are who you are, period.
It is rarely noted that the above is an assumption, a theory as yet unproven. If you take a social constructionist perspective of personality it’s quite different, and may not even exist! A person would exhibit variable, yet self recognizable patterns of behavior heavily influenced by context. Basically you are who you interact with, and on and on you become.
What was causing me to lose sleep however was the nagging question, ‘If my personality is stable enough to predict performance into the future, when exactly did it become stable?’ When I was 5, or perhaps 25? Knowing this would make a big difference in when I should take the assessment right? And if it became stable over a ‘period of time’ then we couldn’t really call it stable could we, it is ‘becoming’ stable, so when will that stop? Good questions to rob me of sleep.
So after losing sleep and getting angry about that, I figured that these statements must be defined differently than how I was defining them. So how might these companies be defining the statement – Our personality assessment predicts performance.
The only reasonable answer I can come up with is that the company believes that if their assessment data is interpreted ‘accurately’ and all the problems noted above are dealt with in this accurate interpretation, then their assessment can indeed predict performance. And guess what, they will train you to interpret accurately. And if things go south on the prediction front, you just didn’t interpret the data well enough.
Actually I kind of hope this is how these claims are being interpreted, even if it’s an unrecognized interpretation. At least there would be some genuine (although I would say misguided) belief that prediction is possible from the data.
Otherwise these claims are just marketing ploys, to get you to buy and believe something about their product that cannot be realized. The enraging part is that it might cost someone their career, or a chance to bring difference and change to an organization.
Can you offer any other perspectives here?