Fairness in Testing through Scientifically Designed Personality Assessment
The validity and fairness of personality assessments have been called into question. Read our blog to learn if they help or harm diversity and inclusion efforts.
Share
No new assessment is released unless it has been validated and meets international standards for reliability and fairness.
Like most people, I am often asked what I do for a living. Once, when I, a psychometrician, was asked, and I explained that I develop pre-employment tests that are used to make hiring decisions, this individual, a manager at an accounting firm, told me that in every interview he conducts he asks the candidate what their favorite restaurant is and why. He went on to explain how the candidate’s response is a good indicator of some obscure trait that he found particularly valuable in new hires.
I was horrified. Aside from my inability to come up with any conceivable way one’s favorite restaurant could be a valid predictor of job performance, I was floored by the massive bias in terms of gender, race, age, cultural background, and disability status a question would illicit. This is the very opposite of everything that psychometricians try to achieve when developing assessments like personality inventories that are used for selection decisions.
The HBO Max documentary “Persona” calls into question the use of personality assessments. In response, we would like to address some key points on why personality assessment, when developed and validated for the specific purpose of selection by trained professionals, creates a fairer and more predictive tool than other alternatives.
The process of developing a pre-employment personality test
Psychometricians are often asked where we come up with the questions we ask in a pre-employment test. We do not just make them up off the top of our heads and hope for the best.
- First, psychometricians are trained professionals with advanced degrees in psychology and statistics.
- Second, developing a new assessment is a lengthy process that involves a thorough review of peer-reviewed literature on the construct of interest, the collection of empirical data on job requirements across a variety of roles and job levels, and study of advancement in measurement theory to determine the ideal best way to assess the construct of interest. Assessments are developed specifically to measure job-relevant traits.
- Third, when generating new written content, extreme care is taken to reduce bias towards any group. At SHL, our global workforce of psychologists aim to ensure cultural bias within the context is minimized and the content is job-relevant.
- Finally, no new assessment is released unless it has been validated and meets international standards for reliability and fairness.
When it comes to personality tests there are several popular versions, like the MBTI, that use a typology to categorize individuals into a particular profile. These types of tests can be valuable when used for self-development but were not designed for (nor should be used in) making hiring decisions.
Do personality tests harm diversity and inclusion?
When watching Persona, one quote really hit home: “Personality tests are ableist, racist, sexist, and classist”. As mentioned earlier, we do everything possible to minimize bias during the assessment development process. We should mention that the documentary does not provide any empirical evidence to substantiate the claims that personality assessments are biased.
During the validation process, we collect data to ensure that scores do not vary meaningfully across age, race, and gender. Data for our personality assessment, the Occupational Personality Questionnaire (which was designed specifically to assess behavior at work), indicate that personality is the type of assessment content that is least likely to disadvantage specific groups. SHL has literally billions of data points collected over two decades demonstrating, consistent with peer-reviewed literature on the topic of personality assessments developed for selection purposes, that the OPQ does not disadvantage any group. We have also initiated an in-depth research program designed to learn more about the experiences of neurodiverse candidates when completing our assessments to inform best practices for the field to optimize their experience.
The suggestion that personality tests hinder diversity and inclusion could not be further from the truth. In fact, they can serve to promote it. Humans possess inherent biases and preferences when making subjective decisions which can perpetuate discrimination. For example, though very popular, unstructured interviews have been shown time and time again to be extremely biased and not valid (Dana, Dawes, & Peterson, 2013). The use of scientifically validated objective personality assessments, on the other hand, will minimize bias from the selection process.
We should note that when making hiring decisions, personality is just one piece of the process. Scores on pre-employment personality assessments can (and should) be considered alongside other job-related abilities and behaviors, as well as the potential to learn job-relevant skills through training. Considered together, a judgment is made about a candidate’s match with the job requirements.
Personality measurement is not “one-size fits all”. Not every job requires the same traits. Hence, simply because someone’s scores on a personality assessment do not get them a particular job, it does not mean that the same scores would not land that person as a top candidate somewhere else.
Developing a pre-employment personality test is a lengthy process that involves scientists and trained professionals, and each assessment must go through thorough reviews to reduce bias and ensure that it measures what it needs to. This is something that organizations need to understand and distinguish from other types of personality tests.
Scientifically developed personality assessments used for selection are not mysterious. There are no trick questions. Properly developed assessments ask about behavior on the job and scores are derived from the responses that candidates provide based on their own perceptions of themselves and their behaviors. Candidates should feel comfortable responding according to their initial reactions about their tendencies rather than overthinking or considering what is the ‘best’ answer to achieving a higher score. There is no “black box” in the OPQ. Questions are measuring exactly what they appear to measure.
So, do pre-employment tests harm our efforts to promote diversity and inclusion in the workplace? Again, it depends on which personality test you use in your hiring process. If you choose the right test that is scientifically validated and meant to be used in a selection process, then the answer is a resounding “no”.
Contact us to learn more about the steps we take to minimize bias in our pre-employment personality tests and how we can assist you in your recruitment program.
