What to Do With Your ‘Homegrown’ Assessment Tool?
Using a content validation process to evaluate the rigor, objectivity, and legal defensibility of assessment tools built within your organization.
In today’s world of “free” survey tools on the Internet, human resources practitioners occasionally stumble upon an internally developed assessment that lacks the rigor, objectivity, or legal defensibility required to make sound employment decisions. These tools are often developed internally by operators with good intentions or by external consultants that lack a thorough understanding of best practices. Below are a few examples of problematic practices:
- A shipping organization leader creates a brief self-report questionnaire asking about physical strength to use in hiring for loader/unloader roles. (Issue: The questionnaire lacks rigor and objectivity.)
- A call center leader administers a situational judgment test to operator candidates based on common customer complaints. (Issue: The test lacks legal defensibility because it hasn’t determined that effective judgment is required upon entry or something that may be trained on the job.)
- A regional multi-unit restaurant manager drafts and implements a set of interview questions to use in hiring front-of-house staff that assesses candidate knowledge and hospitality best practices (Issue: inconsistent hiring practices due to the failure to coordinate with corporate hiring program for front-of-house staff.)
Do all candidates require the knowledge being assessed on day one of the job to perform the job effectively?
These kinds of scenarios keep human resource leaders up at night. Fortunately, there are standardized methods for reshaping homegrown tools into valid, legally defensible assessments with clear linkages to key behavioral outcomes associated with success on the job. They answer key questions that establish the required rigor and objectivity, which ensures legal defensibility. These questions are:
- Does the assessment being used measure the most relevant and important aspects of the job?
- Do all candidates require the knowledge and/or skills being assessed on day one of the job in order to perform the job effectively?
- Are the demands or difficulty level of the assessment commensurate with the expectations of the role?
The underlying rigor, objectivity, and legal defensibility is commonly referred to as content validation. The following provides a brief overview of how SHL helps organizations around the world answer the questions above through content validation.
What Is Content Validation?
In this context, content validation is a process used to demonstrate that the focus of an assessment is representative and appropriately aligned to specific work behaviors, activities, and/or competencies necessary for effective job performance. The goal of content validation is to accumulate evidence that the assessment is illustrative of the tasks, behaviors, and knowledge necessary for performance on the job, and is, therefore, a reasonable data point for selecting someone into a role.
What Is Required for Content Validation?
- Gain a thorough understanding of the role requirements first. To do this, SHL Consultants typically review role materials (literature, job descriptions, etc.) conduct job observations, and/or interviews with those who know the job well to identify the knowledge, skills, abilities and other characteristics (KSAOs) required for successful performance in the role.
- Document the importance of these KSAOs for day-to-day performance and determine whether they are required on day one of the job. At SHL, we typically use focus groups and/or surveys to collect expert ratings of the importance of these KSAOs and whether they are expected upon entry into the role.
- Link the homegrown assessment to the critical KSAOs. Armed with a clear view of the role requirements, SHL Consultants then collaboratively gauge the alignment of each element of the homegrown assessment to the tasks and KSAOs identified in order to deliver the content validation evidence.
- Ensure that the assessment experience is appropriate for the job. We conduct a thorough review of the assessment procedures to ensure the reading level, administration method, and instructions align with the context and expectations for the job.
Only by going through this process, can you ensure your homegrown assessment can withstand legal scrutiny and serve as a valuable tool for your organization. Putting your homegrown assessment through the rigors of content validation creates a sound and legally defensible framework to use tools developed within your organization.
If you have questions about your homegrown assessment tools, SHL may be able to help you find the answers. Please reach out to us to support you in this process.
