SHL US Regulatory Compliance

Based on the New York city regulations further describing compliance with the law, including the guidelines issued on 23 September 2022, we have narrowed down SHL impacted assessments to our AI-scored assessments. The following is a list of assessments subject to the NYC AEDT law:

• Smart Interview on Demand - Workplace (AM)
• SVAR - Spoken English
• SVAR Conversational - Spoken English
• SVAR - Spoken French
• SVAR - Spoken Spanish
• WriteX - Email Writing (Customer Service)
• WriteX - Email Writing (Managerial)
• WriteX - Email Writing (Sales)
• WriteX - Essay Writing
• Automata (C, C++, Java, Python)
• Smart Interview on Demand - UCF
• Conversational Chat Simulation

The NYC guidelines further expand upon the bias audit requirements by providing examples and impact ratio calculations, all of which appear to align to and are consistent with the Uniform Guidelines standards. Our analyses, audit procedure, and reporting processes are based on established best-practices in the field of assessment, and as guidance has been established on the NYC AEDT Law, SHL has adapted our processes to align with those specific requirements.

The Law defines an automated employment decision tool as any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, which is used to substantially assist or replace discretionary decision making for making employment decisions that impact natural persons. The term “automated employment decision tool” does not include a tool that does not automate, support, substantially assist or replace discretionary decision-making processes and that does not materially impact natural persons, including, but not limited to, a junk email filter, firewall, antivirus software, calculator, spreadsheet, database, data set, or other compilation of data.

To be considered an AEDT an assessment must be used in a way that prioritizes the output of the assessment more than other sources of information or overrule human decisions and be derived through a specific group of mathematical, computer-based techniques.

In determining which of SHL’s suite of assessment solutions are impacted by the NYC AEDT law, we first look to the portion of the definition that outlines what the tool must be derived from, and we then consider what the requirements are for those systems. We have also included the April 5, 2023, Rule Amendment’s definition of the phrase “machine learning, statistical modeling, data analytics, or artificial intelligence” in our analysis of applicability. The amendment defines machine learning, statistical modeling, data analytics, or artificial intelligence as a group of mathematical, computer-based techniques:

1. that generate a prediction, meaning an expected outcome for an observation, such as an assessment of a candidate’s fit or likelihood of success, or that generate a classification, meaning an assignment of an observation to a group, such as categorizations based on skill sets or aptitude; and
2. for which a computer at least in part identifies the inputs, the relative importance placed on those inputs, and, if applicable, other parameters for the models in order to improve the accuracy of the prediction or classification

This definition requires a computer to at least in part identify the inputs, and the relative importance placed on those inputs.

Traditional assessments do not meet this definition because trained human experts determine the inputs and set the relative importance placed on those inputs. Although traditional assessments do not meet the definition, AI scored assessments could satisfy this definition with the inputs and the relative importance placed on those inputs identified by a computer during the training process.

We have drawn a reasonable inference therefore that SHL traditional assessments that use traditional assessment methodologies, do not use a computer to select features or the relative importance of features and are not impacted by the NYC AEDT Law.

The April 5, 2023 regulations, further clarified that the phrase “to substantially assist or replace discretionary decision making” means (i) to rely solely on a simplified output (score, tag, classification, ranking, etc.), with no other factors considered; (ii) to use a simplified output as one of a set of criteria where the simplified output is weighted more than any other criterion in the set; or (iii) to use a simplified output to overrule conclusions derived from other factors including human decision-making. The output results of candidates taking the SHL traditional assessment is intended to guide customers/the employer as to the suitability and aptitude of candidates as part of an overall recruitment or development process – they are not to be relied upon solely in making a hiring decision, they are not weighted more than other criterion (i.e., in person interviews and specific skill assessments), and they do not overrule human decision making.

The NYC AEDT Law does not yet require that all SHL assessment to be included. SHL will continue to monitor the NYC AEDT law for changes over time. We will adjust our processes and reporting accordingly to address any Assessment Solutions that are deemed to be impacted by the NYC AEDT Law to ensure that they undergo the required bias audit.

SHL recommends our customers work with their legal teams and/or external counsel to determine their interpretation of the law and how it impacts their organization. SHL is happy to work directly with Customer’s legal team to support compliance based on that interpretation.

SHL’s standard audits will be updated annually, and are available to SHL customers in two scenarios:

• NEW USER: Customers using the AEDT tool for the first time
• HISTORICAL USER: Customers with historical data based on prior use of the AEDT tool, WHERE THAT DATA HAS BEEN SYSTEMATICALLY AND AUTOMATICALLY COLLECTED BY SHL AS A PART OF THE RESEARCH AND BENCHMARKING AUTHORIZATION PROVIDED BY THE CUSTOMER.

Note: Annual updates of the Standard Audits are available to New User customers and Historical User customers who provide Research and Benchmarking* authorization.

*Research and Benchmarking Authorization is authorization that the customer provides to SHL to ask and receive candidate responses to the optional demographic questions asked at the start of the assessment. Candidates are at all times given the option not to answer the optional demographic related questions and can refuse to respond without penalty. Whether candidates answer the optional demographic questions or not, notice of responses or failure to respond is not provided to our customers (i.e., the employer) or otherwise stored as personal information identifying the particular candidate. Where provided, the candidates’ responses to the optional demographic related questions are anonymized and deidentified and used solely for the purpose of research.

SHL has developed an internal system and repository to host audit results. The system is now active, and Customers can engage their account manager to initiate a formal request. Kindly note that SHL is able to conduct an audit only to the extent that adequate and sufficient information is available and/or made available by the Customer.

The use of SHL’s standard audit reports provides all customers a pathway to compliance at no cost. If a customer does not provide Research and Benchmarking authorization along with data (e.g., do not turn on our Research Form in the candidate workflow), they will need to seek compliance outside of SHL’s support.

Where a customer requests SHL’s support to create an audited report for custom assessments or SHL standard assessments that not listed in response to Q1 above, SHL may be able to conduct such an audit subject to a fee. Please engage your account manager to initiate a formal request for the exception. Please note that in order to conduct an audit of SHL Assessments not listed in response to Q1 above, unless this information is collected within the Customer’s content package, customers will have to provide EEOC demographics matched to assessment sessions by either the test_session_id or combination of candidate_id and project_id. Upon customer request and on an annual basis thereafter if requested, subject to applicable fees and written agreement, SHL will engage an independent third-party auditor to conduct the bias audit. The bias audit will entail testing the SHL Assessment Solution to assess disparate impact on persons based on gender, race, and ethnicity. Please not all such requests will be granted.

Our standard audit reports are no cost to customers.

For audits and reports of SHL Assessments not listed in response to Q1 above, the current cost of a basic audit is $12,000 USD per instrument, based on the cost to our independent auditor and our standard consulting rate, and is subject to availability of the data. See Q6 for more details.

This largely depends on the availability of our auditor, but we do not anticipate a turnaround time being longer than 2 weeks.

Since candidates interact with SHL and the SHL Assessment Solution only at the point of testing, Customers are in the best position to fulfill the prior notification requirement of the NYC AEDT Law. That said, the proposed guidance suggests that the notice need only to be published or provided, in a conspicuous manner, prior to the employer using the AEDT as opposed to prior to the candidate using the AEDT. Kindly consult with your legal counsel for the content, timing, and legal requirement of the NYC AEDT Law’s Candidate Notice requirement. We are happy to provide reasonable support to you based on the outcome of your internal legal consultation.

Please be advised that this FAQ represents SHL’s understanding as of the date of this document. We will review and update periodically to align with developments in the NYC AEDT Law.

Any further questions, please contact SHL at SHL Customer Support or your SHL account manager for more information.