Test Validation: How it Helps Foster Merit-Based Decision-making

With the recent push by the EEOC for employers to use merit-based hiring to promote equal opportunity, the federal government is making it clear that employers should be using job-related, skills-based employee selection processes. The concept of ‘merit-based’ means that employers should evaluate candidates based on their knowledge, skills, abilities, and qualifications that relate to the performance of the job in question, rather than their protected group status or other factors unrelated to job performance. Ensuring a selection tool is merit-based is achieved through the process of test validation.

Test validation is not a new concept. In fact, validating selection procedures has been a central focus in the field of Industrial/Organizational Psychology for many decades. The Uniform Guidelines for Employee Selection Procedures (Uniform Guidelines) outlined professional standards for test validation studies in the employment context in 1978. Since then, courts have consistently given deference to the guidelines in evaluating test validity in litigation. In this blog, we discuss how employers can establish the validity of their selection tools and foster merit-based hiring in their organization.

Test validation can be performed in a variety of ways but typically involves one of two methodologies: 1) Content Validation or 2) Criterion-Related Validation. Content validation is applied to demonstrate job-relatedness by showing that the content of the test is representative of the content of the job through a linkage analysis of test content/items to specific tasks performed and knowledge, skills, or abilities (KSAs). Criterion-related validation is applied to demonstrate job-relatedness by showing a statistical relationship between scores on the test and measures of job performance (e.g., productivity, sales performance, safety incidents, or other measures).

Content Validation

Content validation is most appropriate for tests of knowledge, skills, and abilities, or work sample tests (in which a portion of the job is simulated for the test) for which the content of the test can be easily linked to the content of the job. Content validation is not typically appropriate for measures of personality, cognitive ability, or unobservable traits.

First, a job analysis is required as the foundation of the validation work. If a job analysis has been completed within the past few years and meets the requirements of the Uniform Guidelines, the existing job analysis may be used. If a full job analysis is needed, the analysis typically involves:

• a review of existing documentation on the jobs (job descriptions, training materials, etc.);
• subject matter expert (SME) interviews or focus groups to gather further information about the tasks and KSAs relevant to the job;
• surveys to identify essential job elements; and
• and a technical report regarding the findings.

Once a job analysis has been completed, an Industrial Psychologist will typically review the test content for consistency with results of the job analysis. This includes conducting a “linkage analysis” and evaluation with SMEs to show that assessment content represents and measures content required for successful completion in the target position. This phase links each question/task/component of the test to one or more critical KSAs or duties that are required to successfully perform the job.

After this review, the employer can draw preliminary conclusions about the validity of the test, or the extent to which the test is measuring content that is linked to job performance. Content validation is often helpful for job simulation tests, where the employer is evaluating a sample of the candidate’s performance via a case study or simulation of job tasks the candidate will perform if hired.

Criterion-related Validation

A concurrent, criterion-related validation approach is commonly used to evaluate the validity of tests measuring unobservable traits like personality, biographical data (biodata), or cognitive abilities. This method relies upon demonstrating a statistical relationship between test scores and job performance, such as performance ratings, turnover rates, and other objective outcomes.

In a concurrent design, data is collected from a sample of current employees to help evaluate the relationship between test scores and job performance. When identifying the employee sample, employers should seek to include a broad representation of the overall workforce and qualified labor market, to avoid any bias. Once the employee sample is identified, the employer administers the test and gathers corresponding performance metrics for the employee sample, which may include subjective evaluations (e.g., supervisor ratings) or objective measures (e.g., turnover, sales figures).

A statistical analysis is then conducted to determine if there is a statistically significant relationship between the test scores and job performance measures. With criterion-related validity, the employer will be able to quantify the predictive power of the test by examining the degree to which higher test scores are associated with higher job performance, and vice versa. For example, if higher scores on a cognitive ability test are consistently associated with higher supervisor ratings, the test may be a useful tool for selecting high-performing candidates. Similarly, if a personality assessment can identify traits linked to lower turnover, it can help employers make better hiring decisions and reduce turnover-related costs.

Summary

To promote fair and merit-based hiring, the federal government is urging employers to adopt skills-based selection tools that focus on job-related competencies. Central to this approach is the validation of employment tests to ensure they accurately measure the skills that drive job performance. Two primary validation strategies support this effort:

• Content validation ensures that test items directly reflect the tasks and skills required for the role, based on a thorough job analysis.
• Criterion-related validation uses statistical methods to demonstrate a meaningful link between test scores and actual job performance outcomes, such as productivity or retention.

Together, these methods empower employers to make data-driven hiring decisions that are effective and legally defensible. By aligning assessments with real job demands, organizations can enhance workforce quality, reduce turnover, and build a talent pipeline with the right skillsets.

Be sure to follow these best practices when using tests and other assessments:

• Use professionally developed tools that have been validated for your specific jobs.
• Update validation studies periodically to reflect changes in job roles or organizational needs.
• Ensure fairness and legal compliance, especially under EEOC and Uniform Guidelines on Employee Selection Procedures.

Now is the time for employers to act. Review your current hiring practices, invest in validated assessment tools, and ensure your selection processes are grounded in job relevance and fairness. Doing so not only strengthens your workforce — it positions your organization as a leader in responsible, performance-based hiring.