What Does Criterion Validity Actually Show?

What does a criterion-validity claim actually claim?

A criterion-validity claim is evidence that scores from a specified test relate to a outside outcome for a studied group and context. The outcome is called the criterion; criterion-related evidence is the observed relation between it and test scores. A study might ask whether one reasoning-test version relates to a defined later academic measure among a particular group of students. That finding speaks to those scores, that measure, and that group. It does not automatically transfer to another test version, outcome, population, setting, or decision. The 2014 Standards for Educational and Psychological Testing frame validity as evidence and theory supporting a proposed score interpretation for an intended use. Criterion-related evidence is one part of that case, not a lasting badge attached to a test name. The National Research Council’s Performance Assessment for the Workplace: Volume I also explains that a criterion’s suitability affects what a score relationship can mean; its workplace focus offers measurement reasoning, not validation of an intelligence test. A relationship alone does not show that a test measures intelligence comprehensively, establishes norms, explains an outcome, or guarantees an individual result. Ask instead: which score relates to which measured outcome, for whom, and for what decision?

Sources: Standards for Educational and Psychological Testing; Performance Assessment for the Workplace: Volume I, Chapter 9: Evaluating the Quality of Performance Measures: Criterion-Related Validity Evidence

What counts as the criterion in an intelligence-test study?

The criterion is the observable outcome chosen to represent what a test is supposed to predict or relate to. It might be a defined grade measure, a training result, or a measure of job performance. The criterion is a practical stand-in for a broader outcome of interest, not automatically a complete or neutral measure of it. So a claim that an intelligence score relates to “academic success” is incomplete until it identifies what researchers recorded: course grades, scores on an achievement test, completion of a program, or something else. Each captures a different part of performance and may reflect influences beyond the ability the test is meant to assess. The 1991 National Research Council chapter “Evaluating the Quality of Performance Measures: Criterion-Related Validity Evidence” makes this choice central to interpreting a relationship. Its subject is workplace assessment, not intelligence testing, but its measurement point applies: actual job performance is a conceptual target, while available measures are approximations that must be developed and justified. The chapter notes that military training outcomes were often represented by training grades based largely on written tests of cognitive knowledge. Such grades could omit important parts of job proficiency, including psychomotor ability and problem-solving skills. A score’s relationship with that grade therefore says something about performance on the recorded training criterion; by itself, it does not establish a relationship with every aspect of doing the job. Different criteria make different slices visible. School grades record evaluated work under a school’s grading practices. An achievement test samples performance on its specified academic tasks. A training score reflects the training tasks and scoring rules used. A supervisor rating records a judgment about performance, while a hands-on measure records performance on selected practical tasks. Neither a rating nor a hands-on task is automatically a gold standard: the National Research Council chapter explains that a practical measure may cover only a small or atypical fraction of the important work, while a rating can be influenced by factors unrelated to the performance being judged. The choice needs a reason tied to the intended use. The 2014 “Standards for Educational and Psychological Testing” likewise frames validity around evidence supporting a proposed score interpretation and use. That principle makes the criterion part of the argument. Before accepting a result described as criterion-valid, name the test score and the outcome measured outside the test. Then ask whether that outcome represents the target relevant to the decision. “Predicted performance” is too broad if the study actually measured a narrow training score or one kind of school result. A precise criterion does not settle every question about a test, but it lets readers see exactly what relationship the evidence addresses—and where it stops.

Sources: Performance Assessment for the Workplace: Volume I, Chapter 9: Evaluating the Quality of Performance Measures: Criterion-Related Validity Evidence; Standards for Educational and Psychological Testing

How do predictive and concurrent evidence differ?

Predictive and concurrent evidence differ in when the criterion is measured. A predictive study obtains test scores first and measures an external outcome later. A concurrent study obtains test scores and criterion information at about the same time. The 2014 Standards for Educational and Psychological Testing describes both as designs for studying test–criterion relationships. Timing does not establish whether the outcome represents the intended performance.

Suppose a school wants to know whether a reasoning score forecasts performance at the end of the next term. A study that gives the test now and later records a defined outcome directly examines that temporal claim. The result concerns this score and outcome in that setting. It does not show that the score caused later performance or that the relationship transfers to another age group or institution. The Standards says credibility depends on the criterion’s relevance and the interpretation made for the application.

A concurrent study asks how scores relate to criterion information collected at roughly the same time. It can show whether a new measure tracks an accepted measure or relates to a current outcome. The Standards notes that this design avoids temporal changes and can help compare measures. But it does not test directly whether today’s score forecasts a later outcome. Agreement also does not make measures interchangeable: they may capture different parts of the target, and the criterion needs justification.

When a test description says it “predicts success,” check whether the outcome was measured later, how long afterward, and whether that interval resembles the decision you face. Ask what “success” meant and whose outcomes. A same-time association cannot be relabeled a forecast. Conversely, later measurement does not make a finding decisive if the criterion is poorly matched to the intended use. The National Research Council’s 1991 chapter, Performance Assessment for the Workplace: Volume I, treats predictor and criterion as distinct parts of the inference and explains that criteria may imperfectly represent performance.

Match the study’s score, outcome, population, setting, and timing to the claim. For later decisions, seek evidence that follows scores to a relevant later criterion. Predictive evidence can support a bounded forecast; concurrent evidence can support a bounded same-time relationship. Neither alone establishes certainty for one person, causation, or a general claim about intelligence. “Predicts” is useful only when the study measured a later outcome and the proposed decision fits its horizon and context.

Sources: Standards for Educational and Psychological Testing; Performance Assessment for the Workplace: Volume I, Chapter 9: Evaluating the Quality of Performance Measures: Criterion-Related Validity Evidence

What does the school-grades example show—and not show?

Ryan and French’s 1976 study illustrates how narrow a criterion-validity finding can be. Its publisher abstract reports a comparison involving pupils in grades three through five at schools classified at three socioeconomic levels. The researchers examined whether verbal and nonverbal intelligence tests and an achievement test predicted grades. The criterion was school grades, not intelligence itself or a general measure of later success. The abstract says the achievement test generally predicted grades about as well as the verbal intelligence test. Both generally predicted them better than the nonverbal test and were nearly as predictive as pupils’ previous grades. [Long-Term Grade Predictions for Intelligence and Achievement Tests in Schools of Differing SocioEconomic Levels](https://journals.sagepub.com/doi/10.1177/001316447603600245) also reports that patterns differed among the school levels: verbal-test scores accounted for a greater proportion of grade variation in lower-level schools, while nonverbal-test scores accounted for a greater proportion in higher-level schools.

That variation matters because the claim is not simply “this test predicts.” It is “this test score related to this grade outcome among these pupils in these school contexts.” Change the outcome, examinees, or setting and the abstract no longer answers the same question. The result concerns grades in the studied school years. It does not show that either intelligence test predicts every kind of academic achievement, adult work performance, or an individual pupil’s future with certainty. Nor does the abstract show that test scores caused the grades. Prediction here describes a relationship reported in the study; it does not explain why the relationship appeared.

The accessible abstract is enough to see the comparison and its boundary, but not to explain the differences. It gives no sample size or fuller methods. Those omissions limit what can be inferred; they are not a reason to fill gaps with a story about teaching, opportunity, school resources, or grading practices. The socioeconomic-level differences are reported observations, not explanations of their cause.

The practical lesson is to keep the criterion attached to the result. “Predicted later grades” is more informative than “proved valid” because it names the measured outcome and lets readers compare the finding with a proposed use. It cannot establish how a current test will perform in another population or setting, and it says nothing about a browser test whose score conversion was not studied. Use it as an example of bounded prediction only while keeping the grade outcome, pupil age range, test types, school contexts, and limits of the abstract in view.

Sources: Long-Term Grade Predictions for Intelligence and Achievement Tests in Schools of Differing SocioEconomic Levels

How can criterion choice distort the apparent relationship?

A criterion can be deficient if it misses important parts of the target performance, or contaminated if irrelevant influences enter its measurement. Either problem can make a score-outcome relationship more or less representative of the real decision someone wants to make. The National Research Council's 1991 chapter, “Evaluating the Quality of Performance Measures: Criterion-Related Validity Evidence,” makes this order explicit in its discussion of personnel assessment. The criterion is not a neutral container into which performance simply falls; researchers define and measure it, and those choices shape what a relationship can mean. The chapter illustrates deficiency and contamination with military training measures. Training grades were often based largely on written tests of job-related cognitive knowledge. Such grades could omit important parts of job proficiency, including psychomotor ability or problem-solving, and could reflect verbal or other cognitive demands more than actual work did. In this setting, a relationship between an aptitude score and a written grade would concern performance on the recorded training measure. It could not, by itself, show an equally strong relationship with the broader target of doing the job. A hands-on task does not automatically solve the problem. The same chapter notes that a practical measure may cover only a small or atypical fraction of important work tasks, leaving the criterion deficient. Its ratings might also be influenced by factors irrelevant to the performance being judged, which is contamination. There is another risk when predictor and criterion share features. The National Research Council discusses written aptitude and job-knowledge tests: a common paper-and-pencil format may contribute to their observed association, even when that shared format is not central to the work outcome of interest. Shared method features therefore offer a possible explanation for apparent alignment; they do not establish that any particular result is biased. For an intelligence-test claim, the parallel question is whether the outcome partly rewards the same response demands as the test, rather than independently representing the broader ability or performance named in the claim. The practical check is to inspect both sides of the proposed relationship. What important parts of the target does the criterion leave out? What unrelated demands or rating influences might enter it? If the criterion is a narrow proxy, a strong association supports a claim about that proxy under the studied conditions. Extending it to comprehensive intelligence, general success, or another decision requires separate justification. Criterion choice does not make evidence meaningless; it sets the boundary around the conclusion the evidence can responsibly carry.

Sources: Performance Assessment for the Workplace: Volume I, Chapter 9: Evaluating the Quality of Performance Measures: Criterion-Related Validity Evidence

How is criterion evidence different from a norm or a raw score?

Raw performance, a norm-referenced interpretation, and criterion evidence answer different questions. Raw performance records what a person did on the administered items. A norm-referenced interpretation compares that performance with a defined reference group. Criterion evidence asks whether scores relate to an outside outcome, such as later grades. An outcome relationship cannot by itself show how unusual a score is; a comparison group cannot by itself show what the score forecasts. Each label should be tied to the evidence that supports that particular interpretation. To interpret performance relative to other people, a test needs an appropriate norm group: the reference sample against which scores are compared. Without that basis, a raw count remains a count; it does not become an adult IQ score merely because it appears on a familiar-looking scale. A percentile expresses relative standing within a specified comparison group. It is not the percentage of items answered correctly. Answering 70 percent correctly does not by itself reveal a percentile; that depends on scores in the relevant reference group. This numerical example is illustrative, not a test result. Criterion evidence makes a separate claim: scores varied in relation to a measured outcome in a studied context. A study might compare scores with later grades. That finding creates no norm group or population standing. Conversely, a norm does not show that a score predicts a later outcome. The evidence types complement one another but are not interchangeable. The 2014 edition of the *Standards for Educational and Psychological Testing* treats norms and score interpretation separately from evidence about relationships with other variables. Ask what evidence supports each statement. “The examinee answered this many items correctly” describes performance. “The score has this relative standing” requires an appropriate norm comparison. “Scores related to this outside outcome in a study” describes criterion evidence. Thus, evidence that scores predict grades cannot justify an IQ range, percentile, or adult comparison unless separate, suitable score-interpretation evidence supports those claims. Test IQ Free's displayed range is provisional, produced by an unvalidated conversion with no validated adult reference population. It is not a normed score or established adult comparison. Use it for low-stakes practice and review its worked answers, keeping the estimate separate from performance and norm-based interpretation.

Sources: Standards for Educational and Psychological Testing

Does a relationship explain outcomes or guarantee an individual's result?

A relationship between test scores and an outcome describes a pattern in studied data; by itself, it does not explain why the outcome occurred. An association alone cannot show that taking a test caused a later result. The bounded claim is that, in the examined group and setting, scores and the measured criterion varied together in a way relevant to a proposed use. The National Research Council chapter “Evaluating the Quality of Performance Measures: Criterion-Related Validity Evidence” illustrates this with predictor-criterion tables. A table can show how often people in each test-score category fall into each outcome category, preserving information that a single summary may hide. The chapter’s worked table is explicitly hypothetical: its numbers explain the display, not findings about an intelligence test or a real population. Its methodological point still matters: a group pattern does not settle what will happen to one person. A higher score may be associated with a higher average outcome or greater likelihood of a criterion result, while some higher-scoring people do not reach it and some lower-scoring people do. A correlation or regression summary compresses the pattern and can conceal the spread of outcomes around the average. The National Research Council chapter explains that reporting only mean criterion scores can hide prediction errors visible in the underlying data. Thus, “predicts performance” should prompt a question about individual prediction accuracy, not an assumption of exact forecasts. That accuracy depends on details beyond a validity label: the prediction model, measurement precision, outcome variability in the relevant group, and fit between the studied context and the decision at hand. Without those details, readers cannot infer an exact personal outcome from the existence of a relationship. The 2014 Standards for Educational and Psychological Testing frame validity as evidence supporting a proposed interpretation and use; an association alone is not a universal promise. The practical boundary is simple: “scores were associated with an outcome in a sample” does not mean “this score tells me exactly what I will achieve.” Criterion evidence can inform a decision when its outcome and population fit that decision, while uncertainty about an individual remains. A test result describes performance under its conditions and, where studied, its relation to a measured outcome. It does not establish a cause or determine a person’s future.

Sources: Performance Assessment for the Workplace: Volume I, Chapter 9: Evaluating the Quality of Performance Measures: Criterion-Related Validity Evidence; Standards for Educational and Psychological Testing

Why does the intended decision set the boundary?

Evidence is decision-relevant when the score, outcome, examinees, setting, and proposed use resemble those studied closely enough to support the same inference. A finding that scores relate to school grades does not by itself support employee selection, diagnosis, applicant ranking, or interpretation of adults’ performance. The 2014 Standards for Educational and Psychological Testing frame validity as evidence and theory supporting an interpretation of scores for a proposed use. Interpretations for different uses each need support. So “valid test” is incomplete: ask what the score is supposed to mean and what someone plans to do with it. Changing the proposed use changes the argument that must be supported. An employment rule illustrates this boundary in one domain. Under 41 CFR § 60-3.5, criterion-related evidence for a selection procedure consists of data showing prediction of, or significant correlation with, important elements of job performance. The rule distinguishes screening from ranking: evidence that may support pass/fail screening can be insufficient for ranking, particularly where ranking has greater adverse impact. This regulation concerns employment selection, not general intelligence-score interpretation. Keep five links together: exact score, measured criterion, people studied, context, and decision. A grade-prediction finding among pupils does not automatically become evidence about adult job performance; a criterion correlation does not automatically support diagnosis; screening evidence does not automatically justify ranking. Narrow evidence can still be useful. The ETS report The Validity of Aptitude Tests for High Ability Individuals reviewed historical research involving high-ability examinees and aptitude tests. Its abstract concludes that substantial validity is possible when a test fits the sample and predictive task, a relevant criterion is available, and statistical limitations do not reduce the correlation. This is conditional evidence about those reviewed settings, not a current estimate for every test. Fit gives narrow evidence practical value. Neither “one correlation proves everything” nor “a limited result means nothing” is a sound verdict. Ask whether score, criterion, population, context, and decision match. Where they do, criterion evidence may support that bounded use. Where they do not, the finding may inform but cannot carry the decision alone.

Sources: 41 CFR § 60-3.5: Technical standards for validity studies; Standards for Educational and Psychological Testing; The Validity of Aptitude Tests for High Ability Individuals

What can a reader responsibly do with this browser result?

Treat Test IQ Free's result as performance feedback from this particular 50-question reasoning test. Its displayed estimated IQ range is provisional, produced by an unvalidated conversion with no validated adult reference population. It is not a normed score, confidence interval, established adult comparison, or criterion-valid forecast. It can describe how you performed on this form in that sitting; it cannot establish your standing in a defined population or predict an outside outcome.

The Standards for Educational and Psychological Testing frame validity around evidence supporting a proposed score interpretation and use. Criterion-related evidence for this test would need to connect its score to a defined outcome in a relevant population and setting. The sources considered here do not provide such a study for Test IQ Free or its conversion. Findings about other instruments cannot fill that gap: evidence for another score, criterion, group, or decision does not automatically transfer to this result.

The test covers pattern reasoning, number patterns, and spatial reasoning. The free result includes the provisional range and domain feedback, and readers can review worked answers to see how particular operations work. This supports a low-stakes purpose: notice which question types felt easier or harder in this sitting, then practice a method on fresh examples. Review can build familiarity with those operations. It should not be described as raising general intelligence or predicting school, work, or clinical outcomes.

The optional $9 report adds explanation, worked examples and a practice plan around the same provisional estimate. It does not change the conversion, supply validation, or make the estimate more accurate. More detail may clarify feedback, but it is not evidence for a broader inference.

For curiosity, take the free test and review its worked answers as practice material. Do not use its range to diagnose a condition, identify giftedness, or make hiring, school-placement, or consequential decisions. The Standards for Educational and Psychological Testing emphasize that score interpretations need support for their intended use and examinee population. When a decision matters, seek an assessment designed for that purpose and interpreted with relevant evidence by a qualified professional. Use this result to reflect on performance here; choose an appropriate assessment when the decision requires more.

Sources: Standards for Educational and Psychological Testing

What should you check before relying on a criterion-validity claim?

Before relying on a criterion-validity claim, trace it from the reported score to the proposed decision. Record six links: the test score and version; the outside outcome, or criterion; who was studied; the setting; whether the outcome came later or at roughly the same time; and the intended use. The 2014 Standards for Educational and Psychological Testing frame validity as evidence for a particular score interpretation and use. Then inspect the criterion. What did researchers record, and how well does it represent the named outcome? The National Research Council’s chapter on job performance explains that operational measures can omit important parts of a target or include irrelevant influences. A relationship with a partial measure may still be useful, but only for the inference it supports. Do not transfer it silently to another outcome, population, setting, or decision. For Test IQ Free, the assignment supplies no criterion study for this 50-question score or its conversion. Its estimated IQ range remains provisional performance feedback, not a forecast of grades, work, or an individual outcome. This conclusion would change with appropriately designed evidence linking this score to a defined outcome in the relevant population and setting for the proposed use. For curiosity, take the free test and review its worked answers; for consequential decisions, seek an assessment designed and interpreted for that purpose.

Sources: Standards for Educational and Psychological Testing; Performance Assessment for the Workplace: Volume I, Chapter 9: Evaluating the Quality of Performance Measures: Criterion-Related Validity Evidence

Questions readers ask