Norm-Referenced vs. Criterion-Referenced Intelligence Tests

The short answer: the reference point changes the meaning

A norm-referenced intelligence score compares a person’s performance with scores from a defined reference population. A criterion-referenced score describes performance in relation to a defined skill domain or standard. One asks where performance sits among people; the other asks what specified performance has been demonstrated. IQ interpretations are ordinarily norm-referenced. But a raw browser-quiz result, or an IQ-labeled estimate without validated norms, is not automatically either kind of interpretation. The NCME Glossary defines these as different ways to interpret scores, not labels conferred by a number or test format.

To read a result, separate three questions: what was observed, what it is compared with, and what decision that comparison could support. The comparison could be a population, a stated domain or standard, or nothing documented. A statement about relative standing depends on evidence for the reference group and score conversion; a claim of meeting a standard depends on defining that standard and representing its domain. Counting correct answers alone supplies neither interpretation. Before interpreting a number, ask what it is compared with and whether that comparison fits the decision at hand. Without either reference, describe performance on that particular set of questions, not population standing or mastery.

Sources: Glossary - NCME

What is each score actually anchored to?

A test score answers a comparison question only after someone states the comparison. The raw score is the observed result on a particular form, often a count of correct answers. It records what happened on those items; by itself, it does not say how the performance compares with other people or whether a defined skill has been demonstrated. The NCME Assessment Glossary makes this distinction explicit: norm-referenced interpretation compares performance with a specified reference population’s score distribution, while criterion-referenced interpretation relates performance to a defined criterion domain. A norm is information about the distribution for specified groups. A criterion names the domain or performance target against which results are interpreted. | Interpretation | Anchor | Question it can answer | |---|---|---| | Norm-referenced | Scores from a specified reference population | Where does this performance fall in that distribution? | | Criterion-referenced | A defined domain or performance standard | What level of the specified performance does this result indicate? | These are interpretations of performance, not labels that follow automatically from a test’s format. A report that says “above average,” for example, is making a population comparison and should identify the relevant norm group and the scoring basis. A report that says “meets the standard” is making a criterion claim and should identify what competence or domain the standard represents. Merely printing a threshold does not explain how it was chosen or what successful performance means. A percentile rank belongs to the first kind of comparison: it indicates the percentage of scores in a specified distribution that fall below a given score. It is not the percentage of questions answered correctly. If a report says “70th percentile,” that does not mean the person answered 70 percent of the items correctly; it describes position within the stated score distribution. The NCME glossary defines percentile rank in those terms. The ETS Standards for Quality and Fairness also distinguish norm-based reporting from percent-correct reporting and caution that percent-correct results from different tests are not directly comparable. Two tests can differ in content and difficulty, so the same fraction correct does not establish the same standing or competence. The practical check is simple: ask what was observed, what it is compared with, and what conclusion the comparison supports. If a score report names neither a reference population nor a defined domain or standard, the number may still summarize performance on that form, but it has not yet explained relative standing or mastery. The names of score scales do not supply the missing reference point; the documentation must show it.

Sources: Glossary - NCME; Standards for Quality and Fairness

Why is an IQ interpretation usually norm-referenced?

An IQ interpretation typically describes relative standing in an age-appropriate reference population. It does so because broad general cognitive ability is not ordinarily reported as mastery of a finite syllabus with a pass threshold. That is the usual purpose of a professional IQ score, not a rule that every question about cognition must use norms. The NCME Assessment Glossary defines a norm-referenced interpretation as comparing a person’s performance with the distribution of performance in a specified reference population. It defines norms as statistics or tables that summarize score distributions for specified groups, such as people of particular ages or grades. A percentile, when reported, expresses position within that specified score distribution; it does not mean the person answered that percentage of questions correctly. A criterion-referenced interpretation has a different anchor: performance in relation to a defined domain. The same NCME glossary describes examples such as comparison with a cut score or an interpretation tied to a domain. For example, a school assessment might ask whether a learner has demonstrated specified mathematics operations after instruction. This is narrower than general cognitive ability. A criterion is not simply a convenient number; it must refer to the competence the test is meant to describe. Broad reasoning does not come with one agreed adult syllabus whose completion marks general intelligence. There is no single checklist of puzzles or learned content that all adults must master for an examiner to declare general cognitive ability “passed.” This describes the ordinary IQ interpretation; it does not mean a criterion can never be defined for a specific cognitive skill. When the claim concerns broad cognitive functioning, the usual question is how performance compares with relevant people, not whether a mastery threshold was crossed. The Standards for Educational and Psychological Testing, jointly produced by the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education, provide a professional framework for evaluating test interpretations and uses. This framework does not certify every test that reports an IQ-like number. A defensible relative-standing claim needs documentation identifying the instrument, relevant population, and score conversion. Making questions harder or labeling a result “IQ” does not supply that evidence. So when a report makes an IQ claim, check for the named norm group and instrument-specific scoring evidence before reading the result as relative standing. If it instead describes performance on a defined skill, ask what domain and standard support that statement. Task difficulty alone is not a comparison group, and a number alone is not a criterion. These checks clarify what an assessment says; they do not turn a score into a diagnosis, identity, or fixed account of a person’s potential. That distinction keeps interpretation tied to the stated purpose rather than the familiar label printed beside a score.

Sources: Glossary - NCME; Standards for Educational and Psychological Testing (2014 Edition)

Why can the same raw score answer different questions?

A raw score is the direct record of performance on the items administered: for example, how many answers were marked correct. It does not, by itself, say how that result compares with other people or whether a person met a defined standard. The National Council on Measurement in Education’s “Glossary” distinguishes raw scores from norm-referenced and criterion-referenced interpretations. Consider an example: a fictional examinee answers 15 of 20 items correctly. The count stays fixed, but its meaning depends on what question the assessment is meant to answer. If a test publisher has studied an appropriate reference group and supplies a justified conversion for this form and examinee, a report could use that evidence to describe relative standing. The number 15 alone cannot tell us whether that standing is high, typical, or low. We would need the actual conversion and information about the population behind it; inventing a percentile from the count would turn missing evidence into false precision. A different report might ask whether the examinee has demonstrated a specified skill. Suppose, purely for illustration, the test is designed to sample a clearly defined set of operations, and its documented standard says that at least 15 correct responses meet a stated threshold. The report could then describe performance against that criterion, if the items represent the domain and the threshold has a defensible basis. That hypothetical threshold is not evidence about any real test, and it cannot be used to calculate an IQ score. A number chosen for convenience does not become a meaningful standard just because it separates scores into two groups. If neither comparison is documented, the responsible statement remains narrow: this person answered 15 of these 20 items correctly on this exercise. Educational Testing Service’s “Standards for Quality and Fairness” cautions that percent-correct scores do not become comparable across different tests merely by sharing a percentage scale. Different forms may sample different content. Likewise, the same raw count across two forms does not establish equal skill, difficulty, or relative standing. This is why reports should keep observation and interpretation distinct. The raw result describes what happened on the administered items. A norm conversion adds a comparison with a defined population; a criterion interpretation adds a claim about a defined domain or standard. Each added claim needs its own supporting method. When reading a result, first identify the observed performance, then locate the comparison or criterion the report actually documents. If there is none, keep the conclusion at the level of the responses rather than supplying an unstated meaning.

Sources: Standards for Quality and Fairness; Glossary - NCME

What makes a criterion more than a cutoff?

A cutoff is criterion-referenced only when it represents a defensible standard for a specified domain and the test content supports the intended competence claim. A threshold chosen to select a fixed share of people remains relative even if printed as a fixed number. That line needs a reason. Start by asking what competence the result describes. The domain might be a defined set of skills or knowledge, such as interpreting a information or carrying out specified steps. It needs detail to show what counts as evidence. A label such as “proficient” is a conclusion, not a description of the domain; without knowing what it covers, the reader cannot tell what passing means. The next question is whether the assessment samples that domain. The NCME Assessment Glossary describes a content domain as characteristics a test is intended to measure, represented in test specifications. It describes domain or content sampling as selecting items systematically to represent the domain. Its definition of content-related validity evidence asks whether test content is faithful to domain performance and samples it. A collection of questions cannot support a broad mastery claim just because someone sets a passing score: the questions must cover the stated target. A standard also needs a reasoned basis. Someone must decide what level counts and explain how it relates to the domain. The ETS research memorandum “Applications of Criterion-Referencing to Schools” discusses identifying the assessment’s audience, setting objectives, choosing how results will be reported, and reflecting educational objectives in test items. Its abstract also treats setting cutoff scores and converting norm-referenced tests as procedures requiring attention. The report concerns school applications, so it does not establish an adult intelligence standard. It does show that a cutoff is one part of a design and interpretation process, not a self-justifying number. Consider how the same arithmetic threshold can carry different meanings. Imagine two assessments each use 70 percent as the pass line. In one, the line indicates adequate performance on specified essential tasks, and the items sample those tasks. In the other, it separates a desired proportion of examinees from others. The displayed number is identical, but the first is a claim about a domain, while the second is a selection rule based on relative standing. This is an illustration, not evidence about actual tests; the numeral alone cannot establish which interpretation is warranted. A report that says “pass,” “meets standard,” or “proficient” should prompt three questions: Who defined the standard, and for what decision? What performance is it meant to represent? Do the items sample that domain closely enough to support the claim? If documentation answers these questions, a criterion interpretation may be meaningful for that purpose. If it offers only a threshold, that shows where reporting changes, not what competence the person has demonstrated.

Sources: Applications of Criterion-Referencing to Schools; Glossary - NCME

How does the reference point affect test design and score quality?

A test's purpose changes what its designers must show about its scores. If the claim is that a person's performance is higher or lower than other people's, the score needs to support relative standing in the relevant population. If the claim is mastery of a defined skill or standard, the test needs evidence that its content represents the domain and its decision rule supports that classification. Quality evidence must fit the interpretation being made. The 1969 article “Implications of Criterion-Referenced Measurement” contrasts performance relative to others with status against an established standard. Its abstract examines how these approaches affect variability, item construction, reliability, validity, item analysis, reporting, and interpretation. This is a conceptual discussion in educational measurement, not a contemporary study validating an intelligence test. Its point is that the intended comparison changes the questions used to evaluate a test. For a norm-referenced interpretation, developers must define the population the comparison represents and support the score conversion used to describe standing in that group. A score may be consistent and still fail to justify a population comparison if the reference group, conversion, or interpretation is unsuitable. Reliability describes score consistency; it does not by itself establish what a score means or whether a decision is warranted. A criterion-referenced interpretation raises different questions: What counts as the skill or domain? Do the items represent it? What does the standard mean in observable performance, and how was the decision rule justified? A test might apply a cutoff consistently while that cutoff remains weakly connected to the claimed competence. Consistency cannot repair a poorly defined domain or unsupported threshold; a sound criterion also does not guarantee that every classification is precise. The ETS Research Report “Psychometrics” maps related topics in its abstract, including test purposes, item types, reliability, validity, score interpretation, norm- and criterion-referenced approaches, and standard setting. The accessible abstract establishes the report's scope, not which design is superior. Alongside the 1969 discussion, it suggests a practical check: does documentation connect items, scoring, and decisions to the interpretation offered? Neither format is inherently more fair, reliable, or useful. A population comparison can be poorly matched to a person or decision; a criterion can be vague or represented by too narrow a sample of tasks. Ask: What claim is this score making, and what evidence supports it for this population and use? A technical scale, familiar label, or pass/fail line cannot answer that alone. Inspect documentation in sequence: identify the claim; find the comparison group or defined domain; check how responses become the reported score; then ask whether evidence supports the attached decision. For relative standing, look for an appropriate population and documented conversion. For mastery, look for a defined domain, representative content, and reasoned standard. Without those links, a result may describe performance on administered tasks, while broader interpretation remains unsupported.

Sources: Implications of Criterion-Referenced Measurement; Psychometrics

Can one test support both interpretations?

Yes. The same assessment can support norm-referenced and criterion-referenced interpretations, provided each claim has its own suitable method and evidence. A report might locate overall performance relative to an identified population, then describe performance on a defined skill domain. The first claim concerns standing among people in that population; the second concerns what the examinee demonstrated against specified content or a stated standard. Neither follows automatically from the other, even when both draw on the same answers.

The Standards for Educational and Psychological Testing offer a shared professional framework for evaluating testing. The American Educational Research Association’s edition page identifies the Standards as a joint product of AERA, the American Psychological Association, and the National Council on Measurement in Education, and links to the open-access 2014 edition. The accessible abstract for ETS Research Report RR-98-14, “Psychometrics,” treats norm-reference and criterion-referenced approaches alongside validity, reliability, scaling, score interpretation, and standard setting. Its page summarizes topics, not findings about any particular intelligence test. These sources support a practical discipline: assess each interpretation as a separate claim, with evidence matched to its purpose.

Imagine an assessment with a broad reasoning composite and a short section on reading a defined kind of chart. Its report could compare the composite with an appropriate reference group if the test’s norming and conversion support that use. It could also state whether the examinee met a specified chart-reading standard if the section represents that skill and the standard has a defensible basis. A broad composite does not certify chart-reading mastery; success on that limited skill does not establish relative standing in general reasoning. The same responses may inform both statements, but each needs its own support from test content, scoring, and intended use.

Reports may place unlike statements side by side. A norm-based composite asks how performance compares within a stated population; a domain statement asks what specified performance the examinee showed. Read each line for its reference point and evidence. Check whether the population, score conversion, task domain, and decision rule are documented. A percentile or scaled score alone cannot establish a separate competence claim; a pass label alone cannot establish relative standing. The ETS abstract confirms that both approaches and score interpretation are distinct psychometric topics, but it cannot verify a specific report’s quality. Evidence supporting one interpretation does not transfer automatically to another. If documentation supports only a population comparison, treat a separate domain claim as unestablished; if it supports only a bounded skill judgment, do not read it as an IQ ranking.

Sources: Standards for Educational and Psychological Testing (2014 Edition); Testing Standards; Psychometrics

What does a raw score tell you—and what does it leave out?

A raw score is the count or record of responses under a particular test’s scoring rules. It describes what happened on that form: for example, how many items were answered correctly. It does not, on its own, say how the performance compares with other people, whether a defined skill has been mastered, or how the result should be used. The Standards for Quality and Fairness distinguishes raw scores from converted scores and explains that a conversion needs a documented procedure suited to its purpose. A percent-correct figure is still a description of that test form. Imagine two unrelated 20-item quizzes, each with 15 correct answers. Both raw records can be stated as 15 out of 20, or 75 percent. That arithmetic does not show that the quizzes tested equally difficult material, sampled the same skills, or support the same conclusion. The Standards for Quality and Fairness cautions that percent-correct scores from different tests are not comparable simply because they share a percentage scale. An identical fraction can represent different tasks and different coverage; the denominator alone does not make their meanings equivalent. A percentile answers a different question. It locates a score within a defined score distribution for a specified group. It is not the percentage of questions answered correctly, and it is not a direct measure of how much content a person knows. The group matters: a percentile has meaning only with the population and conversion that produced it. The National Council on Measurement in Education’s glossary defines norm-referenced interpretation through comparison with a specified reference population. A scaled score is also a derived representation, not the original count. A provider may convert raw performance to a scale, but its meaning depends on documented construction and purpose. The number alone cannot show whether it supports comparison across forms or answers a question about a skill. A label such as “standard score” does not answer those evidence questions. The distinction matters when a result is used to make a decision. A raw count can guide review of missed items. A norm-based conversion may support a claim about relative standing if its population and method are appropriate. A mastery statement needs a defined skill domain and a defensible standard. A high percent correct or selected cutoff cannot supply those grounds by itself. Before drawing a conclusion, check the test and form, the scoring or conversion method, the population or task domain used as the reference, and the decision the report says the result can support. If the report does not identify these, describe the observed performance narrowly and leave the broader interpretation open.

Sources: Standards for Quality and Fairness; Glossary - NCME

What should an adult do with an online reasoning result?

An online reasoning result should first be read as a record of how you performed on that particular set of questions. Treat it as a comparison with other people only if the provider documents an appropriate reference population and score conversion. Treat it as evidence of mastery only if the provider defines the skill domain, explains the standard, and shows how the questions support that judgment. The NCME Assessment Glossary defines norm-referenced interpretation by comparison with a specified reference population and criterion-referenced interpretation in relation to a defined criterion domain. Without either reference point, the result has a narrower meaning: performance on that quiz under those conditions. Test IQ Free's current product is an adaptive 50-question test covering pattern reasoning, number patterns, and spatial reasoning. It selects questions as the test proceeds, but adaptation describes how questions are chosen; it does not establish who the score is compared with or what level of a defined skill it certifies. The free result displays a provisional estimated IQ range calculated with placeholder conversion parameters. There is no validated adult reference population for that conversion. The range cannot establish relative IQ standing, and it is not a validated judgment that you have met a reasoning standard. The label “IQ estimate” does not supply comparison evidence. A useful next step is to review the worked answers and identify the operation behind a missed question. Note whether you overlooked a rotation, numerical relationship, or spatial transformation, then try a fresh problem using that operation. This makes the result useful as feedback about what to inspect or practice. It does not show that a domain score is a stable trait, diagnose a condition, or measure your worth or fixed potential. Keep the observation specific: “I missed this kind of pattern today” describes what the quiz observed more accurately than “I am bad at reasoning.” The optional $9 report adds explanation, worked examples, applications, and a practice plan; its point estimate and percentile use the same provisional conversion as the free result. Payment adds interpretation and structure, not validity, a norm group, or a criterion standard. For low-stakes practice, you can take the free test and use its feedback without treating the band as an established population comparison. If an institution needs a consequential decision, follow its assessment process or consult a qualified professional. Ask what instrument, comparison population or task standard, and evidence support that use.

Sources: Glossary - NCME

Choose the comparison that matches the question

Choose the comparison that answers the decision. For relative standing, look for a norm-referenced interpretation tied to an appropriate population and documented score conversion. For demonstrated skill, look for a defined domain, defensible standard, and evidence that the test samples that skill. For low-stakes review, raw answers and worked explanations may be enough. Use three checks before acting: What claim does the report make? What is its reference point: a population, a task standard, or neither? Does that evidence fit the consequence you have in mind? The NCME Glossary distinguishes population comparisons from performance against a defined criterion domain. The joint Standards for Educational and Psychological Testing provide a framework for evaluating interpretations and uses. A score label or cutoff cannot answer these checks by itself. Test IQ Free offers a free 50-question reasoning test and worked answers for practice. Its displayed estimated IQ range comes from a provisional conversion without a validated adult reference population. Treat it as feedback on this quiz, not established relative standing or demonstrated mastery. For consequential education, clinical, or employment decisions, follow the institution's assessment process or consult a qualified professional. Let the evidence the result can support determine your next step.

Sources: Glossary - NCME; Standards for Educational and Psychological Testing (2014 Edition)

Questions readers ask