What the 2026 Construct-Signature Study Added to Test Validation

What did the 2026 construct-signature study add?

The 2026 paper “Construct-signature validity (CSV) for intelligence testing” adds a design-based question to validation: do responses show patterns specified in advance from assumptions about examinee ability and item complexity? Its abstract describes three nested checks: relationships among score bands within a subtest, coherence across subtests, and item-level success patterns. The aim is to see whether planned task demands leave predicted signatures in responses, alongside factor-model evidence. The abstract demonstrates CSV through a secondary analysis of norming data for four nonverbal subtests of the Anadolu-Sak Intelligence Scale (ASIS), using data from twelve-year-old children. The findings were mixed: some predicted patterns appeared partly or only in particular ability bands; another expected pattern was absent in the medium-ability band; item-level analysis showed an ability-by-complexity interaction. This is a bounded demonstration, not validation of the full ASIS or other tests. CSV’s contribution is a set of testable predictions that can make questions about task representation more specific. The abstract does not establish that the complexity categories are sound, that patterns will replicate, or that they improve score interpretation. CSV therefore complements factor analysis; the study does not show that factor analysis is inadequate. Broader validity and practical value remain open.

Sources: Construct-signature validity (CSV) for intelligence testing

What does construct-signature validity add beyond a factor model?

Construct-signature validity (CSV) adds a design-based prediction to test validation: if a test’s theory about ability and task complexity is sound, specified patterns across those dimensions should appear in responses. The 2026 paper in volume 116 of Intelligence proposes three checks at the subtest, domain, and item levels. This can ask whether the tasks behave as the test’s design predicts, alongside evidence about how scores relate to one another. It is a complementary source of evidence, not a new stamp that makes a test valid for every purpose. Factor analysis is a family of methods that examines patterns of score covariation against a proposed latent structure: whether observed scores relate as expected if they reflect underlying abilities. CSV asks a different, design-anchored question: do responses vary across pre-specified combinations of examinee ability and item complexity in the predicted way? Neither question makes the other unnecessary. Factor models can be specified at different levels and answer different structural questions; CSV directs attention to whether the intended task demands leave the predicted pattern in the data. The distinction is about the evidence sought, not a contest between a “global” and “local” method. The paper names three nested signatures. The within-subtest signature (SCI) concerns the ordering of relationships among scores for tasks assigned lower, middle, and higher complexity. The within-domain signature (DCI) concerns whether the strongest coherence among subtests shifts toward the complexity level expected for a given ability group. The item-level signature (ICI) concerns ordered chances of success and contrasts across complexity bands. These are proposed response-pattern checks, not three separate IQ scores or independent verdicts. The paper’s accessible abstract lays out those definitions and reports their use in a secondary analysis of norming data; the following sections examine the predictions and results in more detail. This design-to-response link is the method’s distinctive contribution. In principle, an overall fit to a latent structure could coexist with a mismatch between the test’s claimed task demands and observed responses. CSV makes that mismatch a question researchers can specify in advance and inspect across ability levels. That is a rationale for adding the check, not proof that factor analysis cannot detect local problems or that CSV has already shown incremental value. The Standards for Educational and Psychological Testing frame validity as evidence supporting interpretations of scores for intended uses, with evidence considered together. Under that framing, the useful question is not whether a test “has” CSV validity, but what these pre-specified patterns contribute to a particular interpretation. The 2026 proposal makes a potentially falsifiable link between a test’s design account and response data. Its value depends on the quality of that account and on evidence beyond one application; the framework itself does not establish the interpretation.

Sources: Construct-signature validity (CSV) for intelligence testing; Standards for Educational and Psychological Testing

How did the authors make the predictions testable?

The authors made construct-signature validity (CSV) testable by turning a theory about ability and task complexity into predictions that could be checked against response data. In the accessible abstract of “Construct-signature validity (CSV) for intelligence testing,” items from four nonverbal ASIS subtests were assigned in advance to low, medium, or high complexity bands, while children were grouped into three ability bands. The researchers then examined whether score relationships and item success followed specified patterns across those bands. Pre-specification matters because it makes the expected pattern visible before results are interpreted; it reduces the freedom to invent a matching explanation after seeing the data. But it cannot establish that the complexity labels themselves accurately represent the task demands the theory intends to measure. The paper describes three nested checks, not three different IQ scores. The within-subtest signature (SCI) asks whether correlations among scores for lower-, middle-, and higher-complexity items follow an ordered pattern within a subtest, and whether that pattern changes across ability bands. The within-domain signature (DCI) asks whether cross-subtest coherence is strongest at the complexity level predicted to match a group’s ability. The item-level signature (ICI) examines whether the probability of answering items correctly is ordered across complexity bands and whether the specified drops between bands meet the paper’s stated contrasts. They make different predictions about how a design should behave; none, alone, is a complete judgment about the test or a person. Consider an illustrative research team preparing a new assessment blueprint. Before collecting responses, it could describe the operations each task is intended to require, assign tasks to complexity bands under written rules, and state which score relationships should appear across ability groups. Later results could then be compared with those predictions. If a pattern is absent, the discrepancy is informative, but its cause remains open: the construct theory may be incomplete, the coding rules may not capture the intended demands, or the observed data may not support the predicted relationship. Advance specification narrows interpretive flexibility; it does not resolve those alternatives. A bounded example from outside intelligence testing shows why task coding deserves scrutiny. The ETS report “A Complexity Analysis of Items From a Survey of Academic Achievement in the Life Sciences” examined 44 life-science items from the 1985–86 NAEP science assessment. Its abstract reports that educators’ average difficulty judgments were the strongest single predictor of item difficulty. In a model using discrete item attributes without those overall judgments, required knowledge mattered, while rated cognitive demand did not. This study concerned science achievement questions, not ASIS, IQ tests, or CSV, so it cannot establish that CSV’s complexity map is right or wrong. It illustrates a narrower point: an intended or rated demand should not automatically be treated as the difficulty examinees actually experience. A credible application therefore needs transparent coding rules, predictions specified in advance, and evidence that the categories represent the operations the construct account says matter.

Sources: Construct-signature validity (CSV) for intelligence testing; A Complexity Analysis of Items From a Survey of Academic Achievement in the Life Sciences

What did the ASIS demonstration find, and what stayed uncertain?

The demonstration produced a mixed profile, not a single pass-or-fail result. In a secondary analysis of norming data from the Anadolu-Sak Intelligence Scale (ASIS), the researchers examined items from four nonverbal subtests. They classified items in advance into three complexity levels and divided children into three IQ groups. The abstract of “Construct-signature validity (CSV) for intelligence testing” reports that some predicted relationships appeared in some groups, while other predicted patterns were incomplete or absent. That unevenness matters: the method can describe where predictions fit and where they do not, but the abstract does not establish why a mismatch occurred.

For the within-subtest signature (SCI), the predicted ordering appeared in both the low- and high-ability groups. The support was partial, however. The abstract says adjacent correlations were close and the key differences small, so the predicted order was not sharply separated. In the medium-ability group, the observed pattern showed adjacency but not the transition the framework predicted. “Partial” here means that some features aligned with the expected order while the evidence did not show the complete, distinct pattern. This is neither full confirmation nor proof of failed measurement.

The within-domain signature (DCI) also varied by group. Its expected shift in the peak of cross-subtest coherence appeared in the high-ability group, only partly in the low-ability group, and was absent in the medium-ability group, according to the abstract. These comparisons describe fit to prediction, not its cause. The available abstract does not identify whether complexity coding, the subtests selected, the grouping scheme, or another feature accounts for the partial and missing patterns. Assigning a cause would go beyond what this report lets a reader verify.

At item level, the abstract reports that percent-correct profiles and a binomial generalized estimating equation model showed an ability-by-complexity interaction. In plain terms, the relation between the predefined complexity levels and the proportion of correct answers differed across the ability groups analyzed. This adds a finer-grained result alongside the band-score patterns. It is an interaction finding, not a statement that individual scores are reliable, and it does not by itself establish that the complexity labels capture the intended cognitive demands. The abstract supplies no basis here for adding effect sizes, confidence intervals, or a detailed account of which cells drove the interaction.

the demonstration shows how CSV can give a differentiated account: patterns may fit partly in some groups, a predicted transition may be missing in another, and item-level results may still show an interaction. These results concern different summaries of one bounded analysis, so they are not contradictory. Nor do they comprehensively validate ASIS. The abstract describes a secondary analysis of one instrument’s norming data and four nonverbal subtests; it cannot establish stability in new samples, explain the mismatches, or show that using them improves score interpretation or decisions. The next question is whether independent data reproduce the patterns under defensible complexity classifications, and whether the results add practical value.

Sources: Construct-signature validity (CSV) for intelligence testing

What did the item-level signal add to the demonstration?

The item-level result adds a closer view of the proposed pattern. Rather than asking only whether grouped scores relate in an expected order, the item-level signature asks whether the relationship between task-complexity bands and success changes across ability groups. In the abstract for “Construct-signature validity (CSV) for intelligence testing,” the researchers report examining percent-correct profiles and fitting a binomial generalized estimating equation model. Both analyses showed an ability-by-complexity interaction in the ASIS demonstration. In plain terms, the association between the preassigned complexity categories and the probability of a correct response differed across the ability groupings analyzed. The abstract does not provide an effect size or enough cell-by-cell detail to say how large each difference was or reconstruct its direction. This finding comes from the same secondary analysis of norming data for four nonverbal ASIS subtests among twelve-year-old children; it is not a result about intelligence tests generally. This finer grain complements the other signatures because an aggregate relationship can conceal where a proposed design pattern appears or breaks down. Different item-success profiles can point researchers toward task categories or ability bands for closer examination. But the interaction alone does not identify its cause. The complexity labels may or may not capture the intended cognitive demands; other item features may matter; and the pattern may or may not recur in new data. These are follow-up questions, not explanations established by the reported interaction. The abstract supports the statistical pattern under the study’s analysis; it does not establish individual score accuracy or practical decision benefit. Three claims can easily be collapsed. First is the observed result: item success varied in an ability-by-complexity pattern in this analysis. Second is the construct claim: the complexity map represents task demands relevant to the intended ability. Third is the use claim: adding this evidence improves a particular interpretation or decision. The first does not automatically prove the second, and neither alone proves the third. The abstract presents the item-level interaction as one part of a proposed validity framework, alongside within-subtest and within-domain signatures. It does not establish score reliability, a causal mechanism, or a general property of item response models. Read narrowly, the item-level signal makes the design-based proposal more inspectable: researchers can ask where the predicted response pattern appears, then test its interpretation and usefulness separately.

Sources: Construct-signature validity (CSV) for intelligence testing

How does CSV compare with factor analysis without replacing it?

Factor analysis and construct-signature validity (CSV) ask related but distinct questions. Factor analysis examines whether relationships among test scores fit a proposed latent structure. CSV asks whether response patterns specified from test design appear across examinee ability and item complexity. “Construct-signature validity (CSV) for intelligence testing” presents those patterns as an additional source of evidence. A fitted latent structure does not by itself show that tasks represent intended demands; a predicted ability-by-complexity pattern does not establish every use of a test.

Two analyses of the Anadolu-Sak Intelligence Scale (ASIS) illustrate the distinction, but they are not a head-to-head contest. “Factorial Invariance of the Anadolu Sak Intelligence Scale Across Gender” analyzed raw scores from 4,641 participants in the ASIS standardization study. Using multisample confirmatory factor analysis, it examined three age groups: 4–7 (2,528 participants), 8–9 (848), and 10–12 (1,265). The study reported that its specified second-order general-intelligence, or g, factor model was invariant across gender in all three age groups, with partial subtest-intercept invariance for ages 4–7. Invariance here means that the tested model’s specified measurement relationships were sufficiently comparable across the groups analyzed.

The 2026 CSV paper instead used a secondary analysis of norming data for four ASIS nonverbal subtests in children aged twelve. It examined predicted patterns across complexity levels and ability groups. Thus, the invariance analysis asked whether a specified factor structure operated comparably across gender and age strata; CSV asked whether design-based ability-by-complexity signatures appeared in selected responses. The larger participant count in the invariance study does not make it a stronger test of CSV, just as CSV’s later date does not make it a stronger test of invariance. The studies differ in sample, analysis, outcome, and intended inference, so participant totals cannot rank the methods.

The invariance result supports comparability of the particular modeled factor structure across the tested groups. By itself, it does not establish that every item represents its intended demand, that all subgroup comparisons are fair, or that every score interpretation and use is warranted. CSV’s response-pattern evidence cannot establish those broader points either. The 2014 “Standards for Educational and Psychological Testing” frames validity around evidence and theory supporting score interpretations for proposed uses, with a rationale for each interpretation. Under that framework, factor structure and design-linked response patterns can contribute different pieces of evidence; conclusions still depend on the population, score meaning, and decision at issue.

An evaluation can ask whether modeled score relationships are comparable in relevant groups and whether task demands produce the response patterns predicted by the construct account. Each may clarify evidence without settling the rest. The ASIS invariance findings concern the instrument and groups studied; the CSV demonstration concerns one age cohort and four nonverbal subtests. Neither establishes that an unrelated adult browser test has a validated score conversion. Match each validation question to its claim; one favorable model result is not general approval of a test.

Sources: Construct-signature validity (CSV) for intelligence testing; Factorial Invariance of the Anadolu Sak Intelligence Scale Across Gender; Standards for Educational and Psychological Testing

What would establish more than a promising demonstration?

The next step is independent testing of both the predicted response patterns and the complexity map used to define them, followed by evidence that construct-signature validity (CSV) adds useful information for a stated interpretation or decision. The secondary analysis is an initial demonstration, not proof of stable signatures, generality, or improved decisions. The 2014 Standards for Educational and Psychological Testing, published jointly by the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education, frame validity as evidence and theory supporting a particular score interpretation for a proposed use. A structural finding cannot support every interpretation or use by itself. A predicted response pattern is distinct from evidence supporting a consequential placement or clinical judgment. A useful research sequence would begin by making the complexity map inspectable. Researchers could define which task features justify each category, ask reviewers to code items without seeing response results, and report disagreements. This is a proposed safeguard, not a procedure established by the accessible abstract of “Construct-signature validity (CSV) for intelligence testing.” The signature depends on what counts as low, medium, or high demand. If categories shift after results are visible, a match is harder to distinguish from a flexible explanation of the data. Researchers should then specify expected orderings and contrasts in advance, estimate uncertainty, and test whether conclusions change under other defensible item codings or ability bands. This tests dependence on analytic choices. Replication should use new examinees, ages, domains, and instruments, not only further divisions of the original data. The abstract reports one secondary analysis of norming data for twelve-year-old ASIS examinees and four nonverbal subtests; it does not report such broader replication. Replication still leaves the practical question: what does CSV add? Researchers could compare its information with suitable factor-analytic and item-response analyses: does CSV reveal a mismatch those methods miss, or sharpen an interpretation they support? They would then need to show that this added information changes a specified interpretation or decision beneficially. These are proposed tests, not reported findings. A detectable pattern may have no practical value. “Validating Score Interpretations and Uses Through an Argument-Based Approach to Validity” offers a useful lens: examine the chain of inferences from responses to scores and from scores to proposed uses. The 2026 essay “What questions should a contemporary construct validation of an intelligence test address?” adds a conceptual reminder that intended meaning may involve task, situation, and audience as well as test structure. It is not an empirical refutation of CSV; it helps show why a structural result alone cannot settle what a score means across settings. The verdict would strengthen with stable signatures under defensible, predeclared coding and added value for a defined use. Until then, CSV is a promising complementary framework with an initial, bounded demonstration. Further evidence must connect design assumptions, response patterns, and intended use; no single favorable pattern is a validation stamp.

Sources: Standards for Educational and Psychological Testing; Validating Score Interpretations and Uses Through an Argument-Based Approach to Validity; What questions should a contemporary construct validation of an intelligence test address?; Construct-signature validity (CSV) for intelligence testing

What should a test user conclude from one demonstration?

A reader should treat the 2026 ASIS analysis as evidence about a proposed method in one bounded dataset, not as evidence about their own IQ score. The study examined norming data for four nonverbal subtests of the Anadolu-Sak Intelligence Scale, using children aged twelve. It asked whether response patterns followed predictions across ability and item-complexity bands. They provide no direct evidence about an adult taking a different browser test. That boundary matters because validation is an argument connecting evidence to a particular interpretation and use. A result about children’s responses to selected ASIS tasks cannot, by itself, support an interpretation of an adult’s performance on unrelated questions. Different tests, populations, and score interpretations need their own evidence. The abstract for “Construct-signature validity (CSV) for intelligence testing” also describes a secondary analysis, not an independent replication or a study of how people make decisions from scores. Its reported interaction is a finding within that analysis, not a certificate of ASIS validity or evidence for another instrument. The same limit applies to Test IQ Free. Its adaptive 50-question test covers pattern reasoning, number patterns, and spatial reasoning, but the similarity of broad labels such as “reasoning” or “intelligence” does not make its score equivalent to the ASIS result. The free result includes a provisional estimated IQ range generated by a conversion with placeholder parameters and no validated adult reference population. The range is not a normed score or an established comparison with a population. The 2026 study did not evaluate this test, its items, its conversion, or its adult users. It therefore cannot be cited as validation for the displayed estimate. The optional $9 report uses the same provisional conversion for its point estimate and percentile. It adds explanation, worked examples, applications, and a practice plan; payment does not add accuracy or validation. A percentile label does not become a defensible population rank because it appears in a longer report. Readers should interpret any result as performance under the particular test conditions, not as identity, personal worth, diagnosis, or a fixed limit on future ability. For low-stakes curiosity, a free reasoning test can still be useful as a chance to work through questions and review worked answers. That is a practice use, not a claim that the resulting range measures general intelligence accurately. If the decision concerns clinical assessment, educational placement, or another consequential matter, do not use this browser result or the ASIS study to decide it; seek an assessment and interpretation suited to that decision. The conclusion is narrow: CSV offers a useful additional research question, while this demonstration validates neither an unrelated adult score nor a person’s score or an adult browser-test conversion.

Sources: Construct-signature validity (CSV) for intelligence testing

What the study adds—and the next responsible step

The 2026 study, “Construct-signature validity (CSV) for intelligence testing,” adds a testable way to examine whether a test’s design leaves predicted ability-by-complexity patterns in responses. Its ASIS demonstration found mixed evidence in norming data for four nonverbal subtests given to twelve-year-old children. That is a promising complement to other validation evidence, not a validation stamp for the ASIS as a whole or for another test. For any result, keep three questions together: what did the method test, who was studied, and what score use do you have in mind? Here, the method examined planned response patterns in one age-specific dataset; it did not establish validity for an adult browser test or its score conversion. The free Test IQ Free assessment offers 50 reasoning questions and worked answers for low-stakes practice. Its estimated IQ range comes from a provisional conversion with no validated adult reference population, so treat it as performance under those test conditions. The optional $9 report adds explanation and a practice plan, not accuracy or validity. For consequential decisions, seek assessment and interpretation suited to that decision rather than relying on an online estimate.

Sources: Construct-signature validity (CSV) for intelligence testing

Questions readers ask