What the 2023 Raven’s Matrices Study Found About Time Limits

What did the 2023 study add about time limits?

The 2023 study adds a warning: finishing before the deadline does not show that a clock left performance unchanged. In a randomized experiment, 295 undergraduates completed 18 items from Raven’s Advanced Progressive Matrices Set II under unrestricted time, a 20-minute limit, or a 10-minute limit. Even with 20 minutes, nearly everyone finished, yet participants responded faster from early in the task, reported lower confidence, and answered less accurately than participants working without a limit. “Should Intelligence Tests Be Speeded or Unspeeded? A Brief Review of the Effects of Time Pressure on Response Processes and an Experimental Study with Raven’s Matrices,” reports that accuracy was still lower after the analysis accounted for time spent on each item. That matters because it makes a simple explanation based only on unanswered final items or fewer seconds per answer inadequate. This concerns performance on this task under these conditions. It does not tell us how many IQ points a particular person lost, provide a correction for another test, or establish what would happen across ages, languages, or reasoning measures. Treat the clock as part of the administration conditions when interpreting a result; do not turn this comparison into an individual score adjustment.

Sources: Should Intelligence Tests Be Speeded or Unspeeded? A Brief Review of the Effects of Time Pressure on Response Processes and an Experimental Study with Raven’s Matrices

What exactly did the 2023 experiment compare?

The experiment compared three timing conditions on the same 18 Raven Advanced Progressive Matrices (APM) Set II items: unrestricted time, 20 minutes, and 10 minutes. The study included 295 undergraduates from the University of Rennes 2 in France, randomly assigned to one of those conditions. Because the groups worked on the same abridged set and assignment to timing condition was random, differences between their average task performance can be more plausibly attributed to the imposed time condition than differences in comparisons between unrelated groups taking different forms. The study record, “Should Intelligence Tests Be Speeded or Unspeeded? A Brief Review of the Effects of Time Pressure on Response Processes and an Experimental Study with Raven’s Matrices,” describes this as a controlled comparison of speededness and response processes. “Unrestricted” here means the researchers gave participants no time instruction or response-speed instruction. It does not mean that they were explicitly told to work slowly, nor that every participant spent the same amount of time. In the 20-minute condition, participants saw a warning and a remaining-time counter while working on the matrices. A separate 10-minute condition applied a tighter limit. The 20-minute allowance was selected because earlier work suggested nearly everyone could finish. The 20-minute allowance was chosen because earlier evidence suggested it would be sufficient for virtually all participants to complete these 18 items; the 10-minute limit was intended as a more severe pressure condition. In the experiment itself, the 20-minute group used an average of about 10 minutes, and the 10-minute group an average of about 8 minutes. Those group averages show that the maximum allowance was not the same as time actually spent. They do not show that every person in either condition had the same pace or experience. The task used an abridged APM Set II, not all 36 items. Participants answered 18 odd-numbered items in ascending difficulty. Researchers recorded accuracy, response time, confidence, and reported strategies. This permits a focused question: what changed when timing instructions changed while items stayed constant? It does not compare every Raven version, setting, population, or reasoning measure. A timed and an untimed test from separate studies may differ in item selection, delivery, participants, and instructions as well as timing. Random assignment holds the task largely steady while varying timing, clarifying the condition contrast. Each participant was assigned to one condition, not retested under different clocks. This avoids familiarity from answering the same items twice. But random assignment cannot make this sample representative of all test takers or establish the same effect for another form. The unrestricted group had 97 participants; each timed group had 99. These are experiment groups, not a reference sample for converting an individual result. Participants were university undergraduates, with mean ages around 19 to 20, and all were native French speakers. Five of 300 initially recruited students were excluded for not meeting a working-memory task accuracy criterion. The sample sets a boundary: random assignment supports comparisons within this study but does not make it a general adult reference population. The outcome is performance on these items under specified conditions. The experiment supplies no personal score correction, norm, or prediction for another test. When interpreting a Raven-style result, ask which form, item count, and time rule applied. Conclusions beyond those conditions need separate evidence for the relevant test, population, and purpose.

Sources: Should Intelligence Tests Be Speeded or Unspeeded? A Brief Review of the Effects of Time Pressure on Response Processes and an Experimental Study with Raven’s Matrices

Why is finishing the test not proof that the clock was harmless?

Finishing tells you that a participant reached the final item before the deadline. It does not tell you that the deadline left the earlier answers, pace, or confidence untouched. In the 2023 Raven’s matrices experiment, participants under time pressure began speeding up from the first item, including in the milder condition. The abstract reports that this condition allowed enough time for all participants to complete the task at a leisurely pace. Yet the timed groups answered less accurately and reported lower confidence. Thus, completion rate alone misses changes that can occur well before anyone runs out of minutes. [Should Intelligence Tests Be Speeded or Unspeeded? A Brief Review of the Effects of Time Pressure on Response Processes and an Experimental Study with Raven's Matrices] A deadline can affect a test in at least two distinguishable ways. The 2023 experiment found a broad pace change, not merely a last-minute rush. The reported speeding began with the very first item, so the clock was influencing work before the point at which a participant might have needed to skip anything. A final unanswered item is visible in a way that a shifted pace is not; both may both matter to the score, but they describe different routes from administration conditions to observed answers. Here, item response time means the time spent on an individual question. The study’s item-level analysis compared accuracy while accounting for response time. The reported accuracy reduction remained in that analysis. In plain terms, the lower accuracy was not explained only by participants spending fewer seconds on each item. The authors describe this as evidence of an effect beyond speeding itself. It makes an omission-only explanation inadequate: people may finish every item and still show a difference in observed performance under the clock. That statistical result needs careful wording. Accounting for response time is a way to test whether the accuracy difference can be reduced to measured seconds per item. It is not a direct demonstration of the psychological cause of the remaining difference. The study’s abstract also reports lower confidence and poorer strategy use under time pressure, but these observations do not establish that anxiety, attention, motivation, or any single mental process caused the accuracy change. The 20-minute condition matters because it weakens the simple claim that only people who cannot finish are affected. The abstract characterizes the mild limit as leaving enough time for completion at a leisurely pace, yet it still reports speeding across the task and lower accuracy. The contrast is between specified timing conditions, including unrestricted time and two limits; the milder deadline still provides a clock that participants may respond to. So completion is useful information, but it is not a harmlessness test. To interpret a timed result, keep the time limit alongside the item count and the number answered. The experiment does not say that every clock changes every test in the same way, and it offers no personal correction for a result. It does show why “I finished” cannot, by itself, establish that observed performance would have been identical without the deadline.

Sources: Should Intelligence Tests Be Speeded or Unspeeded? A Brief Review of the Effects of Time Pressure on Response Processes and an Experimental Study with Raven’s Matrices

What do strategy findings explain—and what can they not explain?

A strategy finding can make the 2023 result easier to interpret, but it cannot tell us that time pressure caused a particular strategy change. Two response strategies are useful to distinguish. In constructive matching, a person works out what the missing part should be before comparing it with the available choices. In response elimination, the person checks choices against the problem and rules out those that fail. Researchers infer approaches from response patterns and other process measures; they do not observe thought directly. Consider an invented everyday example: a shelf has three spaces, and the first two hold a mug and a bowl. If the goal is to complete a familiar place setting, constructive matching means first thinking “a plate” and then checking whether that object fits the remaining space. Response elimination means inspecting the available objects and crossing out anything that cannot serve that role. The separate study “Intraindividual strategy shifts in Raven’s matrices, and their dependence on working memory capacity and need for cognition” examined two samples of 100 people each completing Raven’s Advanced Progressive Matrices and measures of working memory and need for cognition. The abstract reports shifts toward response elimination as difficulty increased and an association between strategy variation and item-to-item accuracy variation. The reported 78% refers to variance in item-to-item accuracy accounted for in that study’s model. This is not 78% of individual scores or a general estimate for other tests. That finding supplies context for the 2023 experiment’s report of poorer strategy use under time pressure. It makes plausible that a clock may affect approach as well as pace. Yet difficulty and time pressure are different conditions. The earlier strategy study examined variation as items became harder; it did not randomly impose the 2023 study’s time limits. Conversely, the 2023 study compared timing conditions and recorded strategy measures, but its results do not isolate strategy as the cause of the accuracy difference. The available evidence supports a connection worth investigating, not a settled causal chain. Confidence adds another useful clue about the response process. In “Should Intelligence Tests Be Speeded or Unspeeded? A Brief Review of the Effects of Time Pressure on Response Processes and an Experimental Study with Raven’s Matrices,” the 2023 experiment found lower confidence and poorer strategy use in its time-pressure conditions, alongside reduced accuracy. They do not reveal exactly why: confidence is a participant’s judgment, and a reported strategy or strategy score is an indicator of approach, not a direct view of every mental step. The study did not establish whether confidence, strategy, attention, motivation, anxiety, or another process mediated the accuracy effect. The careful conclusion is therefore modest. Strategy research helps explain why response process deserves attention when a clock is present; it does not prove that timing made people abandon one specific approach, or that teaching a strategy would remove the observed effect. For a reader, the practical lesson is to treat the recorded result as performance under its stated conditions. A plausible mechanism can sharpen the next research question without becoming a personal explanation for a score.

Sources: Intraindividual strategy shifts in Raven’s matrices, and their dependence on working memory capacity and need for cognition; Should Intelligence Tests Be Speeded or Unspeeded? A Brief Review of the Effects of Time Pressure on Response Processes and an Experimental Study with Raven’s Matrices

What does broader evidence say about changing the test conditions?

A meta-analysis broadens the question beyond whether a clock changes answers in one sitting. “Process differences as a function of test modifications: Construct validity of Raven’s Advanced Progressive Matrices under standard, abbreviated and/or speeded conditions” synthesized 142 studies involving 26,848 participants. The researchers examined whether time limits and shorter forms changed relationships between APM performance and other ability measures. The abstract reports substantial differences in correlations with spatial visualization, memory, and perceptual speed under speeded conditions and shorter forms. This is evidence that administration changes can matter to the abilities reflected in a score; it is not an estimate of how many answers a clock costs. A correlation answers a different question from an accuracy contrast. It describes how scores vary together across people or measures. The meta-analysis considered convergent and discriminant relationships: whether APM scores related differently to other ability traits when the test was speeded or shortened. The abstract reports the largest differences as ρ̂ = 0.26 for spatial visualization, 0.08 for memory, and 0.34 for perceptual speed. These are differences in reported relationships across conditions, not IQ-point changes, percentages of people affected, or predicted losses for individuals. The contribution concerns construct validity: evidence about what a score can be interpreted to represent under specified conditions. Time limits and shortened forms should remain distinct. A clock can make speed part of the task while leaving every item available. A shortened form removes items, changing the content sampled and the information available from responses. The meta-analysis grouped evidence across standard, abbreviated, and/or speeded conditions, so its result is not a clean comparison of one timing rule while everything else stays fixed. Its abstract says altered conditions may lead examinees to use different strategies and combinations of abilities. That is an interpretation of the pattern, not proof that one specific strategy caused each correlation difference. This broader evidence complements the 2023 experiment without answering the same question. That experiment compared performance under assigned timing conditions on an 18-item APM task. The meta-analysis pooled studies to ask whether modified forms and constraints changed relationships with other abilities. One offers a within-study contrast in observed accuracy and response behavior; the other gives a broader view of how modifications may change what performance represents. They should not be combined into a universal timing penalty. Because the publisher’s full text is restricted, the accessible IDEAS/RePEc record supports the study count, sample total, stated method, abstracted correlation differences, and interpretation; it does not allow detailed audit of study selection, moderator analyses, or each modification’s separate contribution. The practical inference is modest: identify the version and administration conditions before treating two results as comparable. A timed Raven form may serve a defined purpose, but interpretation needs evidence for that form, population, and intended use. The meta-analysis does not show that every timed test is invalid, an untimed score automatically better, or that results can be corrected by adding points. It shows why “Raven score” alone may omit relevant information: length and speed conditions can shape observed performance and its relationships with other measured abilities.

Sources: Process differences as a function of test modifications: Construct validity of Raven’s Advanced Progressive Matrices under standard, abbreviated and/or speeded conditions

How can a timed short form still be useful?

A timed short form can be useful when its practical purpose is clear and evidence supports interpreting its scores for that purpose. That conclusion is compatible with the 2023 experiment: a clock can change performance, while a particular timed version can still show useful measurement properties. The two questions are separate. Evidence that a form measures consistently or relates to another measure does not show that time pressure leaves responses unchanged, and a timing effect does not by itself make every timed form unusable. “Evaluating untimed and timed abridged versions of Raven’s Advanced Progressive Matrices” examined two online 12-item forms in separate studies. The untimed study included 608 participants, with a mean age of 27.89; the timed study included 479, with a mean age of 20.93. Confirmatory factor analyses supported a one-dimensional structure in each sample. The authors reported test–retest values of .65 for the untimed form and .69 for the timed form. Scores also correlated with the Wechsler Abbreviated Scale of Intelligence, Second Edition: for the Perceptual Reasoning Index, r was .50 in the untimed study and .56 in the timed study; for Full-Scale IQ, r was .34 and .41, respectively. These correlations do not establish interchangeable scores or clinical use. The study’s conclusion was qualified. Both forms appeared psychometrically sound in the analyses reported, and the authors noted a practical advantage for the timed form when it would sit within a longer test battery: untimed completion duration varied considerably. They also concluded that the untimed version might be the better abridged form when considering classical test analyses, item response theory analyses, and validity. This is not a blanket endorsement of timed testing. It shows a tradeoff between managing test duration and preserving a form whose score interpretation has support. Which side matters more depends on the intended use and evidence for that form. The design matters when comparing this result with “Should Intelligence Tests Be Speeded or Unspeeded?” That 2023 experiment randomly assigned 295 undergraduates at one French university to unrestricted, 20-minute, or 10-minute conditions while they answered 18 items from APM Set II. It examined the effect of changing time conditions within the same task. By contrast, the 2022 short-form studies examined psychometric properties in separate timed and untimed samples using a 12-item online form. Because the tasks, samples, and designs differ, the studies do not cancel each other out. The 2022 findings cannot establish that timing has no effect in the 2023 task; the 2023 result cannot establish that the 12-item timed form is useless for every supported purpose. The practical reading is conditional: a timed score may be useful when the relevant version has evidence for the population and decision at hand, but it remains performance under that version’s timing rule. Do not treat evidence of usefulness as evidence of unchanged performance, or use a result from another form to make an IQ conversion. For a low-stakes browser quiz, the free Test IQ Free 50-question test can offer practice and worked answers; its displayed IQ range is a provisional estimate from an unvalidated conversion with no validated adult reference population. It is not a normed comparison or a confidence interval. The optional paid report adds explanation, not validity.

Sources: Evaluating untimed and timed abridged versions of Raven’s Advanced Progressive Matrices; Should Intelligence Tests Be Speeded or Unspeeded? A Brief Review of the Effects of Time Pressure on Response Processes and an Experimental Study with Raven’s Matrices

What does a responsible score interpretation require?

A score interpretation needs evidence for the exact test version, population, administration conditions, and intended use. A timed raw result alone does not establish an IQ comparison. The Standards for Educational and Psychological Testing make the key point practical: validity concerns whether evidence supports a proposed interpretation for a specified use. The question is not whether a test has a permanent label of “valid,” but whether the reasoning from this score to this conclusion is supported in this setting. Evidence for one use does not automatically justify another.

A raw score is the count of answers marked correct before comparison with a reference group. It describes observed performance on the administered questions; it does not say where that performance falls among people of a particular age or other defined population. A norm-referenced score interprets a person's result against a specified reference group. That comparison depends on norms: data from the group used to describe how scores are distributed for the relevant test and population. If the group, form, language, or administration differs, the comparison may not carry over. The study Should Intelligence Tests Be Speeded or Unspeeded? therefore tells us that timing conditions affected performance on its task; it does not turn that task's group result into a conversion for another person, form, or online quiz.

Precision concerns how much measurement uncertainty surrounds a score. A reported number may look exact while evidence about its precision, such as an appropriate estimate of measurement error, is absent or unsuitable for the intended decision. Validity evidence concerns the interpretation and use; precision concerns how consistently the score distinguishes performance under relevant conditions. Neither is supplied by the score's appearance or by calling a test an intelligence test. The Standards for Educational and Psychological Testing call for a rationale and supporting evidence for each intended interpretation, including attention to the examinee population and the test's format and administration. When administration changes, such as imposing a time limit, the developer or user needs to consider whether that change alters what the score means and whether comparisons remain warranted. This is a rule for evaluating evidence, not empirical proof that every clock changes every test in the same way. The Raven study is evidence about one particular timing contrast, not a substitute for documentation about a different form.

So ask what the number represents before deciding what it means: which version was administered, to whom its comparison applies, under what timing and other conditions, and for what use its interpretation is supported. A result is not a diagnosis, a measure of personal worth, or a verdict on fixed potential; it records performance in a particular assessment context. Test IQ Free's 50-question reasoning test offers a provisional estimated IQ range from an unvalidated conversion with no validated adult reference population. That display is not a normed result, confidence interval, or established population comparison. Its $9 report adds explanation, not validity. For open practice, review the questions and worked answers as task performance; for consequential decisions, use an appropriately qualified assessment supported for the relevant population and purpose.

Sources: Standards for Educational and Psychological Testing; Should Intelligence Tests Be Speeded or Unspeeded? A Brief Review of the Effects of Time Pressure on Response Processes and an Experimental Study with Raven’s Matrices

Which conclusions travel beyond this sample—and which do not?

The finding travels as a reason to examine timing conditions in other tests, not as proof that every timed test lowers accuracy in the same way or that every person would score higher without a limit. The 2023 experiment supports a bounded statement: timing affected performance on an 18-item portion of Raven’s Advanced Progressive Matrices Set II among 295 undergraduates at one French university. Generalization carries those details with the result. It does not turn “this group, this form, this administration” into “all reasoning tests, all examinees, all purposes.”

The abstract for “Should Intelligence Tests Be Speeded or Unspeeded?” reports that time pressure reduced response times disproportionately on difficult items and among participants higher on measured ability, working memory, or need for cognition. Those are group-level patterns. They do not identify what happened inside any one examinee’s reasoning, establish that a particular person was harmed by a clock, or justify ranking people by those characteristics. The abstract says these response-time differences did not differentially affect ability estimates. A process measure can vary across conditions or groups without establishing a corresponding difference in the estimated ability outcome. The study gives a reason to inspect administration, while resisting a claim that every measured difference translates into a different ability estimate.

The broader APM meta-analysis asks a related but distinct question. Its abstract synthesizes 142 studies involving 26,848 participants and reports that speed constraints and shortened forms affected relationships between APM performance and other ability measures. This supports caution when interpreting changed versions: altering time or length can alter which demands contribute to performance and how scores relate to other measures. It does not replicate the 2023 experiment or show that every modification produces the same change. Because the accessible record is an abstract, conclusions about detailed moderators or study quality should remain limited to what it reports.

These findings make timing an evidence-informed question, not a universal verdict. The 2023 randomized comparison is informative about assigned limits on that task. The meta-analysis indicates that modified APM conditions can matter for construct-validity relationships across a wider literature. Neither supplies a correction factor for a different test or a person’s result. The Standards for Educational and Psychological Testing offer a rule for interpretation: evidence must support the intended meaning and use for the relevant version, population, and administration. The Standards do not prove that the 2023 result applies elsewhere; they explain what evidence a broader claim would need.

A stronger conclusion would require replicated comparisons across relevant forms, timing rules, and populations, with clear reporting of intended score use and evidence supporting that interpretation. Until then, carry the practical question forward: what exact task was timed, for whom, and for what decision? A timed score describes performance under those conditions. Whether it supports a broader comparison depends on evidence for that use; this study cannot tell an individual how many points to add, or whether removing a clock would change their result.

Sources: Should Intelligence Tests Be Speeded or Unspeeded? A Brief Review of the Effects of Time Pressure on Response Processes and an Experimental Study with Raven’s Matrices; Process differences as a function of test modifications: Construct validity of Raven’s Advanced Progressive Matrices under standard, abbreviated and/or speeded conditions; Standards for Educational and Psychological Testing

How should I read a timed reasoning result in practice?

Start by recording what the result actually describes: the test name and version, number of items, time limit, and whether the reported number is simply the count answered correctly or an interpretation compared with a reference group. Then ask whether evidence supports that interpretation for this version, population, administration, and intended use. The Standards for Educational and Psychological Testing treats score meaning as a matter of evidence tied to a proposed use; a number alone does not establish an IQ comparison.

For low-stakes exploration, note the conditions, work through the explanations, and treat your result as performance on those questions under that clock. The 2023 Raven’s matrices experiment gives a reason to include timing in that description, not a formula for adjusting an individual score. It studied one 18-item form and cannot tell you how many points to add, or predict what you would have done under another limit.

If the result may affect placement, diagnosis, or eligibility, take the question to a qualified professional who can select an assessment and interpret it for that purpose. Test IQ Free’s 50-question test is available for practice; its estimated IQ range comes from an unvalidated conversion with no validated adult reference population, so it is not a normed comparison. The optional $9 report adds explanation, not validity.

Sources: Standards for Educational and Psychological Testing

What did the 2023 study ultimately change?

The 2023 study changed the question from “Did everyone finish?” to “Did the clock alter the performance being observed?” A 20-minute limit affected accuracy and response processes on an 18-item Raven’s Advanced Progressive Matrices Set II task. Timing is part of the conditions for interpreting performance; the study offers no correction for an individual score. This does not mean timed tests are never useful. “Should Intelligence Tests Be Speeded or Unspeeded? A Brief Review of the Effects of Time Pressure on Response Processes and an Experimental Study with Raven’s Matrices” examines assigned timing in this task. “Psychometric properties of abridged versions of Raven’s Advanced Progressive Matrices” reports evidence about separate timed and untimed short forms. These answer different questions: a form may serve a defined purpose even when its time limit affects observed performance. For a personal result, note the test and timing conditions; do not infer an IQ change from this experiment. Stronger conclusions require evidence for the exact form, population, and intended use.

Sources: Should Intelligence Tests Be Speeded or Unspeeded? A Brief Review of the Effects of Time Pressure on Response Processes and an Experimental Study with Raven’s Matrices; Evaluating untimed and timed abridged versions of Raven’s Advanced Progressive Matrices

Questions readers ask