What to Do When Two Reasoning-Test Patterns Seem Plausible

If two patterns fit, test both before you decide

When two patterns seem plausible, compare both against the item before choosing. First, apply the directions and any structure the prompt supplies. Second, check whether each rule accounts for every displayed part without contradiction. Third, ask what each rule predicts next and what extra assumptions it needs. If both remain consistent and predict different answers, treat the choice as provisional; use the format’s required response and any stated instructions to decide what to enter. A clear direction or repeated example may settle the apparent tie, so this check is not a reason to doubt every answer. If a candidate rule fails on a displayed step, set it aside rather than quietly changing the operation partway through; a switch is justified only when the prompt gives a reason to expect one.

Keep two questions separate: the answer key tells you which response receives credit under the scoring rule, while logical uniqueness asks whether the prompt and its conventions rule out competing continuations. A keyed answer can be the expected response even when the displayed information alone does not prove it is the only possible one. The sections that follow examine why that difference can arise, then compare exact transition rules in an original sequence example.

Why do a few observations leave room for different continuations?

A finite sequence records observations: the terms actually shown. A rule proposes a relationship among those observations and says how that relationship continues. The next value is a prediction made by applying the rule, not another observation already contained in the list. When the same finite list fits more than one exact rule, the observations by themselves do not determine a unique continuation. Mathematicians call this underdetermination: the available information leaves multiple accounts possible. It describes a logical relationship between data and rules, not how often test questions are ambiguous.

For example, imagine a short list of weekdays: Monday, Tuesday, Wednesday. “Continue with Thursday” follows if the intended rule is consecutive days. But the list does not state that convention; another rule could select weekdays by some other feature and point elsewhere. This is an illustration, not a test item. The consecutive-days reading is familiar and compact, which makes it a sensible default in an ordinary weekday sequence, but familiarity is a convention-based reason to prefer it, not a deduction forced by those three labels alone.

That distinction matters because a reasoning prompt supplies more than a bare list. Directions may name the operation; examples may establish a repeated format; a diagram may constrain how rows, columns, or features relate. Those cues become part of the information a solver should use. If they sharply constrain the rule, the apparent alternatives may disappear. Conversely, if a solver can state two rules that both respect the prompt and lead to different completions, the observations have not settled the choice without some additional convention or preference. This does not make the item defective by itself: it clarifies what comes from the shown data and what comes from the task’s instructions and format.

A rule is an added description, so exact fit and a justified prediction are separate achievements. Many descriptions can pass through finite points if enough special conditions are allowed. That does not make every description equally useful in a test: an interpretation must also fit the directions and familiar format. A finite record alone does not prove its next term; judge what the item supports by inspecting both observations and constraints.

When is a second rule plausible rather than merely imaginable?

A serious competing rule must do more than produce a different answer. It should account for the item as presented, operate the same way wherever it is applied, and make a clear prediction that differs from the first rule. These requirements distinguish a usable rival from a formula invented only because it can be made to fit what is already visible.

Coverage means the rule explains every relevant transition in a sequence or every relevant cell and relation in a diagram. A candidate that handles most of a pattern but leaves one change unexplained has not covered the item. Consistency means the same stated operation or relationship applies across those parts, unless the prompt signals a change. A solver should be able to describe the rule once and use it throughout, rather than silently switching operations at the awkward step. Distinct prediction means the candidate leads to a different completion under the same prompt. If two verbal descriptions ultimately predict the same answer, they may be different ways of explaining one response, but they are not competing answers.

For an invented illustration, suppose a row of symbols adds one mark at each step, while their outlines also change in a repeating order. One account might say the count increases by one and the outline cycles through three forms. A rival that predicts a different next outline is worth considering only if it explains each outline already shown and does not change its cycle to rescue a mismatch. The illustration is not a test item. It shows why a candidate needs a stable account of the displayed structure, not simply a clever description of its endpoint.

A highly elaborate mathematical expression can often be fitted to known observations after they are visible. That ability alone gives it little practical support as the intended test-taking rule: the solver had no reason, from the task format, to choose that expression in advance. A compact relation aligned with the item's familiar structure is generally a stronger candidate, provided it actually fits. Still, coherence does not establish intent. An explicit convention can rule out a rival, and a rule that needs an unsupported extra condition remains weak even if its arithmetic is internally consistent.

Hide the proposed answer and explain the rule from earlier displays alone. If the explanation depends on knowing which option you hope to justify, it may be a post-hoc fit and offers weaker evidence of a consistently applicable pattern.

How should you compare two rules across the whole item?

Compare candidates with the same questions, in the same order. This avoids giving one rule a generous reading while demanding exact detail from its rival. Write each account plainly enough that another solver could apply it without guessing what operation you meant. Then check the full item, not just the last visible change or the answer choices.

| Check | Candidate A | Candidate B | |---|---|---| | Coverage | Does it explain every displayed transition or cell? | Apply the same coverage test. | | Prompt fit | Does it respect the stated directions and layout? | Apply the same prompt constraints. | | Prediction | What exact completion follows from it? | Does it yield a different completion? | | Added assumptions | Which conditions are visible, and which are being supplied? | List its extra conditions separately. |

Start with what is already on the page: repeated changes, positions, labels, directions, and any examples that establish a convention. Mark separately a condition you had to add to make a candidate work. An assumption is not automatically unreasonable; a familiar format may license a convention. But if the rule needs a special exception, a hidden starting point, or a switch that the prompt never signals, record that as an added condition instead of treating it as observed evidence. Then ask whether the same assumption would have been chosen before seeing the answer it produces.

For items with several features, test each feature on its own before combining them. For example, track count independently from orientation, or row movement independently from shading. If the candidate accounts for each visible feature and their combined prediction is clear, state that result. If one feature is unresolved, do not let a good fit on another feature conceal it. This stepwise check makes it easier to locate precisely where two accounts agree or diverge, without assuming that every item has only one operation.

Parsimony favors the account needing fewer added conditions after both candidates fit the prompt; it proves neither truth nor scoring intent. A complex account may be right when multiple operations are cued, while a simple one fails if it contradicts a visible transition. Provisionally prefer the candidate that covers the item, respects its structure, predicts specifically, and needs less unsupported repair. If a genuine tie remains, keep it provisional.

Only after deriving both predictions should you compare them with the response choices, if choices are provided. A choice can expose that you misapplied a stated rule or that one prediction is unavailable in the format. It cannot supply missing steps in a candidate account by itself: selecting a rule because it reaches a listed option risks fitting backward. Keep the written prediction and the prompt evidence visible during this check, so the option functions as a consistency check rather than the source of the rule.

Worked example: what do 2, 4, 8, 14 actually support?

This original sequence is an illustration, not a test item and not evidence about the quality of any test. The four observed terms are 2, 4, 8, and 14. To compare continuations, state each rule as a procedure and check its arithmetic at every transition; a loose phrase such as “multiply, then add more” is not precise enough to test.

Rule A looks at the differences between neighboring terms. From 2 to 4, the increase is +2. From 4 to 8, the increase is +4. From 8 to 14, the increase is +6. The observed increases therefore rise by 2 each time: +2, +4, +6. Continuing that relationship gives a next increase of +8. Add it to the last observed term: 14 + 8 = 22. Under Rule A, the next term is 22. This account applies the same difference rule across all three transitions and extends the observed change in those differences by one step.

Rule B uses a different, fully stated procedure: multiply by 2 on each of the first two transitions, then add 6 on every later transition. First transition: 2 × 2 = 4. Second: 4 × 2 = 8. Third: 8 + 6 = 14. On the next transition the rule is now in its later stage, so 14 + 6 = 20. Under Rule B, the next term is 20. This does fit every displayed term. It does not alternate multiplication and addition with an increasing add-on; the switch happens after the second transition, and the later operation is a fixed addition of 6.

The list alone does not say that a two-step multiplication phase should be followed by fixed addition. Rule B therefore introduces a switch point and a later operation that the four terms do not independently establish. Rule A continues a single regularity visible in the differences, so it is the more economical provisional continuation under an ordinary preference for extending a pattern without an unannounced phase change. That preference is a reason to choose Rule A when a response is required, not a proof that 22 is logically forced. The two procedures demonstrate why exact fit and unique determination are different: each rule reproduces the observations, while their next predictions diverge.

A solver could construct still more rules that match these four terms, and an additional term or an explicit test convention could favor another account. This mathematical possibility does not make every constructed formula equally plausible under ordinary item conventions. The useful comparison is whether each rule is stated in advance of its prediction, covers every transition without being quietly revised, and needs extra conditions not cued by the prompt. Here that comparison favors Rule A provisionally because its continuation uses the observed progression of differences, while Rule B requires a specific transition from doubling to adding six. If the item supplied a cue for that phase change, the balance could change; no such cue appears in this bare list.

When can several visual changes still determine one answer?

Several features can change at once without creating competing answers. The key question is whether the separate features impose compatible constraints that converge on the same completion. Counting one feature correctly is not enough if the candidate violates another visible relation; conversely, two changing features do not create ambiguity when both point to one candidate.

Consider an invented card sequence, not a test item: each card shows an arrow and a group of dots. Across the displayed cards, the arrow turns one quarter-turn clockwise at each step, while the number of dots increases by one. A proposed fourth card must continue both relations: its arrow must have turned clockwise by another quarter-turn, and it must show one more dot than the preceding card. A card that has the right dot count but turns the arrow counterclockwise is only a partial match. It satisfies the count constraint and fails the direction constraint, so it does not complete the stated pattern.

This example separates three cases. First, one feature may be sufficient: if the prompt asks only for a count progression and no other displayed property changes, the count rule may settle the choice. Second, multiple independent features may converge: the count and orientation rules can each be checked, and together they may identify the same candidate. Third, the features may leave divergent predictions: perhaps the count relation is clear but the arrow's direction or starting orientation is not specified well enough to distinguish two candidates. Only the third case leaves a live alternative after every visible constraint has been applied. The number of operations by itself does not tell you which case you have.

Describe each feature separately, predict its requirement, then test the completion against all requirements. Keep only constraints visible in the prompt or supported by convention; do not invent a feature to rescue an answer. Inspect the layout too: positions or orientations may be unclear even when a relation sounds obvious in words. Intended feature count alone cannot establish uniqueness. Another candidate remains viable if it fits the visible constraints and predicts a different completion.

Any accompanying artwork should remain metaphorical and generic: a blank workbench can suggest careful observation without depicting a solvable public puzzle or proprietary item. For this article, the hero still life contrasts a clock and an hourglass across one blank sheet. After this section, the support image shows hands putting a clock aside and opening a blank notebook; it contains no hourglass. Neither image should encode arrows, dot counts, answer choices, grids, or another inferable test item.

Which directions and conventions should constrain your reading?

Directions belong inside the comparison, not in a separate mental note. Before judging which pattern is simpler, translate every explicit instruction into a condition that each candidate must satisfy. If a prompt says to read each row from left to right and compare the same position across rows, a rule based only on the visual diagonal does not meet the stated task, even if it happens to produce one listed choice. If it says to continue an alternating operation, the proposed account must identify the alternating steps and apply them consistently to the shown material. These directions narrow the space of admissible interpretations; they do not merely suggest what the author might have intended.

Examples can establish a convention as well. Suppose a diagram gives two completed transformations and asks for the next one. If both examples rotate a shape one quarter-turn clockwise while preserving its size, the repeated relation is evidence for a clockwise rotation convention. A candidate that enlarges the shape has to explain why the examples do not show that change. This is a fresh illustration, not a test item. Repeated structure can make an implicit rule well supported, especially when the same relation appears in more than one place. The solver should use a clear cue supplied by the item rather than reject it merely because it is not phrased as a sentence.

Layout also carries constraints when positions are stable and readable. In a matrix, a direction to compare columns may make a row-based account irrelevant; labels, alignment, or a marked starting cell can likewise tell you which elements correspond. First identify those relations, then assess whether each candidate explains all of them. If the display makes two plausible correspondences possible, note that uncertainty specifically: it may affect which cells a rule compares, rather than the operation itself. Keep a visible cue separate from a convention inferred from repeated form, and keep both separate from an extra condition added only to rescue a preferred prediction.

Answer choices come later. Derive what each surviving rule predicts from the prompt, then check whether that prediction appears among the available responses. If a rule predicts a response that is absent, the choices may expose a misread direction, an arithmetic slip, or a candidate that the format does not support. A listed answer can therefore help eliminate an inconsistent prediction. But a match does not prove that the matched rule is uniquely implied: another continuation might also fit the directions and displayed examples while predicting something not offered. The choices define the response set for scoring, not every conceivable continuation of the pattern.

Do not invent an operation just because it reaches an option; state it first and verify each displayed relation. Do not disregard a familiar convention consistently demonstrated by the item. Use the strongest present constraints, weigh repeated cues, and mark remaining inferences. What to enter or whether to move on depends on the test's instructions and interface; no universal guessing policy replaces them.

What can a key establish about your reasoning?

A marked answer establishes which response was recorded and, under the scoring key, whether that response receives credit. By itself, it does not show the sequence of observations and inferences that produced the mark. A correct choice can follow from sound rule application, a partial shortcut, elimination, or a guess; an incorrect choice can reflect a reasoning error, a misunderstood direction, or a mismatch between the intended and perceived task. Those possibilities do not make the key useless. They define what kind of evidence it supplies: a scored response, rather than a transcript of thought.

The distinction appears in the primary study “Complement or Contamination: A Study of the Validity of Multiple-Choice Items when Assessing Reasoning Skills in Physics.” The researchers reformulated Swedish national physics constructed-response items as multiple-choice tasks. In one substudy, 12 students gave oral explanations; a second sample included 102 students at five randomly chosen schools. The study reported concern that students could select an option through strategies other than the reasoning the task was meant to assess. This is evidence that a chosen option may not disclose the process behind it in that setting. It does not show that a particular abstract reasoning item has two valid answers, nor does it estimate ambiguity in IQ tests: the participants were school students in Sweden and the tasks concerned physics.

That boundary still leaves a useful, cautious inference for score reading. If response processes can differ from the intended reasoning in a studied assessment format, the bare mark cannot be treated as a detailed explanation of an individual's thinking. For an individual item, inspect the prompt and the work you actually did; for a broader claim about a test score, evidence about the intended interpretation and use is needed. The study does not establish that every multiple-choice response is misleading. Rather, it shows why process claims require evidence suited to process, such as explanations or other response-process investigation, and why a selected option alone has less reach.

The governing principle in “Standards for Educational and Psychological Testing (2014)” is that validity concerns the evidence and theory supporting a specified interpretation and use of test scores. It is not a permanent badge attached to a test name, and a key cannot supply all the evidence needed for a broader interpretation. Whether a score supports a conclusion depends on the intended use and the evidence relevant to that use. The Standards provide professional guidance for evaluating score interpretations; they do not inspect this item, validate a browser quiz, or establish that every keyed response is uniquely forced by its wording.

Single-best-answer scoring nevertheless has a practical advantage: it applies a defined rule consistently, makes responses straightforward to record, and can identify the option expected under the prompt's conventions. A keyed response may be entirely appropriate even when a philosophical account can imagine other continuations outside those conventions. The point is not to equate scoring with proof of logical uniqueness. A key answers, “What response earned credit under this scoring rule?” The separate question, “What does this response show about the examinee's reasoning or ability?” needs evidence beyond the key and must stay within the assessment's supported use.

So one questioned item should be interpreted at the scale it can bear. It may reveal a specific uncertainty about the wording, a candidate rule, or the response process; it cannot alone settle a person's ability, and it does not invalidate every other item or the entire assessment. Likewise, one correct mark supports the narrow statement that the keyed response was credited, not a diagnosis or a broad claim about general intelligence. Keep the observed response, the reasoning you can document, and the intended score interpretation distinct. That separation preserves the information the score does provide while preventing it from being asked to prove more than the available evidence supports.

Sources: Standards for Educational and Psychological Testing (2014); Complement or Contamination: A Study of the Validity of Multiple-Choice Items when Assessing Reasoning Skills in Physics

What does evidence about item flaws let us conclude?

Evidence that assessment writers use quality rules, or that some examination questions contain flaws, cannot by itself tell us whether this particular reasoning question has two defensible answers. These sources answer different questions: what construction guidance recommends, what one provider says it does, and what researchers found in one examined item set. Keeping those categories separate prevents a broad quality concern from being mistaken for an item-level diagnosis.

The empirical result comes from “The impact of item-writing flaws and item complexity on examination item difficulty and discrimination value.” The 2016 study examined 1,925 faculty-authored examination questions prepared for third-year veterinary students and reported multiple item-writing flaws in 37.3% of questions. That is evidence that review-detectable flaws occurred in that particular veterinary assessment context. “Flaw” is not synonymous with “ambiguous”: a coding category can include problems other than competing plausible answers. The percentage is neither an ambiguity rate nor an estimate for IQ or general reasoning tests, whose items, examinees, and assessment settings were not studied there.

“Item-writing rules: Collective wisdom” offers a different kind of evidence. Frey and colleagues reviewed 20 classroom-assessment textbooks and reference works, identified rules that appeared across them, and noted that experimental validation of such rules was sparse. Agreement among instructional texts can describe a professional convention; it does not establish how often a rule prevents ambiguity, whether applying it causes better outcomes, or whether every item that follows it has only one reasonable interpretation. The review therefore supports treating familiar construction rules as guidance rather than as a measured guarantee.

“Guidelines for the Construction of Multiple Choice Questions Tests” is a medical-education recommendation paper. Its accessible abstract and indexed material recommend focused questions, enough information to answer, and a single best response. Because the available record is recommendation guidance rather than an empirical comparison, it shows what the authors advise item writers to aim for. It does not measure the prevalence of ambiguous reasoning items or demonstrate that a particular prompt meets those aims. The limited access also constrains claims here to the recommendations visible in that record.

“ETS K–12 Item Development” is a provider’s description of its own process. ETS says its K–12 item development includes alignment and technical, accessibility, fairness, and editorial review. This documents the review layers ETS reports using in that program; it is not an independent audit of their effectiveness, a guarantee that every item is unambiguous, or evidence about IQ tests. A stated process can show that quality review is an explicit design concern while leaving open what happened to any individual question.

None of these sources labels the disputed question, checks its exact wording and layout, or reports how examinees interpreted it. To judge this item, inspect its full prompt: identify what is stated, what conventions its examples establish, whether each candidate explains all displayed features, and whether the two rules actually predict different responses. That inspection can show whether an alternative survives the visible constraints. A stronger claim about item quality would need evidence tied to this question, such as relevant expert review or pilot responses and their interpretation; general guidance and results from another assessment cannot substitute for it.

The limits do not make construction advice pointless. Focused wording, adequate information, and review are sensible aims because they give examinees a fairer basis for applying the intended task. The evidence here simply cannot turn those aims into a measured guarantee about this item. A plausible second pattern is a reason to examine the prompt closely, not proof from these studies that the question is ambiguous.

Sources: ETS K–12 Item Development; Guidelines for the Construction of Multiple Choice Questions Tests; The impact of item-writing flaws and item complexity on examination item difficulty and discrimination value; Item-writing rules: Collective wisdom

Choose a next step that matches what the item can show

During a timed test, follow its directions: choose the best-supported option provisionally, or move on if allowed. In review, record the observation, candidate rules, predictions, and distinguishing constraint. Practice an unclear operation on fresh examples; repetition does not show that general intelligence has increased.

Record where the predictions split and what missing constraint would decide between them. An additional displayed step, for example, might distinguish continuing differences from a rule that changes operation. Keep this as a review question rather than inventing the step. If an answer is required now, choose under the stated directions and preserve the uncertainty in your notes.

If a score will affect education, clinical care, or another consequential decision, discuss the question with a qualified professional and use an assessment suited to that purpose. For low-stakes practice, take the free Test IQ Free 50-question test and review its worked answers. Its estimated IQ range is provisional, produced by an unvalidated conversion with no validated adult reference population. The optional $9 report adds explanation, not validity.

Questions readers ask