Two prototypes, one task: compare designs beyond looks
Compare two prototypes on the same task, account for presentation order and separate decision accuracy from visual preference.
Article language: English

In this article
Design A is restrained; design B is bold. One person on the team likes the first, another shows the second at every meeting, and a third suggests combining them. The result is design C: a bold headline and a very quiet space where customers are supposed to find the terms. A comparison with people in the intended audience can help, provided you first decide which choice the design needs to make easier.
Compare the two prototypes using the same task and the same criteria. Observation can show where someone finds information, how they understand it and what they choose to do next. A preference for one appearance is separate evidence. On its own, it does not explain whether the offer has become easier to choose.
In a small design study, the aim need not be to announce a winner with a percentage lead. You might instead want to resolve one difference: should the terms of a package appear beside its name or inside an expandable section? To make that decision meaningful, prepare both versions and the way you present them fairly.
Choose a difference you can explain
Imagine a fictional offer of workshops for teams. The basic package includes one meeting and prepared exercises. The extended package adds an individual discussion of the results. In version A, the terms appear on the cards. In version B, they sit below a short summary and an expand button. The question is whether people can find and understand the difference in scope.
Write down your prediction before the study. For example: visible terms will help people new to the offer distinguish the packages without opening more content. Add how you will recognise that effect. The participant should identify the right package for the situation and explain why. Clicking to expand the content is neither a failure nor a success by itself.
If the versions also differ in price, names and the amount of supporting evidence, it becomes harder to attribute a result to the layout. You can deliberately compare complete concepts, but the conclusion will then concern the concepts as a whole. Do not announce that a larger button was specifically responsible. The study should reflect what actually changed.
Prepare the same facts in both versions. If one prototype gives a complete condition while the other gives only a summary, you are comparing the availability of information as well as its layout. That may be the right question, but name it as such. Matching the content matters particularly when the team wants to assess the way the information is presented.
Check how complete each prototype is. One should not offer a working journey while the other ends on a blank screen after the first choice. Differences introduced by the mock-ups can overwhelm the task you intended to study. Record their limitations and walk through both prototypes before meeting participants.
Use the guide to hypothesis cards (in Polish) to record the question and study conditions. The card does not have to be elaborate. Its job is to stop the team changing its definition of success after the favourite version struggles in the first session.
The same person does not start from zero on the second design
You can show both prototypes to the same people or divide them between groups. Each choice changes the conditions of comparison. In the first, a person can compare directly but carries knowledge from the earlier task. In the second, people bring different experience and each sees only one version.
MIT course material on experiment design distinguishes comparisons within the same group from those between groups and describes order effects. It also discusses systematically varying the order of conditions. That explains the mechanism; it does not determine the right sample size for every website project.
When the same people see both versions, you can plan an A-B order in some sessions and B-A in others. That way, the same version is not always presented second. It does not erase what participants remember. After the first attempt, they already know there is an individual discussion and where it may be found.
Do not treat the second attempt as an independent first impression. You can observe the rest of the journey, differences in operation and comparative comments. If the main question is about an independent first reading, the conditions need to be separated accordingly. In a small study, acknowledge the limitation instead of hiding it behind the word “random”.
Keep the task situation equivalent in meaning. “Choose a workshop for a team that needs a later discussion of the results” describes a need. “Open the extended package” gives away the solution and removes part of the question. Prepare the instruction in the same way for both versions, without adding explanations for the one that causes difficulty.
If you use similar tasks with different details, check that they are comparable. Choosing a package for a small team and choosing one for a large organisation may require different reasoning, even if the instructions are the same length. Changing names or numbers does not automatically remove a rule learned earlier. Record what stays the same, what changes and why both cases require the person to find the same important information. Otherwise, a better result may reflect an easier scenario.
With several tasks, watch for participants learning the offer itself. Someone may reach the second package faster because they remember the outcome of the earlier conversation. Record the order and any interventions. An answer given after a condition has been explained has a different status from information found independently in the prototype.
The guide to UX research recruitment (in Polish) explains how to choose people who fit the question. If only experienced training organisers see one version and first-time buyers see the other, the difference in results does not describe the layout alone. When dividing groups, account for criteria relevant to the task and make the limitations clear.

A qualitative comparison with shared tasks and a consistent way of taking notes can reveal obstacles in both proposals. Its conclusions will concern observed problems and hypotheses for the project, without claiming percentage precision the study cannot support.
Record what happened and what the person thought they achieved
Before presenting the versions, give them neutral names such as A and B. Do not describe one as the existing design and the other as the improved one if that knowledge is not needed for the question. The facilitator should avoid defending a solution they created, too. Agree on a common introduction and a response to difficulty beforehand, so the favourite version does not receive extra explanations in a gentler tone.
Prepare a shared observation sheet before the session. For each task, record whether the participant found the information, interpreted it correctly and identified the next step. Add when help was given and any limitation of the prototype. Completion after someone points to a button should be distinguished from completion without help.
GOV.UK's usability benchmarking guidance includes task completion, time and cases where someone believes they have succeeded when they have not. That distinction is useful for an offer. Sample size and the method of comparison still need a separate plan.
In our example, a participant might quickly choose the basic package, believing it includes an individual discussion. The time looks excellent. The decision, however, rests on a misunderstanding. Ask for a brief explanation: “What in the package you chose fits this situation?” Record the basis for the choice before showing the correct answer.
The guide to checking content in someone's own words (in Polish) helps distinguish finding a sentence from understanding what it means. Keep the same questions for both versions. Otherwise, one prototype receives a test of understanding while the other gets only a question about whether it looks good.
Time can be useful if the task and conditions allow a comparison. Record a pause caused by a tool problem or a longer explanation of the situation, though. Do not attribute every second to the page layout. With few participants, a precise observation of an obstacle may be more useful than an average of several very different journeys.
Ask about preference after the tasks if the participant has seen both versions. Ask why and connect the answer to a specific moment. “I prefer B because I don't see so much detail” is an interesting opinion. If that same person did not find a necessary condition in B, keep both pieces of information. A single opinion need not force the project to choose between appearance and accurate meaning. If you also use a post-task survey, the guide to questions about difficulty and satisfaction (in Polish) explains why to show its result separately from actual task performance.
Keep a record of successful routes, too. Version A may explain the difference between packages well, while B leads more comfortably to the form. Results like those may point to a useful combination. It should follow from what the elements do, not an obligation to satisfy everyone on the team.
A conversation about a project with Aeneia Design can start with one unresolved decision and material to compare. The more precise the question, the easier it is to agree on the scope of the prototypes and study.
Decide what comes next, even if neither version wins
Before the study, decide how you will handle conflicting criteria. If a version is faster but leads people to choose the wrong scope, the time saved alone does not make it the better choice. For this task, accurate understanding may be the basic condition and speed a secondary criterion. Another task may have a different priority. Writing that order down protects against fitting the assessment to the result the team likes most.
After the sessions, organise the material by tasks and criteria, not by the order in which designers speak. Show the problem, the evidence behind it and its consequence for the audience. If the same misunderstanding appears in both versions, the fix may concern shared content rather than the card layout.
Distinguish three possible outcomes: one version solves a particular problem better, both need improvement, or the material does not support a decision. The third is not a failed study. It may reveal a poorly chosen task, conditions that differ too much or missing information in the prototype.
Review the findings by presentation order as well. If participants mainly did better with whichever version they saw second, note the possible effect of already knowing the offer. Do not subtract that effect from the time or adjust scores by intuition. The material may call for a different separation of conditions in the next study. That observation is useful even when it leads you to postpone choosing a layout.
Do not build a winning design from every element judged better in isolation. Combining them can change the hierarchy and create a new obstacle. If you move an explanation from A into the journey in B, prepare the whole version and check it again. A button does not work in a vacuum, even when there is plenty of space around it on the board.
Use the guide to design decision logs (in Polish) to record the question, versions, order and reason for the choice. Add what the study did not cover. A month later, it will be easier to distinguish a supported principle from a decision that applied only within the scope tested.
A short five-second test can add information about initial orientation, as explained in the guide to five-second tests (in Polish), but it does not replace carrying out the task. Likewise, clicking through a prototype does not prove that the later implementation works. Each study has its own question and its own kind of result.
Once you choose a direction, prepare the next version with improvements based on the observations. There is no need to promise a final resolution of everyone's tastes. You do need to explain why this layout better supports the customer's decision and what remains to be checked. Then design C can come from a reasoned combination, rather than a diplomatic division of the headline between everyone at the meeting.









