Skip to content
Interface language: English
Design / Signal

Research without a moderator: the task must explain what to do

Plan unmoderated research with clear tasks, stopping points, a full pilot and an evidence-based analysis.

Article language: English

Illustration: a woman in red and golden streaks of light in an open gallery.
In this article

A participant opens a study at 9:17 p.m. The screen says, “Check workshop registration”. Should they assess the offer, choose a time, complete a form or actually register? There is no moderator. The instructions appear to have gone home early, leaving three words and several possible interpretations behind.

An unmoderated study takes place without someone conducting the session live. A participant receives prepared instructions, performs tasks and supplies the result in an agreed way. The team reviews the material later. The convenience of choosing one's own time does not remove the organisational work: the goal, starting state and way to finish must make sense without a live explanation.

Let us plan such a study for a fictional studio called Rondo. We want to see whether a person can choose a workshop for two team members and recognise the chosen time in the summary. We use a test version, with no real registration. The examples below describe a research design, not results collected from participants.

One research question needs a specific task

Write down the decision the material should support. At Rondo we are asking about choosing the right time and number of places, not the general appeal of the brand. If the goal is to understand why people decide to buy training, a short interface test will not answer the whole question. Choose a method based on the team's uncertainty, not an unused subscription to a research platform.

The Government Statistical Service survey development toolkit describes an unmoderated test as useful for a short task and a defined research question. It notes a limit: there is no opportunity to ask follow-up questions during the session. The guide concerns survey materials; our example uses this distinction when planning a self-directed website test.

Rondo's instruction might read: “You are arranging a product photography workshop for two people from your team. Both can arrive on Friday no earlier than 3 p.m. Choose a suitable time and two places. Stop at the summary before submitting the registration.” The test material must contain times that make the task possible. The study should not require someone to guess at an option missing from the prototype.

“Afternoon” alone would leave an ambiguity that matters when assessing the result. Saying that they can arrive from 3 p.m. avoids an argument over whether noon meets the requirement. It describes a need without revealing the name of an interface element. The assessment criterion should not contain a hidden condition known only to the researcher.

Do not add filter names or button labels to the instruction if recognising them is what you want to assess. “Choose the Dates tab, then Participants” turns part of the test into following the researcher's directions. State the situation clearly, but leave the route to be discovered. In the pilot, check whether different interpretations come from the task wording or the interface.

Set the starting data. Does the participant use a prepared account or begin as a new visitor? Are fictional details for both people supplied with the task? Does the form require a telephone number? Prepare suitable test material instead of asking for a colleague's details. An empty required field can stop a session because of study organisation even when the website is working correctly.

Recruiting suitable participants (in Polish) still matters (article in Polish). Someone choosing a workshop for a team should understand that kind of task. Do not replace that criterion with the ability to record a screen. Familiarity with a research platform does not automatically make someone a user of Rondo's service.

Check the platform's own requirements: device, browser, language support and how the material is shared. If the study concerns new customers on phones, do not accidentally limit it to people with computers and a browser extension. Record tool restrictions in the recruitment scope. They do not disappear from the interpretation just because the screener excluded people unable to start a session.

The participant knows where to stop; the researcher checks the result

Before starting, describe how the participant will recognise the end and how the researcher will assess the outcome. At Rondo the participant stops at the summary. The researcher checks the time, workshop subject and two places. Pressing “done” in the platform does not confirm these values. The participant might have stopped after reading the programme or after selecting only one place.

In NN/g's guide to unmoderated testing, Kathryn Whitenton stresses self-contained instructions, an explicit stopping point and a pilot, drawing on practical experience. Your own task should explain where to stop without listing the entire correct sequence of clicks.

Match the evidence to the question. If the choice of time matters, the recording must show the summary clearly, or the participant must provide it in another agreed way. If you are examining hesitation around the number of places, a final screenshot alone will not show the process. Do not collect a face recording just because the tool offers it. Each kind of material should have a role in the analysis.

Before recording starts, explain what area will be recorded and how to end participation. Allow time to prepare the screen without private notifications. If speaking actions aloud is required, check in the pilot whether the instruction draws all attention away from the task. Difficulty talking while working should not automatically become evidence that the site is difficult to use.

Test the whole technical flow under the intended participation conditions. Opening the link, accessing the prototype, returning to instructions, starting the recording and submitting the result all form part of the study journey. When the prototype opens a new tab, make sure the necessary material is still recorded. Do not automatically interpret a missing recording as a decision to abandon the task.

Aeneia conceptual photography

Also prepare for stopping and returning, if you intend to allow it. Check whether the platform resumes the same session or starts a new attempt, and what state the prototype retains. Two responses from one person after reopening a link should not be counted as two participants without checking. If returning is unsupported, explain this before starting and retain a record of the interrupted session.

Separate the research platform's interface from the site under test. A participant may mistake a button for moving to the next task for the next step in the form. Prepare a clear introduction and check it in the pilot. If recording controls need explaining, use a neutral example that does not reveal the solution to Rondo's actual task.

Afterwards, you may ask for a brief rating of perceived difficulty (in Polish) (article in Polish). That report supplements the outcome; it does not replace it. Someone may be satisfied despite choosing Saturday instead of Friday. Link the answer to the specific session so the experience can be compared with what appears in the summary.

The pilot includes opening the collected files

First go through the study yourself, but do not stop after your own successful recording. Invite a person with relevant experience to pilot it on their own equipment. If you explain the task in a moderated trial, record that and revise the material. Then test the version intended to work without that explanation. Otherwise you have tested the instruction together with help that later participants will not receive.

Open every pilot file. Is the picture readable? Is it clear which window contains the task and which contains the study? Does the question after completion refer to the correct step? “Response collected” confirms only part of the flow. A recording of a blank screen can be a correctly saved file and still be useless for the project goal.

Test failure too. What should the participant do if the prototype will not open or no time matches the task? They need a way to mark the problem and finish. Do not force a random choice merely to unlock the study submission button. Such a forced result can later look like a decision about the offer.

At Rondo, a second place might be unavailable at the chosen time. If that is a deliberately studied state, include it in the question and assessment criterion. If it is an accidental effect of a previous session, restore the prepared state before the next attempt. Data shared between participants should not change task conditions without being recorded. The researcher needs to know what was actually available in each session.

Record the instruction version after a change. Do not combine results from before and after a clarification as though everyone received the same task. If the change concerns the substance of the task, earlier sessions may help diagnose the pilot but may not answer the main study question. Preserve that material with an explanation of its role.

Record the version of the website or prototype available to participants too. A change halfway through collection may alter screen names and order. If it must be made, note when and determine which sessions concern which version. A shared study link alone does not prove identical conditions. At Rondo the two-place selection and summary would be especially important.

Aeneia Design

Prepare one task, a prototype and an observable ending. Piloting the entire study journey helps refine the instruction and the material used to assess the service design later. Prepare a self-directed task test

Not every interrupted session describes a website error

During analysis, distinguish correct completion, an incorrect result, a deliberate withdrawal and a session without enough evidence. Preserve the basis for each decision: summary values, a statement or a technical message. Record the rules in the report before classifying sessions. If you later change how interrupted sessions are classified, also show the effect on the number of assessed outcomes. Discarding a difficult session should not quietly improve the success rate. If all you can see is a cursor disappearing, do not invent a reason. The participant might have stepped away from the phone, lost connectivity or stopped for a reason we do not know.

Context helps explain the limits of the material. Observing work in its usual setting (in Polish) (article in Polish) may answer questions a recording alone cannot settle. Do not pretend a later comment reconstructs every thought from the session, however. Mark it as an additional explanation collected after the task, with its own limits of memory and context.

If a group misunderstands the goal of the instruction, improve the research design before judging the website. If the group understands the task but chooses the wrong field, that is stronger material for interface analysis. Compare the evidence instead of deciding from the completion percentage alone. In a small sample, a description of specific difficulties may be more useful than an apparently precise forecast for all customers.

When comparing two prototypes (in Polish) (article in Polish), keep conditions that make the difference understandable. Different task wording, place availability or recording of results will make it harder to attribute the outcome to the design. Do not call a variant better merely because its group more often produced a recoverable final screen.

An unmoderated test can reveal places that need further discussion. If you need to ask flexible follow-up questions about a reason, consider a moderated test (in Polish) (article in Polish). Recordings can help prepare questions, but a number of files cannot replace the missing conversation. A confident-sounding researcher summary will not repair an uninterpretable session.

Rondo's result should show whether the task was understood, which choice was recorded and where the evidence ends. This scope of preparation can be discussed with Aeneia Design. Before a wider launch, repeat the complete pilot after corrections. The next participant will open the study at a time of their choosing too, and still will not be able to ask the absent moderator to explain those three words.

Aeneia / Aeneia editorial

Editorial article. Source links are included in the text.

About Aeneia

Brands we have worked with

  • L’Oréal
  • Garnier
  • Netflix
  • Carrefour
  • Warner Bros. Discovery
  • Biedronka
  • HBO Max
  • Wolt
  • Żabka
  • COSRX
  • Embassy of China in Poland
  • Mixa
Aeneia / Initium

From knowledge to a decision

Tell us what you want to put in order. We will agree on where to start.

Contact Aeneia

We partner with and use solutions from:

  • Google
  • OpenAI
  • Microsoft
  • Stripe
  • Anthropic
  • Linear