Help the first time, independent a week later: how to study learning an interface
Study repeat use of an interface after a break with comparable tasks, records of help, between-session experience and decisions about on-screen hints.
Article language: English

In this article
"After a short explanation everyone completed the task" sounds like good news from a test. But for a tool opened once a month, the second half of the sentence is missing: what happened on the next use? The facilitator will not arrive with coffee and a reminder that the right option was hidden under three dots. The user will be left with the screen, their own notes and work to do.
Studying how people learn an interface requires a test spread over time. Repeating an action more smoothly five minutes later does not answer the question of returning after a break. We do not need to build a memory laboratory, however. We need comparable tasks, explicit rules for help and information about what happened between meetings.
Consider a hypothetical panel for publishing assembly instructions. Every few weeks an editor adds a photo to a step, moves a warning and publishes a new version. This example and the proposed procedure below are fictional. They do not describe a study that took place or results from Aeneia clients.
What should survive until the next visit?
Grossman, Fitzmaurice and Attar distinguish initial performance from changes in performance over time in their taxonomy. They also consider user experience. This is a useful distinction in purpose: a successful first encounter and a smooth return after a break require separate observations.
For our panel, choose a concrete outcome: a person can prepare and publish a revised set of instructions without another person's prompts. We do not expect them to recite button names. They may read labels, use the preview and open ordinary help available in the product. If that is how they should work, looking for an on-screen hint is not automatically a memory failure.
Do distinguish a product hint from the facilitator's help. "Changes are saved in the draft" is part of the interface. A researcher saying "Now please choose publish" becomes additional training. If both are recorded as independent success, we cannot see what the user could actually reproduce.
An earlier cognitive walkthrough of first use (in Polish) helps predict whether successive steps will be understandable to a novice. A return study concerns someone who has already had an experience. Their hesitation need not signal the same obstacle as at the first visit. They may remember the goal but look for a function where they saw it before.
Decide what specifically must remain stable between sessions: the tool's layout, account roles, basic names and kind of action. If navigation changes in between, the next meeting will also test a new design. Such a study may be needed, but do not describe its result as a clean measure of retaining the earlier skill.
Choose the interval according to the rhythm of use and the question. A week may be a reasonable pilot stage, but it does not replace a month-long break if that is typical. Nor should a one-week test be promised as a prediction of return after a year. Record exact dates and the actual interval rather than putting every return in a box labelled "later".
The study by Iwon and Miłosz of the Gutenberg editor in Drupal involved three sessions separated by a week. The authors reset the environment and changed the website's subject between sessions. This is an example of preparing repeatable actions with different content, not proof that every interface must be tested exactly seven days apart.
In our example, the first set of instructions may concern assembling a shelf and the second a stand. Both require replacing a photo, moving a warning before the appropriate step and publishing a version. Changing the product name is not enough if the second instruction has twice as many photos or requires extra permissions. Comparability concerns the difficulty of actions, not decoration.
In a pilot, check that one set of materials does not make the task easier by accident. If the warning on the first page is obvious but its meaning must be established on the second, you are also measuring a content difference. In a larger plan, the order of sets can be swapped between participants. Assignment rules still need to be described: the later visit should not always receive the easier variant.
Prepare a correct end state separately for each version of the task. The warning must appear before the specified step, the new photo must replace the old one, and the reader must receive the correct published version. A "Done" message alone does not establish correctness. A participant can repeat an incorrect sequence efficiently and become increasingly satisfied with it.
Perform the tasks in an environment with fictional content where publishing will not expose real readers to an incorrect instruction. Preserve the real distinction between draft and published views, however. A prototype that shows the same success after every click cannot reveal whether a person recognizes the correct result. Before the session, check both task versions from the instruction reader's perspective too.
The same person, different conditions for help
The first session should show both an independent start and what the participant was taught. Prepare rules for responding when someone stops. You might first ask about their current goal, then offer help, and only at the end point to a specific element. Record every intervention and when it occurred. Do not reconstruct it later from the facilitator's memory.
Also define how the first meeting ends. If the plan includes learning to publish, after the demonstration ask the person to perform a short action of that kind independently. Record whether they reached the chosen level before the break. Without this, later failure could mean either forgetting the route or never having learned it in the first place.

If one person saw a full publishing demonstration and another found everything independently, they bring different initial experiences to the second session. Preserve both paths in the record and account for the difference in interpretation. A comparison of "first attempt and second attempt" without information about training would be too thin.
On return, give the goal and new materials, but do not remind the person of the previous route. Otherwise the start of the session becomes another lesson. The person may ask how to use the tool. The facilitator should follow a previously agreed rule, not give more help when they happen to want a smooth recording.
The wording of the goal can also suggest the route. "Publish the new version with the Share button" repeats part of the training. "Make the reader see the new photo and the warning before assembly" describes the result. Use comparable task wording in all sessions. Otherwise improvement may result from a more helpful researcher instruction rather than the participant's experience.
In their study of gesture learning and memorability, Ali, Morris and Wobbrock also tested recall after a week without practice. They studied specific gesture sets, not web forms. Separating learning from later recall is useful for designing our test; we do not transfer their numerical results to the instruction panel.
Also decide what is allowed between meetings. If the question is memory without further training, additional practice changes the conditions. If the question is real adoption, normal work should not be artificially prohibited. In that case, collect how many times the person used the tool, whether anyone helped and whether the application version changed.
A diary study (in Polish) can help collect events between visits, but it cannot replace observation of the repeated task. Sometimes a short note about use and the source of help is enough. Do not ask someone to describe a tool every day when they had no reason to open it all week.
Remember help from a colleague. In teamwork, asking "Did you do it independently?" may miss an earlier conversation with a colleague. That may be a natural way of working worth studying in a task performed in pairs (in Polish). Do not, however, attribute the entire pair's ability solely to one person's memory.
For each attempt, link the result, assistance and process in the record. Note whether the published set of instructions is correct, where the person stopped, questions and use of available help. Measure time from giving the goal to verified completion, according to the plan. If system waiting time is measured separately, do so in every session. Finishing faster after a researcher prompt is not the same as finishing faster independently.
When a participant gets stuck, you may close the independent part and move to a separate assisted attempt. Record that boundary instead of removing the earlier minutes. The second part will show whether an available hint helps them return to work. The first still shows that under specified conditions they did not reach the end independently. The two observations may lead to different screen changes.
Separately mark inability to perform the task: an expired password, a broken prototype button or missing appropriate data. Do not turn an environment defect into a participant error. If a repair requires starting over, retain both runs and explain why the first cannot be compared directly.
An earlier first-click test (in Polish) can reveal where someone looks for an entry point. A return after a break goes further: it also checks recognition of the effect of publishing. Someone who immediately found the editor may still leave the change only in a draft.
A design decision needs the whole journey
Imagine two possible records, still entirely hypothetical. In the first, a person returns to the editor without help but publishes the previous version. In the second, they spend a long time finding the publish option, use on-screen help and publish the correct instruction. One average time would hide the difference between an incorrect outcome and a slower, correct performance.
The first case calls for work on distinguishing versions and confirming what was just published. The second calls for work on the visibility of the transition from draft to publication. Another animated tour of the whole application might miss both problems. At Aeneia Design, such material can inform the design of specific screens and interactions instead of a general instruction to "simplify the experience".
Keep the results for people who did not return to the second session as separate information about the study's progress. Do not record a failure for a task they never attempted. Nor should you describe only those who returned as all those invited. In a small study, show individual paths and limitations rather than giving an average the appearance of a precise forecast for the whole company.
Above all, compare successive performances by the same person. Someone who did not return cannot be replaced by a new volunteer as their "second session". If a new person appears in the second round, their first attempt is separate material. Report how many people performed both tasks and how many only one; each comparison must make clear whether it concerns the same set of participants.
After the task you can collect a difficulty and satisfaction rating (in Polish). Let it comment on the observation. "That was easy this time" can coexist with an unpublished warning. You can then ask how the person knew that the instruction was available to the reader. Preserve their rating even if the publication state reveals an error.
Use the test to choose what information should still be available a month later. Perhaps a stable action name, a preview of the version to be published or a short hint for a rare operation is needed. The screen should support a return to work. It need not force users to prove that they still remember the training.







