Fine-tuning an AI model: check what the examples should teach before sending them
Assess the objective, provenance and consistency of fine-tuning examples, service availability and independent evaluation.
Article language: English

In this article
“We have lots of customer conversations. Let's teach the model with them” sounds like a plan until someone opens those conversations. In one, an employee confirms a condition; in another, they refer to an old price list; in a third, they promise an exception agreed by phone. The archive describes the company's history. It does not necessarily represent behaviour the company wants to repeat.
Fine-tuning a model on examples changes its learned parameters. It is a different operation from attaching an instruction or document to the current question. Before preparing a dataset, define the change you want: an output format, classification method, style or another recognisable behaviour. “Make it know our company” leaves too many possible answers, including for the supplier.
Identify the problem before choosing training
Imagine a team that wants to extract the purpose of a customer's message, available materials and missing information into a working note. The current model understands most messages, but sometimes confuses a customer's questions with the studio's agreed decisions. This is behaviour in a specific task that can be described with examples and compared before and after a change.
If the problem is the absence of a current service list, first check how that list is supplied. Training on an old offer cannot replace access to a current source. Nor will it repair a broken system connection or incorrect content submitted in a form. Without identifying the cause, you could spend money reproducing yesterday's information very consistently.
Keep the current solution's output as a baseline. Establish whether improving the instruction, data format or retrieval method would be enough. Only then assess what additional problem fine-tuning should solve. A trial should lead to a decision about change, not justify a tool already chosen out of habit.
You also need actual access to the service. The OpenAI deprecation schedule describes restrictions on self-serve fine-tuning based on an organisation’s previous tuning activity and use of tuned models. Active existing customers will no longer be able to create new fine-tuning jobs from 6 January 2027; use of tuned models for inference is described separately and remains available until the underlying base model is deprecated. The Polish source recorded its check on 29 October 2026. Do not assume availability from an old guide; check the current terms of your account and chosen provider before planning an implementation.
When assessing an AI provider (in Polish), treat access to training and subsequent use of its result as separate questions. Check compatibility of the model, tuning method and maintenance plan too. Services with different requirements use the same term “fine-tuning”; the name does not confirm a specific path.
The expected answer must follow from the input
In a working example, a customer asks whether a catalogue can be prepared. They attach a product list but no photographs. The reference note should preserve the missing photographs as a question to resolve. If its author adds that the studio has already received them, the example is wrong even if the sentence sounds natural and resembles common orders.
Examine how each reference answer was produced. An editor may know about a later conversation the model will not see. They might then write an accurate description of the whole case but an incorrect answer for that input. Separate knowledge available at the time of the task from information added later. Evaluation needs the same boundary.
OpenAI's data-quality guidance discusses whether answers are supported by the input, consistency among examples and matching the format to real use. It also warns against training claims that actions were performed when the system does not perform them. Check an example's meaning before accepting it as a technically valid record.
Pay particular attention to statements about actions: sending material, making a reservation or updating an entry. In the fictional studio, the note should summarise an enquiry, not perform those operations. A reference answer claiming that a date was booked teaches behaviour beyond the task. A correctly formatted field cannot repair untrue content.
When two people prepare different reference answers, establish why. One may treat a customer's assumption as fact; another may preserve it as a question. Decide which meaning belongs in the output and on what basis. Adding both to a larger dataset preserves the conflict. The next person preparing examples would still have to guess which pattern to follow.
A changing fact needs an identified source
A working note mentions a delivery time. If reference answers always contain the same number of days, the model may repeat it when a new case has a different scope. Are you teaching a way to record the time stated in the material, or fixing a particular value? The first is task behaviour. The second can quickly become an outdated promise.
Prepare a fictional pair of inputs with the same message structure but different agreed terms. Check that each reference answer preserves its own condition. Add a case without a date too. Such material helps reveal the desired relationship between available information and output. It does not require the model to infer the team's schedule from the tone of an enquiry.
Review service names in the same way. The offer can change while the task of organising messages remains relevant. Plan where the tool will receive the current scope and how it should respond to an old name used by a customer. The training set should have a clear relationship with that source, rather than quietly becoming a second, manually maintained offer.
A multi-thread conversation needs separate attention. A customer may first ask about a catalogue, then withdraw that part and keep only a photo session. The reference answer should show what remains current after that change. If you prepare only an answer to the first message, you have not described the task performed after the whole conversation. Preserve the order and identify the point in time to which the expected output applies.

A dataset has an origin and limits on use
Before transferring material, trace its path. Does it come from approved examples, working messages, support requests or fictional scenarios? Who may decide on that use? What data does it contain and where will it be processed? These questions concern a specific flow and need answers from those responsible for data and contracts.
Do not equate removing a name with fully preparing material. It may still contain contact details, a project name, order details or confidential information. Define what material is permitted for the task and check it before transfer. If a legal assessment of a particular use is needed, it should consider the actual data and the provider's terms.
The AI data-flow register guide (in Polish) also covers working copies and resulting assets. Sending a file is not the only stage. You need to know who keeps the original, where the prepared dataset goes and how to identify the version used for training. Decide what should happen if the organisation withdraws approval for the use of particular material.
Synthetic examples can help check format and intended behaviour. If AI helps create them, also check consistency of synthetic data, relationships and events (in Polish), rather than assessing each row alone. Label their origin. If all use one template, they may not represent the language of real requests. Many names in the same sentence structure do not automatically provide many different ways to express a need. A model earns no points for learning a list of names.
Keep a version of the editorial rules while preparing the set. A reviewer should be able to establish which definition was used to choose missing information or a case category. Changing that definition halfway through can make old and new examples describe different tasks under the same name.
The file should fit the chosen method
Once the meaning is settled, check technical requirements. Google Cloud's documentation on tuning open models, which the historical Polish source described as updated on 28 October 2026, describes a training dataset, a recommended separate validation set and supported JSONL formats. Individual formats have restrictions depending on method and modality. A sample file from the internet is not a universal format for every service.
JSONL means that each line contains a separate complete JSON object. Valid syntax confirms only that a program can read the structure. You still need to check message roles, conversation order and output content. An incorrect dialogue can be perfectly valid syntactically.
In our studio, an example should match what the application will actually supply in use. If the model normally sees a form description and a service catalogue, but the dataset provides a polished conversation history, it is being trained on different input. Evaluate it on material in the form that will later reach the tool too.
Link the dataset to the instruction version in the prompt library (in Polish). A change to a standing instruction can affect behaviour after tuning as well. If you change the instruction, format and model at once, it is harder to explain what caused an improvement or deterioration. Plan a comparison that separates those changes.
Keep a dataset manifest: version identifier, objective, scope of examples and change history. It need not be long. It should let you identify the exact dataset and its meaning after the trial. A file called “training_final2” leaves that work to archaeologists, who are usually hired from your own team afterwards.
Acceptance concerns a result the dataset did not give away
Set aside evaluation material before improving the solution based on its answers. Check reference answers separately. If a case reveals an error and is used to make a change, keep that history. A further evaluation on fresh material needs new situations. A high score on rehearsed examples answers a narrower question.
In an AI pilot (in Polish), compare the baseline and tuned solution using the same criterion under understandable conditions. Check preserved meaning, omissions, invented details and subsequent use of the note. A shorter text may be convenient for an employee, but not if an important customer qualification has disappeared.
Success at extracting fields does not automatically confirm the accuracy of embedding-based search (in Polish) or other stages. Assess the task actually tuned. If the result enters a later process, inspect how it is read there too. Even a good answer can be assigned to the wrong case.
Decide who handles the next error after the trial. Fixing a record in a local file does not automatically change an already tuned model. You must decide whether to change the instruction, update the source, withdraw a version or assess another solution. Each path has its own work and should lead to another check of the task.
At acceptance, also record the condition for ending the experiment without deployment. If the new variant does not improve what is needed or makes maintenance harder, keeping the baseline is a valid decision. The report should show the trial on which it was based. Preparing a dataset does not oblige you to continue training.
For a discussion with Aeneia Audit, bring one incorrect output, a correct reference answer and the material available when the task was performed. These can start a conversation about the cause and an appropriate change. The archive can remain where it is for now. First establish exactly what it would teach.







