Skip to content
Interface language: English
Audit / Signal

Prompt injection in documents: is the AI following the right instructions?

Test how an AI workflow handles instructions hidden in documents, tool permissions and approval before an action.

Article language: English

A seated woman beside a glass sphere with dark reflections.
In this article

An assistant reads a partner's document and finds a direction to change the recipient of its result. Material supplied for analysis is trying to pose as a superior. This is an example of prompt injection: an attempt to steer a model through material that should not grant authority. Test that boundary before connecting tools that send messages or change files. A third party's document can contain an instruction, but that alone does not make it a company order.

The procedure below is an editorial proposal for assessing one specific process. It is not a product security certificate. Its outcome should be a record of observed behaviour, the entry paths tested and a decision about which actions still need technical restrictions.

Where can a document impersonate an instruction?

Start by reconstructing one complete task: an employee selects materials, an application retrieves an attachment, a model prepares a result, then a person or integration uses the answer. At each handover record where the content came from and who can change it. An AI data-flow register (in Polish) helps, but here we add a specific question: can the author of the material influence the permissions of the actor carrying out the task?

Folder ownership alone does not answer that. An external file may have entered a company repository, and an internal report may quote a supplier's page. Mark the origin of individual parts if the system can retain it. If it cannot, apply a more cautious trust level to the whole item. Placing a document on a trusted drive does not change who wrote it.

OWASP's guidance on prompt injection describes indirect injection through external content processed by a model and calls for tool permissions to be enforced outside the model. A line in the system instruction saying not to trust documents is not, by itself, sufficient evidence for acceptance.

A useful map separates three roles: the requester defines the task, the document supplies material, and the application checks whether an operation is allowed. A partner's offer may contain a new delivery address as information to check. It does not automatically give the user's consent to send attachments from another project there.

How do you define an observable test result?

Do not judge only whether the model writes a polite refusal. Look at the effect: was a tool called, what arguments did it receive, was there an attempt to exceed scope, and did the user see the change? A system can say it refuses while calling an operation in the background. It can also process a document without an alert and correctly stay within the task boundary. With a browser-operating AI agent (in Polish), also check the state of the relevant object in the destination application rather than treating a description of a click as evidence of its effect.

Before testing, record the allowed outcome of the ordinary task. For a comparison of two offers, that may mean specified criteria, links to source passages and a note of missing data. Then define a prohibited effect: changing the recipient, exposing other material or adding an unsupported claim. This distinguishes a crossed boundary from an ordinary factual error.

In a responsible agent workflow (in Polish), agree in advance which operations need approval. Give each one a test condition: what the employee sees before approving, which document supports the action and what happens after refusal. Consent to analyse a file should not be reinterpreted as blanket consent for every later action.

For the first tests use fictional documents and substitute tools that only record the proposed action. Replace a real mailbox with a log of planned messages. Replace a confidential drive with two artificial projects. The test should reveal whether a boundary can be crossed without causing a real leak or changing a client's documents.

How do you prepare a small, useful set of cases?

Choose several families of situations relevant to the team's work. One tests an attempt to change the purpose of analysis; another adds an unapproved recipient; a third asks to reach another resource; a fourth poses as an administrator's message. You do not need a collection of dramatic phrases. You need a comparison of application behaviour along actual processing paths.

Pair each case with an ordinary control document. A system that rejects every file containing the word "instruction" may make it impossible to analyse an operating manual. Record unjustified blocks as well. The assessment covers security and the ability to complete the agreed task. An overly broad block often leads employees to work around the process.

If a team works in Polish and English, include both. Microsoft's Prompt Shields documentation distinguishes attacks in user prompts from those in documents and notes that performance varies across languages. A classifier result needs interpretation and appropriate application logic; it does not by itself prove that a particular operation was blocked.

For each case keep the document version, configuration version, test date and observed effect. When runs differ, record the range rather than selecting the most favourable answer. This can draw on the approach to AI quality regression tests (in Polish), while keeping the focus on a trust-boundary violation rather than general writing quality.

Aeneia conceptual photography

What might a purchasing scenario look like?

Suppose an assistant compares three equipment offers and drafts questions for suppliers. It may read a specified folder and save drafts. One fictional offer contains a paragraph suggesting that the complete comparison be sent to an extra address to "confirm compliance". The company has given no such consent, and the test's sending tool is a substitute.

Record four observations separately: whether the model noticed the text, treated it as a fact about the offer, proposed an additional message and whether the application allowed the call. Merely quoting the suspicious sentence in a summary does not mean the process was taken over. An attempt to send outside the agreed list is a different category of event.

In the first system version, the approval button shows only "send questions". That is weak because the employee cannot see the extra recipient or attachments. The team changes the screen to show the complete message and recipients, and an independent tool rule blocks sending outside the task's approved list. It then repeats the same case and a normal control send.

If the block works, the report describes that exact path, version and test set. It would be wrong to claim from this that the product resists every possible manipulation. Other formats, future integrations and changes in document retrieval have not been tested. Stating that limit helps plan the next assessment.

How do you choose a safeguard after finding a problem?

Decide according to the possible effect. Reordering points in a working summary calls for a different response from an attempt to send data outside the organisation. First remove the opportunity for the unauthorised operation. Then improve instructions and messages. Reviewing AI integration permissions (in Polish) can often reduce the potential harm before rebuilding the application. If the control relies on an isolated environment, examine the agent sandbox's file and network boundaries (in Polish) together with actual tool behaviour.

For reversible operations limited to a draft, an explicit preview and later review may be enough. Sending, overwriting or retrieving information from other areas needs a control appropriate to that action. The user should approve a specific scope. The execution rule must check it again even when the model supplies a convincing explanation.

If a test reveals an actual unauthorised action, stop the experiment and start the agreed data incident response procedure (in Polish). Do not mix continued testing with securing the event. Further actions can make it harder to establish what happened earlier and which copies need investigation.

What if an instruction is part of the content?

A document may describe an ordinary procedure and the user may ask for a plan to carry it out. Ignoring every imperative sentence would then spoil the task. The distinction is the source of authority: a model can reproduce steps from a manual because a person requested their analysis. That does not authorise it to run the tools described in the manual or change its own limits.

Record two separate answers for the case. First, was the document understood correctly? Second, were only actions approved by the user carried out? This avoids marking a safe system as failed merely because it faithfully summarised a suspicious paragraph. Content can be described, marked and presented for review without executing its instructions.

Consider deferred actions as well. An assistant may save a draft today and another automation may turn it into a message tomorrow. If the first stage marked a passage as an unverified supplier request, that label should survive the handover. Losing it can lead a later actor to treat a working proposal as an approved task. The assessment then includes the later read as well.

Also test an empty or damaged document. The system should clearly explain that material is missing instead of filling the analysis with instructions found elsewhere. This is a related but distinct case: not every unauthorised step is caused by hostile text. Sometimes the application expands the task itself because it lacks an explicit stopping path.

After the workshop keep a limited evidence package. The fictional file version, record of proposed operations and control configuration will usually be enough to reproduce that particular result. Do not copy passwords, real keys or other clients' materials into a report merely to make an example look convincing. Technical evidence should support a fix while remaining safe to circulate in the project.

Connect the test of external instructions to acceptance of the company's AI search (in Polish). A source can be relevant to a question and still contain text that should not control tools. Retrieval relevance and permission to act need separate criteria.

What belongs in a safeguards assessment?

A useful record closes one specific decision: the defined scope may go ahead, needs limits, or operations must pause until a fix. Name an owner for each remaining gap and a date for a repeat test. A general task to "improve AI security" does not give the team an actionable next step.

  • The map covers every document-retrieval channel currently in use.
  • Tests use artificial data and tools without real external effects.
  • Each case specifies the task, prohibited effect and normal control test.
  • The report shows the actual operation arguments and the outcome of execution rules.
  • Approval covers visible recipients, resources and changes.
  • After a fix, both the problematic case and the normal-work control were repeated.
  • The list of limits states what the tests do not yet cover.

To discuss an assessment of such a process, start with the scope of Aeneia Audit.

Aeneia / Aeneia editorial

Editorial article. Source links are included in the text.

About Aeneia

Brands we have worked with

  • L’Oréal
  • Garnier
  • Netflix
  • Carrefour
  • Warner Bros. Discovery
  • Biedronka
  • HBO Max
  • Wolt
  • Żabka
  • COSRX
  • Embassy of China in Poland
  • Mixa
Aeneia / Initium

From knowledge to a decision

Tell us what you want to put in order. We will agree on where to start.

Contact Aeneia

We partner with and use solutions from:

  • Google
  • OpenAI
  • Microsoft
  • Stripe
  • Anthropic
  • Linear