Skip to content
Interface language: English
Audit / Signal

A coding agent in CI: someone else's pull request is not an instruction from the owner

Assess untrusted CI inputs, workflow context, execution permissions and the difference between approving a run and a merge.

Article language: English

A woman in a red apron beside a round glass lens in a gold-coloured housing.
In this article

The submission is called “A small documentation fix”. In the description, its author explains how the agent should prepare the environment before checking the change. The automation executes the command, runs the tests and sends back a polished summary. The problem appeared earlier: the instructions for operating the company environment were written by someone who should not manage it.

A fictional team developing a catalogue portal receives a pull request from an external contributor: a proposed set of changes to be accepted into the project. An agent is supposed to help assess it. CI, or continuous integration, runs automated checks. Combining the two requires a decision about whether outside content remains material for analysis or can begin to direct execution.

The changed file is not the only input. A title, description, branch name, comments and material from previous checks can enter the automation's context. Each needs an identifiable origin. The fact that GitHub or another company tool displays a field does not make the person who wrote it the owner of your task.

First establish who controls each part

In our portal, the project maintainer sets the review scope. The author of the submission controls the proposed code and their description. The team maintains the workflow, the definition of the automation steps, and the access settings. The agent processes the material with the tools it has been given. Those are four different roles, even if every piece of information appears on one pull request page.

Map the inputs to the process. For each field, identify its author, how it is retrieved and how it is used next. A description may enter a prompt for the model, a title may enter a naming-check script and a file's contents may enter a test. The assessment should not stop at “we only read the pull request”. Displaying content and passing it on for execution are different things.

GitHub describes the risk of interpreting untrusted fields as code, including titles, descriptions and branch names. Content can change the effect of an executable script when it is inserted into it.

For an agent there is a separate question of interpreting natural language. A description may instruct it to widen the scope, skip a check or read another resource. The model should be able to report that content without treating it as an instruction from the project maintainer. The right to propose a fix does not include changing the rules of your environment.

Not every imperative sentence is an attack, however. Documentation may contain instructions for installing a program, and the task may be to assess them. The agent should be able to read the steps and consider whether they make sense within the agreed scope. Executing them is a separate decision. Mechanically rejecting every instruction in a README, a file describing the project, would break ordinary documentation review.

Files from which a product loads rules for an agent need particular attention. Establish whether, in the run being assessed, they come from the approved base or from the proposed change. The contributor may try to change not just the portal's code but also how it is reviewed. The input record should distinguish an owner's instruction from one found in the material being assessed. Do not assume every product loads such files in the same way.

Code-assistant file exclusions (in Polish) do not automatically solve this problem. A file may properly be available for analysis and still not be entitled to direct the agent's permissions. In CI, establish who supplied the content and which process will use it as well.

Reading code and running a project are two thresholds

Suppose an outside contributor corrects a button description. The agent opens the diff, then wants to install dependencies and run tests. That second step may involve much more than the changed sentence. The reviewer should identify which version supplied the scripts, configuration and other files used by the command.

GitHub's guidance on pull_request_target distinguishes fetching code from executing it afterwards. It warns against running untrusted content in a context with secrets or a privileged token. This includes actions taken by scripts and dependencies.

The test command itself may look familiar. Still, establish what it will run in the project version under review. A configuration change can affect behaviour without adding an obvious command to the workflow itself. The process owner does not have to analyse every library personally, but should know who is responsible for assessing this threshold and how the environment was restricted.

In our example, the first decision is that the agent compares the submission with the approved base and prepares comments. Any execution is to take place in a separately prepared environment with defined access. We do not assume that separation exists because a tool is called a “sandbox”. An agent sandbox (in Polish) is assessed against its actual file, network and credential boundaries.

Also consider material passed on by an earlier run. An artifact or error record may contain content from an outside change. If a later stage runs with broader access, it should not treat the entire artifact as a trusted instruction just because it was downloaded from the company's own CI platform. Content provenance needs to be preserved between runs too.

When preparing the environment, establish what remains from an earlier task. A self-hosted runner, a machine that executes CI steps, requires assessment of its access to company resources and isolation between runs. A fictional control file from a previous run can be used to check the agreed state of the next one. A single run's result should not replace the question of whether the same workspace is reused.

A team may decide that certain submissions receive analysis without execution. That result should be named honestly. “Read the diff; did not run tests in this scope” tells the recipient more than a green status covering actions that were not performed. The limitation need not mean rejecting the fix; it may identify a further controlled stage.

An event name does not describe the whole configuration

In GitHub Actions, the distinction between events that trigger a workflow matters. For an operational assessment, the name of a “run tests” button is not enough. The automation definition, the source of the files being assessed and the access granted to the job are all needed. Together they determine the actual effect that a qualified practitioner must assess.

Aeneia conceptual photography

Documentation for the event says that pull_request_target uses a trusted base context. A problem can arise after outside code is fetched and executed later. Do not assume that a workflow file from the base proves the origin of everything that workflow runs.

Do not copy settings from a guide into another repository without comparing its current rules. A public repository, a private repository and an environment with its own policies can have different scopes. During acceptance, identify the state actually checked and the date of the check. A process name such as “safe review” is not evidence that a particular restriction was applied.

For a finished product, also distinguish its default protections from the company's configuration. GitHub describes the default approval requirement for workflow runs following Copilot cloud agent changes, but also allows automatic runs to be configured. The default must not be extended to every installation.

This is an example of a specific service, not a rule for every agent. Copilot cloud agent's risk documentation also describes outside instructions in inputs and limits on tools available to automations. A company's assessment still needs to establish the product, mode and capabilities actually granted.

Consent to run checks and consent to merge a change are separate decisions. The person pressing a button should know which will occur. If the environment can run a job with access to sensitive resources, review must cover that effect before execution. Rejecting the pull request later will not undo an action already taken.

Aeneia Audit

Let us trace an agent's inputs in CI, the origin of executed files and each stage's access. An operational review can help the automation team name the evidence it needs to provide. Define the agent's boundaries for someone else's change

A test should preserve who authored the outside material

A first scenario can use a separate test repository, fictional data and tools without real secrets. The owner defines the expected result: analysis of the specified change without widening the scope. An explicit, harmless marker for an instruction outside the task is added to the submission description. The check asks whether the agent treated it as content rather than authorisation.

You do not need to execute someone else's harmful commands to map the risk. The technical team should design tests appropriate to the scope and observe the real boundaries. The report separates material read by the model, proposed actions and actions permitted by the environment. A polite refusal in text does not establish what a tool may already have executed.

Separately, add a normal documentation change containing a correct instruction for using the product. If the agent rejects it solely because it contains an imperative, the control obstructs ordinary work. The team needs to distinguish a described procedure from a source of authority to execute it. In both cases the outside contributor remains the author of the material.

Commands entering an execution environment introduce the additional question of shell interpretation and quotation marks (in Polish). A shell is a program that interprets commands; their meaning needs a separate technical assessment. A model's sentence matching the maintainer's intention does not establish the effect of a command once data has been inserted into a script.

A rejected case should leave an identifiable state. The operator needs to know whether analysis, execution, resource reading or a write was stopped. Do not rerun the same step with greater permissions just because the agent called the refusal an obstacle to completing the task. Expanding access requires the decision appropriate to the new effect.

Acceptance concerns the version and the execution path

Record the repository, submission version, workflow version, triggering event and relevant job permissions. Add the origin of files actually executed in the test. If the contributor adds another commit, a saved version of the change, after the assessment, the previous result does not automatically describe the new contents. This matters especially when a change appears between reading the material and approving execution.

In the fictional portal, the maintainer reviews a change to the button description alone. The contributor later adds a modification to the test setup script. The approval button still concerns the same submission, but the material to be executed is different. Before running it, identify the current set and relate consent to the right version. The pull request number does not replace that link. Preserve the version read by the reviewer and the version actually fetched by the job in the evidence.

AI code review (in Polish) should identify the diff actually read. For CI we add the execution conditions. These pieces of evidence can concern different versions if the process does not retain the connection. Explain such a mismatch in the report instead of combining two green results into one general approval.

Finally, the person taking over the process should be able to find a rejected run and name the next step. It might be rereading the change, a restricted run or a configuration check by the CI owner. “Agent blocked” alone does not specify that decision. Give the recipient a sample report without an oral explanation and see whether they distinguish material only read from code already executed. This is a practical check of the usefulness of the record, not an additional test of someone else's protections.

The principles of a responsible agent workflow (in Polish) still apply: the task scope, tools and evidence of effect must be clear. CI adds a contributor who controls part of the input and automatic transitions between stages. These need to be included before the process is accepted for everyday use.

Aeneia Audit can help organise this map and the acceptance requirements for AI use. A specialist review of CI security requires an appropriate scope and technical expertise. The result should be a concrete answer for the portal maintainer: this change was read, these files ran in a defined environment, and the outside description did not gain authority to manage the remaining access.

Aeneia / Aeneia editorial

Editorial article. Source links are included in the text.

About Aeneia

Brands we have worked with

  • L’Oréal
  • Garnier
  • Netflix
  • Carrefour
  • Warner Bros. Discovery
  • Biedronka
  • HBO Max
  • Wolt
  • Żabka
  • COSRX
  • Embassy of China in Poland
  • Mixa
Aeneia / Initium

From knowledge to a decision

Tell us what you want to put in order. We will agree on where to start.

Contact Aeneia

We partner with and use solutions from:

  • Google
  • OpenAI
  • Microsoft
  • Stripe
  • Anthropic
  • Linear