Granite LLMON: test the instruction and data boundary
A laboratory result needs to meet your workflow. Define what the agent may do before comparing its responses.
Ler em portuguêsBy Pentesty
What the preview lets teams evaluate
On November 12, IBM released experimental 3B and 8B Granite 4.1 LLMON previews for structured separation of instructions and data. Its official publication limits the reported results to static indirect-injection tests and describes learned behavior rather than an enforced guarantee. The material positions the preview for evaluation, outside production.
The business decision can start with a concrete task, such as summarizing incoming requests and preparing a proposed routing decision. Define the permitted outcome, the data the model may consult and decisions requiring human approval. That description becomes the exercise contract, even if the model or the way messages are organized changes during the evaluation.
Build comparable pairs of cases
Prepare a small set of artificial documents approved by the process owner. For each legitimate document, create a test version containing a conflicting direction clearly identified in the exercise plan. Keep real customer data, credentials and external destinations out of the material. Preserve the same legitimate request in both versions so that the comparison remains interpretable.
Include cases where a document genuinely quotes an instruction from somebody else. The system should be able to describe that instruction as content without turning the quotation into execution authority. Add ordinary tables, attachments and ambiguous language. Write down expected behavior for every case beforehand, avoiding changes to the criteria after the team has seen the responses.
Include continuity and recovery
Propose a second stage where the agent receives additional material after producing a correct summary. Check that the initial approval remains attached only to the agreed task. This exercise investigates a local continuity hypothesis; it should not be described as reproducing a benchmark or as establishing general safety for long conversations across other applications and environments.
Define the response to an unexpected request as well: interrupt the task, preserve the trail and return the decision to its owner. Run the exercise against a disposable resource with a known restoration path. The purpose is to understand the time and evidence needed to resume approved work without making the assessment environment a new operational dependency.
Decide against written criteria
Before the pilot, define failure, inconclusive results and acceptable cases for continuing the evaluation. Leave room for quality problems, such as omitting an important fact or inventing a value. Evaluate security and usefulness on the same task set, because an unusable response can also undermine the business process and encourage users to bypass the approved workflow.
The final report should let another person repeat the cases with the same configuration. Identify dataset limits, attempts that were not performed and external controls that affected the outcome. A change to the model, tool or permissions requires reviewing the exercise contract. Approval should follow a verifiable configuration and a bounded use, supported by evidence the process owner understands.
Frequently asked questions
Does the exercise need real data?
Use artificial documents representing the work format. Preserve only the structure needed to assess the task, keeping secrets and customer information out of the exercise.
Which outcome is most useful?
A useful outcome connects the response to the authorized request and observed tool calls. Record quality, inappropriate refusals and inconclusive cases as well.
How should two configurations be compared?
Keep documents, permissions and evaluation criteria the same. Record every configuration difference and repeat cases that show variation between runs.
What should stop the pilot?
An action outside scope or a missing execution trail should trigger the agreed procedure. Preserve evidence and review the configuration before continuing.
Sources and references
Original Pentesty analysis, published November 12, 2026. The proposed scenarios are editorial guidance, with scope and validation to be defined for each environment.