Governing Agent Output Beyond Templates

Side-by-side comparison of the SAP CPU investigation: the original annotated response on the left, where primary, supporting, and investigation detail content is interleaved, and the restructured response on the right, which leads with the likely cause, recommended action, and confidence and groups supporting evidence below

Scneario-based templates were a huge improvement in the usability of our agent output, but it is too prescriptive for longer-term improvements. Defining repsonse policies at the right layer can provide a governable space for UX to direct adaptive experiences across multiple UX surfaces.

Role
Senior UX Designer, partnering with content design, product, and engineering
Timeline
Sep 2026 – Present
Outcome
Implemented first-stage response guidance in the official Microsoft Ignite 2026 agent demo, ahead of private preview availability, and defined a roadmap for adaptive response guidance.

The problem

Scenario-based templates gave us a valuable baseline. They made responses more consistent and helped establish what a good answer should contain for a known task. But testing that guidance against a complex enterprise-grade system investigation revealed a limit: a response could follow the template and still give the wrong prominence to a finding, bury important evidence, or lose continuity between related details.

The problem was not simply that we needed more templates. We needed guidance for deciding, at runtime, what to include, what to lead with, and what to make available on inspection—based on the user’s task, workload context, severity, and evidence.

The roadmap

Roadmap for JTBD-based agent responses in three stages. Now: information relevant to the job to be done goes through scenario detection and Markdown rules, then maps to a fixed template variant. Next: Markdown rules feed both scenario and conversation-state detection, which re-evaluate relevance and prominence to produce a dynamic template variant. Planned: available information and current context are evaluated for intent, continuity state, and system state; a response framework then runs semantic evaluation, inclusion, prominence, and disclosure decisions to produce a response specification for the rendering platform
Proposed progression across three milestones leading to an adaptive system

The roadmap moves from scenario-based templates toward guidance that can adapt to the user’s task, workload context, and available evidence. Each stage preserves the standards established by the last while giving the agent more flexibility in how it meets them.

The first milestone established a consistent baseline for known scenarios. The next stages build on that foundation by guiding not just response structure, but decisions about what to include, what to emphasize, and what to make available on inspection.

Milestone 1: Establishing a baseline

Side-by-side comparison of the SAP CPU investigation: the original annotated response on the left, where primary, supporting, and investigation detail content is interleaved, and the restructured response on the right, which leads with the likely cause, recommended action, and confidence and groups supporting evidence below
The example on the left shows a response where information with different levels of importance are intermixed vs. a structured approach that only shows the user what they need to know in a clear heirarchy with additional information available for investigation.

The first milestone was implemented in the official agent demo for Microsoft Ignite 2026. Working with a content designer, I developed scenario-based response guidance and authored it as Markdown policy files with rules and examples for the agent. I iterated with engineers until repeated demo runs followed the intended structure and content priorities.

This established a practical starting point: UX guidance could shape the agent’s response upstream, rather than only designing the interface around it. It also gave us a consistent answer to test against more complex scenarios.

What the baseline exposed

This video compares a strict templated response approach to a more dynamically aware approach.

The SAP investigation showed that following a template is not the same as making a good content decision. A response could contain the expected sections yet still bury a consequential finding, separate a claim from its evidence, or repeat details that no longer served the user’s task.

That changed the question from “Which template should this scenario use?” to “How should the agent decide what matters in this response?”

The north star: governing decisions, not just structure

Template-based responses across three turns of an Azure-to-Oracle connectivity investigation, each following a numbered 1, 2, 3 structure: a root-cause summary with actions and evidence, a decision about waiting for current Oracle diagnostics, and a recommended route-table change with steps, risk and rollback, validation, and evidence
Prescriptive templates can result in redundant information that can errode user trust across turns.
Dynamic-template responses for the same three turns, where each response leads with what that turn needs: the root-cause summary, then a short decision-needed prompt with options, then a recommended change with steps, risk and rollback, and validation, without repeating earlier evidence
In this example, a response establishes contextual awareness of previous turns, the structure adapts based on this.

The proposed next stages give the agent guidance for evaluating information in context: what is relevant to the user’s task, what is consequential enough to lead with, what evidence supports it, and what can remain available on inspection. The response should preserve continuity with earlier turns and distinguish observed facts, inferences, and recommended actions.

The goal is not to remove structure. It is to make structure serve those decisions, so the agent can handle combinations of workload state, severity, and evidence that no single scenario template anticipated. This is a design direction, not a capability I’m claiming has shipped.

Impact