Configuration design for an agentic system where a wrong rule is a silent failure
A B2B SaaS document processing product, anonymised. Sole designer, inherited mid-development.
Company and product anonymised to protect confidentiality. Design decisions and visual assets reflect work done in role.
The problem before the problem
The feature had no design when I got involved, only an engineering deadline and a rough concept: checkboxes. For a configuration surface with four rule types and three states each, checkboxes meant a save button, 100+ fields on screen, and no visible relationship between what a user set and what the agent would do. The risk wasn't bad UI. It was a user confidently configuring the wrong thing, the agent learning from it, and the user only discovering the mistake weeks later when their dataset was incomplete. By then, restarting would be the only option. That's the problem the design had to solve before any component decisions were made.
The job was clear
When configuring rules, users need to immediately understand what each rule does and what happens if they change it. No surprises downstream. With that established, the design decision was deliberate: build something familiar. A configuration component drawing on patterns users already knew from the existing system, covering full functionality without overwhelming.
The configuration problem
The configuration page was the hardest problem. Four rule types, three states each, and a configuration summary that needed to stay visible without interrupting the flow. The original approach stepped users through configuration sequentially. The rebuilt version keeps all three states visible at once through tab navigation: active, inactive, excluded. Users immediately understand the consequences of each rule. Every rule affects two things: what gets extracted now and how the agent learns for next time.
What changes when a rule changes
The hardest design problem wasn't the configuration layout. It was consequence visibility. Every rule in this system does two things simultaneously: it controls what gets extracted now, and it shapes what the agent learns for future extractions. A user who changes a rule without understanding both effects doesn't just affect one session. They affect every session until they catch it, and in a document processing workflow, catching it means auditing output, not reading an error message. The consequence modal surfaces exactly what stops, what pauses, and what gets retained before a user commits. Destructive actions require explicit confirmation. This wasn't a safety pattern added at the end. It was the structural requirement that shaped the entire page. The tab navigation keeping all three states visible at once came from the same logic: if users can only see one state at a time, they can't reason about the relationship between them.
Consequence modal showing the effects of excluding a field from extractions
Memory as a rule
One rule type stands apart. The memory rule draws from and writes to a list of known values used for fuzzy matching during extraction. Users can view and edit that list directly from the configuration page.
Beta and what it confirmed
The full feature shipped as beta. Scope was deliberate. Ship what real users could validate, hold back what needed more time. The rule-based configuration component did not exist in the library before this feature. Introducing a new pattern into a shared system is a commitment. If it fails with users, it creates debt everywhere it's been adopted. Validating it before broader adoption was the right call. Beta feedback confirmed the core architecture held. Friction surfaced in onboarding, users needed better instructions for interactions that weren't yet intuitive, not in the configuration itself. That distinction mattered: onboarding friction is addressable with copy and guidance. Structural confusion in the configuration would have required a rebuild. It held.