API and integration architecture for AI-enabled software

Start with the system of record and the decision boundary

Before adding an AI endpoint, map which system owns each important record, where the authoritative state lives, which events enter the workflow and who has permission to act. A generated summary may be useful, but it should not silently become the source of truth. An agent may suggest an action, but the target system should still enforce its own validation, authorisation and data-integrity rules.

Define the product boundary in terms of a user action. For example, retrieve permitted evidence for a case, prepare a draft response for review, classify an incoming request into a queue, or propose a change that an authorised user confirms. This keeps architecture decisions connected to real work. It also prevents a broad instruction such as connect the AI to our systems from producing an integration with unclear access, responsibility or success criteria.

Use narrow tools and validated contracts

Tool and API design should make safe behaviour easier than unsafe behaviour. Prefer small operations with a clear purpose, typed inputs, server-side validation, least-privilege credentials and explicit error responses. Do not expose a generic database query, unrestricted file store or powerful administrative endpoint simply because a model could call it. The application layer should validate requests independently of what the model claims it intended.

Where the workflow makes changes, use idempotency, confirmation and durable request identifiers where appropriate. An AI response may be retried, delayed or duplicated. Design for these normal distributed-system conditions rather than assuming an instruction is executed exactly once. Store enough correlation information to understand which input, user, tool call and downstream result belong together without logging unnecessary sensitive content.

Design failure and fallback behaviour before the happy path

Integration failure is normal: an upstream system may return incomplete data, a token can expire, an event can be delayed, a rate limit can apply or a downstream write can partially succeed. Decide what the user sees, whether the action is retried, how a duplicate is prevented and when work moves to a human queue. The fallback may be a read-only view, a manual process, a deferred task or a safe refusal. It should be chosen deliberately for the workflow.

For AI-assisted tools, separate model uncertainty from integration failure. An answer may be uncertain even when every API call succeeds; a tool call can fail even when the model reasoning appears plausible. Logging and user interface should make these distinct enough that the right owner can act. Otherwise a product team may tune prompts to solve an identity, data-quality or supplier issue.

Release integrations through evidence, not credentials alone

Before production activation, verify authorisation boundaries, representative input/output paths, validation failures, retry behaviour, observability, rollback or disablement and the support route. A test environment helps, but it does not remove the need for a carefully bounded production change. Name who owns the integration after release, how provider changes are assessed and how a user reports unexpected behaviour.

The release record should identify the integration version, permissions, service dependencies, data boundary, acceptance tests and known limitations. This becomes useful when a later provider update, schema change or incident occurs. It is also a more credible foundation for procurement discussion than broad claims that an AI system is integrated, secure or enterprise ready without showing which controls and boundaries actually exist.

A working check before committing the next stage.

• 01

  Map the system of record, user action and downstream side effect.

• 02

  Use narrow validated tools with least-privilege service identities.

• 03

  Document data classes, locations, retention and provider boundaries.

• 04

  Handle retries, duplicates, partial writes and unavailable dependencies.

• 05

  Separate model uncertainty from API or data-quality failures.

• 06

  Release with named ownership, monitoring and disablement path.