Production AI governance

Enterprise

AI features are production features when they can see tenant data or trigger writes. Use this checklist before enabling a real provider.

1. Define the data boundary

• List which fields can enter prompts, tool arguments, extracted documents, and model responses.
• Decide whether each field may leave the tenant’s network or must use a local endpoint.
• Keep prompts and responses out of logs unless the data classification allows it.
• Treat assistant conversations and tool-call records as user or tenant data.
• Define retention, export, and erasure behavior before collecting real turns.

The provenance guide records provider, model, prompt revision, and token usage without storing the resolved prompt or model response in the provenance payload.

2. Lock down access and writes

• Mount only the handlers and providers the app needs.
• Add descriptions and risk declarations to every handler the agent may see.
• Test the role-filtered catalog with a normal tenant user and an operator.
• Start in read-only or approval mode.
• Keep high-risk handlers out of unattended approval rules.
• Verify that accepted writes dispatch as the calling user.

Use Make your app agent-ready and the Assistant walkthrough for the concrete approval model.

3. Control providers and secrets

• Store provider API keys in the deployment secret store.
• Select providers per tenant only after the data boundary is documented.
• Set model, endpoint, and output-token limits deliberately.
• Use the provider matrix to decide between a hosted, compatible, local, or mock backend.
• Test timeout, retry, rate-limit, malformed-response, and provider-outage behavior.

4. Test before activation

• Use the mock provider for deterministic handler and integration tests.
• Keep golden inputs and expected outputs for pipeline steps.
• Draft policy or prompt changes before activation when the workflow supports it.
• Run a dry run without a domain write.
• Check the proposed output with the same schema and access rules used by the domain handler.
• Record a rollback procedure for prompts, policies, and provider changes.

5. Monitor the operating path

Track at least:

• request count, latency, timeout, and provider error rate;
• token usage and tenant cap consumption;
• prompt revision, provider, model, and step outcome;
• proposal, approval, rejection, and executed tool-call counts;
• extraction validation failures and pipeline dry-run mismatches.

Alert on provider failures and cap exhaustion separately. A provider outage is not the same incident as a tenant reaching its budget.

6. Keep a human fallback

AI output is a candidate. Keep a normal form, queue, or handler path available when a user rejects a proposal, validation fails, a provider is unavailable, or an extraction is incomplete. The application owns the final domain write.

Build your own governance layer

You can implement these controls with public Kumiko features: tenant-scoped config and secrets, custom handlers, jobs, audit, cap-counter, and your own provider adapter. Enterprise packages the common revision, approval, provenance, and evaluation patterns so each app does not have to rebuild them.