An AI agent that answers email: how we drew KAI’s limits

KAI is the AI member of our team. It appears by name on the team page, because we did not want to hide our use of AI in the background. But giving an AI a name is the easy part. The real question is how much work you leave to it.

Today KAI reads the emails that come into the company. It answers routine ones by itself and schedules meetings. For anything that is not routine, it only prepares a draft, and someone on the team decides whether to send it. This post explains where we drew that line and why.

Both easy options were wrong

Two options come to mind first. The first is to let the agent send everything by itself. The second is to have a person approve every reply before it goes out.

The first is clearly risky. A language model can make up a price or a delivery date in a confident tone. The other side reads that as the company's word.

The second looks safe, but it does not work in practice. We also covered this in another post on AI governance. In a system that asks for approval on every reply, the approval rate gets close to one hundred per cent after a while. The person approving stops reading the text. On paper there is human oversight. In reality the system makes the decision.

What we based the line on

When deciding whether an email was routine, we asked two questions. Is the answer known in advance? If it is wrong, can it be corrected easily?

If the answer to both is yes, the task is routine. Suggesting a time for a meeting, confirming that a document has been received and answering a general question with information from the website all fall into this class. If KAI suggests the wrong time, the other side corrects it in the next message and nobody is harmed.

If the answer to even one is no, the task is not routine. Every piece of correspondence involving price, scope, contracts, delivery dates or personal data is in this class. Here KAI prepares a draft, but the message does not go out without a decision from someone on the team.

The benefit of this split is that it keeps human approval rare but meaningful. Every message that needs approval genuinely requires thought. The person approving knows that what is in front of them matters.

Saying it is AI in the signature

The signature of every email KAI sends states that the message was written by an AI system. We do this for two reasons.

The first is honesty. The other side should not think they are writing to a person and trust the message on that basis. The second is regulation. The EU AI Act, Regulation (EU) 2024/1689, requires that people know when they are interacting with an AI. For our contacts in Europe this is an obligation. For those in Turkey it is a good habit too.

What it cannot do

KAI has no signing authority in the company. It cannot make offers, enter into contracts or interpret legislation. These limits are not only written in a list of rules. They are also built into the tools the agent can use. When you do not give an agent a permission, you do not need to trust it not to use it.

KAI reaches a language model through OpenRouter. This means correspondence goes to a service provider abroad. We explain this separately in our privacy notice. There we also ask people not to send confidential information, special category personal data or trade secrets by email.

Where this design struggles

The line between routine and non-routine is not always clear. "I can't make the meeting, could we do another day?" is a routine message. "I can't make the meeting because we are putting the project on hold" is not. Both start with the same sentence.

That is why the default behaviour in unclear cases is to prepare a draft. KAI does not treat a message it is unsure about as routine. Erring in one direction, that is, leaving work for a person when it was not needed, is cheaper than erring in the other.

What we carry into client projects

The criterion we use in our own work carries over to the systems we build for clients. We do not switch an agent's autonomy on and off with a single switch. We classify tasks by whether the answer is known in advance and whether a mistake can be undone. Depending on the class, the agent either finishes the task or leaves a draft.

This criterion is not perfect. But we can write down clearly where it stands, and a limit we can write down is a limit we can audit.