The assistant on this site: a public AI chat with hard limits

A demo answers. A public chat has to survive.

The assistant on this site, Aria, answers questions about my work, figures out what a visitor needs, and points them to the right page. As a demo, that is an afternoon: a prompt, a model and a text box.

As something anyone on the internet can use, it is a different job. Anyone can send it anything, as often as they like, and every message costs money. These are the decisions that turned it from the afternoon version into one I can leave running.

It answers from what is published, and nothing else

The assistant’s knowledge is built from this site’s own writeups, notes and service pages each time the site is built. When I publish something, it knows about it; when I retire something, it stops mentioning it. Drafts are excluded.

Its instructions tell it to say it does not know rather than invent a project, a number or a price. The only prices it may quote are the ones published on the site. Those instructions and that knowledge live on the server, not in anything a visitor’s browser downloads.

Spending has a ceiling

A public AI endpoint is an open tab on your bill. Usage is limited at more than one level, so no single visitor, script or unusually busy day can push the cost past a ceiling I chose in advance.

Those limits are part of the design, not an afterthought. If a safeguard cannot do its job, the chat stops and points to the contact page rather than running without it.

Inputs are bounded before they cost anything

Message size and conversation length are checked before anything reaches the model, and only the recent part of a conversation is sent. A long or malformed request is turned away before it is paid for.

Replies stream, and stop when nobody is listening

Replies arrive word by word, so the visitor sees progress within a second or two. If they close the chat, the generation is cancelled rather than finished and paid for.

Every reply has a time limit. Temporary provider errors are retried; a provider that is genuinely down produces a clear message, not a spinner. If a reply breaks halfway, the visitor sees that it failed and where to go instead, rather than an answer that simply stops mid-sentence.

What this says about AI features generally

None of this is the interesting part of an AI feature, and all of it is the difference between a demo and a product. The questions are the same for any AI feature in front of real users:

• What is it allowed to know, and what must it never say?
• What can one person, or one script, cost you?
• What happens when the model, the provider or your own infrastructure fails?
• Will the person using it know what happened?

If an AI feature of yours is live and nobody can answer those, send me a brief. The useful starting point is usually the bill and the failure path, not the prompt.