Adding the Boring Parts That Make It Real

The situation

The exciting AI feature worked, but there was no accounts model, billing, or isolation between customers. In Adding the Boring Parts That Make It Real, I wanted to capture the real operating situation rather than describe the problem in abstract terms. What mattered to me was the moment the existing approach stopped being sufficient in normal use, because that was usually the point where the next architectural requirement became obvious.

The naive solution

Defer the "boring" platform work indefinitely and keep investing in the model-facing feature. The naive option was attractive precisely because it was faster and easier to explain. I could have pushed further in that direction and probably improved the demo, but it would have optimised the visible symptom rather than the underlying system constraint.

The hidden constraint

None of the AI capability is sellable or safe to operate for real customers without accounts, org-scoped data, entitlements, and usage accounting. This constraint was easy to miss early because it did not show up in the first successful run. It appeared once the workflow had to survive repetition, failure, another user, another provider, or a return visit, which is why I now look for these second-order requirements much earlier.

The decision

Build the platform data model deliberately: users, organizations and memberships, sites and projects, plans and entitlements, domains, API keys, usage events, and deployment records. I treated this as an architectural or product decision rather than a prompt tweak. The aim was to create a boundary I could reason about later, instead of relying on model behaviour to remain consistent as providers, templates, and user requests changed.

The implementation

Scoped every site and project route through authenticated organization context, and centralized entitlement rules such as site limits and plan access. Usage records track model, tokens, cost, and BYOK status. I also tried to make the implementation concrete enough that the decision could be inspected in code and runtime behaviour. That meant explicit state, explicit policy, and observable transitions rather than a collection of hidden assumptions spread across prompts and handlers.

The tradeoff

A large amount of unglamorous engineering that does not demo well but is a precondition for charging money and onboarding a second customer. I did not treat the decision as free. In most cases I was deliberately accepting more engineering, state management, or operational complexity in exchange for a smaller failure surface and a product that could survive realistic use.

The evidence

Usage and cost data existed per organization, providing the foundation for customer billing and platform-level cost analysis. I use this evidence as a practical signal, not as a claim that the architecture is universally correct. The useful question was whether the new design changed what the system could reliably do under the conditions that had previously broken it.

The reusable lesson

Budget real time for accounts, tenancy, entitlements, and usage tracking - they are not an afterthought bolted onto a working AI feature, they are what makes it a product. The part I would carry into another build is the decision rule behind the implementation, not the exact technology choice. Vendors and frameworks will change; the underlying question about state, control, isolation, feedback, or user trust is much more reusable.

The consultant's questions

• If two customers used this today, is their data actually isolated, or just visually separated in the UI?
• Can you currently answer "what did this customer cost us this month" with real data?
• What assumption in this part of the system would become dangerous if usage or tenant count increased?
• What evidence would tell me this decision needs to be revisited rather than defended indefinitely?