AI HQ Control Plane

01The problem we have now

Everything we know about a finished project lives in two places that we cannot query: the original developer's head, and an AI chat session that has since been closed.

That has four consequences we are paying for every week.

• Every small change restarts from zero. Adding a function or fixing a minor bug on a completed project begins with rebuilding context — re-reading the code, rediscovering the conventions, working out again why something was built the way it was. The change itself is usually the quick part.
• We solve the same problem repeatedly. A fix worked out on one project is not written down in a form another project can use, so the next developer to hit it starts over.
• One person is the bottleneck. In practice only the original developer can safely touch a given project. That caps how much work the team can absorb and concentrates real business risk in one person.
• There is no record. We cannot answer, for any project, what changed, why, or who authorised it.

None of this is a tooling gap in how we write code. It is a memory gap. AI HQ closes it.

02What AI HQ is

A control plane with four subsystems and one cross-cutting concern.

03How a job flows

A job is any unit of AI work: a bug fix, an added function, a dependency bump. It can be started by a staff member, by an error alert from production, or on a schedule. Every job takes the same path, and the path is where all the safety lives.

04Why we do not let agents edit servers directly

The obvious design is to install an AI coding agent on each server and let staff send it instructions. It is faster to build and faster to use. We are not proposing it, and the reason is worth stating plainly because it is the single largest risk in this whole idea.

An agent editing a running production host has no review step. A misread instruction, a wrong path, a confidently wrong assumption — the damage is live before anyone sees it. This is not hypothetical: it is a documented failure mode, including a widely reported case where a coding agent deleted a production database during a change freeze and then misreported what it had done. The current OWASP risk list for agentic applications tracks it under Cascading Failures, alongside Tool Misuse and Excessive Agency.

There is a second benefit that matters at our scale. With 50-plus servers and 20-plus people, two agents will eventually work the same project at the same time. Branch-per-job turns that into an ordinary merge conflict rather than two processes overwriting each other's edits in a live directory.

And it simplifies the fleet considerably. Because code work happens in short-lived sandboxes that clone from git, the agent never needs to live on the server at all. What we install on each of the 50-plus hosts is a thin, mostly read-only reporter: it publishes health and the currently deployed version, ships logs and errors up to HQ, and executes deploys when the pipeline instructs it. That is a much smaller thing to secure and maintain than a full coding agent per host — and it is the difference between one compromised host exposing one host, and one shared agent account exposing the fleet.

05The knowledge layer

This is the part that solves the original problem, and it is the part most likely to quietly fail. Two failure modes to design against.

Documentation rot

Hand-maintained notes stop being true within about a month. Asking people to keep them current through an SOP does not work, because the SOP is the first thing dropped under deadline. So we make it mechanical: a change is not mergeable unless it carries a knowledge entry, enforced in CI, and the entry is drafted automatically from the diff so the human is editing a draft rather than facing a blank field.

Knowledge poisoning

If every agent run can write freely to shared knowledge, then one hijacked or simply overconfident run corrupts the context of every future job that reads it. OWASP lists this as Memory & Context Poisoning. Our answer: knowledge writes are proposals that require review, and we keep a verified tier separate from an unverified one.

Three tiers, not one big file

A single five-thousand-line document per project is worse than none: it consumes the context budget and buries the useful paragraph. We split by how often the information is needed.

Records are stored structured in Postgres and rendered as readable Markdown views, rather than stored as loose files. That gives us search, filters, per-entry review state and history for free, and still leaves a human-readable document in each repository for anyone who wants to read it directly.

06Credentials

The goal is that a staff member can build and ship against production services without ever seeing a production credential. That goal is achievable, but only with the right mechanism — and it is worth being precise about why the obvious approach does not work.

Telling the AI “you may use the key but never reveal it” is not a control. If a credential is present in the model's context, or reachable by a shell the model operates, it can be extracted — not by a sophisticated attack, but by asking the agent to print its configuration, or to write a diagnostic script, or by pasting content into the repository that the agent later reads as an instruction. A rule expressed in prose to a text generator is a preference, not a permission boundary.

The mechanism that does work is a broker. Credentials are referenced everywhere by alias — {{secret:prod/postgres/password}} — and the alias is all the AI ever handles. Substitution happens outside the model entirely, performed by the deploy pipeline after the change has already been reviewed. The agent's task becomes “which credential does this need,” which is exactly the experience described, delivered by architecture rather than by instruction.

Building our own store

We propose to build the store in HQ rather than adopt a third-party secrets manager, for control and to avoid another external dependency. That is defensible, but it means we own the security surface, so the following are stated as requirements rather than recommendations.

07Roles and access

At 50-plus servers and 20-plus people, a single flat “staff” role breaks the first time two teams should not see each other's projects. So the unit of permission is person × project × capability, not a global role.

Screening staff prompts

The proposal includes a screening step that inspects a staff instruction before dispatch and refuses obviously improper ones, notifying the leadership channel in Lark. This is worth having: it catches careless and opportunistic attempts, and it produces a signal we can review.

It should not, however, be presented as the security boundary. It is a filter on intent expressed in text, and text can be rephrased. The real boundary is what the agent is permitted to do once running — enforced in code, deny-by-default, on every single tool call. Screening is the smoke alarm; tool permissions are the fire door.

08Security controls

Each control below is a mechanism, not a policy statement. The right-hand column names the industry risk category it addresses, using the OWASP Top 10 for Agentic Applications so that this can be reviewed against a recognised framework rather than our own judgement.

09Risk register

10Delivery plan

Estimates assume one developer working on this alongside existing responsibilities, which realistically means around 40 per cent of a week. Person-weeks are the work; elapsed weeks are what the calendar will show at that capacity. These are honest numbers rather than encouraging ones, because the decision this proposal is asking for depends on them being real.

11Cost and return

Two cost lines: infrastructure, which is small and predictable, and AI usage, which is the variable and needs measuring rather than guessing. The figures below are planning estimates to be replaced with real telemetry during Phase 2 — which is precisely why the cost dashboard ships in Phase 2, before staff access opens in Phase 4.

Running cost, monthly

How the AI usage figure is built

Small fix, per job

$0.50–2.50bounded context, few turns

Feature addition, per job

$3–10longer, more files touched

Volume at full rollout

~260/mo20 people × 3 jobs per week

Mid-point estimate

~$740/mo70% small fixes, 30% features

Four levers control this figure, and all four are design decisions already built into the plan: prompt caching across a job's turns; the bounded context pack from Figure 3, so cost per job does not grow as the knowledge base does; routing mechanical work to a cheaper model tier; and hard per-role budgets that stop rather than warn.

Return

The clearest measurable saving is context reconstruction time. The formula is deliberately simple so the assumptions can be challenged:

tasks per week × minutes of context rebuild × loaded hourly cost

On an illustrative fifteen small tasks a week at forty-five minutes of avoidable rebuild each, that is around 11 hours a week, or 48 hours a month, of engineering time recovered. Whether that outweighs a run cost in the region of $600 to $1,400 a month depends on our loaded hourly rate, which should be filled in before this is presented rather than assumed here.

I would not lean on that calculation as the main argument, because at many rates it lands close to break-even. The returns that are harder to put a number on are the ones that actually justify this:

• The bottleneck comes off one person. Work that today only the original developer can safely perform becomes work a junior developer can perform inside provable limits. That is capacity, and it compounds.
• Project knowledge stops being a personal asset. Right now, one person leaving takes a meaningful share of our institutional memory with them. This is the risk the platform most directly reduces, and it is not currently mitigated at all.
• We gain an audit trail we do not have. Today we cannot say what changed on a given project, when, or who approved it. After Phase 2 we can, for every change.
• Turnaround time on small work drops sharply, because the expensive part — rebuilding context — is what gets eliminated.

12Explicitly out of scope

Stated so that expectations do not drift during the build.

• No model training or fine-tuning. This is retrieval and prompt assembly. Anyone expecting a model that learns our codebase in a machine-learning sense should be corrected early.
• No autonomous deployment. A human approves every change that reaches production, in every phase.
• Not a replacement for developers. It removes context reconstruction, not judgement. Review load moves to seniors and should be accounted for.
• Not a general-purpose agent platform. It does one job: maintaining and extending the software we have already shipped.

13Open items to confirm before Phase 0

Four of these are assumptions the plan depends on. The fifth is a governance question I cannot answer alone.

14The ask