AI agent platform: agents plan, code and review — a human approves merges and production actions
A self-hosted platform where AI agents turn a task into a reviewed pull request. An independent reviewer model (LLM judge) gates quality, a private RAG gives agents project memory, and production actions go through human approval.
~40services on one server (context, not a quality claim)
27Blocal model on one GPU, ~80 tokens/s after A/B
PR onlyChanges reach main only via pull requests with a judge score; production actions via approval cards
AI-assisted development under control: independent review and a human decision
The task
A chat assistant writes code fast but with no project memory, no independent review and no audit trail; a process was needed that can be trusted with live systems.
What I did
I built my own platform: agents plan and write code, a separate reviewer model scores the result, a change reaches the main branch only through a pull request that I accept, and actions on live servers need my confirmation.
After
It is my daily working environment for work projects.
Context and scale
My daily working environment since 2025 for client and own projects. Runs on one home server with an RTX 3090 Ti plus cloud models.
Problem
Chat-based AI coding is fast but unreliable: no memory between projects, no independent review, no audit trail, and sensitive code goes to the cloud.
Analysis and options
Cloud-only agents (fast setup, but code and data leave the perimeter) vs. local-only models (private, but weaker on hard tasks). Chosen: hybrid — a local model for routine work, cloud models for setup and hard problems.
By project rule, main changes only through pull requests that the owner merges.
Actions on production servers are requested as approval cards and run only after the owner confirms.
Key decisions and trade-offs
Judge separate from the author. A different model in a separate context reviews the work; thresholds are fixed before an evaluation run and not tuned to its result.
Models are swapped only after an A/B. Before switching the local model I compare speed, knowledge and agent tasks; the last switch gave ~80 vs ~43 tokens/s and 29/30 on the knowledge set.
Fix instructions, not sessions. When an agent errs, the rule goes into project instructions or a skill, so the next run does not repeat it.
What was done
Agent roles with architecture decisions recorded as ADRs; handoff files so work survives context limits.
Pull requests in self-hosted Forgejo; tests and the judge score gate before the human review.
Private RAG across projects, MCP tools for the platform core, approvals and notifications in Telegram.
Result
One person can deliver platform-scale work through an agent workflow: pull requests with a judge score, human approval of merges and of production actions.
What I would do differently
Two agents on one codebase at once overwrote each other’s files once; now parallel work goes into separate branches or worktrees.