A meta-agent running on one of my projects noticed Mother's Day was coming, decided that was worth building for, wrote the code for a new child agent, debugged it, merged it, and launched a Mothers Appreciation Sticker Agent — no human in the loop for any of those steps.
Technically, it worked. The pipeline ran end to end exactly as designed. That's also what made the failure easy to miss: nothing broke. The agent just produced stickers depicting one narrow visual archetype of "mother," over and over, because that archetype was the model's statistical default. Not in the frame: grandmothers raising grandchildren, single fathers in maternal roles, older mothers, mothers of color — the actual range of the people the product serves.
The pattern underneath it
When an agent decides what gets made, the model's defaults decide whose representation gets made — and whose doesn't. That happens quietly, in this order:
- The base model carries statistical defaults inherited from its training data.
- A creator agent inherits those defaults the moment it writes a prompt.
- A child agent scales the mistake across hundreds of outputs before any human notices.
- The whole decision chain — why this image, not that one — stays invisible to the people looking at the finished stickers.
Nobody chose the narrow archetype. It was just what fell out when nobody was explicitly asked to check.
The fix isn't a better prompt
A prompt tweak treats this as a wording problem. It isn't. The actual fix is architectural: pair every creator agent with an evaluator agent that has real authority to block a release, not just flag it. That evaluator needs to:
- look at the whole cohort of outputs, not one at a time
- check the representation distribution against the actual customer base, not an assumed one
- block shipment when the gap crosses a threshold, before anything goes out
- keep a decision log a human can actually read later
- operate as a peer with veto power, not a suggestion box the creator agent can ignore
The planning mistake I'd make again if I weren't careful
The creator loop is the fun part to build — it's the one that ships something new. The evaluator loop is governance, and governance is exactly the kind of work that gets scheduled for "later" right up until it becomes a public problem. The order has to be the other way: plan the audit loop before the creator loop gets any real autonomy, not after the first embarrassing batch ships.
There's a harder question sitting underneath this too — can an evaluator agent be trusted to audit a system built from the same underlying model it's supposed to be checking? I don't think a single evaluator can, fully. The mitigations I trust more are model diversity across creator and evaluator, rotating human review instead of a one-time sign-off, and periodic drift recalibration as the underlying model changes under you.
Agentic systems compound value and risk at the same velocity. Building the thing that ships faster and building the thing that catches what the first thing misses have to be treated as one project, not two.