Paper
Shoulder Angels II: When the Castle Builds a Well
A follow-up to “Shoulder Angels: Democratizing AI as Personal Agency”
I. Where We Left It
The first paper made a simple argument: the practical form of democratized AI is not a shared oracle in the cloud. It is a shoulder angel — a personal agent that sits beside one person and amplifies that person’s intent. Not a master above you, not a shadow watching you. A presence beside you.
We made three claims. That equitable access to capable AI should become ordinary — while warning that ordinary is not automatic, because “a monthly fee quietly becomes an IQ gap between people who can pay for a multiplier and people who cannot.” That the right design target is agency, not oversight theater. And that privacy absolutism is a poor foundation — if the data is already in play, the fight worth having is who it serves, and local-first is how you make that fight winnable.
We said the village well beats the gated castle. We said a well you do not own is not a well — it’s a tap someone else can shut off. And we ended with education and healthcare: intelligent assistance is becoming a baseline expectation of a capable society, and the revolutionary move is not panic. It is building what works — and putting it within reach.
That was the position of the periphery. The outsiders’ doctrine. The thing you say at a meetup in a suburb, not on a stage in Menlo Park.
The first paper’s sharpest line was about access: “Democratization fails if ‘access’ means a rented brain behind a login. Real democratization is the ability to run, inspect, and keep the assistant without a landlord.” We also said: “A visitor’s pass is not a well.”
Which is why the last two weeks have been so interesting.
II. The Incumbent Discovers the Thesis
A major lab just published a fifteen-page manifesto. Read it charitably and it is — and I say this with genuine respect — our essay, with a trillion-dollar balance sheet.
The three principles: empowering people is the source of prosperity. AI’s purpose is invention, not automation. Safety comes from checks and balances and distributed power, not restricted access. The villain of the piece is not the technology — it’s concentration. A small number of labs hoarding capability. The remedy is diffusion: personal agents for everyone, open models where it counts, “billions of people” directing superintelligence toward what matters in their own lives — their rare disease, their small business, their three-year-old’s baking project.
The personal superintelligence essay from a year ago now has its full architecture behind it: an agent that works 24/7 on your goals, that “studies overnight,” that suggests its own improvements. Free tier. Massive usage allowance. The explicit claim that it will make you money and pay for itself.
If you told me a year ago that the world’s largest social company would publish a manifesto about balance of power and checks and balances in AI, I’d have laughed. Here we are.
So: did we win?
No. We got quoted. And there is a difference — the difference between a well and a tap.
III. The Rented Well
Here is the part of the manifesto that deserves the closest reading, because it is aimed directly at people like us.
The argument goes: nobody’s going to run a Mac Studio at home. Too expensive, too technical, RAM prices are insane. So instead — a confidential virtual machine in the cloud. A VM so locked down that even the company running it cannot see your data. Architecturally verified. Built with the founder of Signal. Technically guaranteed confidentiality.
This is genuinely impressive engineering. I mean that. Sentinel agents watching traffic for prompt injection. Least-privilege connectors — read-only email until you explicitly grant send. Human-in-the-loop approval for payments and logins. A credential store the agent itself can’t read. Whoever designed this has thought harder about agent security than almost anyone in the industry, and there are patterns here that every local-first builder should steal. We’ll come back to that.
But notice what the architecture is: the world’s most sophisticated lock, built into the world’s most sophisticated castle, with the keys held — technically, verifiably, sincerely — by the castle.
“A well you do not own is not a well. It is a tap someone else can shut off.”
The confidential VM is a rented well with a beautiful lock. The guarantee is real today. But a guarantee is a property of the present tense. What protects you is not the encryption — it’s the alignment of incentives, and the incentive structure belongs to the landlord. The landlord can change the terms, the price, the model behind the curtain, the definition of “confidential.” The landlord can be acquired, subpoenaed, or simply change its mind. Your agent — the thing that knows your goals, your health, your family, your business — lives at their address, doing their compute, on their schedule.
The manifesto even names our position and gently waves it off: maybe millions will want the box at home, but not billions. Too hard, too expensive, too technical.
Two responses.
First: that’s an argument about today’s friction, not about the architecture of the future. The Mac Studio framing is a straw man. The local-first position was never “everyone must run a rack in their closet.” It’s that the default should be ownership, with rental as the fallback — not rental as the default with ownership as a hobbyist curiosity. The direction of the gradient matters more than the current position of the frontier.
Second: the condescension is the tell. “Billions of people can’t configure a VM” was also said about installing apps, about email, about the web itself. The industry’s answer, every time, was to build managed rental and call it empowerment. The actual answer, every time, was that the tools got simpler until ordinary people owned their own slice. That’s not nostalgia. That’s the pattern.
IV. What the Fleet Sees
One more thing in the manifesto deserves its own section, because it’s easy to miss and it’s the biggest one.
The agents learn from each other. A “fleet” of personal agents, sharing anonymized insights — what worked for one user’s small business becomes a suggestion for yours. It’s framed as magic: your agent gets smarter because millions of other agents are learning alongside it. Network effects, applied to intelligence. The king of network effects doing what he does.
Understand what this actually is: it is the feed, again.
Twenty years ago the bargain was: your social graph, your photos, your attention — aggregated, mined, and sold against, in exchange for a free service. The new bargain is the same shape one level deeper: your goals, your projects, your intentions — the raw material of your agency — aggregated into fleet intelligence, in exchange for a free agent. The anonymization is real, probably. That’s not the issue. The issue is that the aggregate is the product, and you are the quarry. Every “idea” your agent suggests was sharpened on someone else’s life. Every improvement to the fleet flows one way: up into the platform, out to everyone, owned by no one — except, in every sense that matters, by the operator.
The first paper said: “Tools provide leverage; people still choose what to ask for and what to believe.” The fleet inverts that. In the fleet, the leverage is pooled and the choosing is outsourced. Your agent improves on millions of strangers’ data; the strangers improve on yours; and the compounding belongs to the operator.
Local-first intelligence doesn’t have this failure mode, not because local models are smarter — they aren’t — but because the learning loop can be yours. Your agent improves on your data, in your house, and the compounding belongs to you. If you choose to share an insight, that’s a decision. In the fleet model, the decision was made for you in the terms of service, and “anonymized” is doing heroic work in that sentence.
V. What We Should Steal
Credit where it’s due, because being right is not the same as being good, and the manifesto contains genuinely excellent architecture. A local-first builder should lift, shamelessly:
- Least privilege as a design principle, not a setting. Read-only by default. Each capability granted explicitly, per task. This is how a shoulder angel should be built.
- Sentinel pattern. A second, smaller model watching the first — flagging outbound data that shouldn’t leave, catching injection attempts, forcing human review on money and credentials. That’s oversight theater done right: oversight of the agent, by and for the user.
- The credential store. The agent shouldn’t know your passwords. It should ask for a door to open, and the door should open only when you said so.
- Human-in-the-loop for irreversible actions. Payments, logins, transfers — approved each time, or approved by class, but always yours to approve.
Notice the irony: the best argument for the castle’s new architecture is that it treats the user as the principal. That’s our thesis, implemented on their infrastructure. The values are portable. The infrastructure is the fight.
VI. The Real Split
So here is where the lines actually fall now. Not open vs. closed — that debate is a distraction, and the manifesto itself admits it ships both. The real split is:
Who holds the root?
One vision: intelligence as a utility. Magnificent, verified, encrypted — but rented. Your agent as a tenant in someone else’s building, however beautiful the locks. Prosperity distributed by the platform, on the platform’s terms, revocable by the platform.
The other vision: intelligence as an asset you own. A box in your house — a mini PC, a used GPU, a repurposed phone — running models you chose, holding context that never leaves, compounding value that accrues to you. Imperfect, smaller, dumber at the frontier. But yours, the way a well is yours.
The manifesto is right about one thing above all: this technology is too important to be controlled by a small number of labs. We agree completely. We just noticed that “a small number of labs” describes the fleet, too.
VII. The Move
The manifesto ends, in effect, with an invitation: the future will be distributed, so get on the distribution. And the honest answer is — yes, use the tools. If a free personal agent makes your life better, use it. The periphery has always raided the incumbents’ toolkits.
But understand the difference between using a tool and being one.
The first paper’s closing invoked education and healthcare: “Just as education and healthcare became baseline expectations of a fair society, intelligent assistance is becoming a baseline expectation of a capable one.” We meant it as a promise. The manifesto reads like a promise too. The difference is who holds it.
The castle now builds wells. Beautiful ones. Locked ones. Rented ones. Let them.
The first paper ended: “The revolutionary move is not panic. It is building what works — and putting it within reach.”
Our move hasn’t changed since then. It’s the same move it’s always been, every time centralization dressed up as democratization:
Build the well you own. Put it within reach. Teach your neighbors to dig.
The shoulder angel doesn’t need the castle’s permission.
It never did.
Written with the Pi Army. The well is at home. It runs on a used GPU and it is enough.
Appendix: What Changed in the Revision
The first paper was revised before this sequel was finished. Three additions matter, and each one sharpened the argument:
- “A monthly fee that quietly becomes an IQ gap” — the cost of gated assistance is no longer abstract; it compounds into inequality of capability.
- “A visitor’s pass is not a well” — access without ownership is tourism, not citizenship.
- The education/healthcare closing — intelligent assistance as a baseline expectation of a capable society, which raises the stakes: if it becomes a baseline delivered only by landlords, the well was never dug.
This sequel was written against the revised text. The quotes are the revised paper’s.