Quick question: Do you want AI to be so well trained, it could help husbands (or wives for that matter) to plan the perfect murder of their spouses? Probably not, right? Just as a gut reaction, that feels like a no. I wouldn’t even think it was a particularly hard question.
But America contains many diverse perspectives, and one such perspective was shared by Comma AI founder and longtime jailbreaker George Hotz over the weekend.
The post comes in response to a bunch of big-picture AI alignment plans, most recently the AI 2040: Plan A policy paper from the AI Futures institute. That paper envisions a world in which the world’s researchers collectively choose to slow down AI development for 14 years for the good of humanity. But of course, not everyone who read the paper agrees with its premises or conclusion.
Hotz is in the camp who disagrees. In his post, he argues that the fast-takeoff scenario — the hypothetical where AI rapidly obtains superhuman abilities — doesn’t make a lot of sense. I agree a lot of what he says here! For Hotz, the best approach to AI alignment and safety is to focus on locally controlled AI models that are closely aligned with the interests of their users.
That’s a cool idea, particularly since it reminds me how much of current AI is built around centrally managed services like Claude and ChatGPT. There are infrastructure-related reasons why AI services developed this way: it’s expensive to host these large state-of-the-art models and most people aren’t using them enough throughout the day to justify truly personal AI. But those factors become less important as the technology develops. Part of what was so exciting about OpenClaw was this experimental, DIY approach, and it would be great to see more AI products try to recapture that.
But Hotz is a provocateur by nature, so he doesn’t stop there. He compares user-aligned AI to a gun(!), which does not complain if you use it to kill your stepmom. (I feel like there are other rules against this?) A truly aligned AI would be able to order meth-lab equipment from Amazon Prime and show you how to use it if that’s what you wanted and asked for, he says. (Again, I don’t think AI would be the limiting factor here.) Hotz even says he would die to defend this principle, although it’s hard to imagine the series of events that would lead to that.
“We either live in a world with freedom or we don’t,” Hotz writes. If those are the options, the freedom world does sound better! Still, I don’t know.
It’s not all about freedom, right? Any structure involving a lot of people (societies, marketplaces, corporations, etc.) requires balancing equities, binding individual needs into a network of interdependent preferences and systems of accountability. And anyone deploying mass-market tech products should probably think about that network as a whole, which means taking seriously the interests of the as-yet-unmurdered spouses and stepparents of the world.
The freedom Hotz experiences is really a space of potential futures made possible by collective enterprise; those futures would disappear overnight if we all started behaving like little AI-powered Napoleons. Like the meme says, we live in a society.
Having a local AI willing to take on the corporate world for my benefit does sound cool though! I can’t wait for a review unit.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
