\n<\/p>\n
On Monday, Mark Zuckerberg published a 6,500-word manifesto<\/a> about personal AI, largely about the possibilities for the \u201cpersonal superintelligence\u201d systems Meta AI is building. The ideas in the post aren\u2019t totally new. A version of the essay ran in the Wall Street Journal<\/a> two weeks ago, and he\u2019s talked about them on Meta earnings calls before. But this is probably the most detailed version he\u2019s shared.<\/p>\n I\u2019ve seen it described as an anti-doomer essay, but that\u2019s not quite right. It\u2019s more of a description of why Mark Zuckerberg is personally excited about the ways AI is going to change society. Yet, even as Zuckerberg tries to paint a picture of the wonderful future abundant superintelligence will bring, he keeps reminding us of all the ways it\u2019s likely to go wrong.<\/p>\n I find AI exciting too \u2014 that\u2019s why I keep writing about it \u2014 but a lot of the public sees AI as creepy and unpleasant. So it\u2019s worth pinning down exactly what\u2019s happening here, and why Zuckerberg isn\u2019t doing the industry any favors with essays like this.<\/p>\n Social media is still a sensitive topic in the tech industry, and we don\u2019t have the space here to litigate the relative merits of every single complaint people have about Facebook. Suffice it to say, Facebook as a product and Zuckerberg as a person are both unpopular with the U.S. public. A recent survey<\/a> found that 64% of Americans believe social media has been harmful to democracy and a similar percentage<\/a> believe it should be more heavily regulated, numbers that cut evenly across partisan lines. Just this weekend, a court fined the company $567 million for being harmful to children<\/a>. The vibes are bad.<\/p>\n I don\u2019t bring this up to imply that Zuckerberg should withdraw to the wilderness in shame \u2014 but the fallout from social media is one of the central reasons we\u2019re now seeing so much anxiety about the social impact of AI. Whether it\u2019s fair or not, the public does not trust tech executives to make sure new technologies like this have a positive impact on society. Instead of acknowledging that and trying to win back their trust, this essay demonstrates over and over again how the trust was lost in the first place.<\/p>\n A large part of the essay is devoted to hazy generalities about intelligence \u2014 very similar to the hazy generalities Zuckerberg and Dorsey used to give about free speech. Here is one for instance:<\/p>\n As everyone gains more powerful tools, each person will become more capable of shaping the future, not less.\u00a0\u2026 People and institutions with competing interests naturally check and balance each other to lead towards positive outcomes. The best and most realistic path to building a positive AI future is by delivering superintelligence to everyone.<\/p>\n<\/blockquote>\n I guess? I can think of a few examples of conflicting interests leading to bad outcomes, but let\u2019s put that aside. The weirder thing is that this is being presented as an earnest philosophical conclusion, instead of a specific product called \u201cpersonal intelligence\u201d that Meta is bringing to market. If you\u2019re already inclined to distrust this person, you might worry that they\u2019re writing all this to convince themselves that nothing bad can happen.<\/p>\n That feeling got more intense for me when Zuckerberg got to the specific examples, many of which are alarmingly out of touch with the reality of how AI tools are being used. For instance, this is Zuckerberg\u2019s take on AI in education:<\/p>\n Everyone will have a personalized tutor and coach with a PhD in every subject\u00a0and unlimited patience to help you learn anything you want. Students will have extra help in areas they need it that is currently only available to those whose parents can pay. Adults will have a superintelligent learning assistant that knows exactly how to teach you new job skills, new languages, new hobbies, or anything else you\u2019re interested in.<\/p>\n<\/blockquote>\n Of course, the product Zuckerberg is describing already exists: This is a consumer chatbot like ChatGPT, Claude, and Gemini. These tools really are helpful if you want to learn about a certain topic, but the main way they\u2019re used in education is to avoid<\/em> learning, since your personalized tutor can do your homework and write your assigned essay \u2014 and because there\u2019s no robust watermarking system in place, there\u2019s no way for teachers to be sure which essays were generated with AI.<\/p>\n This is a pretty low-stakes example of AI harms, but it\u2019s a real thing that is happening right now<\/a>. Sometimes you design a technology to do good things and it has unintended consequences that make things worse. And the fact that one of the most powerful people in the world refuses to acknowledge this can make people understandably nervous!<\/p>\n Zuckerberg also gives an example of how AI will change the legal system for the better:<\/p>\n As a thought experiment, imagine only one person had a superintelligent lawyer. They would have an unfair advantage in court \u2014 even if they were wrong on the merits. That would lead to a worse society. But now imagine everyone has a superintelligent lawyer. In this case, justice would be carried out much more fairly and efficiently than it is today when there is often an imbalance in skills and resources in litigation.<\/p>\n<\/blockquote>\n Again, that\u2019s a loaded example. Sure, access to legal AI might lead to a more just society, but it also might add more complexity to the bureaucratic hellscape that already exists. Or it might unleash a new wave of vexatious litigants that just clog up the works with the legal equivalent of spam. It\u2019s hard to feel calm about any of this stuff, and the fact that Zuckerberg isn\u2019t<\/em> worried makes me more<\/em> worried.<\/p>\n In other places, even descriptions of Meta\u2019s existing business practices start to take on an oddly abstract character. For instance, this is how Zuckerberg describes Meta\u2019s commitment to a freemium model:<\/p>\n Everyone will have free or affordable access to these tools.\u00a0\u2026 We will offer free versions that will be accessible to billions of people.\u00a0For those who want to pay to use more compute, there will be a dynamic auction mechanism that will guarantee that everyone gets the lowest price possible for the intelligence and compute they\u2019re using while also ensuring the capacity is used for whatever people collectively find most valuable. This will ensure the benefits of superintelligence are distributed widely.<\/p>\n<\/blockquote>\n In broad strokes, I agree with everything he\u2019s saying here. Zuckerberg is right that access is an issue, and he\u2019s right that a freemium model allows for more access than requiring everyone to pay for compute up front. Dynamic markets for spot compute already exist, so it\u2019s not like he\u2019s imagining something completely outside the norm. But there\u2019s a reason every consumer AI product insulates end users from the spot price of the compute they\u2019re using. Surge pricing is a terrible user experience, particularly when it\u2019s deployed on a tool you actually rely on for your work.<\/p>\n Again, I have to assume that Zuckerberg knows this, and he\u2019s not actually planning to set token prices through dynamic auctions. But I\u2019m actually less sure of that now than I was before I read this piece.<\/p>\n There are better ways to talk about this stuff. For all their faults, Sam Altman and Dario Amodei do a pretty good job when they\u2019re directly communicating with the public. They do exactly what Zuckerberg resists here: acknowledging the dangers of AI, emphasizing their own precautions, and trying to convince listeners that they can be trusted. <\/p>\n That communication can only do so much. Sometimes (especially recently<\/a>), the safeguards fail and the inherent danger of the whole enterprise is made clear. But without some kind of trust-building, the industry may not always be able to survive those failures.<\/p>\n<\/div>\n When you purchase through links in our articles, we may earn a small commission<\/a>. This doesn\u2019t affect our editorial independence.<\/em><\/p>\n\n
\n
\n
\n