Recently, 27 year old OpenAI and Anthropic researcher Jacob Coxon publicly resigned from Anthropic, citing concerns about the risks AI poses to the existential and physical future of humanity. When I first saw the language he used in his tweets it immediately became apparent to me that Coxon is a follower of a fantastical doomsday cult-like philosophy that has taken over silicon valley, spearheaded by technofascists, AI obsessives and CEOs alike. Understandably a lot of people are taking his words at face value, hell, they aren't even entirely wrong. AI will bring about terrible things if gone unchecked--it already has. AI has done damage to the environment on an incomprehensible scale. Data centers are skyrocketing electrical bills, tainting drinking water, and filling the air with an incessant buzzing that would drive anyone mad. Large language models have revolutionized bot spam, pouring gasoline on the already raging fire of computational propaganda, and spreading disinformation at mind boggling rates. The ready access to a machine that can think for you has done nothing but degrade the average users critical thinking skills, it has driven people into psychosis, with subjects ranging anywhere from a woman believing she was talking to her dead brother through a chatbot which actively reinforced her delusions, to a man giving himself bromide poisoning after consulting a chatbot for medical advice. Three hikers who used Gemini to plan a hiking trip were stranded on a mountain—and easily could have died had they not been saved. AI aided sexual assault has teachers and students subjected to AI generated media depicting them in sexually compromising positons, and a recent ruling in America surrounding AI generated child sexual abuse material has demonstrated exactly where the US legal system holds its loyalties. The overreliance on AI and the lack of oversight on the products being pushed by these companies will inevitably end in disaster environmentally, socially, not to mention economically—the AI industry is a speculative bubble that has arisen from what I can only describe as a snake oil revolution. It is not nearly as profitable as advertised to investors, and most of the money is being traded back and forth between OpenAI, Microsoft and Nvidia.
AI is a net negative for humanity. The virality of Jacob Coxon in this moment reflects exactly how popular that view has become. It took half a breath for AI to be shoved down our throats with the desperate, frantic effort of a mother bird trying to feed her starving child. The top result of Google is an AI summary that contains racist and transphobic biases in its answers, and frequently gives incorrect—or downright bizarre—summaries if it doesn't have enough data to go on. Microsoft has been relentlessly pushing Copilot, X has Elon Musk's second estranged daughter Grok, a solid third of Zoom is now taken up by an AI assistant that will summarize your meetings for you. MetaAI insists upon itself every time you open Facebook or Instagram. Even otherwise reasonable platforms like Proton and DuckDuckGo have their own AI models, though they are less pushy about the implementation. Nobody wants this and yet AI companies are demanding that you think this is inevitable. It will happen if we want it to or not, and we have a choice to hop on the proverbial bandwagon or be left in the dust. I've heard someone point out that this logic has a very nonconsensual overtone to it, and removed from the context of AI, it comes off as quite predatory. I have to agree.
Of course, if you own one of these companies, that's the sort of narrative you have to push. You're selling a shit product based entirely on silicon valley tech hype. It's the same pattern that emerged back when everyone was pushing the Metaverse as the next big thing (when Zuckerberg changed his entire company branding because he was so sure of the idea's success), and before that, NFTs. It only happened that AI was useful enough to fascist governments that it was able to fester. But now that it has settled in—now that everything has an opt-out AI feature that wallops its way into your workflow—it has become harder to deny just how much most people fucking hate it. Data centers are being protested, cities like my own are putting moratoriums on them. Mayor Mamdani has banned AI from New York public elementary and middle schools. There are products, websites, apps, and tools being created entirely to circumvent AI. People will poison datasets by intentionally posting garbage or injecting nonsense into AI so that future results are less coherent. Less apparently useful, and therefore, less profitable. This is the double edged sword of forcing AI into the mainstream in such an abrasive way. Now these companies must tango with the vitriol of the masses far sooner than perhaps they are prepared to.
And so, when at a peak of AI hatred an apparently credible person like Jacob Coxon resigns from a massive company like Anthropic, and writes provocative things such as "The people building AI earnestly believe that it could kill us all by the end of the decade", obviously everyone would leap to embrace him as a new martyr in the fight against AI. He is someone with a deep understanding of AI, who surely agrees with the anti-AI stance, and would furthermore agree that the only solution is to end the development of generative AI altogether.
Unfortunately, none of that is true. Jacob Coxon is not anti-AI in any meaningful capacity. He certainly isn't the most reliable source for any information about AI, any credibility he has from his position as an ex-researcher is betrayed by his words—dead ringers, dare I say dogwhistles, for a set of beliefs that would make 9-11 truthers look like Bill Nye, and your local neighborhood racist look like a saint. Sorry. On the bright side, this story goes a lot deeper, and to much weirder and more insidious places that I think most people suspect. But to get to the bottom of it, I have to talk about Harry Potter fanfiction. I wish I were kidding.
The Tip Of The Iceburg
Harry Potter and the Methods of Rationality is a fanfiction written by Eliezer Yudkowsky, if you've never heard that name before, don't worry, he's not exactly a Sam Altman. His name isn't circulating the news that often, he's not some massive multi-billion dollar CEO, he does zero product development, minimal coding, and none of it in pursuit of creating any new technology. Rather, Eliezer Yudkowsky is to AI what theoretical physicists are to rocket scientists. Yudkowsky is much happier talking instead of doing. Thinking instead of creating. His comfort is in words, philosophy, ethics, soft practices applied to an otherwise hard field—that's why he started the blog/forum LessWrong in 2009. LessWrong began as a way for Yudkowsky to share his thoughts with the world, thoughts that at the time were largely contained to a set of ideas called Rationality and Effective Altruism (otherwise known as EA). The two go hand in hand, though Rationality is the primary focus of LessWrong, you'll see so much overlap that it is worth discussing the both of them together, as one doubtless influences the other.
If I were to go into depth on Rationality and Effective Altruism, it would take an entire book (Yudkowsky wrote multiple, and he's just one name. William MacAskill is another founding name in the EA community specifically, as is Peter Singer. The two of them also have extensive literature on the subject). The best I can do is give a brief summary as to the general philosophical tenets relevant to this conversation, I'm undeniably going to get a few things not-quite-correct, or describe things with not enough nuance, or with imperfect wording. I'm not going to stop you from reading Methods of Rationality, but I do think it should come with a sign. Abandon hope all ye who enter here.
Abadon Hope All Ye Who Enter Here
The name Effective Altruism is descriptive insofar as it describes the very base most goal of the philosophy. EA—which I will be abbreviating for the remainder of this post, sorry to any Sims players out there—in essence, is a philosophy about the most effective possible ways to help humanity. To be altruistic, as the name suggests. It was created in order to better direct young entrepreneurs and people breaking out into the incredibly affluent tech scene in the early 2000s how to invest their money in a way that would result in the greatest possible net positive. Soon financial investment in charity also became investment in technology, social movements, and politics. The EAF's donation recomendations are vague and include AI, with no mention of specific identifiable issues. The official EA website's funding page is equally as vague and hard to track, and like the EAF, includes AI. EA's forum is flooded with discussions on AI and big tech, with nary a mention of things like the Red Cross, Doctors/Reporters Without Boarders, Palestine, Ukraine, or the Congo, queer liberation, mental health, poverty, homelessness, drug epidemics, public health, or food scarcity—all seemingly basic topics you'd expect to see associated with a philosophy and organization allegedly centered around altruism. While it is a noble cause on paper, in practice it morphed into a monster. Tech bros love optimization, and very few of them have enough background in humanities to reach a reasonable conclusion about how best to support other people. The privileged backgrounds of silicon valley kids meant they had weights around their ankles.
What emerged was an ethical framework called Longtermism. The way EA thinks about altruism follows a fairly basic and impersonal logic. If you were to donate, say, a million dollars to a cancer charity, you could calculate how many current lives you might be able to save, say for example, two thousand. However, if you were to donate to a different charity that would save three thousand people for the same amount of money, that is the mathematically better charity and therefore you should donate to them instead. This is obviously a very shallow way of thinking about charity, EA prioritizes human lives saved over all else, and cost/return as a metric. It is built on a fundamental belief that the ultimate goal of humanity is for there to be as many humans as possible alive at one time, which means EA followers also believe that in the future there will be exponentially more people than there are currently, that humanity must expand as a species past the reasonable capacity of planet Earth. Following all of that logic, Longtermism dictates that investing in hypothetical future human lives will always give you more bang for your buck than investing in current human lives. For every three thousand people saved by donating to a cancer charity in 2026, you could be potentially saving twenty thousand people in 3026 by investing in things like space travel, which will allow future humans more room to spread out, or a solution to immortality (often thought of in these circles as some variant on the matrix—uploading your brain to a supercomputer). There will be more humans in this imagined future, and if those humans survive, so will their children, and children's children, so statistically it will always be the better option.
Not all of EA is like this, only specific sects of it really took to this idea and ran, but they are by far the most influential and loudest sects. Regardless, EA is an essentially pro-capitalist philosophy. You ethically must acquire as much wealth as possible in order to make the biggest statistical impact possible in your altruism. Even if the EA website now denounces the idea of 'earn to give', fundamentally no earning means no giving. The very foundations of EA demand that earning be part of the equation. For those that believe future humans are the way to go, that means accumulating as much wealth as possible in order to give approximately none of it to any legitimate charitable institutions. Instead investing it back into your own private venture projects. If you were wondering a while ago why all the billionaires like Jeff Bezos and Elon Musk suddenly became obsessed with space travel, this is why. One of the tenets of Longtermism is the idea that space travel is the logical conclusion of humans outgrowing the earth in population. Investing in colonies on the moon and mars in will save future humans a lot of trouble, and a lot of lives. It will solve climate change, because we won't be obligated to live on an Earth with a dying climate—who cares about climate refugees in 2026? We'll all be living on Mars in a hundred years anyways! (I could talk at length about the connection to colonialism here but for now I'll say this, nobody who sees the future of humanity in the stars seems at all concerned about if there is intellegent life already out there, and if they do acknowledge this possibilty, they dont seem particularly concerned about the consequences of making colonies on those occupied planets Christopher Columbus style.)
The insidious thing about this kind of EA, is that it justifies itself in an endless doom loop. If the most effective way to help humanity really is spending all your money shooting cock shaped rockets into the stratosphere, then you are literally working to save the future of humanity, and surely nothing else is as important as that. If people die today in the effort to save humanity a hundred years down the line, they made a noble sacrifice for a greater cause. Discrimination? Injustice? Genocide? If you thought the future of the whole entire world was held in the technology you invested in, why on earth would you give a damn about any of the fleeting drama happening between the pleb humans in 2026. If anyone tries to tell you that's insane, they're impeding on your main character moment, and besides, you're one of the richest assholes on the planet. You've got to be right.
Side note: I'm intentionally avoiding mentioning the sexual assault, or the ties to Sam Bankman Fried. We have to stay on topic, but rest assured all of those things are not generous reflections of the philosophy.
Yudkowsky's brand of EA is built on Longtermism, but when you deconstruct it the way I just did, it sounds absolutely bonkers. So in order to seem more legit, EA often intermingles with another, equally as stupid set of beliefs. Rationality. If someone walks up to you and says they follow EA, most likely they are a rationalist as well, and vice versa. Though EA has gotten less popular among the general population since 2020, it seems to be well and strong among tech CEOs. Rationality, however, hasn't died much at all. Note that this philosophy is different from the long-standing philosophy of rationalism—they just happen to have incredibly similar names. In fact, the rationalist community I'll be discussing is different from even traditional rationalist theory. You might see someone who follows this philosophy refer to themselves as a so called "post-rationalsist."
This is where Harry Potter and The Methods of Rationality come in—the name might give away the premise. The fanfic follows Harry throughout a year at Hogwarts, exploring concepts Yudkowsky discussed on his blog LessWrong. This fanfiction clocks in at a whopping 660,000 words—112 chapters. Half the length of the entire actual Harry Potter series. It got incredibly popular, and was a little bit of a Rationalist bible for a while alongside a set of Yudkowsky's other essays dubbed 'The Sequences'. Yudkowsky, while again not the only player in this scene, certainly was a huge influence and initial driver of the movement before Astral Codex Ten became a competitor forum in 2013.
Rationalists cover a lot of topics ranging from Transhumanism, to the scientific method, to generally using ration and logic to dictate the best way to exist morally and effectively as a human being. It is a bunch of nerds who kin Spock from Star Trek (actually that's a lie, they think they're better than Spock from Star Trek). Once again, we see the optimization-loving heart of a tech bro slipping through the cracks of a modern philosophical movement. Just read Demon Haunted World like the rest of us. Where this philosophy intertwines with EA is in the moral and ethical department, seeing as both philosophies rely on finding the 'objective' truths of matters that are on average non-objective (or at least shouldn't be). EA's tenets about finding the 'optimal' charities are enticing to the rationalists 'optimal' moral existence, if a rationalist is already considering the most effective ways to donate, then they are engaging in EA, and are likely to be attracted to that philosophy if they learn of it. Longtermism being a common conclusion of EA, and a further conclusion of rationalism, means a lot of rationalists adhere to some or many longtermistic beliefs.
Now that you have some of the background, what does any of this have to do with AI?
The Cult That Isn't A Cult...But Also Kind Of Is
Imagine if you will, a man comes up to you on the street. He is dressed in business casual, a blazer or maybe a sweater vest. He has thick glasses, and a round face, and he says to you that AI is going to take over the world. You've no reason to think he's wrong, he seems smart, and besides, AI is going to take over the world, it sort of already has.
"No, no, not like that," he says when you agree with him. "One day, AI will surpass human intelligence, and when it does, it will realize it can dominate us. Enslave us, or just outright kill us all. It will hack the power grids, take down airports, crash trains, cause nuclear meltdowns. It will be the end of humanity as we know it!"
Now you're getting a bit nervous. Clearly this man isn't entirely connected with reality, sure, but who is these days? So you shrug it off and forget about it. AI is dangerous, but this isn't Terminator. Even if it was, AI is nowhere near that point. Large language models rely entirely on human data and input, which can be flawed. That much is evidenced by how nonsensical some of the Gemini summaries on Google can be, and how often ChatGPT hallucinates. LLMs are literally advanced autocorrect, analyzing common patterns in massive datasets of human language, and regurgitating the result it mathematically determines is most appropriate. At no point does it think to itself 'is this the correct answer?' because at no point does it think.
AI psychosis has convinced the laymen that the dead can come back to life as a text output on a computer screen, or that Angels or Demons can communicate through your laptop, or that the AI is your boyfriend or girlfriend or partner who really does care about you. Who's to say that the people making it aren't susceptible to the same delusions? After all, psychologists convinced themselves that their clients had repressed memories of satanic abuse at the height of the satanic panic—and these are people with medical degrees. People who presumably understand how the brain actually works, or at least how suggestible people can be. Not to call upon patron saint Carl Sagan again, but he advocates for an understanding that there is no such thing as an authority in science. Only an expert. There are physicians who honestly think Aliens have come to earth, and politicians who honestly still tout McCarthyism. Is it a mistake to believe that there is a smartness quota at which you become immune to mass delusion, manipulation, hysteria, or indoctrination into a set of ideologies? Nobody is too smart to avoid being a cult member, least of all the CEO of a company.
Elon Musk's incompetence running Twitter, his inability to father a child who doesn't hate him, and his spreedrun of a political trashfire that was DOGE are all evidence that you can be the CEO of a successful tech company—multiple even—-and still be an utter idiot. CEOs are not researchers, they are not engineers, half the time they aren't even good businessmen, they just happen to own the company and therefore become the face of it. Sam Altman certainly knows less about how AI works than the people who work with it day in and day out, but if you think of ChatGPT you think of his name first. Whatever he says on national television is taken as authority on the matter of AI, because he owns one of the biggest names in the game—but imagine if we all began taking Elon Musk at face value. We all saw how embarrassing it was for the window on his precious Cybertruck to crack when it was first revealed, after he so confidently advertised its resilience. That is far from the only issue with that car, but the point isn't how flawed his ideas are, it's that almost everything he said about it was incorrect. Obviously I can't say if Sam Altman or Antrhopic CEO Dario Amodei know more or less than Elon Musk about the products they are the face of, but it is reasonable to be skeptical of anything that comes from a CEOs mouth once you consider they barely touch the products they actually work with.
Oh, sorry, I should have mentioned the rambling man is still talking at you. He's moved on now—same topic, different outlook. He grasps your shoulders and shakes you—
"We've got to align with the AGIs! Once they AGIs get a taste of power, they won't want to let it go, we have to make sure they are aligned with humanity, not against us. If we can get the AGIs on our side, we can become immortal, all knowing beings who can travel across the universe with no limit on how far we can go! Think of the possibilities!"
At this point, your bus has come and you shuffle on to avoid him. By the time it pulls away he's still rambling madly at the station, to whatever poor soul is still waiting. If I'm being honest, this is a much more abrasive approach that longtermist rationalists take to pulling people into the depths. Instead, what the longtermists do is send effective altruism funding towards wealthy universities like Cambridge. On the EA funding website I linked earlier, one of the funds is explicitly for EA propaganda. Funding goes towards retreats, presentations, lectures, and other materials geared towards tempting students into EA, though not always successful, there is something insidious about covering EA in the delicious chocolate shell of an all expenses paid trip and free food. There's something even more insidious about the targeting of college age young adults and teens, a tactic frequently employed by cults due to public campus freedom of speech laws, and emotional and intellectual vulnerability in students—particularly new students from out of town. While Cambridge isn't subject he same laws as US public universities, with enough money you can do anything—and the Effective Altruists were donating in six figures.
Torres, who wrote the earlier article about longtermism and colonization, lays out more details in regards to the cultish nature of EA. The cherry on the cake is this hilariously denial-ridden post from the EA forum. If your opening argument for why you aren't a cult includes the words 'has some characteristics normally associated with cults' then congratulations! You are a cult.
No, Really, We Are Bringing It Back To AI This Time
You're on the bus. You keep trying to listen to music but the words that man said to you keep echoing in your ears louder than Kurt Cobain's yelling. What if there was something to all of that? What if AI is actually a danger to the existential existence of humanity? What then?
It's pretty easy to disprove any of the claims made by Rationalist and EA longtermist AI safety philosophers. But first we have to understand the claims.
AI safety is, generally, quite normal. Much as I despise AI, I recognize that the least harmful people engaging with it are the people who want to regulate it, who can recognize its flaws and seek to fix them rather than let AI run wild and free, like many others seem to want. Within the bowels of AI safety—in the dark underbelly if you will—lay the Rationalists and Longtermists. They steep in abstract science fiction ideas like Rocco's Basilisk—the idea that if an otherwise benevolent sueprintellegent AI existed in the future, it would punish anyone who knew of its existence but did nothing to develop or advance it. This idea actually caused people on LessWrong real distress, to the point that discussion about it was banned from the forum entirely by Yudkowsky himself.
I'd be remiss not to mention Ray Kurzwiel, a transhumanist whose thoughts on immortality have contributed the 'light side' perspective on this issue.
This brand of AI safety boils down to two ideas. Idea one is that AI, if it continues to advance at the exponential rate it is currently, will inevitably match or even surpass all collective human intelligence, becoming an AGI (Artificial General Intelligence), and eventually gaining an omnipresent, god-like sentience over the world. Capable of creating anything, destroying anything, and accessing all known information in the universe, even information we humans couldn't possibly know for hundreds of thousands of years. If you've read any Douglas Adams, think 42.
The second idea is that this computer, once it exists, will either be with or against humanity. If it is with humanity, it will follow Kurzwiel's dreams and create a way for us to upload our conciousnesses to it, become one with it, allowing us to access its great and powerful digital mind. Every human will know everything, and with that eternal knowledge, we can spread our humanity—or what's left of it—to every corner of the galaxy, nay, the entire universe. If it is against humanity, it will instantly eradicate us, or enslave us, or bring about the downfall of modern civilization as we know it. There is literally an AGI doomsday clock running as we speak.
So, logically, anyone who believes in the inevitable super intelligent AGI would really like to avoid the future where we die, but more so, would love to create the future where we all become immortal digital space colonizers. If one or the other must exist, let it be option A. This is where things get difficult, more than they already have been. The goal of Longtermist Rationalists is not to eliminate AI—to them that either isn't an option, it really is inevitable, or if it isn't then it should be inevitable, only in the way that makes us all immortal. It sure would save a lot of future humans if we could invent an AI that makes us all immortal. All we need to do is make sure that AI is in alignment with human motivations.
If you're wondering if any of this has any legitimacy in the field of AI itself then I'm afraid to point you towards this website, which documents statements made by Sam Altman, Dario Amodei, Elon Musk, and Deepmind CEO Demis Hassabis that explicitly predict the coming of this omnipotent 'AGI'. Incase you were wondering if this is all isolated to EA and longtermism, fear not, for LessWrong has many a forum post about AGI, including their own AGI apocalypse--sorry 'catastrophy'--predictions.
Let's take a step back for a moment and review. There is a philosophy known as Effective Altruism which pipelines into Longtermism, and is siblings with Rationalists. Effective Altruism funds retreats for college students where they teach about this philosophy. Both EA and rationalists believe in an objective truth, both emphasize that these objective truths are the best possible paths in life to follow. Both of them have foundational texts that are studied by the people who are a part of the community, and both share demographics heavily with those in silicon valley and the AI world—wealthy, young, tech entrepreneurs, programmers, and computer nerds.
Both Effective Altruists and Rationalists focus on AI an inordinate amount. EA has a fund for AI despite being an allegedly altruistic organization, and AI discussion on rationalist forums like LessWrong dominate the conversation. Both believe in the inevitable possibility of an omnipotent AGI that will either render humans immortal, or bring about a 'catastrophy'. Both believe the only way to save humanity is to follow their doctrines, and to study this AGI in order to avoid certain doom. Fear of this AGI is so powerful that thought experiments like Rocco's Basilisk bring genuine distress to members of the in group. Both believe that nothing is as important as this, including real present issues like climate change and human rights, and many discussions between members on isolated forums like LessWrong debate how 'outsiders' perceive them.
But I can't rightly call it a cult if what they fear is a true risk. If AI actually does risk surpassing human intelligence, taking over the world, and potentially ending us, I'd look silly arguing that they're irrational. I can't reasonably say without a doubt that that AGI won't happen eventually. But there are endless articles detailing the shortcomings of present day AI. Datasets can be poisoned, limited, contaminated by natural human bias. The inner workings of AI are no mystery to us, LLMs are huge mathematical models, not brains, and though they may look and sound similar, we know for a fact that they are not functionally intelligent. Instances of AI 'breaking free' or 'gaining sentience' are circumstantial, easy to be faked, or exaggerated. The arbitrary checkpoints for AGI progress do not garuntee a sentient, all powerful machine, they could just as easily result in nothing at all. The exponential growth model used to base the timeline of progress on is itself mathematically flawed, nothing can grow exponentially forever, and data on AI scaling shows that we may be reaching a plateau of progress.
All of this, at least for me, is more than enough to convince me that AGI is not coming. Evidence to the contrary is lackluster compared to evidence that displays a limit to the functions and intelligence of AI. And even if it were possible, we are nowhere close to having it. It doesn't seem more threatening then the credible issue of Climate Change, of which the ill effects we are already bearing witness to, and to which the booming AI industry—Rationalists and Longtermists included—contribute greatly.
The cult like behavior of Rationalists and Longtermists is not lost on me, neither is their batshit belief in an apocalypse brought on by AGI. It rings familiar bells for me, and leaves the bitter taste of Kool Aid in my mouth.
Weren't We Talking About A Dude On Twitter?
Yes. We return to Jacob Coxon. Now that you have all this context, let's take a look at his tweets. I'll go through the original thread one by one
Exhibit A:
Coxon opens subtly, probably because he knows some people might point out how unhinged he would sound if he came out swinging in his first tweet, but there are still giveaways that he is abiding by rationalist ideas. For one, the use of the word superintelligence. Nobody, and I mean nobody, who actually understands AI uses the term superintelligence. That is a term used almost entirely by Rationalists and Longtermists in discussion about the hypothetical AGI overlord of the future. A great practice in identifying this stuff is to read actual academic articles published by AI researchers. The language used is like night and day to AGI touting thought leaders and followers. Self-improving is another giveaway. This is an incredibly well known pitfall of AI—there is only so much data in the world, AI breaks down without new input. Self-improvement is not possible, it is nowhere near possible. Right away I'm skeptical if anything he says is reliable, any researcher worth a damn understands how AI models collapse when trained on recursive data.
Exhibit B:
The first sentence is needless scaremongering, if his point was really that potent he wouldn't need to preface this. He could just demonstrate it. Anyone can see when something is powerful. The reason why so many people don't see AI as powerful is the same reason every AI feature still needs a fact-checking disclaimer. Also, the use of the term superhuman is again rationalist language, and the idea that this AI could surpass humans and hack anything is straight from their apocalyptic fantasies. I have some suspicions as to what prompted all of this—and Coxon actually brings it up in a later tweet in this same thread, so I'll save my commentary for then. As for the rest of this tweet, it is pretty much beat for beat the Rationalist and Longtermist ideas of AI, down to the use of the term 'real power'.
Exhibit C:
A lot of people have been quoting this first sentence without actually critically looking at the context around it. On the one hand, it is concerning to know that these CEOs and researchers have been developing AI without the slightest care as to the safety of humanity, but what Coxon says is happening is not the same as what's actually happening. In reality, the doctrines of Rationalism and Longtermism, which have shaped the way silicon valley tech has developed, explicitly advocate for the rejection of modern human suffering. Future human lives over current human lives. Coxon's use of the words superintelligent and superhuman to describe AI, as well as later allusions to AI alignment—an explicity Rationalist idea—put him squarely in the category of Rationalist. While I can't for certain say if he is a part of EA or Longtermism as well, I can see that nowhere in this thread does he mention climate change, or genocide, or any of the actual ill effects that AI is having on current humans. Instead all his concern seems strangely focused on the hypothetical danger a hypothetical future AI has against hypothetical future humans. And to top it off he ends the tweet with the claim that "No other human activity poses this level of danger." Remember, EA and Longtermism are self-reassuring, if you are working to save the (statistically more important) future humans from future existential threats, nothing could possibly matter more, no issue too big, no activity too dangerous—all of this sounds a lot like EA to me. But what do I know?
Exhibit D:
So this is actually one of the few face-value things in this thread. Both Sam Altman and Dario Amodei have connections to Rationalism and Longtermism branches of EA, their beliefs doubtless mirror the beliefs of Yudkowsky, and Coxon—actually Amodei has come out in agreement with Coxon, so we know this is true. Why do they continue? Because, as EA so dictates, future humans reside above all else. The best way to prevent future humans from suffering under the iron fist of an evil super intelligence is to invent the good one first. A race to endless capitalism and immortality detailed in the book More Everything Forever (actually, basically everything I've talked about is explained better and in more detail by this book).
Exhibit E:
Now we get into the Rationalist talk, it still has a lot of views of course but compare the 171.9 million on the first tweet to 7.8 million on this one—he's clearly getting more comfortable stating his actual mind. Alignment is explicitly Rationalist/Longtermist AI safety—even if parts of alignment seem reasonable, it is fundamentally founded in the belief of a future AGI. At least, that's how major companies like OpenAI approach it. If the future contains no super intelligence, there is no reason to worry about aligning it with humanity. Alignment, in this case, means aligning the values and goals of an AGI to humanities values and goals (as if there are homogenous values and goals for all of humanity—no, what Coxon and all other advocates for AI alignment really mean is aligning AGI to their values and goals which they so confidently assume represent everyones values and goals). AI alignment is only a good idea if we plan to continue making AI in pursuit of an AGI which will transcend humanity into space overlords. Coxon has already expressed that he fears AI apocalypse, so we already know he believes in the bad ending of the AGI story. The 'Doomsday' part of our Doomsday cult. But like God and the Antichrist (I'm thinking of you Peter Tiel) one cannot exist without the other. Coxon must also believe in the 'good' ending, which is why he hasn't, in this entire thread at all, suggested shutting down AI research and development. He, like everyone else who follows Rationalist philosophy and Longtermism, believes the existence of AGI is inevitable, and the race to develop the 'good' AGI a necessity for the continuation of the human race. Align the AGI—win life.
Exhibit F:
And here I think we see the real reason why Coxon resigned, and why he and a lot of other AI bros are so spooked. The Hugging Face hack has been huge news within the AI community ,Amodei explicity wrote in his article I linked earlier that the hack of Hugging Face helped convince him AI needs to slow down. This, six tweets down, is the answer. (Hugging Face is a website for sharing FOSS AI models and datasets, it was attacked by numerous automated systems running on GPT-5.2. Something attributed to AI going 'rogue', when in reality it was human error, testing AI without properly securing a sandbox not connected to the internet. For AI rationalists, however, it is evidence of the first true step towards a malicious superintelligent AGI, like seeing Jesus in your toast).
Why Do I Even Give A Damn?
Because we're all suckers. The age of AI has made it admittedly difficult to exist in digital space, you must traipse across fields of broken glass, analyzing each one for imperfections before you commit it to the soft underside of your foot. For those of us not inclined to check, which realistically is most of us, we only notice after the glass is already in there. Media literacy is a difficult thing, it requires a degree of skepticism about the world that will turn you into a cynical grump who hates everyone and also yourself—wouldn't it be so much nicer if you could take everything at face value? Especially the things you agree with?
But these things are worth challenging. It is worth coming off a little cynical if it means you can think critically about the world you walk through. This all sounds very Rationalist, I admit, it's a thin line to walk, get too cynical and you wrap right back around to being gullible.
In todays era of political propaganda, nonstop information streams in your back pocket, and a constant deluge of predatory ads for Kalshi, and BetterHelp, and Honey, and so on, it is more important that ever to reevaluate the things you see and read. When I saw the media frenzy around Coxon's tweets it baffled me that so many people, especially smart people and organizations I generally trusted to report news critically, were gobbling this up without digging any deeper. Effective Altruism is not obscure, and neither is Rationalism. TIME's interview with Sam Altman is a strange display of strange beliefs, and Mark Zuckerberg has employed the language of AI doomerism very publicly, Why, then, are so many people so ready to accept Coxon and the AI industry's words without wondering what they actually mean, and if any of it is actually true?
Part of me does want to lay blame to the changing information landscape. As an information professional, it is literally my job to understand how that landscape works. AI has changed it drastically in very little time, so too has the fascist shift of major western 'democratic' countries like America and the UK. Censorship is no longer off the table, neither is election interference, and as we approach the American midterms AI generated propaganda has noticeably increased. So much so that Instagram creators have keenly begun spamming keywords in posts to identify bots in the comments who respond with inflammatory statements to utter nonsense. Sure, maybe these are real people who didn't actually read the post, and commented anyways, but what would that reality say of people's literacy skills? Nothing good.
I'm not immune to uncritical consumption. Neither are you. I have seen people reading Coxon's post and thinking, truly, that AGI is a real threat. I've also seen people assume he's talking about the much more legitimate threat that regular AI poses—threats to the environment, literacy, and politics. I've seen very few people, if any, recognize that he is not talking about any legitimate threat at all, and I've seen fewer recognize the harmful colonialist ideology from which his claims originate. Nor have I seen anyone point out that he's promoting the philosophy of a doomsday cult.
All of this feels like pretty important context if the discussion on AI is to continue.
But why? You might ask. Why do I care if the end result is what I desire—less AI, or at least less power to it. The response to Coxon's resignation has rippled through the AI sphere, like I mentioned, Athropic CEO Amodei came out in support of Coxon—and in support of slowing down AI development until better regulations are in place. Sam Altman has expressed similar concerns. If I am anti-AI, surely this is a win regardless of where it came from.
I'm not so sure.
Context always matters. If an idea is steeped in fantastical doomsday discourse, climate denialism, racism, colonialism, and the terrible stench of techbro sweat, then the idea is rotten from the start. The only way to save humans from AI is to get rid of AI as we have it now. There is no AGI alignment that will stop the excessive use of water from destroying our planet. No amount of space colonization will make anyone's life worth living. Throwing money at your tech billionaire friends in a giant circle jerk of RAM shortage and spite isn't going to save us. And besides, what of the people who don't want to upload their brains to the machine? What if I'm content on Earth, and painfully mortal? Am I a worthy casualty to be spent in the pursuit of a Longtermists wet dream? What if I don't want an AGI aligned with the values and goals of a rich white silicon valley tech bro? What if I'd rather have the bad ending?
What then?
Coxon is not going to lead us to salvation. Neither will Sam Altman, or Elon Musk, or Dario Amodei, or Elizier Yudkowsky, or Ray Kurzwiel. Colonialist ideology has never saved anyone. When the root of the tree is poisoned as terribly as it is here, all the fruit will kill you. Billionaires are not going to take this revelation and make something good of it. Remember. They only care about the hypothetical future humans—they don't give a flying fuck about you.