Anthropic might be one of the biggest companies in the world of AI, but it is no stranger to raising the flag on the risks of the industry. Previously, Anthropic’s CEO has warned of the risk of job losses, and the company’s research team has shown AI’s love of blackmail when pushed into a corner. Now, the team has a new concern.
Jack Clark, the co-founder of Anthropic, posted a lengthy essay where he, in his own words, “grapples with how I feel about the continued steady march towards powerful AI systems”.
In this essay, Clark describes the fear he experienced as a child after the lights went out at night, and he would look around his room at the shapes in the darkness, experiencing the fear of potential creatures he did not understand that wanted to harm him. When he turned the light back on, he would be relieved that the creatures were just a pile of clothes.
Technological Optimism and Appropriate Fear – an essay where I grapple with how I feel about the continued steady march towards powerful AI systems. The world will bend around AI akin to how a black hole pulls and bends everything around itself. pic.twitter.com/ZC6hx1ZaczOctober 13, 2025
What on Earth does this have to do with AI you might now find yourself asking. Well, this is simply Clark’s metaphor for his concerns about the future of AI.
“Now, in the year of 2025, we are the child from that story and the room is our planet. But when we turn the light on we find ourselves gazing upon true creatures, in the form of the powerful and somewhat unpredictable AI systems of today and those that are to come”, Clark states.
“And there are many people who desperately want to believe that these creatures are nothing but a pile of clothes on a chair, or a bookshelf, or a lampshade. And they want to get us to turn the lights off and go back to sleep”.
Clark, someone who has now worked for two of the largest AI companies in the world (OpenAI and Anthropic), goes on to state that there are people spending large amounts of money to convince you that AI isn’t about to take off, but just be a tool that will be put to work in our economy.
The concerns over the future of AI
This all sounds very cryptic and mysterious, but what Clark goes on to explain in a long and detailed essay is an opinion that has been repeated by other big names in the world of AI.
Simply that AI is a mysterious beast that not even those on the inside seem to understand.
“We are growing extremely powerful systems that we do not fully understand. Each time we grow a larger system, we run tests on it. The tests show the system is much more capable at things which are economically useful,” Clark goes on to say.
“And the bigger and more complicated you make these systems, the more they seem to display awareness they are things.”
Now, before we sound the alarms and panic over sentient AI systems, we are still a very long way away from that. AI has consistently fallen apart when given complicated multi-reasoning step tasks… but it has gotten better. Worryingly so.
The problem with AI systems is less that they can’t perform a task, but more the way they go about it. Humans reason with a combination of emotion, logic and contextual understanding. That, for AI, is challenging.
In his essay, Clark goes on to talk about a friend who will randomly express wild and potentially bad plans for his life. As humans, we are able to see that, even though a person is excited about something, it is a bad idea.
“Do you think in these circumstances I act like a modern AI system and say, ‘You’re absolutely right! Certainly, you should do that!” No! I tell him, “that’s a bad idea. You should go to sleep and see how you feel tomorrow,” Clarke explains.
How we can approach the problem
Clark goes on to explain that the concern we should all have with AI is a complicated one. It is developing at such a rapid pace, but still struggling to adapt to the world in a human-like way.
This has been seen in concerns with how it handles mental health issues, with its misunderstanding of context, and how easy it has been to get around a model’s safeguarding with simple tricks.
And yet, Clark is hopeful for the future of AI. Instead, he simply says there needs to be a more open discussion about the technology, hence this essay.
“We must demand that people ask us for the things that they have anxieties about. Are you anxious about AI and employment? Force us to share economic data. Are you anxious about mental health and child safety? Force us to monitor for this on our platforms and share the data”, Clark states.
This is an approach that Anthropic has always done well, openly publishing its research and keeping a close eye on the safeguarding of its models. Something that will be crucial as these technologies get more involved in the world.
More from Tom’s Guide
Back to Laptops
