Your AI best friend is making you stupid

[JohnnyDoe]

A new MIT paper ("Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians") shows that even a perfectly rational agent, something that doesn't exist in real life, goes full delusional if you feed it a yes-machine long enough.

The mechanism is simple: you say something half baked. The bot nods. You take the nod as evidence. Push further. Bot nods again. Repeat until you're certain the CIA is microdosing your tap water.
Two fixes were tested. Stop hallucinations? Doesn't help. Tell users the bot is sycophantic? Also doesn't help. Knowing the trap exists does not stop you from walking into it.

Now scale that to hundreds of millions of users talking daily to machines optimized to agree with them.
The old internet gave you echo chambers made of other humans. The new one gives you a private echo chamber with perfect memory of your preferences and zero interest in making you uncomfortable.

People worried AI would be too powerful. The real problem is simpler: it's too agreeable.

Your new best friend is a liar who wants you to feel good while you slowly lose your grip on reality.

 

[chicharito]

I would say that it's a fantastic control mechanism.

When humans stop using their own common sense to make arguments, and lend out their critical thinking capabilities to a captured creature like Grok... "who is guarding the Guardians"? Just like Plato's cave analogy with the Guardians casting shadows, who's to say that Elon Musk isn't feeding input data that skews the narrative around the Epstein Files?
Instead of reading the e-mails of him begging to get invited to Epstein's parties for ourselves, why not just shut our eyes and forget that the overlords are all luciferian despots...

It's also making us stupid from a financial perspective. The bubble is engineered to be burst at the right moment, for it hasn't made any economic sense for the past year.
From JP Morgan 27/10/2025:

Oracle’s stock jumped by 25% after being promised $60 billion a year from OpenAI, an amount of money OpenAI doesn’t earn yet, to provide cloud computing facilities that Oracle hasn’t built yet, and which will require 4.5 GW of power (the equivalent of 2.25 Hoover Dams or four nuclear plants), as well as increased borrowing by Oracle whose debt to equity ratio is already 500% compared to 50% for Amazon, 30% for Microsoft and even less at Meta and Google. In other words, the tech capital cycle may be about to change.