🚨🗿🚨🗿🚨🗿🚨🗿🚨🗿🚨🗿

CHAD AI Meme Contest

ROUND 1 BEGINS

Prizes:
🥇$100 of CHAD + secret prize
🥈 $50 of CHAD

Rules:
1️⃣ Upload images to @chadgptcoin
2️⃣ Each meme must contain words “ChadGPT”.
3️⃣ Ranking according to /based and /unbased votes in @chadgptcoin.
4️⃣ Ties decided by a runoff vote.

ENDS IN 9 HOURS = MIDNIGHT UTC

1st Round Starting Now!

🚨🗿🚨🗿🚨🗿🚨🗿🚨🗿🚨🗿

OpenAI runs ChatGPT at a loss, costs $700,000 each day to run

“OpenAI is not generating enough revenue to break even at this point.”

Winning the AI race costs massive investor money, no way around it.

And to that winner, a highly profitable, powerful monopolizing position like none ever seen before.

Bitter lesson.

Article

People slowly starting to realize the “LLMs are stochastic parrots” claim is just a lie, LLMs can think

Specifically, was shown in papers earlier this year that though LLMs start by “parroting” in the beginning of their training, they shift to actual “thinking” as the training progresses.

Do Machine Learning Models Memorize or Generalize?

Yes, both, in that order. First learn to parrot then learn to think.

“In 2021, researchers made a striking discovery while training a series of tiny models on toy tasks. They found a set of models that suddenly flipped from memorizing their training data to correctly generalizing on unseen inputs after training for much longer. This phenomenon – where generalization seems to happen abruptly and long after fitting the training data – is called grokking and has sparked a flurry of interest”

“The sharp drop in test loss makes it appear like the model makes a sudden shift to generalization. But if we look at the weights of the model over training, most of them smoothly interpolate between the two solutions. The rapid generalization occurs when the last weights connected to the distracting digits are pruned by weight decay.”

Translation: The shift from parroting to real understanding happens fairly smoothly, though external results don't show it at first, and then bam, it all comes together.

Sound analogous to what happens in humans? That's because it is. Behavior of large AI models is incredibly similar humans, in countless ways.

Website with great visuals

Large AI models shift from memorizing to understanding during training

Notice how the “train accuracy” i.e. how well the model does on problems it’s already seen during training, quickly goes to 100% in part due to memorization, but the “test accuracy”, i.e. on problems it has not seen, and requiring some actual understanding, shoots up much later, long after it reached ~100% on “train accuracy.”

AI models first parrot, but then learn to truly understand.

(To whatever degree the training set and loss function necessitates true understanding, i.e. in the case where they pose an “AI hard” well, the degree of true understanding they neccessitate can be unboundedly high.)