2 years after completion of GPT-4 training, OpenAI has finally begun training of GPT-5

Why the huge gap?

What’s for sure is that one major effect this huge delay has been to kill off many AI startups, who’d bet on AI capabilities quickly advancing, and essentially bet on OpenAI continuing training of GPT-5 immediately after GPT-4.

Those upcoming AI startups are exactly where any future OpenAI killer could have been expected to be lurking.

The huge delay has undoubtedly lead to the death of countless AI startups.

I.e.

Did OpenAI delay the start of GPT-5 training by almost 2 years, as a strategic move, to kill off most of their competition?

The purpose of a system is what it does.

OpenAI Announcement

OpenAI CTO Mira Murati just said that the company's models are ‘not that far ahead’ of what the public currently has for free

OpenAI does not have some GPT-5 level model waiting in the wings.

Why did OpenAI totally blow the huge lead they had on the industry, giving everyone else time to catch up to GPT-4?

OpenAI CTO dampens expectations of radically improved AI models in the near future

But at the same time, no other AI giants seem to be picking up the torch, for training of a next-generation foundation model that will likely cost $250M or more.

Who will be the first to launch a 10x better GPT-5 level model?

Article

Ilya Sutskever’s announcement’s of leaving OpenAI, and starting a new company

Safe Superintelligence Inc., primarily based in Tel Aviv.

— Few details but sounds like doubling down on the censorship insanity.

Tweet

Update: AI training compute used for top LLMs grows by ~5x per year

That’s a doubling every ~5 months, far faster than the 18 months of Moore’s Law.

Notice that this claims that Gemini Ultra used more compute than GPT-4,

but Gemini Ultra is reportedly far worse than GPT-4 at coding. Something doesn’t add up.

Conclusion:

Massive increases in raw LLM ability are coming, far beyond GPT-4 ability, but not quite yet.

Website

State of AI

Chatbot convinces woman to kill her dog.

AI leans communist.

Article

Token Cost of GPT-4 level models over time

Cost of 1M tokens has dropped from $180 to $0.75 in ~18 months = 240x cheaper.

— FWIW, none of the cheap ones are quite to the quality of real GPT-4 on coding, the only real job where AI matters right now, and who cares if they’re cheaper when they’re not yet quite good enough to really do the jobs they could.

Industry wasted last 2 years making models cheaper rather than pushing forward the state of the art.