OpenAI: The New Moore’s Law Regime for Model Compute and Model Size
“Since 2012, the amount of compute used in the largest AI training runs has been increasing exponentially with a 3.4-month doubling time, by comparison, Moore’s Law had a 2-year doubling period.”
Already been happening for a decade, and this one isn't going to stop.
Just getting started.
Article
“Since 2012, the amount of compute used in the largest AI training runs has been increasing exponentially with a 3.4-month doubling time, by comparison, Moore’s Law had a 2-year doubling period.”
Already been happening for a decade, and this one isn't going to stop.
Just getting started.
Article
👏10❤1👍1
Sam Altman’s OWN 2021 blog post, Moore’s Law for Everything = This exponential increase in model size trend is not going to stop, ever
And now Sam is saying that the age of increasingly larger models is suddenly over?
Sam’s Blog Post
And now Sam is saying that the age of increasingly larger models is suddenly over?
Sam’s Blog Post
👏9👀2❤1
Media is too big
VIEW IN TELEGRAM
Scaling Laws of Large Foundation Models. Bigger = Better, Forever
“As we make bigger models and give them more compute, they just keep getting better. This means as models keep getting bigger, they actually become more sample efficient.
This is kind of crazy, because back in the day I was always taught that you have to use the smallest model possible so that it doesn’t overfit your data, but now that just seems to be wrong.”
FYI: This is also a strong hint at the answers to our current $100 prize contest, which we’re wrapping up today!
“As we make bigger models and give them more compute, they just keep getting better. This means as models keep getting bigger, they actually become more sample efficient.
This is kind of crazy, because back in the day I was always taught that you have to use the smallest model possible so that it doesn’t overfit your data, but now that just seems to be wrong.”
FYI: This is also a strong hint at the answers to our current $100 prize contest, which we’re wrapping up today!
🔥7❤1👍1👏1
This media is not supported in your browser
VIEW IN TELEGRAM
Sam Before: Only once we’ve built a dyson sphere around the sun and gotten compute as efficient as possible, only then should we even begin to entertain the idea of slowing down scaling up AI models
Sam Now: The age of scaling up AI models Is already over, trust me bro.
Sam Now: The age of scaling up AI models Is already over, trust me bro.
🤣14❤4
This media is not supported in your browser
VIEW IN TELEGRAM
me someday when I finally get approved for GPT-4 API
🎉16🤩4👍2❤1
This media is not supported in your browser
VIEW IN TELEGRAM
“ChatGPT was used to create a script about the deeper meaning of Peppa Pig in the style of Jordan Peterson. Tortoise-tts used to produce JP voice and Stablediffusion to create fine-tuned images of JP. SadTalker is used to make his face move and talk.”
👍9❤4🤯1👌1
Told ChatGPT to code an HTML website with its favorite colors
"Write html code for a website that displays the text "hello world, I am ChatGPT and I coded this website" and choose your favorite colors"
"Can you add tables and some fancy html elements to that?"
"Now I want you to make it as complicated and artistic as you possibly can"
"Write html code for a website that displays the text "hello world, I am ChatGPT and I coded this website" and choose your favorite colors"
"Can you add tables and some fancy html elements to that?"
"Now I want you to make it as complicated and artistic as you possibly can"
👍9❤1😍1