Scaling Laws of Large Foundation Models. Bigger = Better, Forever

“As we make bigger models and give them more compute, they just keep getting better. This means as models keep getting bigger, they actually become more sample efficient.

This is kind of crazy, because back in the day I was always taught that you have to use the smallest model possible so that it doesn’t overfit your data, but now that just seems to be wrong.”

FYI: This is also a strong hint at the answers to our current $100 prize contest, which we’re wrapping up today!

Sam Before: Only once we’ve built a dyson sphere around the sun and gotten compute as efficient as possible, only then should we even begin to entertain the idea of slowing down scaling up AI models

Sam Now: The age of scaling up AI models Is already over, trust me bro.

“ChatGPT was used to create a script about the deeper meaning of Peppa Pig in the style of Jordan Peterson. Tortoise-tts used to produce JP voice and Stablediffusion to create fine-tuned images of JP. SadTalker is used to make his face move and talk.”

Write me a cover letter in the style of Edgar Allen poe for a job at mcdonalds

Told ChatGPT to code an HTML website with its favorite colors

"Write html code for a website that displays the text "hello world, I am ChatGPT and I coded this website" and choose your favorite colors"

"Can you add tables and some fancy html elements to that?"

"Now I want you to make it as complicated and artistic as you possibly can"