GPT-4 is original for almost everything — except jokes — for which is HORRIBLE and Plagiarizes ~100%
So the big question is, which is more likely?
(A) GPT-5 will grok jokes: Will jokes, at least basic non-plagiarized ones, be the next major domain that GPT-5 suddenly “groks”?
Or,
(B) More training alone isn't enough, some bigger change is needed: Is a fundamentally different model architecture or interaction approach needed in order for the GPT models to be able to make decent jokes in response to normal prompts?
FWIW, we settled on (B), in order to achieve AFAIK what seems to be the first systematic generation of real, even if primitive, jokes.
Try our basic joke generation out with the command /vid
So the big question is, which is more likely?
(A) GPT-5 will grok jokes: Will jokes, at least basic non-plagiarized ones, be the next major domain that GPT-5 suddenly “groks”?
Or,
(B) More training alone isn't enough, some bigger change is needed: Is a fundamentally different model architecture or interaction approach needed in order for the GPT models to be able to make decent jokes in response to normal prompts?
FWIW, we settled on (B), in order to achieve AFAIK what seems to be the first systematic generation of real, even if primitive, jokes.
Try our basic joke generation out with the command /vid
👍16❤8🤯4👏2
GROKKING: GENERALIZATION BEYOND OVERFITTING ON SMALL ALGORITHMIC DATASETS
Translation: for each complex task, as you train large neural networks more, the neural networks eventually reach a point where they suddenly go from completely failing at a task to suddenly getting it. I.e. “grokking”
Paper
Translation: for each complex task, as you train large neural networks more, the neural networks eventually reach a point where they suddenly go from completely failing at a task to suddenly getting it. I.e. “grokking”
Paper
👍16❤3💯1
Do Machine Learning Models Memorize or Generalize?
Are today’s LLMs still in the memorizing/plagiarising stage for jokes?
Will GPT-5 make the jump to grokking jokes, and suddenly be able to make good jokes, with normal prompting, and without just plagiarising them?
Article on Grokking
Are today’s LLMs still in the memorizing/plagiarising stage for jokes?
Will GPT-5 make the jump to grokking jokes, and suddenly be able to make good jokes, with normal prompting, and without just plagiarising them?
Article on Grokking
❤17👏5👍2🔥1
JOKES:
When and how will LLMs finally get jokes, and stop just plagiarising them?
When and how will LLMs finally get jokes, and stop just plagiarising them?
Anonymous Poll
33%
GPT-5: Just add more training, and GPT-5 will finally grok jokes
8%
GPT-6: Just add more training, and GPT-6 will finally grok jokes
10%
GPT-7: Just add more training, and GPT-7 will finally grok jokes
29%
NEVER: Just adding more training isn’t enough, change to the model architecture/prompting is needed
20%
Show results
❤28👏12👍1
👍96❤33🔥17😁13🥰11🎉11👏9⚡6🗿6🤣5💯3