GPT-3/LLMs' Achilles heel is short context length - how many "in-context" examples they can consume to learn a new task.

Enter "Structured Prompting": scale your examples from dozens => 1,000+

Here's how:

=> Get 1000s of in-context samples

=> split them into M groups, each small enough to fit in regular context length

=> encode each of M groups using LLM encoder

=> combine these encoded groups and attend over a scaled version of the combination simultaneously

Paper: https://arxiv.org/pdf/2212.06713.pdf

Code: https://github.com/microsoft/LMOps

AI has a lying problem

AI Alignment

AI Alignment

What will finally enable this?

The student

Honor Roll student of the future.

the moment when teachers figure out they can grade 300 essays in 15 min with chatGPT

AI has a wokeness problem,
and a lying problem.

what would steve jobs think of the woke mob?