Crazy people implemented MicroGPT by Andrej Karpathy entirely in FPGA logic. 🧠⚡

Without a graphics processor. 🚫📺
Without PyTorch. 🐍❌
Without inference loops on the central processor. 🚫🔄

Just a transformer, "embedded" in the hardware, generating 50,000+ tokens per second. ⚡🚀

The model is small, but that's not the point: inference doesn't have to exist only in a software environment. 💾➡️🔌

The goal wasn't to create the largest possible model. 📉
The goal was to present the entire inference path of the transformer in a form readable for hardware: memory, counters, state machines, accumulators, lookup tables, and multi-cycle arithmetic blocks. ⚙️🔧

The basic scheme uses fixed Q4.12 arithmetic and weights stored in ROM. 📜🔢

Most of the model boils down to one repetitive operation: matrix-vector multiplication. 🧮 Therefore, a reusable 16-channel stream block for matrix-vector calculations was implemented, and then it was temporarily multiplexed to Q/K/V, MLP, and the output layer of the language model. 🔄🧩

The most interesting thing turned out to be the attention mechanism. 🧐✨

In Python, it's a single neat equation. 🐍➕
In RTL, it turns into a schedule: generating Q/K/V, going through scalar products, tracking the maximum, approximating the exponential, accumulating, dividing, mixing V, and then reverse projection. 📅📉

https://github.com/Luthiraa/TALOS-V2 🔗🛠️

https://t.iss.one/DataScienceN