One of the key moments when I truly understood how transformers work: 🧠✨

"Stop thinking of a transformer as a conveyor belt, where each layer transforms the output of the previous one."

"Start thinking of it as a residual flow." 🌊

Each block in a transformer has a residual connection that adds the input of the block to its output:

x' = f(x) + x

Because of this addition, the attention mechanism or MLP within the layer – the function f in the formula above – actually does NOT transform the input. Instead, it calculates the information that needs to be ADDED to the input before passing it to the next block! ➕

Imagine the main part of the transformer as a shared whiteboard. 📝 Each block reads what it needs from it and adds its own notes. All changes are additions.

Furthermore, layers can exchange information over distances. A block in the first layer can write information, and a block in the fifth layer can read it, even though there is no direct connection between them. 🔗

I owe these ideas to an older article by Anthropic about the architecture of transformers:
transformer-circuits.pub/2021/framework

#Transformers #DeepLearning #AI #MachineLearning #NeuralNetworks #Tech

✨ Join Best TG Channels https://t.iss.one/addlist/0f6vfFbEMdAwODBk

⭐️ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A

🔖 A Comprehensive Resource for Machine Learning Practice

The platform offers:

🫡 1200+ machine learning tasks;
🫡 Mathematics, from linear algebra to PCA;
🫡 Deep Learning, NLP, and Computer Vision;
🫡 Labs with real datasets;
🫡 Learning paths and interview preparation;
🫡 Projects using GPT, RL, and CUDA.

Almost everything can be accessed directly in your browser, and basic tasks are available for free.

⛓️ Link to the platform
https://enghusseinsheikho.substack.com/p/a-comprehensive-resource-for-machine?r=3hwnr8&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

#MachineLearning #DeepLearning #DataScience #AI #Coding #Tech

✨ Join Best TG Channels https://t.iss.one/addlist/0f6vfFbEMdAwODBk

⭐️ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A