Question 6 (Advanced):
Which of the following attention mechanisms is used in transformers?

A) Hard Attention
B) Additive Attention
C) Self-Attention
D) Bahdanau Attention

#Transformers #NLP #DeepLearning #AttentionMechanism #AI

Question 10 (Advanced):
In the Transformer architecture (PyTorch), what is the purpose of masked multi-head attention in the decoder?

A) To prevent the model from peeking at future tokens during training
B) To reduce GPU memory usage
C) To handle variable-length input sequences
D) To normalize gradient updates

#Python #Transformers #DeepLearning #NLP #AI

✅ By: https://t.iss.one/DataScienceQ

Question 32 (Advanced - NLP & RNNs):
What is the key limitation of vanilla RNNs for NLP tasks that led to the development of LSTMs and GRUs?

A) Vanishing gradients in long sequences
B) High GPU memory usage
C) Inability to handle embeddings
D) Single-direction processing only

#Python #NLP #RNN #DeepLearning

✅ By: https://t.iss.one/DataScienceQ

❓ Interview question :
What is the Transformer architecture, and why is it considered a breakthrough in NLP?

❓ Interview question :
How does self-attention enable Transformers to capture long-range dependencies in text?

❓ Interview question :
What are the main components of a Transformer model?

❓ Interview question :
Why are positional encodings essential in Transformers?

❓ Interview question :
How does multi-head attention improve Transformer performance compared to single-head attention?

❓ Interview question :
What is the purpose of feed-forward networks in the Transformer architecture?

❓ Interview question :
How do residual connections and layer normalization contribute to training stability in Transformers?

❓ Interview question :
What is the difference between encoder and decoder in the Transformer model?

❓ Interview question :
Why can Transformers process sequences in parallel, unlike RNNs?

❓ Interview question :
How does masked self-attention work in the decoder of a Transformer?

❓ Interview question :
What is the role of key, query, and value in attention mechanisms?

❓ Interview question :
How do attention weights determine which parts of input are most relevant?

❓ Interview question :
What are the advantages of using scaled dot-product attention in Transformers?

❓ Interview question :
How does position-wise feed-forward network differ from attention layers in Transformers?

❓ Interview question :
Why is pre-training important for large Transformer models like BERT and GPT?

❓ Interview question :
How do fine-tuning and transfer learning benefit Transformer-based models?

❓ Interview question :
What are the limitations of Transformers in terms of computational cost and memory usage?

❓ Interview question :
How do sparse attention and linear attention address scalability issues in Transformers?

❓ Interview question :
What is the significance of model size (e.g., number of parameters) in Transformer performance?

❓ Interview question :
How do attention heads in multi-head attention capture different types of relationships in data?

#️⃣ tags: #Transformer #NLP #DeepLearning #SelfAttention #MultiHeadAttention #PositionalEncoding #FeedForwardNetwork #EncoderDecoder

By: t.iss.one/DataScienceQ 🚀