✅ Correct answer: A) Vanishing gradients in long sequences

# Vanilla RNN vs LSTM comparison
import torch.nn as nn

rnn = nn.RNN(input_size=100, hidden_size=50, num_layers=1)
lstm = nn.LSTM(input_size=100, hidden_size=50, num_layers=1)

# Forward pass for 10 timesteps
inputs = torch.randn(10, 1, 100)  # (seq_len, batch, input_size)
h_rnn = torch.zeros(1, 1, 50)     # Initial hidden state
h_lstm = (torch.zeros(1, 1, 50), torch.zeros(1, 1, 50))  # LSTM state

out_rnn, _ = rnn(inputs, h_rnn)     # Prone to vanishing gradients
out_lstm, _ = lstm(inputs, h_lstm)  # Better long-term memory

### Key Problems with Vanilla RNNs:
1. Gradient Issues:
- Error signals decay exponentially over timesteps
- Tanh/Sigmoid activations compound the problem

2. LSTM/GRU Solutions:
| Mechanism | Purpose |
|-----------------|----------------------------------|
| Forget Gate | Controls what to remember |
| Input Gate | Regulates new information |
| Cell State | Highway for long-term gradients |

### Practical Impact:

# Training a sentiment analyzer
rnn_model = nn.RNN(embed_dim, hidden_dim)       # Fails beyond 50 words  
lstm_model = nn.LSTM(embed_dim, hidden_dim)     # Handles 500+ words  

*Modern Alternative*: Transformers (no recurrent connections at all)

5 minutes of work - 127,000$ profit!

Opened access to the Jay Welcome Club where the AI bot does all the work itself💻

Usually you pay crazy money to get into this club, but today access is free for everyone!

23,432% on deposit earned by club members in the last 6 months📈

Just follow Jay's trades and earn! 👇

https://t.iss.one/+mONXtEgVxtU5NmZl

Tired of endless job boards and low offers?
Unlock access to exclusive remote jobs from top startups—some with salaries $100k+ and early-bird roles at $50/h and above.
New high-paying openings posted daily—tech, marketing, design, and more.
Ready to upgrade your career from anywhere?
Check today’s top jobs now before they’re gone!

#إعلان InsideAds