# 📚 PyTorch Tutorial for Beginners - Part 2/6: Deep Neural Networks & Training Techniques
#PyTorch #DeepLearning #MachineLearning #NeuralNetworks #Training

Welcome to Part 2 of our comprehensive PyTorch series! This lesson dives deep into building and training neural networks, covering architectures, activation functions, optimization, and more.

---

## 🔹 Recap & Setup

import torch
import torch.nn as nn
import torch.optim as optim
import matplotlib.pyplot as plt
from torch.utils.data import DataLoader, TensorDataset

# Check GPU
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print(f"Using device: {device}")

---

## 🔹 Deep Neural Network (DNN) Architecture
### 1. Key Components
| Component | Purpose | PyTorch Implementation |
|--------------------|-------------------------------------------------------------------------|------------------------------|
| Input Layer | Receives raw features | nn.Linear(input_dim, hidden_dim) |
| Hidden Layers | Learn hierarchical representations | Multiple nn.Linear + Activation |
| Output Layer | Produces final predictions | nn.Linear(hidden_dim, output_dim) |
| Activation | Introduces non-linearity | nn.ReLU(), nn.Sigmoid(), etc. |
| Loss Function | Measures prediction error | nn.MSELoss(), nn.CrossEntropyLoss() |
| Optimizer | Updates weights to minimize loss | optim.SGD(), optim.Adam() |

### 2. Building a DNN

class DNN(nn.Module):
    def __init__(self, input_size, hidden_sizes, output_size):
        super().__init__()
        layers = []
        
        # Hidden layers
        prev_size = input_size
        for hidden_size in hidden_sizes:
            layers.append(nn.Linear(prev_size, hidden_size))
            layers.append(nn.ReLU())
            prev_size = hidden_size
            
        # Output layer (no activation for regression)
        layers.append(nn.Linear(prev_size, output_size))
        
        self.net = nn.Sequential(*layers)
        
    def forward(self, x):
        return self.net(x)

# Example: 3-layer network (input=10, hidden=[64,32], output=1)
model = DNN(10, [64, 32], 1).to(device)
print(model)

---

## 🔹 Activation Functions
### 1. Common Choices
| Activation | Formula | Range | Use Case | PyTorch |
|-----------------|----------------------|------------|------------------------------|------------------|
| ReLU | max(0, x) | [0, ∞) | Hidden layers | nn.ReLU() |
| Leaky ReLU | max(0.01x, x) | (-∞, ∞) | Avoid dead neurons | nn.LeakyReLU() |
| Sigmoid | 1 / (1 + e^(-x)) | (0, 1) | Binary classification | nn.Sigmoid() |
| Tanh | (e^x - e^(-x)) / ... | (-1, 1) | RNNs, some hidden layers | nn.Tanh() |
| Softmax | e^x / sum(e^x) | (0, 1) | Multi-class classification | nn.Softmax() |

### 2. Visual Comparison

x = torch.linspace(-5, 5, 100)
activations = {
    "ReLU": nn.ReLU()(x),
    "LeakyReLU": nn.LeakyReLU(0.1)(x),
    "Sigmoid": nn.Sigmoid()(x),
    "Tanh": nn.Tanh()(x)
}

plt.figure(figsize=(12, 4))
for i, (name, y) in enumerate(activations.items()):
    plt.subplot(1, 4, i+1)
    plt.plot(x.numpy(), y.numpy())
    plt.title(name)
plt.tight_layout()
plt.show()

---

🌟 Vision Transformer (ViT) Tutorial – Part 1: From CNNs to Transformers – The Revolution in Computer Vision

Let's start: https://hackmd.io/@husseinsheikho/vit-1

#VisionTransformer #ViT #DeepLearning #ComputerVision #Transformers #AI #MachineLearning #NeuralNetworks #ImageClassification #AttentionIsAllYouNeed

✉️ Our Telegram channels: https://t.iss.one/addlist/0f6vfFbEMdAwODBk

📱 Our WhatsApp channel: https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A

✨ NeRFs Explained: Goodbye Photogrammetry? ✨

📖 Table of Contents NeRFs Explained: Goodbye Photogrammetry? How Do NeRFs Work? Block #A: We Begin with a 5D Input Block #B: The Neural Network and Its Output Block #C: Volumetric Rendering The NeRF Problem and Evolutions Summary and Next Steps…...

🏷️ #3DComputerVision #3DReconstruction #DeepLearning #NeuralNetworks #Photogrammetry #Tutorial

✨ Adversarial Learning with Keras and TensorFlow (Part 3): Exploring Adversarial Attacks Using Neural Structured Learning (NSL) ✨

📖 Table of Contents Adversarial Learning with Keras and TensorFlow (Part 3): Exploring Adversarial Attacks Using Neural Structured Learning (NSL) Introduction to Advanced Adversarial Techniques in Machine Learning Harnessing NSL for Robust Model Training: Insights from Part 2 Deep Dive into…...

🏷️ #AdversarialLearning #DeepLearning #ImageProcessing #Keras #MachineLearning #NeuralNetworks #NeuralStructuredLearning #TensorFlow #Tutorial

🤖🧠 The Little Book of Deep Learning – A Complete Summary and Chapter-Wise Overview

🗓️ 08 Oct 2025
📚 AI News & Trends

In the ever-evolving world of Artificial Intelligence, deep learning continues to be the driving force behind breakthroughs in computer vision, speech recognition and natural language processing. For those seeking a clear, structured and accessible guide to understanding how deep learning really works, “The Little Book of Deep Learning” by François Fleuret is a gem. This ...

#DeepLearning #ArtificialIntelligence #MachineLearning #NeuralNetworks #AIGuides # FrancoisFleuret

📌 I Measured Neural Network Training Every 5 Steps for 10,000 Iterations

🗂 Category: MACHINE LEARNING

🕒 Date: 2025-11-15 | ⏱️ Read time: 9 min read

A deep dive into the mechanics of neural network training. This detailed analysis meticulously measures key training metrics every 5 steps over 10,000 iterations, providing a high-resolution view of the learning process. The findings offer granular insights into model convergence and the subtle dynamics often missed by standard monitoring, making it a valuable read for ML practitioners and researchers seeking to better understand how models learn.

#NeuralNetworks #MachineLearning #DeepLearning #DataAnalysis #ModelTraining

📌 Neural Networks Are Blurry, Symbolic Systems Are Fragmented. Sparse Autoencoders Help Us Combine Them.

🗂 Category: DEEP LEARNING

🕒 Date: 2025-11-27 | ⏱️ Read time: 17 min read

Neural networks and symbolic AI models compress information in fundamentally different ways, leading to "blurry" continuous representations versus "fragmented" discrete ones. Sparse Autoencoders (SAEs) offer a promising bridge between these two paradigms. By learning sparse, interpretable features from the dense activations within neural networks, SAEs can help translate continuous data into more structured, symbolic-like components. This approach aims to combine the robust pattern recognition of neural systems with the logical reasoning capabilities of symbolic AI, advancing the quest for more understandable and capable models.

#SparseAutoencoders #AIInterpretability #NeuralNetworks #SymbolicAI #NeuroSymbolic