Models Demystified
A Practical Guide from Linear Regression to Deep Learning
by Michael Clark & Seth Berry

This book is designed to guide readers on a comprehensive journey into the world of data science and modeling. For those just beginning their exploration, it offers:
- A solid foundation in the basics of modeling, presented from a practical and accessible perspective.
- A versatile toolkit of models and concepts that can be immediately applied to real-world problems.
- A balanced approach that integrates both statistical and machine learning methodologies.

For readers already experienced in modeling, the book provides:
- Deeper context and insights into familiar models.
- An introduction to new and advanced models that expand your knowledge.
- Enhanced understanding of how to select the most appropriate model for a given task and where to focus your efforts.

Above all, this book aims to highlight the common threads that connect different models, offering readers a clear and intuitive understanding of how they function and interrelate. Whether you're a beginner or a seasoned practitioner, this resource is crafted to deepen your expertise and broaden your perspective on the art and science of modeling.

Table of contents:

Preface
1  Introduction
2  Thinking About Models
3  The Foundation
4  Understanding the Model
5  Understanding the Features
6  Model Estimation and Optimization
7  Estimating Uncertainty
8  Generalized Linear Models
9  Extending the Linear Model
10  Core Concepts in Machine Learning
11  Common Models in Machine Learning
12  Extending Machine Learning
13  Causal Modeling
14  Dealing with Data
15  Danger Zone
16  Parting Thoughts

Link: Site

Navigational hashtags: #armknowledgesharing #armbooks #armsites
General hashtags: #ml #machinelearning #r #python #linear #metrics #featureengineering #optimization #mle #glm #dl #deeplearning

@data_science_weekly

Deep Learning
by Ian Goodfellow, Yoshua Bengio and Aaron Courville

The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames.

Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models.

Table of Contents:

Part I: Applied Math and Machine Learning Basics
2 Linear Algebra
3 Probability and Information Theory
4 Numerical Computation
5 Machine Learning Basics
Part II: Modern Practical Deep Networks
6 Deep Feedforward Networks
7 Regularization for Deep Learning
8 Optimization for Training Deep Models
9 Convolutional Networks
10 Sequence Modeling: Recurrent and Recursive Nets
11 Practical Methodology
12 Applications
Part III: Deep Learning Research
13 Linear Factor Models
14 Autoencoders
15 Representation Learning
16 Structured Probabilistic Models for Deep Learning
17 Monte Carlo Methods
18 Confronting the Partition Function
19 Approximate Inference
20 Deep Generative Models

Links:
- Site
- Book

Navigational hashtags: #armknowledgesharing #armbooks #armsites
General hashtags: #dl #deeplearning

@data_science_weekly

🤗 AI Agents Course

This free course will take you on a journey, from beginner to expert, in understanding, using and building AI agents.

In this course, you will:
- 📖 Study AI Agents in theory, design, and practice.
- 🧑‍💻 Learn to use established AI Agent libraries such as smolagents, LangChain, and LlamaIndex.
- 💾 Share your agents on the Hugging Face Hub and explore agents created by the community.
- 🏆 Participate in challenges where you will evaluate your agents against other students’.
- 🎓 Earn a certificate of completion by completing assignments.

And more!

At the end of this course you’ll understand how Agents work and how to build your own Agents using the latest libraries and tools.

Here is the general syllabus for the course:
Onboarding
Set you up with the tools and platforms that you will use.
Agent Fundamentals
Explain Tools, Thoughts, Actions, Observations, and their formats. Explain LLMs, messages, special tokens and chat templates. Show a simple use case using python functions as tools.
Frameworks
Understand how the fundamentals are implemented in popular libraries : smolagents, LangGraph, LLamaIndex
Use Cases
Let’s build some real life use cases (open to PRs 🤗 from experienced Agent builders)
Final Assignment
Build an agent for a selected benchmark and prove your understanding of Agents on the student leaderboard 🚀

Link: Course

Navigational hashtags: #armcourses
General hashtags: #nlp #llm #agents

@data_science_weekly

Machine Learning in Production by Carnegie Mellon University

This is a course for those who want to build software products with machine learning, not just models and demos. We assume that you can train a model or build prompts to make predictions, but what does it take to turn the model into a product and actually deploy it, have confidence in its quality, and successfully operate and maintain it at scale?

The course is designed to establish a working relationship between software engineers and data scientists: both contribute to building ML-enabled systems but have different expertise and focuses. To work together they need a mutual understanding of their roles, tasks, concerns, and goals and build a working relationship. This course is aimed at software engineers who want to build robust and responsible products meeting the specific challenges of working with ML components and at data scientists who want to understand the requirements of the model for production use and want to facilitate getting a prototype model into production; it facilitates communication and collaboration between both roles. The course is a good fit for student looking at a career as an ML engineer. The course focuses on all the steps needed to turn a model into a production system in a responsible and reliable manner.

It covers topics such as:
- How to design for wrong predictions the model may make?
How to assure safety and security despite possible mistakes? How to design the user interface and the entire system to operate in the real world?
- How to reliably deploy and update models in production?
How can we test the entire machine learning pipeline? How can MLOps tools help to automate and scale the deployment process? How can we experiment in production (A/B testing, canary releases)? How do we detect data quality issues, concept drift, and feedback loops in production?
- How to scale production ML systems?
How do we design a system to process huge amounts of training data, telemetry data, and user requests? Should we use stream processing, batch processing, lambda architecture, or data lakes?
- How to test and debug production ML systems?
How can we evaluate the quality of a model’s predictions in production? How can we test the entire ML-enabled system, not just the model? What lessons can we learn from software testing, automated test case generation, simulation, and continuous integration for testing for production machine learning?
- Which qualities matter beyond a model’s prediction accuracy?
How can we identify and measure important quality requirements, including learning and inference latency, operating cost, scalability, explainablity, fairness, privacy, robustness, and safety? Does the application need to be able to operate offline and how often do we need to update the models? How do we identify what’s important in a ML-enabled product in a production setting for a business? How do we resolve conflicts and tradeoffs?
How to work effectively in interdisciplinary teams?
How can we bring data scientists, software engineers, UI designers, managers, domain experts, big data specialists, operators, legal council, and other roles together and develop a shared understanding and team culture?

Link: Course

Navigational hashtags: #armcourses
General hashtags: #ml #dl #machinelearning #deeplearning #mlsystemdesign #mlops #mlsysdes

@data_science_weekly

MIT 6.S191 Introduction to Deep Learning

An efficient and high-intensity bootcamp designed to teach you the fundamentals of deep learning as quickly as possible!

MIT's introductory program on deep learning methods with applications to natural language processing, computer vision, biology, and more! Students will gain foundational knowledge of deep learning algorithms, practical experience in building neural networks, and understanding of cutting-edge topics including large language models and generative AI. Program concludes with a project proposal competition with feedback from staff and panel of industry sponsors. Prerequisites assume calculus (i.e. taking derivatives) and linear algebra (i.e. matrix multiplication), we'll try to explain everything else along the way! Experience in Python is helpful but not necessary.

Link: Course

Navigational hashtags: #armcourses
General hashtags: #dl #deeplearning #llm #cv #nlp

@data_science_weekly

What happens when...
you type google.com into your browser's address box and press enter?

This repository is an attempt to answer the age-old interview question "What happens when you type google.com into your browser's address box and press enter?"

Except instead of the usual story, we're going to try to answer this question in as much detail as possible. No skipping out on anything.

This is a collaborative process, so dig in and try to help out! There are tons of details missing, just waiting for you to add them! So send us a pull request, please!

Link: GitHub

Navigational hashtags: #armrepo
General hashtags: #systemdesign

@data_science_weekly

🤗 Deep Reinforcement Learning Course

This course will teach you about Deep Reinforcement Learning from beginner to expert. It’s completely free and open-source!

In this course, you will:
- 📖 Study Deep Reinforcement Learning in theory and practice.
- 🧑‍💻 Learn to use famous Deep RL libraries such as Stable Baselines3, RL Baselines3 Zoo, Sample Factory and CleanRL.
- 🤖 Train agents in unique environments such as SnowballFight, Huggy the Doggo 🐶, VizDoom (Doom) and classical ones such as Space Invaders, PyBullet and more.
- 💾 Share your trained agents with one line of code to the Hub and also download powerful agents from the community.
- 🏆 Participate in challenges where you will evaluate your agents against other teams. You’ll also get to play against the agents you’ll train.
- 🎓 Earn a certificate of completion by completing 80% of the assignments.

And more!

At the end of this course, you’ll get a solid foundation from the basics to the SOTA (state-of-the-art) of methods.

The course is composed of:
- A theory part: where you learn a concept in theory.
- A hands-on: where you’ll learn to use famous Deep RL libraries to train your agents in unique environments. These hands-on will be Google Colab notebooks with companion tutorial videos if you prefer learning with video format!
- Challenges: you’ll get to put your agent to compete against other agents in different challenges. There will also be a leaderboard for you to compare the agents’ performance.

Link: Course

Navigational hashtags: #armcourse
General hashtags: #reinforcementlearning #rl #deeprl #agents #hf #huggingface

@data_science_weekly

Machine Learning from Scratch by Danny Friedman

This book is for readers looking to learn new machine learning algorithms or understand algorithms at a deeper level. Specifically, it is intended for readers interested in seeing machine learning algorithms derived from start to finish. Seeing these derivations might help a reader previously unfamiliar with common algorithms understand how they work intuitively. Or, seeing these derivations might help a reader experienced in modeling understand how different algorithms create the models they do and the advantages and disadvantages of each one.

This book will be most helpful for those with practice in basic modeling. It does not review best practices—such as feature engineering or balancing response variables—or discuss in depth when certain models are more appropriate than others. Instead, it focuses on the elements of those models.

Link: Book

Navigational hashtags: #armbooks
General hashtags: #ml #machinelearning

@data_science_weekly

Deep Learning Models by Sebastian Raschka

A collection of various deep learning architectures, models, and tips for TensorFlow and PyTorch in Jupyter Notebooks:
- Traditional Machine Learning
- Multilayer Perceptrons
- Convolutional Neural Networks
- Basic
- Concepts
- AlexNet
- DenseNet
- Fully Convolutional
- LeNet
- MobileNet
- Network in Network
- VGG
- ResNet
- Transformers
- Ordinal Regression and Deep Learning
- Normalization Layers
- Metric Learning
- Autoencoders
- Fully-connected Autoencoders
- Convolutional Autoencoders
- Variational Autoencoders
- Conditional Variational Autoencoders
- Generative Adversarial Networks (GANs)
- Graph Neural Networks (GNNs)
- Recurrent Neural Networks (RNNs)
- Many-to-one: Sentiment Analysis / Classification
- Many-to-Many / Sequence-to-Sequence
- Model Evaluation
- K-Fold Cross-Validation
- Data Augmentation
- Tips and Tricks
- Transfer Learning
- Visualization and Interpretation
- PyTorch Workflows and Mechanics
- PyTorch Lightning Examples
- Custom Datasets
- Training and Preprocessing
- Improving Memory Efficiency
- Parallel Computing
- Other
- Autograd
- TensorFlow Workflows and Mechanics
- Custom Datasets
- Training and Preprocessing
- Related Libraries

Link: GitHub

Navigational hashtags: #armtutorials
General hashtags: #ml #machinelearning #dl #deeplearning #pytorch #tensorflow #tf #pytorchlightning

@data_science_weekly

MLOps guide by Chip Huyen

A collection of materials from introductory to advanced. This is roughly the path she would follow if she were to start my MLOps journey again.

Table of contents:
- ML + engineering fundamentals
- MLOps
- Overview
- Intermediate
- Advanced
- Career
- Case studies
- Bonus

Link: Guide

Navigational hashtags: #armtutorials
General hashtags: #ml #mlops

@data_science_weekly

Deep Learning. Foundations and Concepts by Chris Bishop and Hugh Bishop

"Deep Learning is Springer Nature’s bestselling book of 2024, cementing its position as a cornerstone resource in the field of artificial intelligence."
- Springer Nature

This book offers a comprehensive introduction to the central ideas that underpin deep learning. It is intended both for newcomers to machine learning and for those already experienced in the field. Covering key concepts relating to contemporary architectures and techniques, this essential book equips readers with a robust foundation for potential future specialization. The field of deep learning is undergoing rapid evolution, and therefore this book focusses on ideas that are likely to endure the test of time.

The book is organized into numerous bite-sized chapters, each exploring a distinct topic, and the narrative follows a linear progression, with each chapter building upon content from its predecessors. This structure is well-suited to teaching a two-semester undergraduate or postgraduate machine learning course, while remaining equally relevant to those engaged in active research or in self-study.

A full understanding of machine learning requires some mathematical background and so the book includes a self-contained introduction to probability theory. However, the focus of the book is on conveying a clear understanding of ideas, with emphasis on the real-world practical value of techniques rather than on abstract theory. Complex concepts are therefore presented from multiple complementary perspectives including textual descriptions, diagrams, mathematical formulae, and pseudo-code.

Link: Book

Navigational hashtags: #armbooks
General hashtags: #dl #deeplearning

@data_science_weekly

Full Speed Python by João Miguel Jones Ventura

This book aims to teach the Python programming language using a practical approach. Its method is quite simple: after a short introduction to each topic, the reader is invited to learn more by solving the proposed exercises.

These exercises have been used extensively in author's web development and distributed computing classes at the Superior School of Technology of Setúbal. With these exercises, most students are up to speed with Python in less than a month. In fact, students of the distributed computing course, taught in the second year of the software engineering degree, become familiar with Python's syntax in two weeks and are able to implement a distributed client-server application with sockets in the third week.

Link: Book

Navigational hashtags: #armbooks
General hashtags: #python #programming

@data_science_weekly

Pen and Paper Exercises in Machine Learning by Michael Gutmann

This is a collection of (mostly) pen-and-paper exercises in machine learning. Each exercise comes with a detailed solution. The following topics are covered:
- Linear Algebra
- Optimisation
- Directed Graphical Models
- Undirected Graphical Models
- Expressive Power of Graphical Models
- Factor Graphs and Message Passing
- Inference for Hidden Markov Models
- Model-based Learning (including ICA and unnormalised models)
- Sampling and Monte-Carlo Integration
- Variational Inference

Link: GitHub

Navigational hashtags: #armrepo
General hashtags: #math #mathematics #linearalgebra

@data_science_weekly

Practical Deep Learning for Coders by fast.ai

Practical Deep Learning for Coders 2022 part 1, recorded at the University of Queensland, covers topics such as how to:
- Build and train deep learning models for computer vision, natural language processing, tabular analysis, and collaborative filtering problems
- Create random forests and regression models
- Deploy models
- Use PyTorch, the world’s fastest growing deep learning software, plus popular libraries like fastai and Hugging Face

There are 9 lessons, and each lesson is around 90 minutes long. The course is based on their 5-star rated book, which is freely available online.

Link: Course

Navigational hashtags: #armcourses
General hashtags: #dl #deeplearning #ml #machinelearning

@data_science_weekly

LLM Engineering Essentials by Nebius Academy

Gain the skills to build LLM-powered services that work. Master LLM APIs and self-hosted LLMs as you code, experiment, and create a platform for custom AI-powered NPCs.

During the course you will:
1. Understand the fundamentals of LLM APIs and workflows to create a chatbot based on your favorite fantasy character
2. Learn to work with self-hosted LLMs, encoders, and vector stores, and build a RAG system
3. Explore monitoring tools like Prometheus and Grafana. Optimize and fine-tune your LLM-powered service

Syllabus:

Week 1. LLM Basics
Week 2. LLM Workflows
Week 3. Context
Week 4. Self-served LLMs
Week 5. Optimization and Monitoring
Week 6. Fine-Tuning

Link: GitHub

Navigational hashtags: #armcourses
General hashtags: #llm #largelanguagemodels

@data_science_weekly

The Elements of Statistical Learning: Data Mining, Inference, and Prediction by Trevor Hastie, Robert Tibshirani, and Jerome Friedman

This book describes the important ideas in a variety of fields such as medicine, biology, finance, and marketing in a common conceptual framework. While the approach is statistical, the emphasis is on concepts rather than mathematics. Many examples are given, with a liberal use of colour graphics. It is a valuable resource for statisticians and anyone interested in data mining in science or industry. The book's coverage is broad, from supervised learning (prediction) to unsupervised learning. The many topics include neural networks, support vector machines, classification trees and boosting - the first comprehensive treatment of this topic in any book.

This major new edition features many topics not covered in the original, including graphical models, random forests, ensemble methods, least angle regression & path algorithms for the lasso, non-negative matrix factorisation, and spectral clustering. There is also a chapter on methods for "wide'' data (p bigger than n), including multiple testing and false discovery rates.

Links:
- Book Homepage
- PDF

Navigational hashtags: #armbooks
General hashtags: #ml #machinelearning #supervised #unsupervised

@data_science_weekly

The Book of Statistical Proofs

A centralized, open and collaboratively edited archive of statistical theorems for the computational sciences!

Link: Site

Navigational hashtags: #armbooks #armsites
General hashtags: #statistics

@data_science_weekly

Machine Learning Refined. Foundations, Algorithms, and Applications by Jeremy Watt, Reza Borhani and Aggelos K. Katsaggelos

Now more than ever, it is crucial to understand the core foundations of AI and machine learning. True mastery of a subject means understanding its tenets from multiple, complementary angles. 

Ideally, this means being able to explain what you know intuitively.  
- Being able to draw a picture of an idea plainly on a cocktail napkin.
- Being able to recall key formulae that rigorously support or define an idea. 
- And finally, being able to apply a concept practically, in code.
 
This book aims to lead you towards this mastery of AI fundamentals by explaining every concept intuitively first, visually second, mathematically third, and fourth in code. In that order.  For every major concept.

Links:
- Site
- GitHub

Navigational hashtags: #armbooks #armsites
General hashtags: #ml #machinelearning #optimization #regression #classification #nn #neuralnetworks #trees

@data_science_weekly

Introduction to Machine Learning by Laurent Younes

This book introduces the mathematical foundations and techniques that lead to the development and analysis of many of the algorithms that are used in machine learning.

It starts with an introductory chapter that describes notation used throughout the book and serve at a reminder of basic concepts in calculus, linear algebra and probability and also introduces some measure theoretic terminology, which can be used as a reading guide for the sections that use these tools. The introductory chapters also provide background material on matrix analysis and optimization. The latter chapter provides theoretical support to many algorithms that are used in the book, including stochastic gradient descent, proximal methods, etc.

After discussing basic concepts for statistical prediction, the book includes an introduction to reproducing kernel theory and Hilbert space techniques, which are used in many places, before addressing the description of various algorithms for supervised statistical learning, including linear methods, support vector machines, decision trees, boosting, or neural networks.

The subject then switches to generative methods, starting with a chapter that presents sampling methods and an introduction to the theory of Markov chains.

The following chapter describe the theory of graphical models, an introduction to variational methods for models with latent variables, and to deep-learning based generative models.

The next chapters focus on unsupervised learning methods, for clustering, factor analysis and manifold learning.

The final chapter of the book is theory-oriented and discusses concentration inequalities and generalization bounds.

Links:
- arXiv
- pdf

Navigational hashtags: #armbooks
General hashtags: #ml #machinelearning #optimization #regression #classification #nn #neuralnetworks #trees

@data_science_weekly

Multimodal Deep Learning

In the last few years, there have been several breakthroughs in the methodologies used in Natural Language Processing (NLP) as well as Computer Vision (CV). Beyond these improvements on single-modality models, large-scale multi-modal approaches have become a very active area of research.

In this seminar, authors reviewed these approaches and attempted to create a solid overview of the field, starting with the current state-of-the-art approaches in the two subfields of Deep Learning individually.

- Further, modeling frameworks are discussed where one modality is transformed into the other Chapter 3.1 and Chapter 3.2), as well as models in which one modality is utilized to enhance representation learning for the other (Chapter 3.3 and Chapter 3.4). To conclude the second part, architectures with a focus on handling both modalities simultaneously are introduced (Chapter 3.5).
- Finally, they also cover other modalities (Chapter 4.1 and Chapter 4.2) as well as general-purpose multi-modal models (Chapter 4.3), which are able to handle different tasks on different modalities within one unified architecture. One interesting application (Generative Art, Chapter 4.4) eventually caps off this booklet.

Links:
- Book

Navigational hashtags: #armbooks #armsite
General hashtags: #dl #deeplearning #nlp #cv

@data_science_weekly

The Tensor Cookbook by Thomas Dybdahl Ahle

What are Tensor Diagrams? Machine learning involves a lot of tensor manipulation, and it's easy to lose track of the larger structure when manipulating high-dimensional data using notation designed for vectors and matrices.
Graphical notation (first introduced by Roger Penrose in 1971) reduces the mental overhead and makes the connections "come alive":

In short, each edge is the index of a tensor, and connecting two edges contracts the tensors over this dimension. After a bit of practice, this becomes incredibly intuitive.
The Tensor Cookbook aims to popularize tensor diagrams by rewriting the classical "Matrix Cookbook". You can think of it as a reference book, skip around for some cool diagrams, or a crash course full of exercises to practice your skill.

Links:
- Book
- Site

Navigational hashtags: #armbooks #armsite
General hashtags: #tensor #matrix #derivative #statistics #probability #ml

@data_science_weekly