Data Science & Machine Learning
73.3K subscribers
791 photos
2 videos
68 files
690 links
Join this channel to learn data science, artificial intelligence and machine learning with funny quizzes, interesting projects and amazing resources for free

For collaborations: @love_data
Download Telegram
Many data scientists don't know how to push ML models to production. Here's the recipe ๐Ÿ‘‡

๐—ž๐—ฒ๐˜† ๐—œ๐—ป๐—ด๐—ฟ๐—ฒ๐—ฑ๐—ถ๐—ฒ๐—ป๐˜๐˜€

๐Ÿ”น ๐—ง๐—ฟ๐—ฎ๐—ถ๐—ป / ๐—ง๐—ฒ๐˜€๐˜ ๐——๐—ฎ๐˜๐—ฎ๐˜€๐—ฒ๐˜ - Ensure Test is representative of Online data
๐Ÿ”น ๐—™๐—ฒ๐—ฎ๐˜๐˜‚๐—ฟ๐—ฒ ๐—˜๐—ป๐—ด๐—ถ๐—ป๐—ฒ๐—ฒ๐—ฟ๐—ถ๐—ป๐—ด ๐—ฃ๐—ถ๐—ฝ๐—ฒ๐—น๐—ถ๐—ป๐—ฒ - Generate features in real-time
๐Ÿ”น ๐— ๐—ผ๐—ฑ๐—ฒ๐—น ๐—ข๐—ฏ๐—ท๐—ฒ๐—ฐ๐˜ - Trained SkLearn or Tensorflow Model
๐Ÿ”น ๐—ฃ๐—ฟ๐—ผ๐—ท๐—ฒ๐—ฐ๐˜ ๐—–๐—ผ๐—ฑ๐—ฒ ๐—ฅ๐—ฒ๐—ฝ๐—ผ - Save model project code to Github
๐Ÿ”น ๐—”๐—ฃ๐—œ ๐—™๐—ฟ๐—ฎ๐—บ๐—ฒ๐˜„๐—ผ๐—ฟ๐—ธ - Use FastAPI or Flask to build a model API
๐Ÿ”น ๐——๐—ผ๐—ฐ๐—ธ๐—ฒ๐—ฟ - Containerize the ML model API
๐Ÿ”น ๐—ฅ๐—ฒ๐—บ๐—ผ๐˜๐—ฒ ๐—ฆ๐—ฒ๐—ฟ๐˜ƒ๐—ฒ๐—ฟ - Choose a cloud service; e.g. AWS sagemaker
๐Ÿ”น ๐—จ๐—ป๐—ถ๐˜ ๐—ง๐—ฒ๐˜€๐˜๐˜€ - Test inputs & outputs of functions and APIs
๐Ÿ”น ๐— ๐—ผ๐—ฑ๐—ฒ๐—น ๐— ๐—ผ๐—ป๐—ถ๐˜๐—ผ๐—ฟ๐—ถ๐—ป๐—ด - Evidently AI, a simple, open-source for ML monitoring

๐—ฃ๐—ฟ๐—ผ๐—ฐ๐—ฒ๐—ฑ๐˜‚๐—ฟ๐—ฒ

๐—ฆ๐˜๐—ฒ๐—ฝ ๐Ÿญ - ๐——๐—ฎ๐˜๐—ฎ ๐—ฃ๐—ฟ๐—ฒ๐—ฝ๐—ฎ๐—ฟ๐—ฎ๐˜๐—ถ๐—ผ๐—ป & ๐—™๐—ฒ๐—ฎ๐˜๐˜‚๐—ฟ๐—ฒ ๐—˜๐—ป๐—ด๐—ถ๐—ป๐—ฒ๐—ฒ๐—ฟ๐—ถ๐—ป๐—ด

Don't push a model with 90% accuracy on train set. Do it based on the test set - if and only if, the test set is representative of the online data. Use SkLearn pipeline to chain a series of model preprocessing functions like null handling.

๐—ฆ๐˜๐—ฒ๐—ฝ ๐Ÿฎ - ๐— ๐—ผ๐—ฑ๐—ฒ๐—น ๐——๐—ฒ๐˜ƒ๐—ฒ๐—น๐—ผ๐—ฝ๐—บ๐—ฒ๐—ป๐˜

Train your model with frameworks like Sklearn or Tensorflow. Push the model code including preprocessing, training and validation scripts to Github for reproducibility.

๐—ฆ๐˜๐—ฒ๐—ฝ ๐Ÿฏ - ๐—”๐—ฃ๐—œ ๐——๐—ฒ๐˜ƒ๐—ฒ๐—น๐—ผ๐—ฝ๐—บ๐—ฒ๐—ป๐˜ & ๐—–๐—ผ๐—ป๐˜๐—ฎ๐—ถ๐—ป๐—ฒ๐—ฟ๐—ถ๐˜‡๐—ฎ๐˜๐—ถ๐—ผ๐—ป

Your model needs a "/predict" endpoint, which receives a JSON object in the request input and generates a JSON object with the model score in the response output. You can use frameworks like FastAPI or Flask. Containzerize this API so that it's agnostic to server environment

๐—ฆ๐˜๐—ฒ๐—ฝ ๐Ÿฐ - ๐—ง๐—ฒ๐˜€๐˜๐—ถ๐—ป๐—ด & ๐——๐—ฒ๐—ฝ๐—น๐—ผ๐˜†๐—บ๐—ฒ๐—ป๐˜

Write tests to validate inputs & outputs of API functions to prevent errors. Push the code to remote services like AWS Sagemaker.

๐—ฆ๐˜๐—ฒ๐—ฝ ๐Ÿฑ - ๐— ๐—ผ๐—ป๐—ถ๐˜๐—ผ๐—ฟ๐—ถ๐—ป๐—ด

Set up monitoring tools like Evidently AI, or use a built-in one within AWS Sagemaker. I use such tools to track performance metrics and data drifts on online data.

Data Science Resources
๐Ÿ‘‡๐Ÿ‘‡
https://whatsapp.com/channel/0029Va8v3eo1NCrQfGMseL2D

Like if you need similar content ๐Ÿ˜„๐Ÿ‘
๐Ÿ‘12โค1
Here is the list of few projects (found on kaggle). They cover Basics of Python, Advanced Statistics, Supervised Learning (Regression and Classification problems) & Data Science

Please also check the discussions and notebook submissions for different approaches and solution after you tried yourself.

1. Basic python and statistics

Pima Indians :- https://www.kaggle.com/uciml/pima-indians-diabetes-database
Cardio Goodness fit :- https://www.kaggle.com/saurav9786/cardiogoodfitness
Automobile :- https://www.kaggle.com/toramky/automobile-dataset

2. Advanced Statistics

Game of Thrones:-https://www.kaggle.com/mylesoneill/game-of-thrones
World University Ranking:-https://www.kaggle.com/mylesoneill/world-university-rankings
IMDB Movie Dataset:- https://www.kaggle.com/carolzhangdc/imdb-5000-movie-dataset

3. Supervised Learning

a) Regression Problems

How much did it rain :- https://www.kaggle.com/c/how-much-did-it-rain-ii/overview
Inventory Demand:- https://www.kaggle.com/c/grupo-bimbo-inventory-demand
Property Inspection predictiion:- https://www.kaggle.com/c/liberty-mutual-group-property-inspection-prediction
Restaurant Revenue prediction:- https://www.kaggle.com/c/restaurant-revenue-prediction/data
IMDB Box office Prediction:-https://www.kaggle.com/c/tmdb-box-office-prediction/overview

b) Classification problems

Employee Access challenge :- https://www.kaggle.com/c/amazon-employee-access-challenge/overview
Titanic :- https://www.kaggle.com/c/titanic
San Francisco crime:- https://www.kaggle.com/c/sf-crime
Customer satisfcation:-https://www.kaggle.com/c/santander-customer-satisfaction
Trip type classification:- https://www.kaggle.com/c/walmart-recruiting-trip-type-classification
Categorize cusine:- https://www.kaggle.com/c/whats-cooking

4. Some helpful Data science projects for beginners

https://www.kaggle.com/c/house-prices-advanced-regression-techniques

https://www.kaggle.com/c/digit-recognizer

https://www.kaggle.com/c/titanic

5. Intermediate Level Data science Projects

Black Friday Data : https://www.kaggle.com/sdolezel/black-friday

Human Activity Recognition Data : https://www.kaggle.com/uciml/human-activity-recognition-with-smartphones

Trip History Data : https://www.kaggle.com/pronto/cycle-share-dataset

Million Song Data : https://www.kaggle.com/c/msdchallenge

Census Income Data : https://www.kaggle.com/c/census-income/data

Movie Lens Data : https://www.kaggle.com/grouplens/movielens-20m-dataset

Twitter Classification Data : https://www.kaggle.com/c/twitter-sentiment-analysis2

Share with credits: https://t.iss.one/sqlproject

ENJOY LEARNING ๐Ÿ‘๐Ÿ‘
๐Ÿ‘5
Data Science Learning Plan

Step 1: Mathematics for Data Science (Statistics, Probability, Linear Algebra)

Step 2: Python for Data Science (Basics and Libraries)

Step 3: Data Manipulation and Analysis (Pandas, NumPy)

Step 4: Data Visualization (Matplotlib, Seaborn, Plotly)

Step 5: Databases and SQL for Data Retrieval

Step 6: Introduction to Machine Learning (Supervised and Unsupervised Learning)

Step 7: Data Cleaning and Preprocessing

Step 8: Feature Engineering and Selection

Step 9: Model Evaluation and Tuning

Step 10: Deep Learning (Neural Networks, TensorFlow, Keras)

Step 11: Working with Big Data (Hadoop, Spark)

Step 12: Building Data Science Projects and Portfolio

Data Science Resources
๐Ÿ‘‡๐Ÿ‘‡
https://whatsapp.com/channel/0029Va4QUHa6rsQjhITHK82y

Like for more ๐Ÿ˜„
๐Ÿ‘4โค1
Practice projects to consider:

1. Implement a basic search engine:
Read a set of documents and build an index of keywords. Then, implement a search function that returns a list of documents that match the query.

2. Build a recommendation system: Read a set of user-item interactions and build a recommendation system that suggests items to users based on their past behavior.

3. Create a data analysis tool: Read a large dataset and implement a tool that performs various analyses, such as calculating summary statistics, visualizing distributions, and identifying patterns and correlations.

4. Implement a graph algorithm: Study a graph algorithm such as Dijkstra's shortest path algorithm, and implement it in Python. Then, test it on real-world graphs to see how it performs.
โค4๐Ÿ‘1
Hey Guys๐Ÿ‘‹,

The Average Salary Of a Data Scientist is 14LPA 

๐๐ž๐œ๐จ๐ฆ๐ž ๐š ๐‚๐ž๐ซ๐ญ๐ข๐Ÿ๐ข๐ž๐ ๐ƒ๐š๐ญ๐š ๐’๐œ๐ข๐ž๐ง๐ญ๐ข๐ฌ๐ญ ๐ˆ๐ง ๐“๐จ๐ฉ ๐Œ๐๐‚๐ฌ๐Ÿ˜

We help you master the required skills.

Learn by doing, build Industry level projects

๐Ÿ‘ฉโ€๐ŸŽ“ 1500+ Students Placed
๐Ÿ’ผ 7.2 LPA Avg. Package
๐Ÿ’ฐ 41 LPA Highest Package
๐Ÿค 450+ Hiring Partners

Apply for FREE๐Ÿ‘‡ :
https://tracking.acciojob.com/g/PUfdDxgHR

( Limited Slots )
โค4๐Ÿ‘2
A-Z of essential data science concepts

A: Algorithm - A set of rules or instructions for solving a problem or completing a task.
B: Big Data - Large and complex datasets that traditional data processing applications are unable to handle efficiently.
C: Classification - A type of machine learning task that involves assigning labels to instances based on their characteristics.
D: Data Mining - The process of discovering patterns and extracting useful information from large datasets.
E: Ensemble Learning - A machine learning technique that combines multiple models to improve predictive performance.
F: Feature Engineering - The process of selecting, extracting, and transforming features from raw data to improve model performance.
G: Gradient Descent - An optimization algorithm used to minimize the error of a model by adjusting its parameters iteratively.
H: Hypothesis Testing - A statistical method used to make inferences about a population based on sample data.
I: Imputation - The process of replacing missing values in a dataset with estimated values.
J: Joint Probability - The probability of the intersection of two or more events occurring simultaneously.
K: K-Means Clustering - A popular unsupervised machine learning algorithm used for clustering data points into groups.
L: Logistic Regression - A statistical model used for binary classification tasks.
M: Machine Learning - A subset of artificial intelligence that enables systems to learn from data and improve performance over time.
N: Neural Network - A computer system inspired by the structure of the human brain, used for various machine learning tasks.
O: Outlier Detection - The process of identifying observations in a dataset that significantly deviate from the rest of the data points.
P: Precision and Recall - Evaluation metrics used to assess the performance of classification models.
Q: Quantitative Analysis - The process of using mathematical and statistical methods to analyze and interpret data.
R: Regression Analysis - A statistical technique used to model the relationship between a dependent variable and one or more independent variables.
S: Support Vector Machine - A supervised machine learning algorithm used for classification and regression tasks.
T: Time Series Analysis - The study of data collected over time to detect patterns, trends, and seasonal variations.
U: Unsupervised Learning - Machine learning techniques used to identify patterns and relationships in data without labeled outcomes.
V: Validation - The process of assessing the performance and generalization of a machine learning model using independent datasets.
W: Weka - A popular open-source software tool used for data mining and machine learning tasks.
X: XGBoost - An optimized implementation of gradient boosting that is widely used for classification and regression tasks.
Y: Yarn - A resource manager used in Apache Hadoop for managing resources across distributed clusters.
Z: Zero-Inflated Model - A statistical model used to analyze data with excess zeros, commonly found in count data.

Like for more ๐Ÿ˜„
๐Ÿ‘13โค8
๐Ÿ˜‚๐Ÿ˜‚
๐Ÿ˜29๐Ÿ‘5๐Ÿ˜ข3๐Ÿคฉ1
Accenture Data Scientist Interview Questions!

1st round-

Technical Round

- 2 SQl questions based on playing around views and table, which could be solved by both subqueries and window functions.

- 2 Pandas questions , testing your knowledge on filtering , concatenation , joins and merge.

- 3-4 Machine Learning questions completely based on my Projects, starting from
Explaining the problem statements and then discussing the roadblocks of those projects and some cross questions.

2nd round-

- Couple of python questions agains on pandas and numpy and some hypothetical data.

- Machine Learning projects explanations and cross questions.

- Case Study and a quiz question.

3rd and Final round.

HR interview

Simple Scenerio Based Questions.

Data Science Resources
๐Ÿ‘‡๐Ÿ‘‡
https://whatsapp.com/channel/0029Va4QUHa6rsQjhITHK82y

Like if you need similar content ๐Ÿ˜„๐Ÿ‘
๐Ÿ‘5โค1
๐ŸŒŸ Embark on a Journey of Discovery and Innovation with @DeepLearning_ai! and @MachineLearning_Programming ๐ŸŒŸ

What We Offer:
* ๐Ÿง  Deep Dives into AI & ML
.
* ๐Ÿค– Latest in Deep Learning.
* ๐Ÿ“Š Data Science Mastery.
* ๐Ÿ‘ Computer Vision & Image Processing.
* ๐Ÿ“š Exclusive Access to Research Papers.

Why Us?
* Connect with experts and enthusiasts.
* Stay updated, stay ahead.
* Empower your knowledge and career in tech.

Ready for a deep dive? Click here to explore, learn, and grow with
@DeepLearning_ai

@MachineLearning_Programming!

Step into the futureโ€”today.
๐Ÿ‘5โค1๐Ÿ”ฅ1๐ŸŽ‰1๐Ÿคฉ1
Probability for Data Science
๐Ÿ”ฅ7โค5๐Ÿ‘4
Resume key words for data scientist role explained in points:

1. Data Analysis:
   - Proficient in extracting, cleaning, and analyzing data to derive insights.
   - Skilled in using statistical methods and machine learning algorithms for data analysis.
   - Experience with tools such as Python, R, or SQL for data manipulation and analysis.

2. Machine Learning:
   - Strong understanding of machine learning techniques such as regression, classification, clustering, and neural networks.
- Experience in model development, evaluation, and deployment.
   - Familiarity with libraries like TensorFlow, scikit-learn, or PyTorch for implementing machine learning models.

3. Data Visualization:
   - Ability to present complex data in a clear and understandable manner through visualizations.
   - Proficiency in tools like Matplotlib, Seaborn, or Tableau for creating insightful graphs and charts.
   - Understanding of best practices in data visualization for effective communication of findings.

4. Big Data:
   - Experience working with large datasets using technologies like Hadoop, Spark, or Apache Flink.
   - Knowledge of distributed computing principles and tools for processing and analyzing big data.
   - Ability to optimize algorithms and processes for scalability and performance.

5. Problem-Solving:
   - Strong analytical and problem-solving skills to tackle complex data-related challenges.
   - Ability to formulate hypotheses, design experiments, and iterate on solutions.
   - Aptitude for identifying opportunities for leveraging data to drive business outcomes and decision-making.


Resume key words for a data analyst role

1. SQL (Structured Query Language):
   - SQL is a programming language used for managing and querying relational databases.
   - Data analysts often use SQL to extract, manipulate, and analyze data stored in databases, making it a fundamental skill for the role.

2. Python/R:
   - Python and R are popular programming languages used for data analysis and statistical computing.
   - Proficiency in Python or R allows data analysts to perform various tasks such as data cleaning, modeling, visualization, and machine learning.

3. Data Visualization:
   - Data visualization involves presenting data in graphical or visual formats to communicate insights effectively.
   - Data analysts use tools like Tableau, Power BI, or Python libraries like Matplotlib and Seaborn to create visualizations that help stakeholders understand complex data patterns and trends.

4. Statistical Analysis:
   - Statistical analysis involves applying statistical methods to analyze and interpret data.
   - Data analysts use statistical techniques to uncover relationships, trends, and patterns in data, providing valuable insights for decision-making.

5. Data-driven Decision Making:
   - Data-driven decision making is the process of making decisions based on data analysis and evidence rather than intuition or gut feelings.
   - Data analysts play a crucial role in helping organizations make informed decisions by analyzing data and providing actionable insights that drive business strategies and operations.

Data Science Interview Resources
๐Ÿ‘‡๐Ÿ‘‡
https://whatsapp.com/channel/0029Va4QUHa6rsQjhITHK82y

Like for more ๐Ÿ˜„
๐Ÿ‘13โค2
ML Interview Question โฌ‡๏ธ

โžก๏ธ Logistic Regression

The interviewer asked to explain Logistic Regression along with its:

๐Ÿ”ท Cost function
๐Ÿ”ท Assumptions
๐Ÿ”ท Evaluation metrics

Here is the step by step approach to answer:

โ˜‘๏ธ Cost function: Point out how logistic regression uses log loss for classification.

โ˜‘๏ธ Assumptions: Explain LR assumes features are independent and they have a linear link.

โ˜‘๏ธ Evaluation metrics: Discuss accuracy, precision, and F1-score to measure performance.

Knowing every concept is important but more than that, it is important to convey our knowledge๐Ÿ’ฏ

Data Science Resources
๐Ÿ‘‡๐Ÿ‘‡
https://whatsapp.com/channel/0029Va4QUHa6rsQjhITHK82y

Like if you need similar content ๐Ÿ˜„๐Ÿ‘
๐Ÿ‘10
๐Ÿš€ Top 10 Tools Data Scientists Love! ๐Ÿง 

In the ever-evolving world of data science, staying updated with the right tools is crucial to solving complex problems and deriving meaningful insights.

๐Ÿ” Hereโ€™s a quick breakdown of the most popular tools:

1. Python ๐Ÿ: The go-to language for data science, favored for its versatility and powerful libraries.
2. SQL ๐Ÿ› ๏ธ: Essential for querying databases and manipulating data.
3. Jupyter Notebooks ๐Ÿ““: An interactive environment that makes data analysis and visualization a breeze.
4. TensorFlow/PyTorch ๐Ÿค–: Leading frameworks for deep learning and neural networks.
5. Tableau ๐Ÿ“Š: A user-friendly tool for creating stunning visualizations and dashboards.
6. Git & GitHub ๐Ÿ’ป: Version control systems that every data scientist should master.
7. Hadoop & Spark ๐Ÿ”ฅ: Big data frameworks that help process massive datasets efficiently.
8. Scikit-learn ๐Ÿงฌ: A powerful library for machine learning in Python.
9. R ๐Ÿ“ˆ: A statistical programming language that is still a favorite among many analysts.
10. Docker ๐Ÿ‹: A must-have for containerization and deploying applications.

I have curated the best interview resources to crack Data Science Interviews
๐Ÿ‘‡๐Ÿ‘‡
https://whatsapp.com/channel/0029Va4QUHa6rsQjhITHK82y

Like if you need similar content ๐Ÿ˜„๐Ÿ‘
๐Ÿ‘6โค1
What ๐— ๐—Ÿ ๐—ฐ๐—ผ๐—ป๐—ฐ๐—ฒ๐—ฝ๐˜๐˜€ are commonly asked in ๐—ฑ๐—ฎ๐˜๐—ฎ ๐˜€๐—ฐ๐—ถ๐—ฒ๐—ป๐—ฐ๐—ฒ ๐—ถ๐—ป๐˜๐—ฒ๐—ฟ๐˜ƒ๐—ถ๐—ฒ๐˜„๐˜€?

These are fair game in interviews at ๐˜€๐˜๐—ฎ๐—ฟ๐˜๐˜‚๐—ฝ๐˜€, ๐—ฐ๐—ผ๐—ป๐˜€๐˜‚๐—น๐˜๐—ถ๐—ป๐—ด & ๐—น๐—ฎ๐—ฟ๐—ด๐—ฒ ๐˜๐—ฒ๐—ฐ๐—ต.

๐—™๐˜‚๐—ป๐—ฑ๐—ฎ๐—บ๐—ฒ๐—ป๐˜๐—ฎ๐—น๐˜€
- Supervised vs. Unsupervised Learning
- Overfitting and Underfitting
- Cross-validation
- Bias-Variance Tradeoff
- Accuracy vs Interpretability
- Accuracy vs Latency

๐— ๐—Ÿ ๐—”๐—น๐—ด๐—ผ๐—ฟ๐—ถ๐˜๐—ต๐—บ๐˜€
- Logistic Regression
- Decision Trees
- Random Forest
- Support Vector Machines
- K-Nearest Neighbors
- Naive Bayes
- Linear Regression
- Ridge and Lasso Regression
- K-Means Clustering
- Hierarchical Clustering
- PCA

๐— ๐—ผ๐—ฑ๐—ฒ๐—น๐—ถ๐—ป๐—ด ๐—ฆ๐˜๐—ฒ๐—ฝ๐˜€
- EDA
- Data Cleaning (e.g. missing value imputation)
- Data Preprocessing (e.g. scaling)
- Feature Engineering (e.g. aggregation)
- Feature Selection (e.g. variable importance)
- Model Training (e.g. gradient descent)
- Model Evaluation (e.g. AUC vs Accuracy)
- Model Productionization

๐—›๐˜†๐—ฝ๐—ฒ๐—ฟ๐—ฝ๐—ฎ๐—ฟ๐—ฎ๐—บ๐—ฒ๐˜๐—ฒ๐—ฟ ๐—ง๐˜‚๐—ป๐—ถ๐—ป๐—ด
- Grid Search
- Random Search
- Bayesian Optimization

๐— ๐—Ÿ ๐—–๐—ฎ๐˜€๐—ฒ๐˜€
- [Capital One] Detect credit card fraudsters
- [Amazon] Forecast monthly sales
- [Airbnb] Estimate lifetime value of a guest

I have curated the best interview resources to crack Data Science Interviews
๐Ÿ‘‡๐Ÿ‘‡
https://whatsapp.com/channel/0029Va4QUHa6rsQjhITHK82y

Like if you need similar content ๐Ÿ˜„๐Ÿ‘
๐Ÿ‘3โค2
Lol ๐Ÿ˜‚
๐Ÿ˜26๐Ÿ‘4๐Ÿค”2โค1