HandsOnLLM/Hands-On-Large-Language-Models
Official code repo for the O'Reilly Book - "Hands-On Large Language Models"
Language:Jupyter Notebook
Total stars: 194
Stars trend:
#jupyternotebook
#artificialintelligence, #book, #largelanguagemodels, #llm, #llms, #oreilly, #oreillybooks
Official code repo for the O'Reilly Book - "Hands-On Large Language Models"
Language:Jupyter Notebook
Total stars: 194
Stars trend:
16 Sep 2024
5pm ▊ +6
6pm ▊ +6
7pm ▉ +7
8pm ▎ +2
9pm ▍ +3
10pm ▌ +4
11pm ▍ +3
17 Sep 2024
12am ▏ +1
1am ▍ +3
2am ▋ +5
3am ██▎ +18
4am ██▏ +17#jupyternotebook
#artificialintelligence, #book, #largelanguagemodels, #llm, #llms, #oreilly, #oreillybooks
👍5❤1
Tools Every AI Engineer Should Know
1. Data Science Tools
Python: Preferred language with libraries like NumPy, Pandas, Scikit-learn.
R: Ideal for statistical analysis and data visualization.
Jupyter Notebook: Interactive coding environment for Python and R.
MATLAB: Used for mathematical modeling and algorithm development.
RapidMiner: Drag-and-drop platform for machine learning workflows.
KNIME: Open-source analytics platform for data integration and analysis.
2. Machine Learning Tools
Scikit-learn: Comprehensive library for traditional ML algorithms.
XGBoost & LightGBM: Specialized tools for gradient boosting.
TensorFlow: Open-source framework for ML and DL.
PyTorch: Popular DL framework with a dynamic computation graph.
H2O.ai: Scalable platform for ML and AutoML.
Auto-sklearn: AutoML for automating the ML pipeline.
3. Deep Learning Tools
Keras: User-friendly high-level API for building neural networks.
PyTorch: Excellent for research and production in DL.
TensorFlow: Versatile for both research and deployment.
ONNX: Open format for model interoperability.
OpenCV: For image processing and computer vision.
Hugging Face: Focused on natural language processing.
4. Data Engineering Tools
Apache Hadoop: Framework for distributed storage and processing.
Apache Spark: Fast cluster-computing framework.
Kafka: Distributed streaming platform.
Airflow: Workflow automation tool.
Fivetran: ETL tool for data integration.
dbt: Data transformation tool using SQL.
5. Data Visualization Tools
Tableau: Drag-and-drop BI tool for interactive dashboards.
Power BI: Microsoft’s BI platform for data analysis and visualization.
Matplotlib & Seaborn: Python libraries for static and interactive plots.
Plotly: Interactive plotting library with Dash for web apps.
D3.js: JavaScript library for creating dynamic web visualizations.
6. Cloud Platforms
AWS: Services like SageMaker for ML model building.
Google Cloud Platform (GCP): Tools like BigQuery and AutoML.
Microsoft Azure: Azure ML Studio for ML workflows.
IBM Watson: AI platform for custom model development.
7. Version Control and Collaboration Tools
Git: Version control system.
GitHub/GitLab: Platforms for code sharing and collaboration.
Bitbucket: Version control for teams.
8. Other Essential Tools
Docker: For containerizing applications.
Kubernetes: Orchestration of containerized applications.
MLflow: Experiment tracking and deployment.
Weights & Biases (W&B): Experiment tracking and collaboration.
Pandas Profiling: Automated data profiling.
BigQuery/Athena: Serverless data warehousing tools.
Mastering these tools will ensure you are well-equipped to handle various challenges across the AI lifecycle.
#artificialintelligence
1. Data Science Tools
Python: Preferred language with libraries like NumPy, Pandas, Scikit-learn.
R: Ideal for statistical analysis and data visualization.
Jupyter Notebook: Interactive coding environment for Python and R.
MATLAB: Used for mathematical modeling and algorithm development.
RapidMiner: Drag-and-drop platform for machine learning workflows.
KNIME: Open-source analytics platform for data integration and analysis.
2. Machine Learning Tools
Scikit-learn: Comprehensive library for traditional ML algorithms.
XGBoost & LightGBM: Specialized tools for gradient boosting.
TensorFlow: Open-source framework for ML and DL.
PyTorch: Popular DL framework with a dynamic computation graph.
H2O.ai: Scalable platform for ML and AutoML.
Auto-sklearn: AutoML for automating the ML pipeline.
3. Deep Learning Tools
Keras: User-friendly high-level API for building neural networks.
PyTorch: Excellent for research and production in DL.
TensorFlow: Versatile for both research and deployment.
ONNX: Open format for model interoperability.
OpenCV: For image processing and computer vision.
Hugging Face: Focused on natural language processing.
4. Data Engineering Tools
Apache Hadoop: Framework for distributed storage and processing.
Apache Spark: Fast cluster-computing framework.
Kafka: Distributed streaming platform.
Airflow: Workflow automation tool.
Fivetran: ETL tool for data integration.
dbt: Data transformation tool using SQL.
5. Data Visualization Tools
Tableau: Drag-and-drop BI tool for interactive dashboards.
Power BI: Microsoft’s BI platform for data analysis and visualization.
Matplotlib & Seaborn: Python libraries for static and interactive plots.
Plotly: Interactive plotting library with Dash for web apps.
D3.js: JavaScript library for creating dynamic web visualizations.
6. Cloud Platforms
AWS: Services like SageMaker for ML model building.
Google Cloud Platform (GCP): Tools like BigQuery and AutoML.
Microsoft Azure: Azure ML Studio for ML workflows.
IBM Watson: AI platform for custom model development.
GitHub/GitLab: Platforms for code sharing and collaboration.
Bitbucket: Version control for teams.
8. Other Essential Tools
Docker: For containerizing applications.
Kubernetes: Orchestration of containerized applications.
MLflow: Experiment tracking and deployment.
Weights & Biases (W&B): Experiment tracking and collaboration.
Pandas Profiling: Automated data profiling.
BigQuery/Athena: Serverless data warehousing tools.
Mastering these tools will ensure you are well-equipped to handle various challenges across the AI lifecycle.
#artificialintelligence
👍7
10 Python Libraries Every AI Engineer Should Know
1. Hugging Face Transformers
A powerful library for using and fine-tuning pre-trained transformer models for NLP. Learn more: Hugging Face NLP Course
2. Ollama
A framework for running and managing open-source LLMs locally with ease. Learn video: Ollama Course
3. OpenAI Python SDK
The official toolkit for integrating OpenAI models into Python applications. Learn more: The official developer quickstart guide
4. Anthropic SDK
A client library for seamless interaction with Claude and other Anthropic models. Learn more: Anthropic Python SDK
5. LangChain
A framework for building LLM applications with modular and extensible components. Learn more: DeepLearning.AI
6. LlamaIndex
A toolkit for integrating custom data sources with LLMs for better retrieval. Learn more: Building Agentic RAG with LlamaIndex
7. SQLAlchemy
A Python SQL toolkit and ORM for efficient and maintainable database interactions. Learn more: SQLAlchemy Unified Tutorial
8. ChromaDB
An open-source vector database optimized for AI-powered search and retrieval. Learn more: Getting Started - Chroma Docs
9. Weaviate
A cloud-native vector search engine for efficient semantic search at scale. Learn more: 101T Work with: Text data
10. Weights & Biases
A platform for tracking, visualizing, and optimizing ML experiments.
Learn more: Effective MLOps: Model Development
#artificialintelligence
1. Hugging Face Transformers
A powerful library for using and fine-tuning pre-trained transformer models for NLP. Learn more: Hugging Face NLP Course
2. Ollama
A framework for running and managing open-source LLMs locally with ease. Learn video: Ollama Course
3. OpenAI Python SDK
The official toolkit for integrating OpenAI models into Python applications. Learn more: The official developer quickstart guide
4. Anthropic SDK
A client library for seamless interaction with Claude and other Anthropic models. Learn more: Anthropic Python SDK
5. LangChain
A framework for building LLM applications with modular and extensible components. Learn more: DeepLearning.AI
6. LlamaIndex
A toolkit for integrating custom data sources with LLMs for better retrieval. Learn more: Building Agentic RAG with LlamaIndex
7. SQLAlchemy
A Python SQL toolkit and ORM for efficient and maintainable database interactions. Learn more: SQLAlchemy Unified Tutorial
8. ChromaDB
An open-source vector database optimized for AI-powered search and retrieval. Learn more: Getting Started - Chroma Docs
9. Weaviate
A cloud-native vector search engine for efficient semantic search at scale. Learn more: 101T Work with: Text data
10. Weights & Biases
A platform for tracking, visualizing, and optimizing ML experiments.
Learn more: Effective MLOps: Model Development
#artificialintelligence
👍4
🔰 How to become a data scientist in 2025?
👨🏻💻 If you want to become a data science professional, follow this path! I've prepared a complete roadmap with the best free resources where you can learn the essential skills in this field.
🔢 Step 1: Strengthen your math and statistics!
✏️ The foundation of learning data science is mathematics, linear algebra, statistics, and probability. Topics you should master:
✅ Linear algebra: matrices, vectors, eigenvalues.
🔗 Course: MIT 18.06 Linear Algebra
✅ Calculus: derivative, integral, optimization.
🔗 Course: MIT Single Variable Calculus
✅ Statistics and probability: Bayes' theorem, hypothesis testing.
🔗 Course: Statistics 110
➖➖➖➖➖
🔢 Step 2: Learn to code.
✏️ Learn Python and become proficient in coding. The most important topics you need to master are:
✅ Python: Pandas, NumPy, Matplotlib libraries
🔗 Course: FreeCodeCamp Python Course
✅ SQL language: Join commands, Window functions, query optimization.
🔗 Course: Stanford SQL Course
✅ Data structures and algorithms: arrays, linked lists, trees.
🔗 Course: MIT Introduction to Algorithms
➖➖➖➖➖
🔢 Step 3: Clean and visualize data
✏️ Learn how to process and clean data and then create an engaging story from it!
✅ Data cleaning: Working with missing values and detecting outliers.
🔗 Course: Data Cleaning
✅ Data visualization: Matplotlib, Seaborn, Tableau
🔗 Course: Data Visualization Tutorial
➖➖➖➖➖
🔢 Step 4: Learn Machine Learning
✏️ It's time to enter the exciting world of machine learning! You should know these topics:
✅ Supervised learning: regression, classification.
✅ Unsupervised learning: clustering, PCA, anomaly detection.
✅ Deep learning: neural networks, CNN, RNN
🔗 Course: CS229: Machine Learning
➖➖➖➖➖
🔢 Step 5: Working with Big Data and Cloud Technologies
✏️ If you're going to work in the real world, you need to know how to work with Big Data and cloud computing.
✅ Big Data Tools: Hadoop, Spark, Dask
✅ Cloud platforms: AWS, GCP, Azure
🔗 Course: Data Engineering
➖➖➖➖➖
🔢 Step 6: Do real projects!
✏️ Enough theory, it's time to get coding! Do real projects and build a strong portfolio.
✅ Kaggle competitions: solving real-world challenges.
✅ End-to-End projects: data collection, modeling, implementation.
✅ GitHub: Publish your projects on GitHub.
🔗 Platform: Kaggle🔗 Platform: ods.ai
➖➖➖➖➖
🔢 Step 7: Learn MLOps and deploy models
✏️ Machine learning is not just about building a model! You need to learn how to deploy and monitor a model.
✅ MLOps training: model versioning, monitoring, model retraining.
✅ Deployment models: Flask, FastAPI, Docker
🔗 Course: Stanford MLOps Course
➖➖➖➖➖
🔢 Step 8: Stay up to date and network
✏️ Data science is changing every day, so it is necessary to update yourself every day and stay in regular contact with experienced people and experts in this field.
✅ Read scientific articles: arXiv, Google Scholar
✅ Connect with the data community:
🔗 Site: Papers with code
🔗 Site: AI Research at Google
👨🏻💻 If you want to become a data science professional, follow this path! I've prepared a complete roadmap with the best free resources where you can learn the essential skills in this field.
🔢 Step 1: Strengthen your math and statistics!
✏️ The foundation of learning data science is mathematics, linear algebra, statistics, and probability. Topics you should master:
✅ Linear algebra: matrices, vectors, eigenvalues.
🔗 Course: MIT 18.06 Linear Algebra
✅ Calculus: derivative, integral, optimization.
🔗 Course: MIT Single Variable Calculus
✅ Statistics and probability: Bayes' theorem, hypothesis testing.
🔗 Course: Statistics 110
➖➖➖➖➖
🔢 Step 2: Learn to code.
✏️ Learn Python and become proficient in coding. The most important topics you need to master are:
✅ Python: Pandas, NumPy, Matplotlib libraries
🔗 Course: FreeCodeCamp Python Course
✅ SQL language: Join commands, Window functions, query optimization.
🔗 Course: Stanford SQL Course
✅ Data structures and algorithms: arrays, linked lists, trees.
🔗 Course: MIT Introduction to Algorithms
➖➖➖➖➖
🔢 Step 3: Clean and visualize data
✏️ Learn how to process and clean data and then create an engaging story from it!
✅ Data cleaning: Working with missing values and detecting outliers.
🔗 Course: Data Cleaning
✅ Data visualization: Matplotlib, Seaborn, Tableau
🔗 Course: Data Visualization Tutorial
➖➖➖➖➖
🔢 Step 4: Learn Machine Learning
✏️ It's time to enter the exciting world of machine learning! You should know these topics:
✅ Supervised learning: regression, classification.
✅ Unsupervised learning: clustering, PCA, anomaly detection.
✅ Deep learning: neural networks, CNN, RNN
🔗 Course: CS229: Machine Learning
➖➖➖➖➖
🔢 Step 5: Working with Big Data and Cloud Technologies
✏️ If you're going to work in the real world, you need to know how to work with Big Data and cloud computing.
✅ Big Data Tools: Hadoop, Spark, Dask
✅ Cloud platforms: AWS, GCP, Azure
🔗 Course: Data Engineering
➖➖➖➖➖
🔢 Step 6: Do real projects!
✏️ Enough theory, it's time to get coding! Do real projects and build a strong portfolio.
✅ Kaggle competitions: solving real-world challenges.
✅ End-to-End projects: data collection, modeling, implementation.
✅ GitHub: Publish your projects on GitHub.
🔗 Platform: Kaggle🔗 Platform: ods.ai
➖➖➖➖➖
🔢 Step 7: Learn MLOps and deploy models
✏️ Machine learning is not just about building a model! You need to learn how to deploy and monitor a model.
✅ MLOps training: model versioning, monitoring, model retraining.
✅ Deployment models: Flask, FastAPI, Docker
🔗 Course: Stanford MLOps Course
➖➖➖➖➖
🔢 Step 8: Stay up to date and network
✏️ Data science is changing every day, so it is necessary to update yourself every day and stay in regular contact with experienced people and experts in this field.
✅ Read scientific articles: arXiv, Google Scholar
✅ Connect with the data community:
🔗 Site: Papers with code
🔗 Site: AI Research at Google
#ArtificialIntelligence #AI #MachineLearning #LargeLanguageModels #LLMs #DeepLearning #NLP #NaturalLanguageProcessing #AIResearch #TechBooks #AIApplications #DataScience #FutureOfAI #AIEducation #LearnAI #TechInnovation #AIethics #GPT #BERT #T5 #AIBook #data
❤8
Tools Every AI Engineer Should Know
1. Data Science Tools
Python: Preferred language with libraries like NumPy, Pandas, Scikit-learn.
R: Ideal for statistical analysis and data visualization.
Jupyter Notebook: Interactive coding environment for Python and R.
MATLAB: Used for mathematical modeling and algorithm development.
RapidMiner: Drag-and-drop platform for machine learning workflows.
KNIME: Open-source analytics platform for data integration and analysis.
2. Machine Learning Tools
Scikit-learn: Comprehensive library for traditional ML algorithms.
XGBoost & LightGBM: Specialized tools for gradient boosting.
TensorFlow: Open-source framework for ML and DL.
PyTorch: Popular DL framework with a dynamic computation graph.
H2O.ai: Scalable platform for ML and AutoML.
Auto-sklearn: AutoML for automating the ML pipeline.
3. Deep Learning Tools
Keras: User-friendly high-level API for building neural networks.
PyTorch: Excellent for research and production in DL.
TensorFlow: Versatile for both research and deployment.
ONNX: Open format for model interoperability.
OpenCV: For image processing and computer vision.
Hugging Face: Focused on natural language processing.
4. Data Engineering Tools
Apache Hadoop: Framework for distributed storage and processing.
Apache Spark: Fast cluster-computing framework.
Kafka: Distributed streaming platform.
Airflow: Workflow automation tool.
Fivetran: ETL tool for data integration.
dbt: Data transformation tool using SQL.
5. Data Visualization Tools
Tableau: Drag-and-drop BI tool for interactive dashboards.
Power BI: Microsoft’s BI platform for data analysis and visualization.
Matplotlib & Seaborn: Python libraries for static and interactive plots.
Plotly: Interactive plotting library with Dash for web apps.
D3.js: JavaScript library for creating dynamic web visualizations.
6. Cloud Platforms
AWS: Services like SageMaker for ML model building.
Google Cloud Platform (GCP): Tools like BigQuery and AutoML.
Microsoft Azure: Azure ML Studio for ML workflows.
IBM Watson: AI platform for custom model development.
7. Version Control and Collaboration Tools
Git: Version control system.
GitHub/GitLab: Platforms for code sharing and collaboration.
Bitbucket: Version control for teams.
8. Other Essential Tools
Docker: For containerizing applications.
Kubernetes: Orchestration of containerized applications.
MLflow: Experiment tracking and deployment.
Weights & Biases (W&B): Experiment tracking and collaboration.
Pandas Profiling: Automated data profiling.
BigQuery/Athena: Serverless data warehousing tools.
Mastering these tools will ensure you are well-equipped to handle various challenges across the AI lifecycle.
#artificialintelligence
1. Data Science Tools
Python: Preferred language with libraries like NumPy, Pandas, Scikit-learn.
R: Ideal for statistical analysis and data visualization.
Jupyter Notebook: Interactive coding environment for Python and R.
MATLAB: Used for mathematical modeling and algorithm development.
RapidMiner: Drag-and-drop platform for machine learning workflows.
KNIME: Open-source analytics platform for data integration and analysis.
2. Machine Learning Tools
Scikit-learn: Comprehensive library for traditional ML algorithms.
XGBoost & LightGBM: Specialized tools for gradient boosting.
TensorFlow: Open-source framework for ML and DL.
PyTorch: Popular DL framework with a dynamic computation graph.
H2O.ai: Scalable platform for ML and AutoML.
Auto-sklearn: AutoML for automating the ML pipeline.
3. Deep Learning Tools
Keras: User-friendly high-level API for building neural networks.
PyTorch: Excellent for research and production in DL.
TensorFlow: Versatile for both research and deployment.
ONNX: Open format for model interoperability.
OpenCV: For image processing and computer vision.
Hugging Face: Focused on natural language processing.
4. Data Engineering Tools
Apache Hadoop: Framework for distributed storage and processing.
Apache Spark: Fast cluster-computing framework.
Kafka: Distributed streaming platform.
Airflow: Workflow automation tool.
Fivetran: ETL tool for data integration.
dbt: Data transformation tool using SQL.
5. Data Visualization Tools
Tableau: Drag-and-drop BI tool for interactive dashboards.
Power BI: Microsoft’s BI platform for data analysis and visualization.
Matplotlib & Seaborn: Python libraries for static and interactive plots.
Plotly: Interactive plotting library with Dash for web apps.
D3.js: JavaScript library for creating dynamic web visualizations.
6. Cloud Platforms
AWS: Services like SageMaker for ML model building.
Google Cloud Platform (GCP): Tools like BigQuery and AutoML.
Microsoft Azure: Azure ML Studio for ML workflows.
IBM Watson: AI platform for custom model development.
GitHub/GitLab: Platforms for code sharing and collaboration.
Bitbucket: Version control for teams.
8. Other Essential Tools
Docker: For containerizing applications.
Kubernetes: Orchestration of containerized applications.
MLflow: Experiment tracking and deployment.
Weights & Biases (W&B): Experiment tracking and collaboration.
Pandas Profiling: Automated data profiling.
BigQuery/Athena: Serverless data warehousing tools.
Mastering these tools will ensure you are well-equipped to handle various challenges across the AI lifecycle.
#artificialintelligence
❤4