Data Science Jupyter Notebooks
10.8K subscribers
268 photos
31 videos
9 files
674 links
Explore the world of Data Science through Jupyter Notebooksโ€”insights, tutorials, and tools to boost your data journey. Code, analyze, and visualize smarter with every post.
Download Telegram
๐Ÿ”ฅ Trending Repository: MinerU

๐Ÿ“ Description: Transforms complex documents like PDFs into LLM-ready markdown/JSON for your Agentic workflows.

๐Ÿ”— Repository URL: https://github.com/opendatalab/MinerU

๐ŸŒ Website: https://opendatalab.github.io/MinerU/

๐Ÿ“– Readme: https://github.com/opendatalab/MinerU#readme

๐Ÿ“Š Statistics:
๐ŸŒŸ Stars: 45.7K stars
๐Ÿ‘€ Watchers: 183
๐Ÿด Forks: 3.8K forks

๐Ÿ’ป Programming Languages: Python - Dockerfile

๐Ÿท๏ธ Related Topics:
#python #pdf #parser #ocr #pdf_converter #extract_data #document_analysis #pdf_parser #layout_analysis #ai4science #pdf_extractor_rag #pdf_extractor_llm #pdf_extractor_pretrain


==================================
๐Ÿง  By: https://t.iss.one/DataScienceM
โค1
๐Ÿ”ฅ Trending Repository: MineContext

๐Ÿ“ Description: MineContext is your proactive context-aware AI partner๏ผˆContext-Engineering+ChatGPT Pulse๏ผ‰

๐Ÿ”— Repository URL: https://github.com/volcengine/MineContext

๐Ÿ“– Readme: https://github.com/volcengine/MineContext#readme

๐Ÿ“Š Statistics:
๐ŸŒŸ Stars: 1.6K stars
๐Ÿ‘€ Watchers: 12
๐Ÿด Forks: 72 forks

๐Ÿ’ป Programming Languages: Python - HTML - JavaScript - CSS

๐Ÿท๏ธ Related Topics:
#electron #react #python #agent #memory #python3 #embedding_models #rag #vector_database #vision_language_model #proactive_ai #context_engineering


==================================
๐Ÿง  By: https://t.iss.one/DataScienceM
โค1
๐Ÿ”ฅ Trending Repository: waha

๐Ÿ“ Description: WAHA - WhatsApp HTTP API (REST API) that you can configure in a click! 3 engines: WEBJS (browser based), NOWEB (websocket nodejs), GOWS (websocket go)

๐Ÿ”— Repository URL: https://github.com/devlikeapro/waha

๐ŸŒ Website: https://waha.devlike.pro/

๐Ÿ“– Readme: https://github.com/devlikeapro/waha#readme

๐Ÿ“Š Statistics:
๐ŸŒŸ Stars: 4.4K stars
๐Ÿ‘€ Watchers: 38
๐Ÿด Forks: 922 forks

๐Ÿ’ป Programming Languages: TypeScript - JavaScript - HTML - Dockerfile - Shell - Makefile

๐Ÿท๏ธ Related Topics:
#bot #whatsapp #python_bot #whatsapp_web #whatsapp_bot #http_api #whatsapp_api #whatsapp_chat #ai_bot #whatsapp_automation #whatsapp_web_api


==================================
๐Ÿง  By: https://t.iss.one/DataScienceM
๐Ÿ”ฅ Trending Repository: openarm

๐Ÿ“ Description: A fully open-source humanoid arm for physical AI research and deployment in contact-rich environments.

๐Ÿ”— Repository URL: https://github.com/enactic/openarm

๐ŸŒ Website: https://openarm.dev

๐Ÿ“– Readme: https://github.com/enactic/openarm#readme

๐Ÿ“Š Statistics:
๐ŸŒŸ Stars: 1.1K stars
๐Ÿ‘€ Watchers: 36
๐Ÿด Forks: 131 forks

๐Ÿ’ป Programming Languages: MDX - TypeScript

๐Ÿท๏ธ Related Topics:
#python #open_source #machine_learning #reinforcement_learning #robot #robotics #genesis #robot_arm #force_feedback #imitation_learning #humanoid_robot #ros2 #bilateral_teleoperation #teleoperation #mujoco #moveit2 #gravity_compensation #openarm


==================================
๐Ÿง  By: https://t.iss.one/DataScienceM
๐Ÿ”ฅ Trending Repository: reflex

๐Ÿ“ Description: ๐Ÿ•ธ๏ธ Web apps in pure Python ๐Ÿ

๐Ÿ”— Repository URL: https://github.com/reflex-dev/reflex

๐ŸŒ Website: https://reflex.dev

๐Ÿ“– Readme: https://github.com/reflex-dev/reflex#readme

๐Ÿ“Š Statistics:
๐ŸŒŸ Stars: 26.1K stars
๐Ÿ‘€ Watchers: 177
๐Ÿด Forks: 1.6K forks

๐Ÿ’ป Programming Languages: Python - JavaScript - PowerShell - Shell - CSS - Dockerfile

๐Ÿท๏ธ Related Topics:
#python #open_source #gui #framework #web


==================================
๐Ÿง  By: https://t.iss.one/DataScienceM
๐Ÿ”ฅ Trending Repository: maltrail

๐Ÿ“ Description: Malicious traffic detection system

๐Ÿ”— Repository URL: https://github.com/stamparm/maltrail

๐Ÿ“– Readme: https://github.com/stamparm/maltrail#readme

๐Ÿ“Š Statistics:
๐ŸŒŸ Stars: 7.4K stars
๐Ÿ‘€ Watchers: 237
๐Ÿด Forks: 1.2K forks

๐Ÿ’ป Programming Languages: Python - JavaScript - CSS - HTML

๐Ÿท๏ธ Related Topics:
#python #security #sensor #malware #intrusion_detection #network_monitoring #attack_detection


==================================
๐Ÿง  By: https://t.iss.one/DataScienceM
๐Ÿ”ฅ Trending Repository: skyvern

๐Ÿ“ Description: Automate browser-based workflows with LLMs and Computer Vision

๐Ÿ”— Repository URL: https://github.com/Skyvern-AI/skyvern

๐ŸŒ Website: https://www.skyvern.com

๐Ÿ“– Readme: https://github.com/Skyvern-AI/skyvern#readme

๐Ÿ“Š Statistics:
๐ŸŒŸ Stars: 14.7K stars
๐Ÿ‘€ Watchers: 84
๐Ÿด Forks: 1.3K forks

๐Ÿ’ป Programming Languages: Python - TypeScript - MDX - Jinja - JavaScript - Shell

๐Ÿท๏ธ Related Topics:
#python #api #workflow #automation #browser #computer #vision #gpt #browser_automation #rpa #playwright #llm


==================================
๐Ÿง  By: https://t.iss.one/DataScienceM
๐Ÿ”ฅ Trending Repository: best-of-ml-python

๐Ÿ“ Description: ๐Ÿ† A ranked list of awesome machine learning Python libraries. Updated weekly.

๐Ÿ”— Repository URL: https://github.com/lukasmasuch/best-of-ml-python

๐ŸŒ Website: https://ml-python.best-of.org

๐Ÿ“– Readme: https://github.com/lukasmasuch/best-of-ml-python#readme

๐Ÿ“Š Statistics:
๐ŸŒŸ Stars: 22.3K stars
๐Ÿ‘€ Watchers: 444
๐Ÿด Forks: 3K forks

๐Ÿ’ป Programming Languages: Not available

๐Ÿท๏ธ Related Topics:
#python #nlp #data_science #machine_learning #deep_learning #tensorflow #scikit_learn #keras #ml #data_visualization #pytorch #transformer #data_analysis #gpt #automl #jax #data_visualizations #gpt_3 #chatgpt


==================================
๐Ÿง  By: https://t.iss.one/DataScienceM
๐Ÿ”ฅ Trending Repository: social-analyzer

๐Ÿ“ Description: API, CLI, and Web App for analyzing and finding a person's profile in 1000 social media \ websites

๐Ÿ”— Repository URL: https://github.com/qeeqbox/social-analyzer

๐Ÿ“– Readme: https://github.com/qeeqbox/social-analyzer#readme

๐Ÿ“Š Statistics:
๐ŸŒŸ Stars: 13.9K stars
๐Ÿ‘€ Watchers: 374
๐Ÿด Forks: 1.2K forks

๐Ÿ’ป Programming Languages: JavaScript

๐Ÿท๏ธ Related Topics:
#nodejs #javascript #python #cli #profile #social_media #osint #analysis #analyzer #pentesting #username #pentest #nodejs_cli #information_gathering #security_tools #reconnaissance #social_analyzer #person_profile #sosint


==================================
๐Ÿง  By: https://t.iss.one/DataScienceM
โค1
In Python, building AI-powered Telegram bots unlocks massive potential for image generation, processing, and automationโ€”master this to create viral tools and ace full-stack interviews! ๐Ÿค–

# Basic Bot Setup - The foundation (PTB v20+ Async)
from telegram.ext import Application, CommandHandler, MessageHandler, filters

async def start(update, context):
await update.message.reply_text(
"โœจ AI Image Bot Active!\n"
"/generate - Create images from text\n"
"/enhance - Improve photo quality\n"
"/help - Full command list"
)

app = Application.builder().token("YOUR_BOT_TOKEN").build()
app.add_handler(CommandHandler("start", start))
app.run_polling()


# Image Generation - DALL-E Integration (OpenAI)
import openai
from telegram.ext import ContextTypes

openai.api_key = os.getenv("OPENAI_API_KEY")

async def generate(update: Update, context: ContextTypes.DEFAULT_TYPE):
if not context.args:
await update.message.reply_text("โŒ Usage: /generate cute robot astronaut")
return

prompt = " ".join(context.args)
try:
response = openai.Image.create(
prompt=prompt,
n=1,
size="1024x1024"
)
await update.message.reply_photo(
photo=response['data'][0]['url'],
caption=f"๐ŸŽจ Generated: *{prompt}*",
parse_mode="Markdown"
)
except Exception as e:
await update.message.reply_text(f"๐Ÿ”ฅ Error: {str(e)}")

app.add_handler(CommandHandler("generate", generate))


Learn more: https://hackmd.io/@husseinsheikho/building-AI-powered-Telegram-bots

#Python #TelegramBot #AI #ImageGeneration #StableDiffusion #OpenAI #MachineLearning #CodingInterview #FullStack #Chatbots #DeepLearning #ComputerVision #Programming #TechJobs #DeveloperTips #CareerGrowth #CloudComputing #Docker #APIs #Python3 #Productivity #TechTips


https://t.iss.one/DataScienceM ๐Ÿฆพ
Please open Telegram to view this post
VIEW IN TELEGRAM
โค4
๐Ÿ”ฅ Trending Repository: mem0

๐Ÿ“ Description: Universal memory layer for AI Agents; Announcing OpenMemory MCP - local and secure memory management.

๐Ÿ”— Repository URL: https://github.com/mem0ai/mem0

๐ŸŒ Website: https://mem0.ai

๐Ÿ“– Readme: https://github.com/mem0ai/mem0#readme

๐Ÿ“Š Statistics:
๐ŸŒŸ Stars: 42.1K stars
๐Ÿ‘€ Watchers: 203
๐Ÿด Forks: 4.5K forks

๐Ÿ’ป Programming Languages: Python - TypeScript - MDX - Jupyter Notebook - JavaScript - Shell

๐Ÿท๏ธ Related Topics:
#python #application #state_management #ai #memory #chatbots #memory_management #agents #hacktoberfest #ai_agents #long_term_memory #rag #llm #chatgpt #genai


==================================
๐Ÿง  By: https://t.iss.one/DataScienceM
โค1
#YOLOv8 #ComputerVision #ObjectDetection #Python #AI

Audience Analysis with YOLOv8: Counting People & Estimating Gender Ratios

This lesson demonstrates how to use the YOLOv8 model to perform a computer vision task: analyzing an image of a crowd to count the total number of people and estimate the ratio of men to women.

---

Step 1: Setup and Installation

First, we need to install the necessary libraries. ultralytics for the YOLOv8 model, opencv-python for image manipulation, and cvlib for a simple, pre-trained gender classification model.

#Setup #Installation

# Open your terminal or command prompt and run:
pip install ultralytics opencv-python cvlib tensorflow


---

Step 2: Loading Models and Image

We will load two models: the official YOLOv8 model pre-trained for object detection, and we'll use cvlib for gender detection. We also need to load the image we want to analyze. Make sure you have an image named crowd.jpg in the same directory.

#DataLoading #Model

import cv2
from ultralytics import YOLO
import cvlib as cv
import numpy as np

# Load the YOLOv8 model (pre-trained on COCO dataset)
model = YOLO('yolov8n.pt')

# Load the image
image_path = 'crowd.jpg' # Make sure this image exists
img = cv2.imread(image_path)

# Check if the image was loaded correctly
if img is None:
print(f"Error: Could not load image from {image_path}")
else:
print("Image and YOLOv8 model loaded successfully.")


---

Step 3: Person Detection with YOLOv8

Now, we'll run the YOLOv8 model on our image to detect all objects and then filter those results to keep only the ones identified as a 'person'.

#PersonDetection #Inference

# Run inference on the image
results = model(img)

# A list to store the bounding boxes of detected people
person_boxes = []

# Process the results
for result in results:
boxes = result.boxes
for box in boxes:
# Get class id and check if it's a person (class 0 in COCO)
if model.names[int(box.cls)] == 'person':
# Get bounding box coordinates
x1, y1, x2, y2 = map(int, box.xyxy[0])
person_boxes.append((x1, y1, x2, y2))

# Print the total number of people found
total_people = len(person_boxes)
print(f"Total people detected: {total_people}")


---

Step 4: Gender Classification

For each detected person, we will crop their bounding box from the image. Then, we'll use cvlib to detect a face within that crop and predict the gender. This is a multi-step pipeline.

#GenderClassification #CV
๐Ÿ”ฅ Trending Repository: xiaomusic

๐Ÿ“ Description: ไฝฟ็”จๅฐ็ˆฑ้Ÿณ็ฎฑๆ’ญๆ”พ้Ÿณไน๏ผŒ้Ÿณไนไฝฟ็”จ yt-dlp ไธ‹่ฝฝใ€‚

๐Ÿ”— Repository URL: https://github.com/hanxi/xiaomusic

๐ŸŒ Website: https://xdocs.hanxi.cc/

๐Ÿ“– Readme: https://github.com/hanxi/xiaomusic#readme

๐Ÿ“Š Statistics:
๐ŸŒŸ Stars: 6.4K stars
๐Ÿ‘€ Watchers: 22
๐Ÿด Forks: 637 forks

๐Ÿ’ป Programming Languages: Python

๐Ÿท๏ธ Related Topics:
#python #music #docker #vue #docker_compose #xiaomi #pdm #xiaoai #xiaoai_speaker #xiaomusic


==================================
๐Ÿง  By: https://t.iss.one/DataScienceM
#YOLOv8 #ComputerVision #HomeSecurity #ObjectTracking #AI #Python

Lesson: Tracking Suspicious Individuals Near a Home at Night with YOLOv8

This tutorial demonstrates how to build an advanced security system using YOLOv8's object tracking capabilities. The system will detect people in a night-time video feed, track their movements, and trigger an alert if a person loiters for too long within a predefined "alert zone" (e.g., a driveway or porch).

---

#Step 1: Project Setup and Dependencies

We will use ultralytics for YOLOv8 and its built-in tracker, opencv-python for video processing, and numpy for defining our security zone.

pip install ultralytics opencv-python numpy

Create a Python script (e.g., security_tracker.py) and import the necessary libraries. We'll also use defaultdict to easily manage timers for each tracked person.

import cv2
import numpy as np
from ultralytics import YOLO
from collections import defaultdict
import time

# Hashtags: #Setup #Python #OpenCV #YOLOv8


---

#Step 2: Model Loading and Zone Configuration

We will load a standard YOLOv8 model capable of detecting 'person'. The key is to define a polygon representing the area we want to monitor. We will also set a time threshold to define "loitering". You will need a video file of your target area, for example, night_security_footage.mp4.

# Load the YOLOv8 model
model = YOLO('yolov8n.pt')

# Path to your night-time video file
VIDEO_PATH = 'night_security_footage.mp4'

# Define the polygon for the alert zone.
# IMPORTANT: You MUST adjust these [x, y] coordinates to fit your video's perspective.
# This example defines a rectangular area for a driveway.
ALERT_ZONE_POLYGON = np.array([
[100, 500], [800, 500], [850, 250], [50, 250]
], np.int32)

# Time in seconds a person can be in the zone before an alert is triggered
LOITERING_THRESHOLD_SECONDS = 5.0

# Dictionaries to store tracking data
# Stores the time when a tracked object first enters the zone
loitering_timers = {}
# Stores the IDs of individuals who have triggered an alert
alert_triggered_ids = set()

# Hashtags: #Configuration #AIModel #SecurityZone


---

#Step 3: Main Loop for Tracking and Zone Monitoring

This is the core of the system. We will read the video frame by frame and use YOLOv8's track() function. This function not only detects objects but also assigns a unique ID to each one, allowing us to follow them across frames.

cap = cv2.VideoCapture(VIDEO_PATH)

while cap.isOpened():
success, frame = cap.read()
if not success:
break

# Run YOLOv8 tracking on the frame, persisting tracks between frames
results = model.track(frame, persist=True)

# Get the bounding boxes and track IDs
boxes = results[0].boxes.xywh.cpu()
track_ids = results[0].boxes.id.int().cpu().tolist()

# Visualize the results on the frame
annotated_frame = results[0].plot()

# Draw the alert zone polygon on the frame
cv2.polylines(annotated_frame, [ALERT_ZONE_POLYGON], isClosed=True, color=(0, 255, 255), thickness=2)

# Hashtags: #RealTime #ObjectTracking #VideoProcessing

(Note: The code below should be placed inside the while loop of Step 3)

---

#Step 4: Implementing Loitering Logic and Alerts

Inside the main loop, we'll iterate through each tracked person. We check if their position is inside our alert zone. If it is, we start or update a timer. If the timer exceeds our threshold, we trigger an alert for that person's ID.
#PyQt5 #SQLite #DesktopApp #Pharmacy #Barcode #Python

Lesson: Building a Pharmacy Management System with PyQt5 and Barcode Scanning

This tutorial will guide you through creating a complete desktop application for managing a pharmacy. The system will use a SQLite database for inventory, and a Point of Sale (POS) interface that uses barcode scanning to add drugs to a sale and automatically deducts stock upon completion.

---

#Step 1: Database Setup (database.py)

First, we create a dedicated file to handle all SQLite database operations. This keeps our data logic separate from our UI logic. Create a file named database.py.

import sqlite3

DB_NAME = 'pharmacy.db'

def connect():
return sqlite3.connect(DB_NAME)

def setup_database():
conn = connect()
cursor = conn.cursor()
cursor.execute('''
CREATE TABLE IF NOT EXISTS drugs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
barcode TEXT NOT NULL UNIQUE,
quantity INTEGER NOT NULL,
price REAL NOT NULL,
expiry_date TEXT NOT NULL
)
''')
conn.commit()
conn.close()

def add_drug(name, barcode, quantity, price, expiry_date):
conn = connect()
cursor = conn.cursor()
try:
cursor.execute("INSERT INTO drugs (name, barcode, quantity, price, expiry_date) VALUES (?, ?, ?, ?, ?)",
(name, barcode, quantity, price, expiry_date))
conn.commit()
except sqlite3.IntegrityError:
return False # Barcode already exists
finally:
conn.close()
return True

def get_all_drugs():
conn = connect()
cursor = conn.cursor()
cursor.execute("SELECT id, name, barcode, quantity, price, expiry_date FROM drugs ORDER BY name")
drugs = cursor.fetchall()
conn.close()
return drugs

def find_drug_by_barcode(barcode):
conn = connect()
cursor = conn.cursor()
cursor.execute("SELECT id, name, barcode, quantity, price, expiry_date FROM drugs WHERE barcode = ?", (barcode,))
drug = cursor.fetchone()
conn.close()
return drug

def update_drug_quantity(drug_id, sold_quantity):
conn = connect()
cursor = conn.cursor()
cursor.execute("UPDATE drugs SET quantity = quantity - ? WHERE id = ?", (sold_quantity, drug_id))
conn.commit()
conn.close()

# Hashtags: #SQLite #DatabaseDesign #DataPersistence #Python


---

#Step 2: Main Application and Inventory Management UI

Create the main application file, main.py. We will set up the main window with tabs for "Point of Sale" and "Inventory Management". We will fully implement the inventory tab first, allowing users to view and add drugs to the database.
๐Ÿ”ฅ Trending Repository: pytorch

๐Ÿ“ Description: Tensors and Dynamic neural networks in Python with strong GPU acceleration

๐Ÿ”— Repository URL: https://github.com/pytorch/pytorch

๐ŸŒ Website: https://pytorch.org

๐Ÿ“– Readme: https://github.com/pytorch/pytorch#readme

๐Ÿ“Š Statistics:
๐ŸŒŸ Stars: 94.5K stars
๐Ÿ‘€ Watchers: 1.8k
๐Ÿด Forks: 25.8K forks

๐Ÿ’ป Programming Languages: Python - C++ - Cuda - C - Objective-C++ - CMake

๐Ÿท๏ธ Related Topics:
#python #machine_learning #deep_learning #neural_network #gpu #numpy #autograd #tensor


==================================
๐Ÿง  By: https://t.iss.one/DataScienceM
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, "myDynamicElement"))
)

โ€ข Get the page source after JavaScript has executed.
dynamic_html = driver.page_source

โ€ข Close the browser window.
driver.quit()


VII. Common Tasks & Best Practices

โ€ข Handle pagination by finding the "Next" link.
next_page_url = soup.find('a', text='Next')['href']

โ€ข Save data to a CSV file.
import csv
with open('data.csv', 'w', newline='', encoding='utf-8') as f:
writer = csv.writer(f)
writer.writerow(['Title', 'Link'])
# writer.writerow([title, url]) in a loop

โ€ข Save data to CSV using pandas.
import pandas as pd
df = pd.DataFrame(data, columns=['Title', 'Link'])
df.to_csv('data.csv', index=False)

โ€ข Use a proxy with requests.
proxies = {'http': 'https://10.10.1.10:3128', 'https': 'https://10.10.1.10:1080'}
requests.get('https://example.com', proxies=proxies)

โ€ข Pause between requests to be polite.
import time
time.sleep(2) # Pause for 2 seconds

โ€ข Handle JSON data from an API.
json_response = requests.get('https://api.example.com/data').json()

โ€ข Download a file (like an image).
img_url = 'https://example.com/image.jpg'
img_data = requests.get(img_url).content
with open('image.jpg', 'wb') as handler:
handler.write(img_data)

โ€ข Parse a sitemap.xml to find all URLs.
# Get the sitemap.xml file and parse it like any other XML/HTML to extract <loc> tags.


VIII. Advanced Frameworks (Scrapy)

โ€ข Create a Scrapy spider (conceptual command).
scrapy genspider example example.com

โ€ข Define a parse method to process the response.
# In your spider class:
def parse(self, response):
# parsing logic here
pass

โ€ข Extract data using Scrapy's CSS selectors.
titles = response.css('h1::text').getall()

โ€ข Extract data using Scrapy's XPath selectors.
links = response.xpath('//a/@href').getall()

โ€ข Yield a dictionary of scraped data.
yield {'title': response.css('title::text').get()}

โ€ข Follow a link to parse the next page.
next_page = response.css('li.next a::attr(href)').get()
if next_page is not None:
yield response.follow(next_page, callback=self.parse)

โ€ข Run a spider from the command line.
scrapy crawl example -o output.json

โ€ข Pass arguments to a spider.
scrapy crawl example -a category=books

โ€ข Create a Scrapy Item for structured data.
import scrapy
class ProductItem(scrapy.Item):
name = scrapy.Field()
price = scrapy.Field()

โ€ข Use an Item Loader to populate Items.
from scrapy.loader import ItemLoader
loader = ItemLoader(item=ProductItem(), response=response)
loader.add_css('name', 'h1.product-name::text')


#Python #WebScraping #BeautifulSoup #Selenium #Requests

โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
By: @DataScienceN โœจ
โค2