Part 5: Specialized Web Scraping – Social Media, Mobile Apps, Dark Web, and Advanced Data Extraction

Duration: ~60 minutes

Link A: https://hackmd.io/@husseinsheikho/WS-5A

Link B: https://hackmd.io/@husseinsheikho/WS-5B

#SocialMediaScraping #MobileScraping #DarkWeb #FinancialData #MediaExtraction #AuthScraping #ScrapingSaaS #APIReverseEngineering #EthicalScraping #DataScience

✨ How to Drop Null Values in pandas ✨

📖 Learn how to use .dropna() to drop null values from pandas DataFrames so you can clean missing data and keep your Python analysis accurate.

🏷️ #basics #datascience #python

✨ Polars vs pandas: What's the Difference? ✨

📖 Discover the key differences in Polars vs pandas to help you choose the right Python library for faster, more efficient data analysis.

🏷️ #intermediate #datascience #python

The Python library itertools contains many useful functions. 🐍✨

One of them is compress(), which returns an iterator over the elements from data, for which the corresponding element in selectors is equal to True. 🔍💻

Here's an example: 📝👇

#Python #Programming #Itertools #Coding #Tech #DataScience

Cheat sheet on the basics of Python: 🐍📚

basic syntax and language rules 📝
scalar types — basic data types (int, float, bool, str, NoneType) 🔢

datetime — working with date and time 📅⏰

data structures — Python data structures (list, tuple, dict, set) 🗄

list — mutable lists for storing data collections 📋
tuple — immutable sequences of values 🔒
dict (hash map) — storing data in a key-value format 🗝
set — unique elements without order 🔘

slicing — obtaining parts of sequences through indices and step ✂️

module/library — connecting modules and libraries 🔌

help functions — using help() and dir() to explore the Python API 🛠

#Python #Coding #DataScience #Programming #Tech #DevCommunity

⚡️ How Redis counts billions of unique values while barely using memory

There's an algorithm called HyperLogLog. It allows you to roughly estimate how many unique elements have passed through the system, using about 12 KB of memory.

The idea is simple: Redis doesn't store the elements themselves.

It does the following:

- Takes an element
- Calculates a hash from it
- Uses part of the hash as a cell number
- Checks the other part to see how many consecutive zeros it contains
- If the new number is larger than the old one, it updates the cell

Why does this work?

Because a long series of zeros in the hash is rare.

For example:

- 1 consecutive zero - quite common
- 5 consecutive zeros - less common
- 10 consecutive zeros - about a 1 in 1024 chance
- 20 consecutive zeros - a very rare event

If Redis sees a very rare pattern, it means that many different elements have likely passed through it.

Redis uses 16,384 small counters. Each stores the maximum "rarity" it has seen for its group of elements.

Then Redis combines these values mathematically to get an estimate of unique elements.

Not an exact number, but a very close approximation.

The main trick of HyperLogLog:

it can handle millions or even billions of values, but memory hardly increases at all.

That's why Redis can count unique users, IPs, requests, or events without huge tables and lists.

#Redis #HyperLogLog #DataScience #Tech #BigData #MemoryEfficiency

✨ Join Best TG Channels https://t.iss.one/addlist/0f6vfFbEMdAwODBk

⭐️ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A

🚀 Level up your AI & Data Science skills with HelloEncyclo — a growing all-in-one platform featuring hands-on courses in LLMs, Deep Learning, MLOps, Data Engineering, and more.
✅ 13 courses live + 40+ coming soon
🎯 One access, lifetime updates
🔑 Use code: PRESALE-BOOK-WAVE-2GFG
👉 https://helloencyclo.com/?ref=HUSSEINSHEIKHO