PostgreSQL Pro | Database Mastery
1.32K subscribers
1 photo
28 links
🐘 PostgreSQL Mastery Hub

🎯 What you get:
- Daily optimization tips
- Performance guides
- Real-world solutions
- Query debugging help
- Production best practices

πŸ“ˆ Join 500+ developers improving their PostgreSQL skills
Download Telegram
πŸ“Œ Tutorial: Full-Text Search in PostgreSQL – Powering Advanced Search

πŸ”Ή Introduction:
If you need to search large text fields (like descriptions or documents), full-text search in PostgreSQL is an efficient way to implement advanced search features directly in your database. This functionality enables quick searching within text, ideal for applications like document management, e-commerce, and more!

1️⃣ Setting Up Full-Text Search:

PostgreSQL provides the tsvector and tsquery data types for storing and querying searchable text.

- tsvector: Stores searchable text in a preprocessed form.
- tsquery: Represents search queries that can be matched against tsvector data.

Example:

CREATE TABLE articles (
article_id SERIAL PRIMARY KEY,
title TEXT,
content TEXT,
content_tsvector TSVECTOR
);


Here, content_tsvector will store the processed version of content for fast searching.

2️⃣ Populating tsvector with Text Data:

The to_tsvector function converts text to tsvector, making it searchable.

Example:

UPDATE articles
SET content_tsvector = to_tsvector('english', content);


This command processes content in English, creating a searchable tsvector.

3️⃣ Basic Text Search:

To perform a search, use to_tsquery, which converts the search term into a tsquery.

Example:

SELECT title
FROM articles
WHERE content_tsvector @@ to_tsquery('english', 'database');


This query searches for "database" within the content_tsvector field.

4️⃣ Adding Triggers for Automatic Updates:

To keep content_tsvector up-to-date with content, set up a trigger.

CREATE FUNCTION update_tsvector() RETURNS TRIGGER AS $$
BEGIN
NEW.content_tsvector := to_tsvector('english', NEW.content);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER tsvectorupdate BEFORE INSERT OR UPDATE
ON articles FOR EACH ROW
EXECUTE FUNCTION update_tsvector();


This trigger automatically updates content_tsvector whenever content changes.

5️⃣ Phrase Searches with plainto_tsquery:

If you want a search term treated as a single phrase, plainto_tsquery converts it into a tsquery.

Example:

SELECT title
FROM articles
WHERE content_tsvector @@ plainto_tsquery('english', 'full text search');


This query searches for the exact phrase "full text search."

6️⃣ Highlighting Search Results:

The ts_headline function highlights search terms in results.

Example:

SELECT title, ts_headline('english', content, to_tsquery('full & text & search')) AS preview
FROM articles
WHERE content_tsvector @@ to_tsquery('english', 'full & text & search');


This highlights occurrences of "full text search" in the content preview.

7️⃣ Ranking Results by Relevance:

Use ts_rank or ts_rank_cd to rank results based on how well they match the query.

Example:

SELECT title, ts_rank(content_tsvector, to_tsquery('database')) AS rank
FROM articles
WHERE content_tsvector @@ to_tsquery('database')
ORDER BY rank DESC;


This ranks articles based on their relevance to the term "database."

8️⃣ Combining Full-Text Search with SQL Filters:

You can combine full-text search with other SQL conditions for more refined queries.

Example:

SELECT title
FROM articles
WHERE content_tsvector @@ to_tsquery('database')
AND publication_date > '2023-01-01';


This searches for articles containing "database" published after January 1, 2023.

9️⃣ Indexing for Full-Text Search:

For faster full-text search, create a GIN index on the tsvector column.

Example:

CREATE INDEX idx_content_tsvector ON articles USING gin(content_tsvector);


This index improves search performance, especially with large datasets.

@postgres
πŸ‘1
πŸ“Œ PostgreSQL Partitioning: Efficient Big Data Management

πŸ”Ή Why Partitioning?
Partitioning splits large tables into smaller, manageable tables (partitions) for faster queries and easier data handling. PostgreSQL supports:
- Range Partitioning: Use for date ranges.
- List Partitioning: Organize by fixed values (e.g., regions).
- Hash Partitioning: Distributes data evenly.

1️⃣ Example: Range Partitioning by Year
Perfect for time-based data like sales records.

CREATE TABLE sales (
sale_id SERIAL PRIMARY KEY,
sale_date DATE,
amount NUMERIC
) PARTITION BY RANGE (sale_date);

CREATE TABLE sales_2022 PARTITION OF sales
FOR VALUES FROM ('2022-01-01') TO ('2023-01-01');


2️⃣ Example: List Partitioning by Region
Great for location-specific data.

CREATE TABLE customers (
customer_id SERIAL PRIMARY KEY,
region TEXT,
name TEXT
) PARTITION BY LIST (region);

CREATE TABLE customers_us PARTITION OF customers FOR VALUES IN ('US');
CREATE TABLE customers_eu PARTITION OF customers FOR VALUES IN ('EU');


3️⃣ Querying and Indexing Partitions
Queries automatically target relevant partitions, boosting speed. Add indexes to each partition as needed:
CREATE INDEX idx_sales_date_2022 ON sales_2022 (sale_date);


πŸ”Ή Key Benefits of Partitioning
- Optimized Querying: Only relevant partitions are scanned, reducing load.
- Easy Maintenance: Manage partitions individually, which is especially helpful with large tables.
- Simple Archiving: Detach or drop old data partitions easily.

πŸ”š Conclusion
Partitioning is a great solution for managing large datasets, providing faster performance, and simplifying maintenance tasks in PostgreSQL. Give it a try!

@postgres
πŸ‘2
πŸ“Œ PostgreSQL and JSON: The Best of Both Worlds

πŸ”Ή Why JSON in PostgreSQL?
PostgreSQL combines relational power with NoSQL flexibility using JSON and JSONB (binary JSON). Store, query, and manipulate semi-structured data easily.

1️⃣ Creating a JSON Column
Store JSON data directly in a table:

CREATE TABLE products (
id SERIAL PRIMARY KEY,
name TEXT,
details JSONB
);


2️⃣ Inserting JSON Data

INSERT INTO products (name, details) 
VALUES ('Laptop', '{"brand": "TechCorp", "price": 1200}');


3️⃣ Querying JSON Data
Access fields with the -> and ->> operators:

SELECT details->'brand' AS brand FROM products; -- JSON object
SELECT details->>'price' AS price FROM products; -- Text value


4️⃣ Filtering with JSON
Use @> to find rows containing specific JSON fields:

SELECT name FROM products 
WHERE details @> '{"brand": "TechCorp"}';


5️⃣ Updating JSON Fields
Modify parts of a JSON object using jsonb_set:

UPDATE products 
SET details = jsonb_set(details, '{price}', '1100')
WHERE name = 'Laptop';


6️⃣ Indexing JSON for Speed
For faster queries, add a GIN index:

CREATE INDEX idx_products_details ON products USING gin(details);


πŸ”Ή When to Use JSON?


Ideal for flexible, semi-structured data.
Combine structured and unstructured data in the same table.

πŸ”š Conclusion:
PostgreSQL’s JSON features allow you to enjoy the flexibility of NoSQL with the power of SQL. Use it to enhance your database workflows!


@postgres
πŸ‘1
πŸ“Œ PostgreSQL Full-Text Search: Build Fast and Accurate Searches

πŸ”Ή Why Full-Text Search?
Full-text search in PostgreSQL lets you search textual data with precision and speed. It’s perfect for applications like blogs, e-commerce, or document repositories.

1️⃣ Setting Up Full-Text Search
Create a table with searchable content:

CREATE TABLE articles (
id SERIAL PRIMARY KEY,
title TEXT,
content TEXT
);


2️⃣ Adding a Search Vector
Enhance search with a generated search vector column:

ALTER TABLE articles ADD COLUMN search_vector tsvector;
UPDATE articles
SET search_vector = to_tsvector('english', title || ' ' || content);


This processes the text for efficient searching.

3️⃣ Indexing for Speed
Boost search performance with a GIN index:

CREATE INDEX idx_search_vector ON articles USING gin(search_vector);


4️⃣ Searching with to_tsquery
Find articles with specific words:

SELECT title, content 
FROM articles
WHERE search_vector @@ to_tsquery('postgres & tutorial');


This searches for articles containing both β€œpostgres” and β€œtutorial.”

5️⃣ Highlighting Results
Make results more readable with ts_headline:

SELECT title, ts_headline('english', content, to_tsquery('postgres')) AS snippet
FROM articles
WHERE search_vector @@ to_tsquery('postgres');


6️⃣ Automating Updates with Triggers
Ensure search_vector stays up-to-date:

CREATE OR REPLACE FUNCTION update_search_vector() 
RETURNS TRIGGER AS $$
BEGIN
NEW.search_vector := to_tsvector('english', NEW.title || ' ' || NEW.content);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER trigger_update_search_vector
BEFORE INSERT OR UPDATE ON articles
FOR EACH ROW EXECUTE FUNCTION update_search_vector();


πŸ”Ή Benefits of Full-Text Search


Fast and scalable search capabilities.
Built-in support for ranking and highlighting results.
No need for external search engines for basic use cases.

πŸ”š Conclusion
PostgreSQL’s full-text search provides a powerful way to integrate efficient search functionality into your database. Whether for small projects or enterprise solutions, it’s worth exploring!


@postgres
πŸ‘2❀1πŸ”₯1
πŸ“Œ PostgreSQL CTEs: Simplify Your SQL Queries

πŸ”Ή What Are CTEs?
CTEs (Common Table Expressions) allow you to break complex queries into smaller, readable chunks. They’re temporary result sets that exist only for the duration of a query.

πŸ”Ή Why Use CTEs?


Make queries easier to read and maintain.
Reuse logic within the same query.
Handle recursive operations elegantly.

1️⃣ Basic Syntax
Define a CTE with WITH and use it in the main query:

WITH cte_example AS (
SELECT department, AVG(salary) AS avg_salary
FROM employees
GROUP BY department
)
SELECT department, avg_salary
FROM cte_example
WHERE avg_salary > 50000;


This calculates the average salary per department and filters those above 50k.

2️⃣ Chaining Multiple CTEs
Combine multiple CTEs for more complex logic:

WITH sales_data AS (
SELECT product_id, SUM(quantity) AS total_quantity
FROM sales
GROUP BY product_id
),
top_selling AS (
SELECT product_id
FROM sales_data
WHERE total_quantity > 100
)
SELECT *
FROM products
WHERE id IN (SELECT product_id FROM top_selling);


This identifies top-selling products and retrieves their details.

3️⃣ Recursive CTEs
Recursive CTEs are useful for hierarchical data, like organizational charts or file directories.

WITH RECURSIVE employee_hierarchy AS (
SELECT id, name, manager_id
FROM employees
WHERE manager_id IS NULL
UNION ALL
SELECT e.id, e.name, e.manager_id
FROM employees e
JOIN employee_hierarchy eh ON e.manager_id = eh.id
)
SELECT * FROM employee_hierarchy;


This creates a hierarchy of employees reporting to managers.

πŸ”Ή When to Use CTEs?


Complex queries with nested logic.
Breaking down large queries for better readability.
Hierarchical or recursive data structures.

πŸ”š Conclusion
CTEs are a great tool to simplify SQL queries and make them easier to understand. Mastering CTEs will save you time and effort when working with complex datasets!

@postgres
πŸš€ Master Your Data with PostgreSQL: The Ultimate Database Solution! 🐘

If you're looking for a powerful, reliable, and flexible database system, PostgreSQL is your go-to choice! Whether you're a developer, data engineer, or just someone who loves working with data, PostgreSQL has everything you need to build scalable and efficient applications. Let’s dive into why PostgreSQL is a game-changer! πŸ’‘

---

### Why PostgreSQL?
βœ… Open Source & Free: PostgreSQL is completely free to use and open source, meaning you can customize it to fit your needs without breaking the bank.
βœ… Rock-Solid Reliability: Trusted by companies like Apple, Spotify, and Instagram for mission-critical applications.
βœ… Scalability: Handles everything from small projects to massive datasets with ease.
βœ… Extensibility: Add custom functions, data types, and even write code in multiple programming languages (PL/pgSQL, Python, JavaScript, and more!).
βœ… ACID Compliance: Ensures data integrity and consistency, even in complex transactions.

---

### Key Features You’ll Love
✨ Advanced SQL Support: PostgreSQL supports complex queries, window functions, and common table expressions (CTEs) for advanced data analysis.
✨ JSON & NoSQL Capabilities: Store and query JSON documents alongside traditional relational data.
✨ Full-Text Search: Build powerful search functionality into your applications.
✨ Geospatial Data: Use PostGIS to handle location-based data for maps, GPS, and more.
✨ Concurrency & Performance: Handles multiple users and high workloads without breaking a sweat.

---

### Getting Started with PostgreSQL
1️⃣ Installation: Download and install PostgreSQL from [postgresql.org](https://www.postgresql.org/). It’s available for Linux, macOS, and Windows.
2️⃣ Learn the Basics: Start with simple SQL queries, then explore advanced features like triggers, stored procedures, and partitioning.
3️⃣ Tools & Ecosystem: Use tools like pgAdmin, DBeaver, or VS Code extensions to manage your databases effortlessly.

---

### Pro Tips for PostgreSQL Users
πŸ”§ Indexing: Use indexes (B-tree, GIN, GiST) to speed up queries.
πŸ”§ Backup & Recovery: Regularly use pg_dump and pg_restore to safeguard your data.
πŸ”§ Extensions: Explore the rich ecosystem of extensions like PostGIS, pg_partman, and more to extend PostgreSQL’s functionality.

---

### Join the PostgreSQL Community!
PostgreSQL has a vibrant and supportive community. Whether you’re a beginner or an expert, there’s always something new to learn. Check out forums, blogs, and conferences to stay updated!

πŸ“š Resources:
- Official Docs: [postgresql.org/docs](https://www.postgresql.org/docs/)
- Tutorials: [pgTutorial](https://www.pgtutorial.com/)
- Community: [PostgreSQL Slack](https://postgres-slack.herokuapp.com/)

@postgres
Please open Telegram to view this post
VIEW IN TELEGRAM
πŸš€ PostgreSQL Indexes Demystified: Pick the Right One 🐘

---

### Why Indexes?
βœ… Speed up lookups, joins, and ORDER BY
βœ… Reduce I/O and CPU on hot queries
βœ… Enforce uniqueness & data quality

---

### The Quick Map
πŸ”Ή B-tree (default): equality/range on scalar types; ideal for =, <, >, BETWEEN, ORDER BY.
πŸ”Ή GIN: many-to-many & containment (JSONB, arrays, full-text).
πŸ”Ή GiST: proximity & custom types (geo, ranges, trigram).
πŸ”Ή BRIN: huge append-only tables with natural order (timestamps, IDs).

---

### Quick Start (copy & adapt)
-- B-tree: common filters / sorts
CREATE INDEX idx_user_created ON users (created_at DESC);

-- Composite + deterministic order
CREATE INDEX idx_orders_ct_id ON orders (created_at DESC, id DESC);

-- INCLUDE to cover SELECT list
CREATE INDEX idx_invoice_cover ON invoices (customer_id) INCLUDE (total);

-- Expression index (avoid runtime functions)
CREATE INDEX idx_lower_email ON users ((lower(email)));

-- JSONB containment (GIN)
CREATE INDEX idx_prod_tags_gin ON products USING GIN (tags);

-- BRIN for massive time-series
CREATE INDEX idx_logs_brin ON logs USING BRIN (ts);


---

### Check It’s Used
EXPLAIN (ANALYZE, BUFFERS)
SELECT * FROM orders
WHERE created_at >= now() - interval '7 days'
ORDER BY created_at DESC
LIMIT 50;

πŸ‘€ Look for Index Scan/Only Scan, low rows, no external sort.

---

### Rules of Thumb
🧭 Match index order to your WHERE + ORDER BY.
πŸ” Add a tiebreaker (e.g., id) for stable pagination.
βœ‚οΈ Use partial indexes for sparse filters:
CREATE INDEX idx_active_only ON users (last_login) WHERE active = true;

🧩 Prefer exact types (avoid implicit casts).
πŸ“° For JSONB: @> + GIN; for full-text: GIN on to_tsvector(...).
🧹 Maintain: VACUUM (ANALYZE), watch bloat, reindex if needed.

---

### Anti-Patterns to Avoid
❌ β€œOne index per column” on wide tables
❌ Functions in predicates without matching expression index
❌ Random BRIN on tiny or highly shuffled tables
❌ Over-indexing writes-heavy tables (insert/update cost!)

---

### Mini Checklist
β˜‘οΈ Query stable? (shape won’t change tomorrow)
β˜‘οΈ Columns & order mirror filter/sort?
β˜‘οΈ Can a partial or covering index cut heap lookups?
β˜‘οΈ Verified with EXPLAIN (ANALYZE)?

#PostgreSQL #SQL #Database #Performance #DBA #DevOps

@postgres
πŸ”₯1
🧩 JSONB in Production: Fast Patterns That Scale 🐘



Why

βœ… Flexibility of JSON with the reliability of Postgres.
βœ… Powerful indexing & operators for speed.




Query Patterns
undefined
-- Containment (has all keys/values)
SELECT id FROM products
WHERE attrs @> '{"color":"black","size":"M"}';

-- Property path filter
SELECT id FROM orders
WHERE (data ->> 'status') = 'paid';

-- Any of these tags?
SELECT id FROM articles
WHERE (tags ?| array['pg','sql','tips']);


Indexes that Matter
undefined
-- General-purpose GIN for JSONB containment / existence
CREATE INDEX idx_prod_attrs_gin ON products USING GIN (attrs);

-- Expression index for a hot field
CREATE INDEX idx_orders_status ON orders ((data ->> 'status'));

-- Partial index to target common filter
CREATE INDEX idx_paid_recent ON orders ((data ->> 'status'))
WHERE (data ->> 'status') = 'paid';




Do βœ…

Index by access pattern: containment β†’ GIN; single field β†’ expression index.
Keep JSONB lean (no huge blobs); normalize stable keys where it helps.
Use EXPLAIN (ANALYZE, BUFFERS) to confirm index usage.




Don’t ❌

Don’t store everything in one giant JSONB; joins on normalized tables can be faster.
Don’t cast at runtime without a matching expression index.
Don’t forget VACUUM (ANALYZE); JSONB updates can cause churn.




Mini Checklist

β˜‘οΈ Chosen operators: @>, ?, ?|, ->, ->> fit the query?
β˜‘οΈ Indexes aligned with filters/containment?
β˜‘οΈ Verified with EXPLAIN (ANALYZE) under realistic work_mem?


#PostgreSQL #JSONB #SQL #Database #Performance #DevOps @postgres
❀1πŸ”₯1
🐘 PostgreSQL Pro Tip: The Power of Partial Indexes

Ever noticed your queries slowing down even with proper indexing? Here's a game-changer that many developers overlook: partial indexes.

Instead of indexing every row, you can create indexes on just the data you actually query:

-- Instead of a full index on status
CREATE INDEX idx_orders_status ON orders (status);

-- Create a partial index for active orders only
CREATE INDEX idx_active_orders
ON orders (created_at, customer_id)
WHERE status = 'active';


Why this rocks:
βœ… Smaller index size = faster queries
βœ… Less storage overhead
βœ… Faster INSERT/UPDATE operations
βœ… Perfect for filtering "hot" data

Real-world example:
If 90% of your orders are completed and you mostly query active ones, why index the completed orders at all?

-- Lightning-fast queries on active orders
SELECT * FROM orders
WHERE status = 'active'
AND customer_id = 12345;


⚑ Pro insight: Partial indexes are especially powerful for soft-deleted records, status-based queries, and time-based data filtering.

Have you used partial indexes in your projects? Share your experience below! πŸ‘‡

#PostgreSQL #DatabaseOptimization #SQL #Performance

@postgres
2❀1
πŸ” PostgreSQL Hidden Gem: EXPLAIN (ANALYZE, BUFFERS)

Think you know EXPLAIN? Think again! Most developers stop at EXPLAIN ANALYZE, but there's a secret weapon for true query optimization.

The magic command:

EXPLAIN (ANALYZE, BUFFERS) 
SELECT * FROM users u
JOIN orders o ON u.id = o.user_id
WHERE u.created_at > '2024-01-01';


What BUFFERS reveals:
πŸ“Š Shared hit - Data found in RAM (fast!)
πŸ’Ύ Shared read - Data loaded from disk (slow!)
πŸ“ Shared dirtied - Pages modified in memory
πŸ’½ Temp read/written - Temporary files used

Real example output:

Buffers: shared hit=1247 read=89 dirtied=12


🎯 What this tells you:


High hit ratio (1247/1336 = 93%) = Good! Data is cached
Many reads = Consider adding more memory or better indexes
Temp files = Query needs more work_mem or optimization

Quick wins:
βœ… Shared hit > 95% = Your query is well-cached
❌ Lots of shared reads = Time to optimize or increase shared_buffers
⚠️ Temp files appearing = Increase work_mem or rewrite the query

Pro tip: Run the query twice - first run loads data into cache, second run shows true performance!

Try this on your slowest query and share what you discover! πŸš€

#PostgreSQL #QueryOptimization #Performance #EXPLAIN

@postgres
❀1
Channel name was changed to Β«PostgreSQL Pro | Database MasteryΒ»
Welcome to @postgres! 🐘

To get maximum value:
1. πŸ”” Enable notifications
2. πŸ“Œ Check daily tips at 10 AM UTC
3. πŸ’¬ Ask questions anytime
4. πŸ“š Check pinned messages for guides

Today's tip below πŸ‘‡

@postgres
PostgreSQL Pro | Database Mastery pinned Β«Welcome to @postgres! 🐘 To get maximum value: 1. πŸ”” Enable notifications 2. πŸ“Œ Check daily tips at 10 AM UTC 3. πŸ’¬ Ask questions anytime 4. πŸ“š Check pinned messages for guides Today's tip below πŸ‘‡ @postgresΒ»
πŸ’¬ Community Q&A Thursday!

Let's solve some real PostgreSQL challenges together! Here are this week's most interesting questions from our community:

Q1: "My COUNT(*) queries are taking forever on large tables. Help!"

βœ… Quick fix:
-- Instead of exact count
SELECT COUNT(*) FROM huge_table; -- Slow!

-- Use estimate for large tables
SELECT reltuples::BIGINT
FROM pg_class
WHERE relname = 'huge_table'; -- Instant!

For pagination? Use LIMIT/OFFSET without total count, or cache the count!

Q2: "Should I use UUID or BIGSERIAL for primary keys?"

πŸ”‘ The answer: It depends!
- BIGSERIAL: 8 bytes, sequential, better for JOINs
- UUID: 16 bytes, globally unique, better for distributed systems

Pro tip: You can have both! BIGSERIAL for internal JOINs, UUID for external APIs.

Q3: "My database backup is 50GB but the actual data seems much smaller. Why?"

πŸ“¦ Common culprits:
- Table bloat from dead tuples β†’ Run VACUUM FULL
- Index bloat β†’ Use REINDEX
- Unused indexes β†’ Check with pg_stat_user_indexes
- Old WAL files β†’ Check your WAL retention settings

🎯 Your turn!
Drop your PostgreSQL questions below! No question is too simple or too complex. Let's learn together!

Best question gets featured in tomorrow's post! πŸ†

#PostgreSQL #Community #DatabaseHelp #SQL

@postgres
πŸ“Š Friday PostgreSQL Digest & What's Coming!

This week's highlights:

πŸ”Έ Tuesday: Learned to read EXPLAIN BUFFERS to spot cache misses and optimize memory usage
πŸ”Έ Wednesday: Community Q&A - solved COUNT(*) performance, UUID vs BIGSERIAL debate, and backup bloat issues

πŸ† Top community insight:
"Using partial indexes on our orders table reduced query time from 2.3s to 45ms!" - Thanks for sharing your success story!

πŸ“ˆ Quick stat:
Did you know? PostgreSQL 16 can be up to 35% faster for aggregate queries compared to PostgreSQL 14!

---

πŸš€ COMING NEXT WEEK:

Monday: "The Art of PostgreSQL Connection Pooling"
- Why your app crashes at 100 connections
- PgBouncer vs connection pooling libraries
- The perfect pool size formula

Tuesday: "JSON vs JSONB: The Ultimate Guide"
- Performance benchmarks you need to see
- When to use each type
- Hidden JSONB operators that will blow your mind

Wednesday: πŸ”₯ Special Deep-Dive Coming!
Advanced content alert! We're preparing something special about query optimization that typically costs $$$ in consulting fees. Stay tuned...

Thursday: Another Community Q&A session

πŸ’‘ Weekend Challenge:
Try implementing a partial index in your project this weekend. Share your before/after query times on Monday!

Have a fantastic weekend, PostgreSQL warriors! 🐘

What topics would you like to see covered next week? Comment below! πŸ‘‡

#PostgreSQL #WeeklyDigest #DatabaseTips #Learning

@postgres
🎯 Weekend Special: PostgreSQL Performance Checklist

Perfect for your weekend maintenance window! Here's a 10-point health check for your PostgreSQL database:

1. Check for unused indexes eating space:

SELECT schemaname, tablename, indexname, idx_scan
FROM pg_stat_user_indexes
WHERE idx_scan = 0
ORDER BY pg_relation_size(indexrelid) DESC;


2. Find your slowest queries:

SELECT mean_exec_time, calls, query
FROM pg_stat_statements
ORDER BY mean_exec_time DESC
LIMIT 5;


3. Identify table bloat:

SELECT tablename, 
pg_size_pretty(pg_total_relation_size(tablename::regclass)) as size,
ROUND(((pg_total_relation_size(tablename::regclass) -
pg_relation_size(tablename::regclass))::numeric /
pg_total_relation_size(tablename::regclass))::numeric * 100) as bloat_ratio
FROM pg_tables
WHERE schemaname = 'public'
ORDER BY pg_total_relation_size(tablename::regclass) DESC;


4. βœ… Quick wins (< 5 minutes each):


Run ANALYZE on your main tables
Check log_min_duration_statement is set (recommend 100ms)
Verify shared_buffers is 25% of available RAM
Ensure autovacuum is ON

5. πŸ” Look for missing indexes:

SELECT schemaname, tablename, attname, n_distinct, correlation
FROM pg_stats
WHERE n_distinct > 100
AND correlation < 0.1
AND schemaname = 'public'
ORDER BY n_distinct DESC;


πŸ’ͺ Weekend Challenge:
Run this checklist on your database. Found something interesting? Share your findings!

🎁 Bonus tip:
Schedule this checklist as a monthly cron job and send results to your email!

Which check revealed the most issues in your database? Let us know! πŸ‘‡

#PostgreSQL #Performance #Maintenance #WeekendWork

@postgres
πŸ”₯1
🧠 Sunday PostgreSQL Wisdom: Understanding MVCC

Let's take a Sunday deep breath and understand something fundamental - how PostgreSQL handles concurrent access without locking readers!

The Magic: MVCC (Multi-Version Concurrency Control)

Imagine a library where instead of taking books away to edit them, you create new versions while others keep reading the old ones. That's MVCC!

How it works in simple terms:

-- Transaction 1 starts
BEGIN;
UPDATE products SET price = 100 WHERE id = 1;
-- Row isn't changed yet for others!

-- Meanwhile, Transaction 2 can still read
SELECT price FROM products WHERE id = 1;
-- Sees the OLD price! No waiting!

-- Transaction 1 commits
COMMIT;
-- NOW Transaction 2 sees the new price

πŸ€” Why should you care?

1. No read locks = Your SELECT queries never wait
2. Better performance = Readers and writers don't block each other
3. The cost = Dead tuples that need VACUUMing

The hidden columns in every row:
SELECT xmin, xmax, ctid, * FROM your_table;

- `xmin: Transaction ID that created this row
-
xmax: Transaction ID that deleted this row (or 0)
-
ctid`: Physical location (page, item)

πŸ’‘ Real-world impact:
- That's why COUNT(*) is slow - it checks visibility for each row!
- That's why you need VACUUM - to clean old versions
- That's why PostgreSQL can do true serializable isolation

πŸŽ“ Sunday experiment:
Open two psql sessions and try the example above. Watch how isolation levels affect what you see!

Question for reflection:
Would you prefer databases to lock readers for consistency, or use MVCC with its cleanup overhead? There's no perfect answer!

Share your MVCC "aha!" moments below! πŸ’¬

#PostgreSQL #MVCC #DatabaseInternals #SundayLearning

@postgres
πŸ‘1
🏊 The Art of PostgreSQL Connection Pooling

Your app crashes at 100 connections? Here's why and how to fix it:

The Problem:
Each PostgreSQL connection uses ~10MB RAM. 100 connections = 1GB just for connections!

The Solution: Connection Pooling

-- Check your current connections
SELECT count(*) as connections,
state
FROM pg_stat_activity
GROUP BY state;

-- Find connection hogs
SELECT usename,
application_name,
count(*)
FROM pg_stat_activity
GROUP BY usename, application_name
ORDER BY count(*) DESC;


🎯 Quick Wins:

1. PgBouncer (external pooler)


Transaction mode: 1000+ app connections β†’ 20 DB connections
Session mode: For apps needing prepared statements
Statement mode: Maximum efficiency

2. Application Pooling

// Node.js example
const pool = new Pool({
max: 20, // Don't exceed 100
idleTimeoutMillis: 30000,
connectionTimeoutMillis: 2000,
})


The Magic Formula:

pool_size = (cpu_cores * 2) + disk_spindles


For most apps: 20-30 connections total!

πŸ’‘ Pro insight: More connections β‰  better performance. After 50 connections, you're likely making things WORSE due to context switching.

Which pooling method do you use? Share your experience! πŸ‘‡

#PostgreSQL #ConnectionPooling #Performance #Scaling

@postgres
πŸ‘1
🎭 JSON vs JSONB: The Ultimate PostgreSQL Guide

Think they're the same? Think again! Here's what you're missing:

JSON: The Purist

-- Stores exact text representation
-- Preserves whitespace, duplicates, order
-- Faster input (no parsing)
-- No indexing support
CREATE TABLE logs (data JSON);


JSONB: The Powerhouse

-- Binary storage, parsed on input
-- Removes duplicates, sorts keys
-- 5-10x faster queries
-- Full indexing support!
CREATE TABLE events (data JSONB);


Performance Reality Check:

-- JSONB with GIN index
CREATE INDEX idx_data ON events USING GIN (data);

-- Lightning fast queries
SELECT * FROM events
WHERE data @> '{"user_id": 123}'; -- Uses index!

-- JSON? Full table scan every time 😒


πŸ”₯ Hidden JSONB Operators:


@> Contains
<@ Contained by
? Key exists
?| Any keys exist
?& All keys exist
|| Concatenate
- Delete key/element
#- Delete at path

When to use JSON:


Logging raw API responses
Need to preserve exact format
Write-heavy, read-never workloads

When to use JSONB (99% of cases):


Any querying needed
Performance matters
Need indexing
Data aggregation

πŸ’Ž Secret weapon:

-- Partial JSONB index for massive performance
CREATE INDEX idx_active_users ON events (data->>'user_id')
WHERE data->>'status' = 'active';


Converted JSON to JSONB? What performance gains did you see? πŸš€

#PostgreSQL #JSON #JSONB #Database #Performance

@postgres
πŸ‘1