Topic: Python File Handling — Reading, Writing, and Managing Files (Beginner to Advanced)

---

What is File Handling?

• File handling allows Python programs to read from and write to external files — such as .txt, .csv, .json, etc.

• Python uses built-in functions like open(), read(), and write() to interact with files.

---

Opening a File

file = open("example.txt", "r")  # "r" = read mode
content = file.read()
file.close()

---

Using with Statement (Best Practice)

• Automatically handles file closing:

with open("example.txt", "r") as file:
    content = file.read()

---

File Modes

• "r" — read (default)
• "w" — write (creates or overwrites)
• "a" — append (adds to the end)
• "x" — create (fails if file exists)
• "b" — binary mode
• "t" — text mode (default)

---

Writing to Files

with open("output.txt", "w") as file:
    file.write("Hello, world!")

• Note: "w" overwrites existing content.

---

Appending to Files

with open("output.txt", "a") as file:
    file.write("\nNew line added.")

---

Reading Line by Line

with open("example.txt", "r") as file:
    for line in file:
        print(line.strip())

---

Working with File Paths

• Use os.path or pathlib for platform-independent paths.

from pathlib import Path

file_path = Path("folder") / "file.txt"
with open(file_path, "r") as f:
    print(f.read())

---

Advanced Tip: Reading and Writing CSV Files

import csv

with open("data.csv", "w", newline="") as file:
    writer = csv.writer(file)
    writer.writerow(["name", "age"])
    writer.writerow(["Alice", 30])

with open("data.csv", "r") as file:
    reader = csv.reader(file)
    for row in reader:
        print(row)

---

Summary

• Use open() with correct mode to read/write files.

• Prefer with statement to manage files safely.

• Use libraries like csv, json, or pickle for structured data.

• Always handle exceptions like FileNotFoundError for robust file operations.

---

Exercise

• Write a Python program that reads a list of names from names.txt, sorts them alphabetically, and saves the result in sorted_names.txt.

---

#Python #FileHandling #ReadWrite #DataProcessing #ProgrammingTips

https://t.iss.one/DataScience4

# Interview Power Move: Parallel Merging
from concurrent.futures import ThreadPoolExecutor
from PyPDF2 import PdfMerger

def parallel_merge(pdf_list, output, max_workers=4):
    chunks = [pdf_list[i::max_workers] for i in range(max_workers)]
    temp_files = []
    
    def merge_chunk(chunk, idx):
        temp = f"temp_{idx}.pdf"
        merger = PdfMerger()
        for pdf in chunk:
            merger.append(pdf)
        merger.write(temp)
        return temp
    
    with ThreadPoolExecutor() as executor:
        temp_files = list(executor.map(merge_chunk, chunks, range(max_workers)))
    
    # Final merge of chunks
    final_merger = PdfMerger()
    for temp in temp_files:
        final_merger.append(temp)
    final_merger.write(output)

parallel_merge(["doc1.pdf", "doc2.pdf", ...], "parallel_merge.pdf")

# Pro Tip: Validate PDFs before merging
from PyPDF2 import PdfReader

def is_valid_pdf(path):
    try:
        with open(path, "rb") as f:
            reader = PdfReader(f)
            return len(reader.pages) > 0
    except:
        return False

valid_pdfs = [f for f in pdf_files if is_valid_pdf(f)]
merger.append(valid_pdfs)  # Only merge valid files

# Real-World Case Study: Invoice Processing Pipeline
import glob
from PyPDF2 import PdfMerger

def process_monthly_invoices():
    # 1. Download invoices from SFTP
    download_invoices("sftp://vendor.com/invoices/*.pdf")
    
    # 2. Validate and sort
    invoices = sorted(
        [f for f in glob.glob("invoices/*.pdf") if is_valid_pdf(f)],
        key=lambda x: extract_invoice_date(x)
    )
    
    # 3. Merge with cover page
    merger = PdfMerger()
    merger.append("cover_template.pdf")
    for inv in invoices:
        merger.append(inv, outline_item=get_client_name(inv))
    
    # 4. Add metadata and encrypt
    merger.add_metadata({"/InvoiceCount": str(len(invoices))})
    merger.encrypt(owner_pwd="finance_team_2023")
    merger.write(f"Q3_Invoices_{datetime.now().strftime('%Y%m')}.pdf")
    
    # 5. Upload to secure storage
    upload_to_s3("secure-bucket/processed/", "Q3_Invoices.pdf")

process_monthly_invoices()

By: https://t.iss.one/DataScience4

#Python #PDFProcessing #DocumentAutomation #PyPDF2 #CodingInterview #BackendDevelopment #FileHandling #DataEngineering #TechJobs #Programming #SystemDesign #DeveloperTips #CareerGrowth #CloudComputing #Docker #Microservices #Productivity #TechTips #Python3 #SoftwareEngineering