💡 Pandas Cheatsheet

A quick guide to essential Pandas operations for data manipulation, focusing on creating, selecting, filtering, and grouping data in a DataFrame.

1. Creating a DataFrame
The primary data structure in Pandas is the DataFrame. It's often created from a dictionary.

import pandas as pd

data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 32, 28],
        'City': ['New York', 'Paris', 'New York']}
df = pd.DataFrame(data)

print(df)
#       Name  Age       City
# 0    Alice   25   New York
# 1      Bob   32      Paris
# 2  Charlie   28   New York

• A dictionary is defined where keys become column names and values become the data in those columns. pd.DataFrame() converts it into a tabular structure.

2. Selecting Data with .loc and .iloc
Use .loc for label-based selection and .iloc for integer-position based selection.

# Select the first row by its integer position (0)
print(df.iloc[0])

# Select the row with index label 1 and only the 'Name' column
print(df.loc[1, 'Name'])

# Output for df.iloc[0]:
# Name       Alice
# Age           25
# City    New York
# Name: 0, dtype: object
#
# Output for df.loc[1, 'Name']:
# Bob

• .iloc[0] gets all data from the row at index position 0.
• .loc[1, 'Name'] gets the data at the intersection of index label 1 and column label 'Name'.

3. Filtering Data
Select subsets of data based on conditions.

# Select rows where Age is greater than 27
filtered_df = df[df['Age'] > 27]
print(filtered_df)
#       Name  Age       City
# 1      Bob   32      Paris
# 2  Charlie   28   New York

• The expression df['Age'] > 27 creates a boolean Series (True/False).
• Using this Series as an index df[...] returns only the rows where the value was True.

4. Grouping and Aggregating
The "group by" operation involves splitting data into groups, applying a function, and combining the results.

# Group by 'City' and calculate the mean age for each city
city_ages = df.groupby('City')['Age'].mean()
print(city_ages)
# City
# New York    26.5
# Paris       32.0
# Name: Age, dtype: float64

• .groupby('City') splits the DataFrame into groups based on unique city values.
• ['Age'].mean() then calculates the mean of the 'Age' column for each of these groups.

#Python #Pandas #DataAnalysis #DataScience #Programming

━━━━━━━━━━━━━━━
By: @DataScienceM ✨

💡 SciPy: Scientific Computing in Python

SciPy is a fundamental library for scientific and technical computing in Python. Built on NumPy, it provides a wide range of user-friendly and efficient numerical routines for tasks like optimization, integration, linear algebra, and statistics.

import numpy as np
from scipy.optimize import minimize

# Define a function to minimize: f(x) = (x - 3)^2
def f(x):
    return (x - 3)**2

# Find the minimum of the function with an initial guess
res = minimize(f, x0=0)

print(f"Minimum found at x = {res.x[0]:.4f}")
# Output:
# Minimum found at x = 3.0000

• Optimization: scipy.optimize.minimize is used to find the minimum value of a function.
• We provide the function (f) and an initial guess (x0=0).
• The result object (res) contains the solution in the .x attribute.

from scipy.integrate import quad

# Define the function to integrate: f(x) = sin(x)
def integrand(x):
    return np.sin(x)

# Integrate sin(x) from 0 to pi
result, error = quad(integrand, 0, np.pi)

print(f"Integral result: {result:.4f}")
print(f"Estimated error: {error:.2e}")
# Output:
# Integral result: 2.0000
# Estimated error: 2.22e-14

• Numerical Integration: scipy.integrate.quad calculates the definite integral of a function over a given interval.
• It returns a tuple containing the integral result and an estimate of the absolute error.

from scipy.linalg import solve

# Solve the linear system Ax = b
# 3x + 2y = 12
#  x -  y = 1

A = np.array([[3, 2], [1, -1]])
b = np.array([12, 1])

solution = solve(A, b)
print(f"Solution (x, y): {solution}")
# Output:
# Solution (x, y): [2.8 1.8]

• Linear Algebra: scipy.linalg provides more advanced linear algebra routines than NumPy.
• solve(A, b) efficiently finds the solution vector x for a system of linear equations defined by a matrix A and a vector b.

from scipy import stats

# Create two independent samples
sample1 = np.random.normal(loc=5, scale=2, size=100)
sample2 = np.random.normal(loc=5.5, scale=2, size=100)

# Perform an independent t-test
t_stat, p_value = stats.ttest_ind(sample1, sample2)

print(f"T-statistic: {t_stat:.4f}")
print(f"P-value: {p_value:.4f}")
# Output (will vary):
# T-statistic: -1.7432
# P-value: 0.0829

• Statistics: scipy.stats is a powerful module for statistical analysis.
• ttest_ind calculates the T-test for the means of two independent samples.
• The p-value helps determine if the difference between sample means is statistically significant (a low p-value, e.g., < 0.05, suggests it is).

#SciPy #Python #DataScience #ScientificComputing #Statistics

━━━━━━━━━━━━━━━
By: @DataScienceM ✨