Python for Data Analysts — Complete Guide

Setup & Your First Python Analysis

# Import the core libraries — do this at the top of every notebook
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

# Load an orders CSV file into a pandas DataFrame
df = pd.read_csv('orders.csv')

# First look at the data
print(df.shape)          # (rows, columns) — e.g. (50000, 12)
print(df.dtypes)         # data type of each column
print(df.head(5))        # first 5 rows
print(df.info())         # column names, types, and null counts
print(df.describe())     # count, mean, std, min, max for numeric columns

DataFrames — Selecting, Filtering & Sorting

# Select a single column (returns a Series)
df['city']
df['amount_inr']

# Select multiple columns (returns a DataFrame)
df[['order_id', 'city', 'amount_inr', 'status']]

# Filter rows — equivalent of SQL WHERE
noida_orders = df[df['city'] == 'Noida']
delivered     = df[df['status'] == 'delivered']
high_value    = df[df['amount_inr'] > 2000]

# Multiple conditions — use & (AND) and | (OR), wrap each condition in ()
noida_delivered = df[(df['city'] == 'Noida') & (df['status'] == 'delivered')]
metro_cities    = df[df['city'].isin(['Delhi', 'Mumbai', 'Bengaluru', 'Hyderabad'])]

# Sort
df.sort_values('amount_inr', ascending=False).head(10)   # top 10 orders by value
df.sort_values(['city', 'amount_inr'], ascending=[True, False])

# Rename columns
df = df.rename(columns={'amt': 'amount_inr', 'dt': 'order_date'})

# Add a new calculated column
df['gst_amount']   = df['amount_inr'] * 0.18
df['total_amount'] = df['amount_inr'] + df['gst_amount']

groupby — Aggregating by Category

# Total revenue by city
df.groupby('city')['amount_inr'].sum()

# Multiple aggregations at once — revenue, order count, and average order value
city_summary = df.groupby('city').agg(
    total_revenue   = ('amount_inr', 'sum'),
    order_count     = ('order_id',   'count'),
    avg_order_value = ('amount_inr', 'mean'),
    max_order       = ('amount_inr', 'max'),
).reset_index()

print(city_summary.sort_values('total_revenue', ascending=False))

# Group by multiple columns — revenue by city AND category
pivot = df.groupby(['city', 'category'])['amount_inr'].sum().reset_index()
pivot = pivot.rename(columns={'amount_inr': 'revenue_inr'})

# Monthly revenue trend
df['order_month'] = pd.to_datetime(df['order_date']).dt.to_period('M')
monthly = df.groupby('order_month')['amount_inr'].sum().reset_index()
monthly.columns = ['month', 'revenue_inr']
print(monthly)

# Percentage share of each category
df['category_share_pct'] = (
    df.groupby('category')['amount_inr'].transform('sum')
    / df['amount_inr'].sum() * 100
).round(1)

Data Cleaning in pandas

# --- Check the data quality first ---
print(df.isnull().sum())              # count nulls per column
print(df.duplicated().sum())          # count duplicate rows
print(df['city'].value_counts())      # see all unique cities and counts
print(df['amount_inr'].describe())    # check for negative values or outliers

# --- Handle missing values ---
df['delivery_date'].fillna('Pending', inplace=True)  # fill with a string
df['rating'].fillna(df['rating'].mean(), inplace=True)  # fill with column mean
df = df.dropna(subset=['order_id', 'customer_id'])   # drop rows where key cols are null

# --- Remove duplicates ---
df = df.drop_duplicates(subset=['order_id'])          # keep first occurrence
print(f"Removed {len(df) - df.drop_duplicates().shape[0]} duplicate rows")

# --- Fix data types ---
df['order_date']  = pd.to_datetime(df['order_date'])   # string → datetime
df['amount_inr']  = pd.to_numeric(df['amount_inr'], errors='coerce')  # str → number
df['pincode']     = df['pincode'].astype(str).str.zfill(6)  # ensure 6 digits

# --- Standardise text ---
df['city']     = df['city'].str.strip().str.title()   # remove spaces, title case
df['status']   = df['status'].str.lower().str.strip()
df['category'] = df['category'].str.replace('  ', ' ').str.strip()

# --- Fix inconsistent city names ---
city_map = {
    'New Delhi': 'Delhi', 'delhi': 'Delhi', 'DELHI': 'Delhi',
    'Bengaluru': 'Bangalore', 'blr': 'Bangalore',
    'Mumbai': 'Mumbai', 'bombay': 'Mumbai',
}
df['city'] = df['city'].replace(city_map)

# --- Detect and handle outliers ---
Q1  = df['amount_inr'].quantile(0.25)
Q3  = df['amount_inr'].quantile(0.75)
IQR = Q3 - Q1
outliers = df[(df['amount_inr'] < Q1 - 1.5 * IQR) | (df['amount_inr'] > Q3 + 1.5 * IQR)]
print(f"Outliers detected: {len(outliers)} rows")
df_clean = df[(df['amount_inr'] >= Q1 - 1.5 * IQR) & (df['amount_inr'] <= Q3 + 1.5 * IQR)]

merge & join — Combining DataFrames

# Load two DataFrames
orders    = pd.read_csv('orders.csv')
customers = pd.read_csv('customers.csv')
products  = pd.read_csv('products.csv')

# INNER JOIN — only orders with a matching customer (default)
orders_with_customers = pd.merge(
    orders,
    customers,
    on='customer_id',         # matching column name is same in both DFs
    how='inner'
)

# LEFT JOIN — all orders, even if customer is missing
orders_with_customers = pd.merge(
    orders,
    customers,
    on='customer_id',
    how='left'
)

# Join when column names differ between DataFrames
orders_with_products = pd.merge(
    orders,
    products,
    left_on='product_id',     # column name in left DF
    right_on='prod_id',       # column name in right DF
    how='left'
)

# Stack multiple DataFrames vertically (like SQL UNION ALL)
# Combine 5 regional order files
import glob
all_files = glob.glob('orders_region_*.csv')
df_all    = pd.concat([pd.read_csv(f) for f in all_files], ignore_index=True)
print(f"Combined shape: {df_all.shape}")

# Check for join correctness — row count should match left DF for LEFT JOIN
print(f"Orders: {len(orders)}, After merge: {len(orders_with_customers)}")
# If after > before, you have duplicate customer_id values in the customers file

Visualisation — matplotlib & seaborn

import matplotlib.pyplot as plt
import seaborn as sns

# Set style
sns.set_theme(style='whitegrid', palette='husl')

# --- Bar chart: Revenue by city ---
city_rev = df.groupby('city')['amount_inr'].sum().sort_values(ascending=False).head(8)
plt.figure(figsize=(10, 5))
city_rev.plot(kind='bar', color='#1d4ed8', edgecolor='white')
plt.title('Revenue by City — August 2026', fontsize=14, fontweight='bold')
plt.xlabel('City')
plt.ylabel('Revenue (₹)')
plt.xticks(rotation=45, ha='right')
plt.tight_layout()
plt.savefig('revenue_by_city.png', dpi=150)
plt.show()

# --- Line chart: Monthly revenue trend ---
monthly = df.groupby(df['order_date'].dt.to_period('M'))['amount_inr'].sum()
plt.figure(figsize=(10, 4))
monthly.plot(kind='line', marker='o', color='#16a34a', linewidth=2)
plt.title('Monthly Revenue Trend', fontsize=14, fontweight='bold')
plt.ylabel('Revenue (₹)')
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()

# --- Box plot: Order value distribution by category ---
plt.figure(figsize=(10, 5))
sns.boxplot(data=df, x='category', y='amount_inr', palette='Set2')
plt.title('Order Value Distribution by Category', fontsize=14, fontweight='bold')
plt.xticks(rotation=30, ha='right')
plt.tight_layout()
plt.show()

# --- Heatmap: Revenue by city and category ---
pivot_table = df.pivot_table(
    values='amount_inr', index='city', columns='category', aggfunc='sum', fill_value=0
)
plt.figure(figsize=(12, 6))
sns.heatmap(pivot_table / 1e5, annot=True, fmt='.1f', cmap='YlOrRd',
            cbar_kws={'label': 'Revenue (₹ Lakhs)'})
plt.title('Revenue Heatmap — City × Category (₹ Lakhs)', fontsize=14, fontweight='bold')
plt.tight_layout()
plt.show()

# --- Histogram: Order value distribution ---
plt.figure(figsize=(8, 4))
plt.hist(df['amount_inr'], bins=50, color='#7c3aed', edgecolor='white', alpha=0.8)
plt.axvline(df['amount_inr'].median(), color='red', linestyle='--', label=f"Median: ₹{df['amount_inr'].median():.0f}")
plt.title('Distribution of Order Values', fontsize=14, fontweight='bold')
plt.xlabel('Order Amount (₹)')
plt.legend()
plt.tight_layout()
plt.show()

EDA — Exploratory Data Analysis Workflow

# Complete EDA workflow on an Indian e-commerce orders dataset

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

df = pd.read_csv('orders.csv', parse_dates=['order_date'])

# === STEP 1: Understand the dataset ===
print("Shape:", df.shape)
print("\nData types:\n", df.dtypes)
print("\nNull counts:\n", df.isnull().sum())
print("\nDuplicates:", df.duplicated().sum())

# === STEP 2: Univariate analysis — one variable at a time ===

# Numeric columns
print(df[['amount_inr', 'quantity', 'delivery_days']].describe())

# Categorical columns
for col in ['city', 'category', 'status', 'payment_method']:
    print(f"\n{col}:")
    print(df[col].value_counts(normalize=True).round(2) * 100)  # as %

# === STEP 3: Bivariate analysis — relationships between variables ===

# AOV by category
df.groupby('category')['amount_inr'].agg(['mean', 'median', 'count']).round(0)

# Return rate by category
df['is_returned'] = (df['status'] == 'returned').astype(int)
return_rate = df.groupby('category')['is_returned'].mean() * 100
print(return_rate.sort_values(ascending=False))

# Delivery time by city
df.groupby('city')['delivery_days'].agg(['mean', 'median']).sort_values('mean', ascending=False)

# === STEP 4: Time-based analysis ===
df['month'] = df['order_date'].dt.to_period('M')
df['weekday'] = df['order_date'].dt.day_name()

monthly_revenue = df.groupby('month')['amount_inr'].sum()
print("Revenue trend:\n", monthly_revenue)

weekday_order = ['Monday','Tuesday','Wednesday','Thursday','Friday','Saturday','Sunday']
day_revenue = df.groupby('weekday')['amount_inr'].sum().reindex(weekday_order)
print("\nRevenue by day of week:\n", day_revenue)

# === STEP 5: Outlier detection ===
Q1 = df['amount_inr'].quantile(0.25)
Q3 = df['amount_inr'].quantile(0.75)
IQR = Q3 - Q1
outliers = df[(df['amount_inr'] < Q1 - 1.5*IQR) | (df['amount_inr'] > Q3 + 1.5*IQR)]
print(f"\nOutliers: {len(outliers)} rows ({len(outliers)/len(df)*100:.1f}%)")

# === STEP 6: Correlation analysis ===
numeric_cols = df.select_dtypes(include=[np.number])
corr = numeric_cols.corr()
plt.figure(figsize=(8, 6))
sns.heatmap(corr, annot=True, fmt='.2f', cmap='coolwarm', center=0)
plt.title('Correlation Matrix')
plt.tight_layout()
plt.show()

Do data analysts in India need to learn Python?

Python is increasingly required for data analyst roles in India, particularly in tech companies, startups, analytics consulting firms, and MNCs. It is less commonly required in traditional manufacturing, FMCG, and mid-size businesses, where Excel and SQL are sufficient. The pattern in 2026: junior analyst roles in tech companies list Python as required or preferred; MIS analyst roles in traditional industries rarely require it. The most important Python skills for analysts are pandas (data manipulation), matplotlib or seaborn (visualisation), and a basic understanding of EDA. You do not need to know machine learning or deep learning to get your first data analyst job — but Python proficiency will open more doors and higher-paying roles as you progress.

What is the difference between pandas and numpy for data analysis?

NumPy (Numerical Python) provides multi-dimensional arrays and fast mathematical operations on numbers. It is the foundation that pandas is built on. As a data analyst, you rarely use NumPy directly — you use pandas, which gives you labelled columns, mixed data types, and built-in functions for the kinds of operations analysts do daily. pandas provides DataFrames (think: a spreadsheet in Python), with functions for reading files, filtering rows, grouping by category, aggregating (sum, mean, count), merging tables (like SQL JOINs), and handling missing values. For a data analyst, learn pandas first and thoroughly. NumPy knowledge helps when doing numerical operations on arrays or when you encounter it in data engineering or machine learning contexts.

What Python libraries should a data analyst learn?

Priority order for data analysts in India: (1) pandas — data loading, cleaning, transformation, groupby, merge; (2) matplotlib — basic charts, customisation; (3) seaborn — statistical visualisations, prettier defaults; (4) numpy — array operations, used as pandas dependency; (5) openpyxl or xlrd — reading and writing Excel files from Python; (6) sqlalchemy or pymysql — connecting Python to a SQL database to run queries and load results into pandas. Secondary libraries useful in specific contexts: plotly (interactive charts), scikit-learn (machine learning), scipy (statistical tests). As an analyst, your priority is producing correct analysis fast — master pandas, seaborn, and the ability to connect Python to SQL before exploring the wider ecosystem.

Is Python or SQL more important for data analysts in India?

SQL is more universally required than Python for data analyst roles in India. Almost every data analyst role — across industries, company sizes, and cities — requires SQL. Python is additionally required at tech companies, analytics firms, and for roles that involve modelling or automation. If you are choosing where to invest time: learn SQL first and get proficient. Then learn Python. The combination of SQL + Python + Excel covers the skill set tested in most data analyst interviews in India in 2026. Python without SQL is rare and limiting — data lives in databases, and SQL is how you get it out. Python then processes it further.