Feature Engineering Basics

# Example: raw columns — "hire_date", "today"
# Engineered feature:
df["tenure_years"] = (pd.Timestamp.today() - df["hire_date"]).dt.days / 365.25

# Another example:
df["salary_band"] = pd.cut(df["salary"],
    bins=[0, 30000, 60000, 100000, float("inf")],
    labels=["Entry", "Mid", "Senior", "Lead"]
)

Raw data rarely tells the full story. Tenure in years is more meaningful than a hire date. A salary band creates a categorical insight a raw number cannot. Feature engineering is how you convert data into insight.

# pd.cut — custom boundaries:
df["age_group"] = pd.cut(df["age"],
    bins=[0, 25, 35, 45, 100],
    labels=["<25", "25-35", "35-45", "45+"]
)

# pd.qcut — equal frequency buckets:
df["salary_quartile"] = pd.qcut(df["salary"], q=4,
    labels=["Q1", "Q2", "Q3", "Q4"]
)

cut() divides by value ranges; qcut() divides into equal-frequency groups (each bucket has the same number of rows). Use qcut() when you want balanced segments.

# Label encoding (ordinal):
from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
df["city_encoded"] = le.fit_transform(df["city"])

# One-hot encoding (nominal):
df_encoded = pd.get_dummies(df, columns=["city"], drop_first=True)

# Manual mapping:
df["gender_num"] = df["gender"].map({"Male": 0, "Female": 1})

Use label encoding only for ordinal variables (Low/Medium/High). Use one-hot encoding for nominal variables (city, product) where there is no natural order. drop_first=True avoids the dummy variable trap.

# Multiply two features:
df["revenue_per_customer"] = df["revenue"] / df["customers"]
df["salary_per_year_exp"] = df["salary"] / (df["experience"] + 1)

# Ratio features:
df["conversion_rate"] = df["conversions"] / df["visitors"]

# Polynomial features:
df["exp_squared"] = df["experience"] ** 2

Ratios are often more meaningful than raw numbers — revenue per customer reveals efficiency, not just scale. Adding 1 in the denominator prevents division by zero. Squared features capture non-linear relationships.

from sklearn.preprocessing import StandardScaler, MinMaxScaler

# Standardisation (Z-score) — mean=0, std=1:
scaler = StandardScaler()
df["salary_std"] = scaler.fit_transform(df[["salary"]])

# Normalisation (Min-Max) — scales to [0,1]:
scaler = MinMaxScaler()
df["salary_norm"] = scaler.fit_transform(df[["salary"]])

# Manual:
df["salary_norm"] = (df["salary"] - df["salary"].min()) / (df["salary"].max() - df["salary"].min())

Standardise for algorithms that assume zero-mean data (linear regression, PCA, K-means). Normalise when you need values in [0,1] (neural networks, KNN). Never fit the scaler on test data — fit on train, transform both.