Logistic Regression in Python – Building Classifier
After preparing and cleaning your dataset, the next important step in any machine learning project is building a classifier. This is where the model learns patterns from data and becomes capable of making predictions on new, unseen inputs.
In this guide, you will learn how to build a Logistic Regression classifier in Python using Scikit-Learn, understand how it works internally, and see how it makes predictions in real-world scenarios.
What is a Classifier in Machine Learning?
A classifier is a type of machine learning model that predicts categorical outcomes instead of continuous values.
In simple terms, it answers questions like:
- Yes or No
- True or False
- 0 or 1
- Spam or Not Spam
Examples of Classification Problems
| Input Data | Output Prediction |
|---|---|
| Customer data | Buy / Not Buy |
| Email content | Spam / Not Spam |
| Medical records | Disease / No Disease |
| Banking history | Loan Approved / Rejected |
Logistic Regression is one of the most widely used algorithms for these types of problems.
How Logistic Regression Classifier Works
Logistic Regression builds a classifier in three main steps:
1. Linear Combination of Features
The model first combines input features using weights:
- Age
- Salary
- Other numerical variables
This creates a single value called z.
2. Sigmoid Function (Probability Conversion)
The value z is passed through the sigmoid function to convert it into a probability between 0 and 1.
P = 1 / (1 + e^(-z))
This step is what makes Logistic Regression a probabilistic model.
3. Decision Threshold
Finally, the probability is converted into a class label:
- If P ≥ 0.5 → Class 1
- If P < 0.5 → Class 0
This is how the final classification is made.
Step 1: Import Required Libraries
We start by importing essential Python libraries.
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
These libraries handle data processing, model training, and evaluation.
Step 2: Load the Dataset
We load the dataset using Pandas.
data = pd.read_csv("data/customers.csv")
print(data.head())
This step helps us understand the structure of the dataset.
Step 3: Define Features and Target Variable
We separate input features (X) and output labels (y).
X = data[['Age', 'Salary']]
y = data['Purchased']
- X → Input features
- y → Target variable (what we want to predict)
Step 4: Split the Dataset
We split the data into training and testing sets.
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.25,
random_state=42
)
Why this is important:
- Training data is used to teach the model
- Test data is used to evaluate performance
Step 5: Feature Scaling
Feature scaling ensures that all variables contribute equally.
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
Why scaling matters:
- Prevents large values from dominating
- Improves model performance
- Ensures stable learning process
Step 6: Build Logistic Regression Classifier
Now we create and train the model.
model = LogisticRegression()
model.fit(X_train, y_train)
What happens here:
The model learns patterns such as:
- How age affects purchasing behavior
- How salary influences decision-making
- Relationships between features and output
Step 7: Make Predictions
After training, we test the model.
y_pred = model.predict(X_test)
print(y_pred)
Example Output:
[0 1 1 0 1]
Meaning:
- 0 → Customer will NOT purchase
- 1 → Customer will purchase
Step 8: Predict Probabilities
Logistic Regression also provides probability values.
probs = model.predict_proba(X_test)
print(probs[:5])
Example Output:
[[0.81 0.19]
[0.20 0.80]
[0.35 0.65]]
Interpretation:
- First column → Probability of class 0
- Second column → Probability of class 1
This is useful for decision-making systems.
Step 9: Decision Function (Internal Logic)
The model also calculates raw scores before applying sigmoid.
print(model.decision_function(X_test))
Higher values indicate stronger confidence for Class 1.
Step 10: Evaluate the Classifier
After predictions, we evaluate performance.
Accuracy Score
from sklearn.metrics import accuracy_score
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy:", accuracy)
Confusion Matrix
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_pred)
print(cm)
This shows correct and incorrect predictions.
Classification Report
from sklearn.metrics import classification_report
print(classification_report(y_test, y_pred))
This includes:
- Precision
- Recall
- F1-score
- Support
Complete Logistic Regression Classifier Code
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
# Load dataset
data = pd.read_csv("data/customers.csv")
# Features and target
X = data[['Age', 'Salary']]
y = data['Purchased']
# Split data
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.25, random_state=42
)
# Feature scaling
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
# Build classifier
model = LogisticRegression()
model.fit(X_train, y_train)
# Predictions
y_pred = model.predict(X_test)
# Evaluation
print("Accuracy:", accuracy_score(y_test, y_pred))
Advantages of Logistic Regression Classifier
- Simple and fast to implement
- Works well for binary classification
- Provides probability outputs
- Highly interpretable model
- Strong baseline for ML projects
Limitations
- Assumes linear decision boundary
- Not suitable for complex non-linear data
- Sensitive to outliers
- Requires feature scaling
Real-World Applications
Logistic Regression is widely used in:
1. Healthcare
- Disease prediction
- Risk classification
2. Finance
- Credit scoring
- Fraud detection
3. Marketing
- Customer churn prediction
- Campaign effectiveness
4. Cybersecurity
- Spam detection
- Fraud detection systems
Best Practices for Building Classifiers
- Always scale numerical features
- Use proper train-test split
- Avoid data leakage
- Evaluate using multiple metrics
- Save trained models for reuse
Conclusion
Building a Logistic Regression classifier in Python is a fundamental step in machine learning. It allows you to train a model that can classify data into categories and provide probability-based predictions.
With Scikit-Learn, you can easily build, train, and evaluate classifiers for real-world applications such as customer prediction, fraud detection, and medical diagnosis.
Mastering this process gives you a strong foundation for more advanced machine learning models.
0 Comments