Logistic Regression in Python Tutorial – Classification with Scikit-Learn

Logistic Regression in Python Tutorial

Logistic Regression is one of the most widely used machine learning algorithms for classification problems. Despite its name, it is not used for predicting continuous values. Instead, it is designed to predict categorical outcomes such as Yes/No or 0/1.

Because of its simplicity, speed, and interpretability, Logistic Regression is often the first algorithm used in machine learning with Python and Scikit-Learn.

In this tutorial, you will learn:

• What Logistic Regression is
• How it works
• How to implement it in Python
• How to evaluate classification performance

---

What is Logistic Regression?

Logistic Regression is a supervised learning algorithm used for classification tasks. It predicts the probability that a given input belongs to a specific class.

Unlike Linear Regression (which predicts numbers), Logistic Regression predicts probabilities between 0 and 1.

Real-world examples:

• Email → Spam or Not Spam
• Customer → Buy or Not Buy
• Patient → Disease or No Disease
• Student → Pass or Fail

Output interpretation:

• 0.90 → High probability of Class 1
• 0.50 → Uncertain prediction
• 0.10 → Low probability of Class 1

---

Why is it Called Logistic Regression?

The term comes from the logistic (sigmoid) function, which converts any real number into a probability value between 0 and 1.

The model works by:

1. Calculating a linear equation from input features
2. Passing the result through the sigmoid function
3. Producing a probability for classification

This is why Logistic Regression is widely used for binary classification problems in machine learning.

---

How Logistic Regression Works (Simple Explanation)

Logistic Regression follows three main steps:

Step 1: Input Features

The model takes input variables such as:

• Study hours
• Age
• Salary
• Experience

---

Step 2: Linear Calculation

The algorithm combines all features using weights to produce a score.

---

Step 3: Sigmoid Function

The score is converted into a probability between 0 and 1.

Final prediction is made using a threshold (usually 0.5):

• ≥ 0.5 → Class 1
• < 0.5 → Class 0

---

Install Required Python Libraries

Before starting, install the required packages:

pip install numpy pandas matplotlib scikit-learn

---

Import Required Libraries

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

---

Create a Simple Dataset (Study Hours vs Pass/Fail)

data = {
    'Hours': [1,2,3,4,5,6,7,8,9,10],
    'Pass':  [0,0,0,0,1,1,1,1,1,1]
}

df = pd.DataFrame(data)
print(df)

---

Prepare Features and Labels

X = df[['Hours']]
y = df['Pass']

• X → Input feature
• y → Target label

---

Split Data into Training and Testing Sets

X_train, X_test, y_train, y_test = train_test_split(
    X, y,
    test_size=0.2,
    random_state=42
)

Why this step is important:

• Prevents overfitting
• Ensures fair evaluation
• Tests model on unseen data

---

Train Logistic Regression Model

model = LogisticRegression()
model.fit(X_train, y_train)

The model learns the relationship between study hours and exam results.

---

Make Predictions

predictions = model.predict(X_test)
print(predictions)

Output example:

[1 0]

• 1 → Pass
• 0 → Fail

---

Predict Probabilities

probabilities = model.predict_proba(X_test)
print(probabilities)

Example output:

[[0.12 0.88]
 [0.91 0.09]]

Interpretation:

• 88% chance of passing
• 9% chance of passing

This is useful for decision-making systems and risk analysis.

---

Evaluate Model Accuracy

accuracy = accuracy_score(y_test, predictions)
print("Accuracy:", accuracy)

Example output:

Accuracy: 1.0

---

Logistic Regression with Iris Dataset (Real Example)

from sklearn.datasets import load_iris

iris = load_iris()

X = iris.data
y = iris.target

X_train, X_test, y_train, y_test = train_test_split(
    X, y,
    test_size=0.2,
    random_state=42
)

model = LogisticRegression(max_iter=200)
model.fit(X_train, y_train)

predictions = model.predict(X_test)

print("Accuracy:", accuracy_score(y_test, predictions))

---

Model Evaluation Metrics

Logistic Regression can be evaluated using multiple metrics:

Accuracy

Measures overall correctness.

Precision

Measures correctness of positive predictions.

Recall

Measures how many actual positives were found.

F1 Score

Balances precision and recall.

Confusion Matrix

Shows detailed prediction results.

from sklearn.metrics import precision_score, recall_score, f1_score, confusion_matrix

---

Advantages of Logistic Regression

• Simple and easy to understand
• Fast training and prediction
• Works well for binary classification
• Produces probability outputs
• Efficient for large datasets
• Strong baseline model in machine learning

---

Limitations of Logistic Regression

• Assumes linear relationships
• Not suitable for highly complex data
• Sensitive to outliers
• Requires feature engineering
• Performs poorly on non-linear problems

---

Real-World Applications

Logistic Regression is widely used in:

• Email spam detection
• Fraud detection systems
• Medical diagnosis
• Credit risk scoring
• Customer churn prediction
• Sentiment analysis
• Marketing prediction systems

---

Best Practices for Logistic Regression

• Always scale features when needed
• Handle missing values properly
• Remove irrelevant features
• Use train-test split correctly
• Evaluate with multiple metrics
• Handle class imbalance carefully

---

When Should You Use Logistic Regression?

Use Logistic Regression when:

• The problem is classification
• You need interpretable results
• The dataset is structured and clean
• You want fast training
• You need probability-based outputs

It is often used as a baseline model before trying advanced algorithms like Decision Trees, Random Forest, or Neural Networks.

---

Conclusion

Logistic Regression is one of the most important algorithms in machine learning for solving classification problems. It is simple, fast, and highly interpretable, making it ideal for beginners and professionals working on real-world data science projects.

By using Python and Scikit-Learn, you can easily build Logistic Regression models for tasks such as prediction, classification, and probability estimation.

Understanding Logistic Regression gives you a strong foundation for advanced machine learning techniques and real-world AI applications.