Logistic Regression in Python – Getting Data
Data is the foundation of every machine learning system. Without high-quality data, even the most advanced algorithms such as Logistic Regression cannot produce reliable results.
Before building a classification model in Python, the first and most important step is data collection and understanding the dataset. This process ensures that your model learns from accurate, relevant, and well-structured information.
In this guide, you will learn how to collect, load, inspect, and prepare data for Logistic Regression using Python and Pandas.
Why Data is Important in Machine Learning
Machine learning models learn patterns directly from data. If the data is incorrect or incomplete, the model will also produce incorrect predictions.
Key reasons data quality matters:
- Improves prediction accuracy
- Reduces model errors
- Ensures consistency in training
- Prevents misleading results
- Builds reliable AI systems
A common principle in data science is:
Garbage In, Garbage Out (GIGO)
This means poor-quality data always leads to poor-quality results.
Understanding Logistic Regression Data
Logistic Regression is used for binary classification problems, where the output has two possible classes.
Common Examples:
| Problem | Output |
|---|---|
| Email filtering | Spam / Not Spam |
| Medical diagnosis | Sick / Healthy |
| Loan approval | Approved / Rejected |
| Customer churn | Leave / Stay |
| Product purchase | Buy / Not Buy |
Typically, the target variable is encoded as:
- 0 = Negative class
- 1 = Positive class
Sources of Data for Machine Learning
There are multiple ways to collect data for Logistic Regression projects.
1. CSV Files (Most Common Format)
CSV files are widely used in machine learning because they are simple and easy to handle.
Example:
Age,Salary,Purchased
25,30000,0
35,65000,1
45,85000,1
Advantages:
- Easy to create and edit
- Lightweight format
- Supported by Pandas
2. Excel Files
Many businesses store data in Excel spreadsheets.
Examples:
- Sales reports
- Customer databases
- Financial records
pd.read_excel("customers.xlsx")
3. Databases (SQL Systems)
Large-scale applications store data in databases such as:
- MySQL
- PostgreSQL
- SQLite
- SQL Server
Example query:
SELECT * FROM customers;
Databases are commonly used in production-level machine learning systems.
4. APIs (Web Data Sources)
Data can also be collected from online APIs.
Examples:
- Weather APIs
- Financial APIs
- E-commerce APIs
- Social media APIs
APIs are useful for real-time machine learning applications.
5. Public Datasets
Popular platforms include:
- Kaggle
- UCI Machine Learning Repository
- Government open data portals
- Academic datasets
These are widely used for learning and practice.
Creating a Sample Dataset
For demonstration, we will use a simple customer dataset.
Create a file named:
data/customers.csv
Dataset:
Age,Salary,Purchased
22,25000,0
25,30000,0
30,45000,0
35,60000,1
40,70000,1
45,85000,1
50,95000,1
This dataset represents customer purchase behavior.
Installing Required Library (Pandas)
Pandas is essential for data handling in Python.
pip install pandas
Loading Data in Python
Now we load the dataset using Pandas.
import pandas as pd
data = pd.read_csv("data/customers.csv")
print(data.head())
Viewing the Dataset
First Rows:
print(data.head())
Last Rows:
print(data.tail())
This helps verify that data is loaded correctly.
Understanding Dataset Structure
Use .info() to inspect dataset details.
print(data.info())
Shows:
- Number of rows
- Column names
- Data types
- Missing values
Checking Data Types
print(data.dtypes)
Output:
Age int64
Salary int64
Purchased int64
Correct data types are essential for machine learning models.
Statistical Summary
print(data.describe())
Provides:
- Mean
- Minimum and maximum values
- Standard deviation
- Data distribution overview
Checking Missing Values
print(data.isnull().sum())
If values exist, they must be handled before training.
Handling Missing Values
Remove missing values:
data = data.dropna()
Fill missing values:
data.fillna(data.mean(numeric_only=True), inplace=True)
Removing Duplicate Data
Duplicates can reduce model quality.
print(data.duplicated().sum())
Remove duplicates:
data = data.drop_duplicates()
Selecting Features and Target
Separate input and output variables:
X = data[['Age', 'Salary']]
y = data['Purchased']
Explanation:
- X → Features (input data)
- y → Target (output label)
Loading Data from Excel Files
Install dependency:
pip install openpyxl
Load Excel data:
data = pd.read_excel("customers.xlsx")
Loading Data from SQL Databases
Example using SQLite:
import sqlite3
import pandas as pd
conn = sqlite3.connect("customers.db")
data = pd.read_sql_query(
"SELECT * FROM customers",
conn
)
Saving Cleaned Dataset
After preprocessing, save the dataset:
data.to_csv("data/cleaned_customers.csv", index=False)
Benefits:
- Faster reuse
- Consistent preprocessing
- Better workflow organization
Best Practices for Data Collection
- Use trusted data sources
- Collect relevant features only
- Ensure data accuracy
- Maintain structured datasets
- Document dataset origin
- Store raw data separately
Common Mistakes to Avoid
Many beginners make these errors:
- Ignoring missing values
- Using duplicate data
- Not inspecting dataset properly
- Mixing data formats incorrectly
- Training model on unclean data
- Using irrelevant features
Avoiding these improves model performance significantly.
Machine Learning Data Workflow
A typical workflow includes:
1. Collect Data
2. Load Data
3. Inspect Data
4. Clean Data
5. Handle Missing Values
6. Remove Duplicates
7. Select Features
8. Train Model
Conclusion
Data collection is the first and most critical step in building a Logistic Regression model in Python. High-quality data ensures that machine learning algorithms learn meaningful patterns and produce accurate predictions.
In this guide, you learned how to collect data from multiple sources such as CSV files, Excel, databases, and APIs. You also learned how to load, inspect, and prepare datasets using Pandas.
With properly collected and structured data, you are now ready to move forward into data preprocessing, feature engineering, and Logistic Regression model training using Scikit-Learn.
0 Comments