Python Pandas Tutorial: How to Drop Columns in a DataFrame

Pandas is a popular open-source data manipulation library for Python that provides powerful tools for working with structured data. One common operation when dealing with data is dropping columns from a DataFrame. In this tutorial, we will explore how to use the drop() function in Pandas to remove columns from a DataFrame. We’ll cover the syntax, options, and provide multiple examples to illustrate its usage.

Table of Contents

1. Introduction to the drop() function
2. Basic Syntax of drop()
3. Examples of Dropping Columns

• Example 1: Drop a Single Column
• Example 2: Drop Multiple Columns

1. Dropping Columns Based on Conditions
2. Inplace vs. Non-Inplace Operation
3. Conclusion

1. Introduction to the drop() function

The drop() function in Pandas is used to remove specified columns from a DataFrame. It provides a flexible way to eliminate unnecessary or irrelevant columns, thereby reducing memory usage and simplifying data analysis. The drop() function doesn’t modify the original DataFrame by default, but rather returns a new DataFrame with the specified columns removed.

2. Basic Syntax of drop()

The basic syntax of the drop() function is as follows:

new_dataframe = old_dataframe.drop(columns=['column_name_1', 'column_name_2', ...])

Here:

• old_dataframe is the DataFrame from which you want to drop columns.
• column_name_1, column_name_2, … are the names of the columns you want to drop.
• new_dataframe is the resulting DataFrame with the specified columns removed.

3. Examples of Dropping Columns

Example 1: Drop a Single Column

Let’s start with a simple example where we have a DataFrame and want to drop a single column from it.

import pandas as pd

# Creating a sample DataFrame
data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 22],
        'Gender': ['Female', 'Male', 'Male']}
df = pd.DataFrame(data)

print("Original DataFrame:")
print(df)

# Dropping the 'Age' column
new_df = df.drop(columns=['Age'])

print("\nDataFrame after dropping 'Age' column:")
print(new_df)

Output:

Original DataFrame:
      Name  Age  Gender
0    Alice   25  Female
1      Bob   30    Male
2  Charlie   22    Male

DataFrame after dropping 'Age' column:
      Name  Gender
0    Alice  Female
1      Bob    Male
2  Charlie    Male

In this example, the ‘Age’ column was dropped from the original DataFrame, and the resulting DataFrame new_df only contains the ‘Name’ and ‘Gender’ columns.

Example 2: Drop Multiple Columns

You can also drop multiple columns using the drop() function. Let’s consider a scenario where we have a DataFrame with more columns and we want to remove two specific columns.

import pandas as pd

# Creating a sample DataFrame
data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 22],
        'Gender': ['Female', 'Male', 'Male'],
        'Country': ['USA', 'Canada', 'UK']}
df = pd.DataFrame(data)

print("Original DataFrame:")
print(df)

# Dropping the 'Age' and 'Country' columns
new_df = df.drop(columns=['Age', 'Country'])

print("\nDataFrame after dropping 'Age' and 'Country' columns:")
print(new_df)

Output:

Original DataFrame:
      Name  Age  Gender Country
0    Alice   25  Female     USA
1      Bob   30    Male  Canada
2  Charlie   22    Male      UK

DataFrame after dropping 'Age' and 'Country' columns:
      Name  Gender
0    Alice  Female
1      Bob    Male
2  Charlie    Male

Here, both the ‘Age’ and ‘Country’ columns were dropped, resulting in a DataFrame containing only the ‘Name’ and ‘Gender’ columns.

4. Dropping Columns Based on Conditions

The drop() function also allows you to drop columns based on specific conditions. For instance, you might want to drop columns with a certain data type or columns with missing values above a certain threshold.

import pandas as pd

# Creating a sample DataFrame
data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, None, 22],
        'Score': [95, 85, 70]}
df = pd.DataFrame(data)

print("Original DataFrame:")
print(df)

# Dropping columns with missing values (NaN)
threshold = len(df) * 0.5  # Drop columns with more than 50% missing values
new_df = df.dropna(axis=1, thresh=threshold)

print("\nDataFrame after dropping columns with missing values:")
print(new_df)

Output:

Original DataFrame:
      Name   Age  Score
0    Alice  25.0     95
1      Bob   NaN     85
2  Charlie  22.0     70

DataFrame after dropping columns with missing values:
      Name  Score
0    Alice     95
1      Bob     85
2  Charlie     70

In this example, the dropna() function was used to drop columns with missing values (NaN) above a certain threshold. The resulting DataFrame new_df only contains columns that have fewer missing values.

5. Inplace vs. Non-Inplace Operation

By default, the drop() function returns a new DataFrame with the specified columns removed, leaving the original DataFrame unchanged. If you want to modify the original DataFrame in-place, you can use the inplace parameter.

import pandas as pd

# Creating a sample DataFrame
data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 22],
        'Gender': ['Female', 'Male', 'Male']}
df = pd.DataFrame(data)

print("Original DataFrame:")
print(df)

# Dropping the 'Age' column in-place
df.drop(columns=['Age'], inplace=True)

print("\nDataFrame after dropping 'Age' column in-place:")
print(df)

Output:

Original DataFrame:
      Name  Age  Gender
0    Alice   25  Female
1      Bob   30    Male
2  Charlie   22    Male

DataFrame after dropping 'Age' column in-place:
      Name  Gender
0    Alice  Female
1      Bob    Male
2  Charlie    Male

In the above example, the inplace=True parameter was used, so the ‘Age’ column was dropped from the original DataFrame itself.

6. Conclusion

In this tutorial, we explored how to use the drop() function in Pandas to remove columns from a DataFrame. We covered the basic syntax of the function, provided examples of dropping single and multiple columns, demonstrated how to drop columns based on conditions, and discussed the difference between inplace

and non-inplace operations.

Being able to drop unnecessary columns from a DataFrame is a crucial skill when performing data analysis and preprocessing tasks. By using the drop() function effectively, you can streamline your data manipulation process and work with more focused and relevant datasets.