• Post author:
  • Post category:Pandas
  • Post last modified:March 27, 2024
  • Reading time:16 mins read
You are currently viewing Pandas Drop Index Column Explained

How to drop the index from Pandas DataFrame? Pandas DataFrames and Pandas Series always have an index, when not specified index while creating, Pandas always creates an index column starting with 0 and incrementing by 1. When you display DataFrmae or Series to the console it displays alongside the column but it is not a column.

Note that in Pandas, technically you can’t drop the index from DataFrame or Series; when you use df.reset_index(drop=True) it actually drops the existing index and resets with the new default index with simple sequential numbers. However, you can drop the index while importing from CSV or exporting DataFrame to a CSV file.

1. Quick Examples of Drop Index from DataFrame & Series

Following are quick examples of how to drop the index column and index level from DataFrame.


# Below are quick examples

# Example 1 - Drop Index and create with new one
df2 = df.reset_index(drop=True)

# Example 2 - Drop Index inplace
df2 = df.reset_index(drop=True, inplace=True)

# Example 3 - Reset the index by setting existing index as column
df2 = df.reset_index()

# Example 4 - Drop index while exporting to CSV
df.to_csv("c:/tmp/courses.csv",index=False)

# Example 5 - Drop index while reading a CSV
df = pd.read_csv("c:/tmp/courses.csv", index_col=False)

# Example 6 - Drop Index level from MultiIndex
df=df.droplevel(0, axis=0) 

# Example 7 - Drop index from series
s2 = s2.reset_index(drop=True)

Pandas Index is an immutable sequence used for indexing DataFrame and Series. pandas.Index is a basic object that stores axis labels for all pandas objects.

DataFrame is a two-dimensional data structure, immutable, heterogeneous tabular data structure with labeled axis rows, and columns. pandas DataFrame consists of three components principal, data, rows, and columns. In DataFrame the row labels are called index.

Let’s create a Pandas DataFrame from Dict.


import pandas as pd

# Create data
technologies = {
    'Courses':["Spark","PySpark","Hadoop","Python"],
    'Fee' :[20000,25000,26000,22000],
    'Duration':['30day','40days','45days', '50days']
               }

# Create dataframe
indexes=['r1','r2','r3','r4']
df = pd.DataFrame(technologies,index=indexes)
print(df)

Yields below output.

pandas drop index

2. Pandas Drop Index Column from DataFrame

As I said above, technically you can’t drop the index column from the pandas DataFrame however, if you do not want the existing index, you can drop it and re-create it with the default index by using reset_index(). Let’s see it with an example


# Drop Index and create with new one
df2 = df.reset_index(drop=True)
print(df2)

Yields below output.

pandas drop index column

This by default doesn’t update the existing DataFrame, you can use inplace=True to update the current DataFrame.


# Drop Index inplace
df.reset_index(drop=True, inplace=True)
print(df)

Yields the same output as above.

2.1. Reset Index without Dropping

Here, drop=True is used to completely drop the index from the DataFrame. However, if you want to set the index as a column and create a new index do not use this param.


# Reset the index by setting existing index as column
df2 = df.reset_index()
print(df2)

I will leave this to you to run and explore the output.

3. Drop the Index Column While Exporting to CSV

By default exporting a pandas DataFrame to CSV includes column names on the first row, row index on the first column, and writes a file with a comma-separated delimiter to separate columns. pandas.DataFrame.to_csv() method provides parameters to ignore an index and header while writing.

To export a DataFrame to CSV by dropping an index in Pandas, use the index=False param. Here, the to_csv() a method is used to write Pandas DataFrame to CSV. The below example demonstrates the Pandas drop index.


# Remove index while exporting
df.to_csv("c:/tmp/courses.csv",index=False)

4. Remove Index Column While Importing a CSV

Similarly, you can also drop the index while reading a CSV file into DataFrame.


# Drop Index while reading a CSV file
df = pd.read_csv("c:/tmp/courses.csv", index_col=False)

5. Pandas Drop Index Level from Multi-Index

Use droplevel() to drop an index level from a multi-index DataFrame, First, let’s create a Multindex DataFrame in Pandas.


# Create DataFrame with MultIndex
multi_index = pd.MultiIndex.from_tuples([("r0", "rA"),
                                       ("r1", "rB")],
                                       names=['index1','index2'])
cols = pd.MultiIndex.from_tuples([("Gasoline", "Toyoto"), 
                                  ("Gasoline", "Ford"), 
                                  ("Electric", "Tesla"),
                                  ("Electric", "Nio")])
data=[[100,300, 900,400 ], [200,500, 300,600]]

df = pd.DataFrame(data, columns=cols,index=multi_index)
print(df)

# Drop Index from MultiIndex
df=df.droplevel(0, axis=0) 
print(df)

Yields below output.

Now, let’s perform Pandas drop index level from MultiIndex by using DataFrame.columns.droplevel() and MultiIndex.droplevel() methods.

Using MultiIndex.droplevel() you can drop single or more levels from multi-level rows/column index. Use axis=1 param to drop columns. To drop row-level use axis=0. The below example drops the first index from DataFrame.


# Drop Index level from MultiIndex
df=df.droplevel(0, axis=0) 
print(df)

Yields below output.

Pandas drop index level

6. Drop Index from Series

Like DataFrame, Pandas Series contains an Index that you can’t drop however if you want to reset the index starting from zero again, use the following approach.


# Create pandas Series
data = ['python','php','java']
s2 = pd.Series(data, index=['r1', 'r2','r3'])
print(s2)

Now, let’s drop the index


# Drop index from series
s2 = s2.reset_index(drop=True)
print(s2)

Frequently Asked Questions on Pandas Drop Index Column

What is the purpose of dropping the index column in Pandas?

The index in Pandas is used to uniquely identify each row in a DataFrame. Sometimes, when you perform operations on a DataFrame or import data, an additional index column is created. Dropping the index column can be useful when you want to reset the index or if you want to remove the default index column that was created during data manipulation.

How do I drop the index column in Pandas?

You can drop the index column in Pandas using the reset_index() method. For example, df = df.reset_index(drop=True)

How can I drop the index column without resetting the index?

You can drop the index column without resetting the index using the drop() method. For example, df = df.drop('index_column_name', axis=1)

What if I want to drop the index and reset it to the default integer index?

You can achieve this by using the reset_index() method with the drop parameter set to True. For example, df = df.reset_index(drop=True)

What if I want to drop the index column for a specific subset of rows?

You can use the reset_index() method with appropriate slicing to reset the index for specific rows. For example, df.loc[condition, :] = df.loc[condition, :].reset_index(drop=True)

7. Conclusion

In this article, you have learned how to perform Pandas drop Index column, and index level from DataFrame and Series. As I explained technically you can’t drop the index from DataFrame however you can remove the existing one and create a new index. Also, learned how to drop the index while reading and writing a CSV file.

Related Articles

References

Naveen Nelamali

Naveen Nelamali (NNK) is a Data Engineer with 20+ years of experience in transforming data into actionable insights. Over the years, He has honed his expertise in designing, implementing, and maintaining data pipelines with frameworks like Apache Spark, PySpark, Pandas, R, Hive and Machine Learning. Naveen journey in the field of data engineering has been a continuous learning, innovation, and a strong commitment to data integrity. In this blog, he shares his experiences with the data as he come across. Follow Naveen @ LinkedIn and Medium