| { One stop for all Spark Examples }

Remove NaN From Pandas Series

Post author:Malli
Post category:Pandas / Python
Post last modified:January 17, 2023

Spread the love

Pandas Series.dropna() function is used to remove NaN values from the pandas Series. It returns new series with the same values as the original but without any NaN values. In this article, I will explain how to remove NaN from Series in Pandas by using dropna() and other methods with examples.

1. Quick Examples of Remove NaN From Series

If you are in hurry below are some quick examples of how to remove NaN from the pandas series.


# Below are the quick examples

# Example 1: use dropna() to remove nan values from a pandas series
ser2 = ser.dropna()

# Example 2: use isnull() to remove nan values from a pandas series
ser2 = ser[~ser.isnull()]

2. Syntax of Series.dropna() Function

Following is the syntax of Series.dropna() function.


# Syntax of Series.dropna() function
Series.dropna(axis=0, inplace=False, how=None)

2.1 Parameter of dropna()

Following are the parameters of the dropna().

axis – {0 or ‘index’} by default Value 0: There is only one axis to drop values from.
inplace – boolean, default Value: False: If True, do an operation in place and return None.
how – str, this optional parameter: Not in use.

2.2 Return Value of dropna()

It returns the pandas Series without NaN values.

3. Create Pandas Series

Pandas Series is a one-dimensional, Index-labeled data structure that is available only in the Pandas library. It can store all the datatypes such as strings, integers, float, and other python objects. We can access each element in the Series with the help of corresponding default indices.

Note : Series data structure is the same as the NumPy array data structure but only one difference is that array indices are integers and start with 0, whereas in series, the index can be anything even strings. The labels do not need to be unique but they must be of hashable type.

Now, let’s create pandas series using a list. Note that NaN in pandas and represent by using NumPy np.nan.


import pandas as pd
import numpy as np
 
# Create the Series 
ser = pd.Series(['Java', 'Spark', np.nan, 'PySpark', np.nan,'Pandas','NumPy', np.nan,'Python'])
print(ser)

Yields below output.


# Output
0       Java
1      Spark
2        NaN
3    PySpark
4        NaN
5     Pandas
6      NumPy
7        NaN
8     Python
dtype: object

4. Use dropna() Method To Remove NaN Values From Series

Using dropna() method we can remove the NaN values from the series. Let’s use Series.dropna() method to remove NaN (missing) values from the original Series to get a new series. This method returns a new Series after removing all NAN values.


# Use dropna() to remove nan values from a pandas series
ser2 = ser.dropna()
print(ser2)

Yields below output.


# Output
0       Java
1      Spark
3    PySpark
5     Pandas
6      NumPy
8     Python
dtype: object

5. Use isnull() Method To Remove NaN Values From Series

We can also use Series.isnull() on the original Series to get a new Series with only boolean values and the same dimensions as the original. The boolean Series contains True if the value in the original is NaN and False otherwise for each element and use this Series on the original series to remove all NaN values.


# Use isnull() to remove nan values from a pandas series
ser2 = ser[~ser.isnull()]
print(ser2)

Yields the same output as above.

6. Complete Example For Remove NaN From Series


import pandas as pd
import numpy as np

# Create the Series  
ser = pd.Series(['Java', 'Spark', np.nan, 'PySpark', np.nan,'Pandas','NumPy', np.nan,'Python'])
print(ser)

# Example 1: use dropna() to remove nan values from a pandas series
ser2 = ser.dropna()
print(ser2)

# Example 2: use isnull() to remove nan values from a pandas series
ser2 = ser[~ser.isnull()]
print(ser2)

7. Conclusion

In this article, you have learned how to remove NaN values from Series in Pandas by using Series.dropna(), and Series.isnull() functions with examples.

Happy Learning !!

References

https://pandas.pydata.org/docs/reference/api/pandas.Series.dropna.html

Tags: pandas Series