Pandas Series.filter() function is used to return the subset of values from Series that satisfies the condition. The filter() is applied with help of the index labels or on the values themselves. We can filter or subset the values of the pandas series using various functions.
In this article, I will explain filter() syntax, parameters, and how to filter the values or rows from the Pandas Series and also filtering values by using where(), isin(), loc[], and lambda functions.
Key Points –
- The
filter()function in Pandas Series allows for the extraction of elements based on user-defined conditions, providing a concise way to retrieve specific data points. - By utilizing the
filter()function, users can perform targeted data manipulation tasks, efficiently isolating and working with subsets of the Series that meet desired criteria without the need for extensive manual filtering or iteration. - The
filter()function facilitates the creation of subsets from a larger Series, aiding in data analysis tasks where isolating specific data points is essential. - By using the
filter()function, analysts and data scientists can efficiently explore and analyze data by focusing on subsets that are relevant to their research or problem-solving objectives. - The
filter()function in Pandas Series is used to select and filter data based on specific criteria, allowing users to focus on relevant information.
1. Quick Examples of Series filter() Function
If you are in hurry below are some quick examples of the Pandas Series filter() function.
# Quick examples of series filter() function
# Example 1: use Series.filter() function
# To filter a pandas series
ser2 = ser.filter(regex = '. .')
# Example 2: filter() index by labels
ser2 = ser.filter(items = ['Spark', 'Python'])
# Example 3 : use loc[] and lambda
# To filter a pandas series
ser2 = ser.loc[lambda x : x == 23000]
# Example 4: use loc[] property & OR condition
ser2 = ser.loc[lambda x : (x 28000)]
# Example 5: use where() function to filter series
ser2 = ser.where(ser < 25000).dropna()
# Example 6: use isin() function to filter series
ser2 = ser[ser.isin([23000,28000])]
2. Syntax of Series.filter() Function
Following is the syntax of the create Series.filter() function.
# Syntax of Series.filter() function
Series.filter(items=None, like=None, regex=None, axis=None)
2.1 Parameter of filter()
Following are the parameters of the filter().
items– A list of labels to filter on the specified axis.like– A string that is used to filter labels based on a substring match.regex– A regular expression (regex) to filter labels based on pattern matching.axis– {0 or ‘index’, 1 or ‘columns’, None}, default None. When not specified it used columns. The axis along which the filtering will be applied. By default, it is set toNone, which means the filtering is done on the index.
2.2 Return Value of filter()
It returns the value of the filter() the same type as the input object.
3. Create Pandas Series
Pandas Series is a one-dimensional, Index-labeled data structure that is available only in the Pandas library. It can store all the datatypes such as strings, integers, float, and other Python objects. We can access each element in the Series with the help of corresponding default indices.
Note : Series data structure is the same as the NumPy array data structure but only one difference that is arrays indices are integers and start with 0, whereas in series, the index can be anything even strings. The labels do not need to be unique but they must be of hashable type.
Now, let’s create pandas series using list of values.
import pandas as pd
# Create the Series
ser = pd.Series([20000,25000,23000,28000,55000,23000])
# Create the Index
index = ['Java','Spark','PySpark','Pandas','python NumPy','Python']
# Set the index
ser.index = index
print(ser)
Yields below output.
# Output:
Java 20000
Spark 25000
PySpark 23000
Pandas 28000
python NumPy 55000
Python 23000
dtype: int64
4. Use Series.filter() Function To Filter a Pandas Series
By using Series.filter() function you can filter the Series by index labels or by values. When you use index labels to files you can use regular expressions by using “regex”. The following example filters values from the given series object whose index label name has a space.
# Use Series.filter() function to filter a pandas series
ser2 = ser.filter(regex = '. .')
print(ser2)
Yields below output.
# Output:
python NumPy 55000
dtype: int64
5. Filter Series by Index Labels
By default pandas.Series.filter() select the indexes by labels you specified using item, like, and regex parameters. The following example filters series with the list of index labels Spark and Python.
# Filter() index by labels
ser2 = ser.filter(items = ['Spark', 'Python'])
print(ser2)
Yields below output.
# Output:
Spark 25000
Python 23000
dtype: int64
6. Use loc[] & Lambda to Filter a Pandas Series
You can also filter the Pandas Series using Series.loc[] along with lambda function. The following example returns values from a series where values are equal to 23000.
# Use loc[] and lambda to filter a pandas series
ser2 = ser.loc[lambda x : x == 23000]
print(ser2)
Yields below output.
# Output:
PySpark 23000
Python 23000
dtype: int64
Alternatively, you can also apply an “OR” condition with the “loc[]” property. The following example filters values that are less than 23000 or values greater than 28000. For examples.
# Use loc[] property & OR condition
ser2 = ser.loc[lambda x : (x < 23000 or x > 28000)]
print(ser2)
Yields below output.
# Output:
Java 20000
python NumPy 55000
dtype: int64
7. Use where() Function To Filter Series
Similarly, we can also use where() function to filter a series by values using expressions. Using the where() function to filter a Series (ser) where the values are less than 25000 and then dropping the NaN values using dropna().
# Use where() function to filter series
ser2 = ser.where(ser < 25000).dropna()
print(ser2)
The above program will produce a Series (ser2) where values greater than or equal to 25000 will be replaced with NaN, and values less than 25000 will be retained. The dropna() method is optional in this case since where() already handles the NaN values, and you can choose whether to include it based on your specific requirements. This example yields the below output.
# Output:
Pandas 28000.0
python NumPy 55000.0
dtype: float64
8. Use isin() Function To Filter Series
By use isin() function is used to get the values from the series that are present in the list of values. For instance, ser.isin([23000, 30000]) creates a boolean mask indicating whether each element in the original Series (ser) is in the specified list [23000, 28000]. The boolean mask is then used to filter the original Series using boolean indexing, resulting in ser2 containing only the elements that match the specified values.
# Use isin() function to filter series
ser2 = ser[ser.isin([23000,28000])]
print(ser2)
Yields below output.
# Output:
PySpark 23000
Pandas 28000
Python 23000
dtype: int64
9. Complete Example
import pandas as pd
# Create the Series
ser = pd.Series([20000,25000,23000,28000,55000,23000])
# Create the Index
index_ = ['Java','Spark','PySpark','Pandas','python NumPy','Python']
# Set the index
ser.index = index_
print(ser)
# Use Series.filter() function to filter a pandas series
ser2 = ser.filter(regex = '. .')
print(ser2)
# Filter() index by labels
ser2 = ser.filter(items = ['Spark', 'Python'])
print(ser2)
# Use loc[] and lambda to filter a pandas series
ser2 = ser.loc[lambda x : x == 23000]
print(ser2)
# Use loc[] property & OR condition
ser2 = ser.loc[lambda x : (x 28000)]
print(ser2)
# Use where() function to filter series
ser2 = ser.where(ser < 25000).dropna()
print(ser2)
# Use isin() function to filter series
ser2 = ser[ser.isin([23000,28000])]
print(ser2)
Frequently Asked Questions on Series filter() Function
The filter() function in Pandas Series is designed to selectively extract and filter data based on specified conditions, allowing users to focus on subsets of data that meet particular criteria.
The function works by creating a boolean mask based on the specified condition. It then applies this mask to the Series, retaining only the elements that satisfy the condition.
The filter() function can be applied to both the index and values of a Pandas Series. By default, it filters based on the index, but users can specify the axis parameter to filter along the columns (axis=1).
The filter() function is commonly used in data analysis tasks where there is a need to extract subsets of data based on specific conditions, enabling users to focus on relevant information and perform targeted analysis on a Pandas Series.
The filtering conditions in filter() are mutually exclusive. Users can choose to use either items, like, or regex to specify the filtering criteria, and only one of them should be used at a time.
The filter() function is a versatile tool for filtering data, offering a more structured and parameterized approach compared to boolean indexing. While boolean indexing is powerful, the filter() function provides additional options for label-based filtering, substring matching, and regular expression filtering. The choice between them depends on the specific requirements of the analysis.
Conclusion
In this article, you have learned the how to filter the Pandas Series by using filter(), where(), isin(), and loc[] with lambda function by using examples.
Happy Learning !!
Related Articles
- Pandas filter by column value
- Pandas Series.max() Function
- Pandas Series sum() Function
- Pandas Series.mean() Function
- How to Use NOT IN Filter in Pandas
- Pandas Filter DataFrame Rows on Dates
- Pandas Filter Rows Using IN Like SQL
- Pandas Filter DataFrame by Multiple Conditions
- pandas DataFrame filter() – Usage & Examples
- Pandas Filter DataFrame by Substring criteria
- Pandas Series groupby() Function with Examples
- Pandas Series unique() Function with Examples
- Pandas Filter Rows with NAN Value from DataFrame Column
- How to Get the Length of a Series in Pandas?