Pandas Get Unique Values in Column

You can get unique values in column/multiple columns from pandas DataFrame using unique() or Series.unique() functions. unique() from Series is used to get unique values from a single column and the other one is used to get from multiple columns.

The unique() function removes all duplicate values on a column and returns a single value for multiple same values.

Note that Uniques are returned in order of appearance. if you want to sort, use sort() function to sort single or multiple columns of DataFrame.

Key Points –

• The unique() method returns the unique values in a column as a NumPy array, which is useful for quickly identifying distinct entries.
• The values returned by unique() are in the order in which they appear in the DataFrame, not sorted by default.
• The unique() method returns values in the original data type of the column, whether it’s numeric, string, or another type.
• unique() includes NaN values in the result if they are present in the column, treating them as a distinct value.
• You can use unique() on a subset of the DataFrame by selecting a specific column, making it easier to focus on a particular feature or attribute.

Related: Find Duplicate Rows from Pandas DataFrame

Quick Examples of Getting Unique Values in Columns

If you are in a hurry, below are some quick examples of how to get unique values in a single column and multiple columns in DataFrame.

# Quick examples of getting unique values in columns

# Example 1: Find unique values of a column
print(df['Courses'].unique())
print(df.Courses.unique())

# Example 2: Convert to list
print(df.Courses.unique().tolist())

# Example 3: Unique values with drop_duplicates
df.Courses.drop_duplicates()

# Example 4: Using pandas.unique() 
# To unique values in multiple columns
df2 = pd.unique(df[['Courses', 'Fee']].values.ravel('K'))

# Example 5: Using pandas.unique() 
# To unique values
df2 = pd.unique(df[['Courses']].values.ravel())

# Example 6: Find the unique values in multiple columns
# Using numpy.unique()
df2 = np.unique(df[['Courses', 'Duration']].values)

# Example 7: Use numpy.unique() 
# To unique values in multiple columns 
column_values = df[['Courses', 'Duration']].values
df2 = np.unique(column_values)

# Example 8: Using Set() in pandas DataFrame
df2 = set(df.Courses.append(df.Fee).values)

# Example 9: Using set() method
df2 = set(df.Courses) | set(df.Fee)

# Example 10: To get unique values in one series/column
df2 = df['Courses'].unique()

# Example 11: Using pandas.concat 
# To extend one column to multiple columns
df2 = pd.concat([df['Courses'],df['Duration'],df['Fee']]).unique()

# Example 12: Use Series.drop_duplicates() 
# To get unique values
print(df.Courses.drop_duplicates())

Now, let’s create a DataFrame with duplicate values, execute these examples and validate the results. Our DataFrame contains column names Courses, Fee, Duration, and Discount.

# Create DataFrame
import pandas as pd
import numpy as np
technologies = {
    'Courses':["Spark","PySpark","Python","pandas","Python","Spark","pandas"],
    'Fee' :[20000,25000,22000,30000,22000,20000,30000],
    'Duration':['30days','40days','35days','50days','40days','30days','50days'],
    'Discount':[1000,2300,1200,2000,2300,1000,2000]
              }
df = pd.DataFrame(technologies)
print("Create DataFrame:\n", df)

Yields below output.

Pandas Get Unique Values in Column

Unique is also referred to as distinct, you can get unique values in the column using pandas Series.unique() function, since this function needs to call on the Series object, use df['column_name'] to get the unique values as a Series.

Syntax:

# Syntax of unique()
Series.unique(values)

Let’s see an example.

# Find unique values of a specified column
print("Get unique values from specified column:\n", df['Courses'].unique())

Yields Series object as output. This eliminates all duplicates and returns only unique values from the Courses column.

Find Unique Values in Multiple Columns

In case you want to get unique values on multiple columns of DataFrame use pandas.unique() function, using this you can also get unique values of a single column.

Syntax:

# Syntax
pandas.unique(values)

Let’s see an example. Since the unique() function takes values, you need to get the value of a column using df[columns_list].values.ravel().

# Using pandas.unique() to unique values in multiple columns
df2 = pd.unique(df[['Courses', 'Fee']].values.ravel())
print("Get unique values from multiple columns:\n",df2)

# Output:
# Get unique values from multiple columns
# ['Spark' 20000 'PySpark' 25000 'Python' 22000 'pandas' 30000]

If you want to get all unique values for one column and then the second column use the argument ‘K‘ to the ravel() function. The argument 'K' tells the method to flatten the array in the order of the elements. This can be significantly faster than using the method’s default ‘C‘ order.

# Using pandas.unique() to unique values in multiple columns
df2 = pd.unique(df[['Courses', 'Fee']].values.ravel('k'))
print("Get unique values from multiple columns:\n", df2)

# Output:
# Get unique values from multiple columns
# ['Spark' 'PySpark' 'Python' 'pandas' 20000 25000 22000 30000]

To get unique values of a single column.

# Using pandas.unique() to unique values
df2 = pd.unique(df[['Courses']].values.ravel())
print("Get unique values from specified column:\n",df2)

# Output:
# Get unique values from specified column:
# ['Spark' 'PySpark' 'Python' 'pandas']

Using Numpy.unique()

If you are using Numpy, use unique() method to eliminate duplicate values.

import numpy as np
# Find the unique values in multiple columns using numpy.unique()
df2 = np.unique(df[['Courses', 'Duration']].values)
print("Get unique values from specified columns:\n", df2)

# Use numpy.unique() to unique values in multiple columns 
column_values = df[['Courses', 'Duration']].values
df2 = np.unique(column_values)
print("Get unique values from multiple columns:\n", df2)

# Output:
# Get unique values from multiple columns:
# ['30days' '35days' '40days' '50days' 'PySpark' 'Python' 'Spark' 'pandas']

Using set() to Eliminate Duplicates

The set() function also removes all duplicate values and gets only unique values. We can use this set() function to get unique values from DataFrame single or multiple columns.

# Using Set() in pandas DataFrame
df2 = set(df.Courses.append(df.Fee).values)
print("Get unique values from multiple columns:\n", df2)

# Using set() method
df2 = set(df.Courses) | set(df.Fee)
print("Get unique values from multiple columns:\n", df2)

# Output:
# Get unique values from multiple columns:
# {20000, 25000, 'pandas', 30000, 22000, 'PySpark', 'Python', 'Spark'}

Using pandas.concat() and Unique() Methods

Using unique() and pandas.concat() combination to get unique values of multiple columns.

# Using pandas.concat to extend one column to multiple columns
df2 = pd.concat([df['Courses'],df['Duration'],df['Fee']]).unique()
print(f"Get Unique Values from three Columns: {df2}")

Yields below output.

# Output:
Get Unique Values from three Columns: ['Spark' 'PySpark' 'Python' 'pandas' '30days' '40days' '35days' '50days'
20000 25000 22000 30000]

Use Series.drop_duplicates()

Finally, you can get the unique values of a column using drop_duplicates() function of Series object. After dropping duplicates, it returns a Series object with unique values.

# Use Series.drop_duplicates() to get unique values
print("Get unique values from specified column:\n", df.Courses.drop_duplicates())

# Output:
Get unique values from specified column:
# 0      Spark
# 1    PySpark
# 2     Python
# 3     pandas
# Name: Courses, dtype: object

Complete Example of pandas Get Unique Values in Columns

import pandas as pd
import numpy as np
technologies = {
    'Courses':["Spark","PySpark","Python","pandas","Python","Spark","pandas"],
    'Fee' :[20000,25000,22000,30000,22000,20000,30000],
    'Duration':['30days','40days','35days','50days','40days','30days','50days'],
    'Discount':[1000,2300,1200,2000,2300,1000,2000]
              }
df = pd.DataFrame(technologies)
print(df)

# Find unique values of a column
print(df['Courses'].unique())
print(df.Courses.unique())

# Convert to List
print(df.Courses.unique().tolist())

# Unique values with drop_duplicates
df.Courses.drop_duplicates()
print(df)

# Using pandas.unique() to unique values in multiple columns
df2 = pd.unique(df[['Courses', 'Fee']].values.ravel('K'))
print(df2)

# Using pandas.unique() to unique values
df2 = pd.unique(df[['Courses']].values.ravel())
print(df2)

# Find the unique values in multiple columns using numpy.unique()
df2 = np.unique(df[['Courses', 'Duration']].values)
print(df2)

# Use numpy.unique() to unique values in multiple columns 
column_values = df[['Courses', 'Duration']].values
df2 = np.unique(column_values)
print(df2)

# Using Set() in pandas DataFrame
df2 = set(df.Courses.append(df.Fee).values)
print(df2)

# Using set() method
df2 = set(df.Courses) | set(df.Fee)
print(df2)

# To get unique values in one series/column
df2 = df['Courses'].unique()
print(df2)

# Using pandas.concat to extend one column to multiple columns
df2 = pd.concat([df['Courses'],df['Duration'],df['Fee']]).unique()
print(df2)

# Use Series.drop_duplicates() to get unique values
print(df.Courses.drop_duplicates())

Frequently Asked Questions of Get Unique Values From DataFrame

Conclusion

In this article, you have learned how to get unique values from single column/multiple columns in DataFrame using unique(), concat(), Series.unique(), and Numpy.unique() functions with examples.

Happy Learning !!

Related Articles

• Select Rows From List of Values in Pandas DataFrame
• Pandas Get List of All Duplicate Rows
• Delete Last Row From Pandas DataFrame
• Change the Order of Pandas DataFrame Columns
• Append a List as a Row to Pandas DataFrame
• Get First Row of Pandas DataFrame?
• Pandas Get Last Row from DataFrame?
• Pandas Get Row Number of DataFrame
• Get First N Rows of Pandas DataFrame

References

• https://pandas.pydata.org/docs/reference/api/pandas.unique.html