Pandas apply() Function to Single & Multiple Column(s)

Using pandas.DataFrame.apply() method you can execute a function to a single column, all and multiple list of columns, in this article I will cover how to apply() a function on values of a selected single, multiple, all columns, For example, let’s say we have three columns and would like to apply a function on a single column without touching other two columns and return a DataFrame with three columns.

1. Quick Examples of Apply Function to a Single Column

If you are in a hurry, below are some of the quick examples of how to apply a function to a single column in pandas DataFrame.


# Below are quick examples
# Using Dataframe.apply() to apply function add column
def add_3(x):
   return x+3
df2 = df.apply(add_3)
print(df2)

# Using apply function single column
def add_4(x):
   return x+4
df["B"] = df["B"].apply(add_4)
print(df)

# apply a lambda function to each column
df2 = df.apply(lambda x : x + 10)
print(df2)

# Using Dataframe.apply() and lambda function
df["A"] = df["A"].apply(lambda x: x-2)
print(df)

# Using Dataframe.apply() & [] operator
df['A'] = df['A'].apply(np.square)
print(df)

# Using numpy.square() and [] operator
df['A'] = np.square(df['A'])
print(df)

# Apply function NumPy.square() to square the values of two rows 
#'A'and'B
df2 = df.apply(lambda x: np.square(x) if x.name in ['A','B'] else x)
print(df2)

# Apply function single column using transform() 
def add_2(x):
    return x+2
df = df.transform(add_2)
print(df)

# Using DataFrame.map() to Single Column
df['A'] = df['A'].map(lambda A: A/2.)
print(df)

# Using DataFrame.assign() and Lambda
df2 = df.assign(B=lambda df: df.B/2)
print(df2)

2. pandas.DataFrame.apply() Syntax

If you are a learner let’s see the syntax of apply() method and executing some examples of how to apply it on a single column, multiple, and all columns. Our DataFrame contains column names A, B, and C.

Below is a syntax of pandas.DataFrame.apply()


DataFrame.apply(func, axis=0, raw=False, result_type=None, args=(), **kwargs)

Let’s create a sample DataFrame to work with some examples.


import pandas as pd
import numpy as np
data = [(3,5,7), (2,4,6),(5,8,9)]
df = pd.DataFrame(data, columns = ['A','B','C'])
print(df)

Yields below output.


   A  B  C
0  3  5  7
1  2  4  6
2  5  8  9

3. Pandas Apply Function to Single Column

We will create a function add_3() which adds value 3 column value and use this on apply() function. To apply it to a single column, qualify the column name using df["col_name"]. The below example applies a function to a column B.


# Using apply function single column
def add_4(x):
   return x+4
df["B"] = df["B"].apply(add_4)
print(df)

Yields below output.


   A   B  C
0  3   9  7
1  2   8  6
2  5  12  9

4. Pandas Apply Function to All Columns

In some cases we would want to apply a function on all pandas columns, you can do this using apply() function. Here the add_3() function will be applied to all DataFrame columns.


# Using Dataframe.apply() to apply function add column
def add_3(x):
   return x+3
df2 = df.apply(add_3)
print(df2)

Yields below output.


   A   B   C
0  6   8  10
1  5   7   9
2  8  11  12

5. Pandas Apply Function to Multiple List of Columns

Similarly using apply() method, you can apply a function on a selected multiple list of columns. In this case, the function will be applied to only selected columns without touching the rest of the columns.


# apply() function on selected list of multiple columns
df = pd.DataFrame(data, columns = ['A','B','C'])
df[['A','B']] = df[['A','B']].apply(add_3)
print(df)

Yields below output


   A   B  C
0  6   8  7
1  5   7  6
2  8  11  9

6. Apply Lambda Function to Each Column

You can also apply a lambda function using apply() method, the Below example, adds 10 to all column values.


# apply a lambda function to each column
df2 = df.apply(lambda x : x + 10)
print(df2)

Yields below output.


    A   B   C
0  13  15  17
1  12  14  16
2  15  18  19

7. Apply Lambda Function to Single Column

You can apply the lambda function for a single column in the DataFrame. The following example subtracts every cell value by 2 for column A – df["A"]=df["A"].apply(lambda x:x-2).


# Using Dataframe.apply() and lambda function
df["A"] = df["A"].apply(lambda x: x-2)
print(df)

Yields below output.


   A  B  C
0  1  5  7
1  0  4  6
2  3  8  9

Similarly, you can also apply the Lambda function to all & multiple columns in pandas, I will leave this to you to explore.

8. Using pandas.DataFrame.transform() to Apply Function Column

Using DataFrame.apply() method & lambda functions the resultant DataFrame can be any number of columns whereas with transform() function the resulting DataFrame must have the same length as the input DataFrame.


# Using DataFrame.transform() 
def add_2(x):
    return x+2
df = df.transform(add_2)
print(df)

Yields below output.


   A   B   C
0  5   7   9
1  4   6   8
2  7  10  11

9. Using pandas.DataFrame.map() to Single Column

Here is another alternative using map() method.


# Using DataFrame.map() to Single Column
df['A'] = df['A'].map(lambda A: A/2.)
print(df)

Yields below output.


     A  B  C
0  1.5  5  7
1  1.0  4  6
2  2.5  8  9

10. pandas.DataFrame.assign() to Apply Lambda Function

You can also try assign()


# Using DataFrame.assign() and Lambda
df2 = df.assign(B=lambda df: df.B/2)
print(df2)

Yields below output.


   A    B  C
0  3  2.5  7
1  2  2.0  6
2  5  4.0  9

11. Using Numpy function on single Column

Use df['A']=df['A'].apply(np.square) to select the column from DataFrame as series using the [] operator and apply NumPy.square() method.


# Using Dataframe.apply() & [] operator
df['A'] = df['A'].apply(np.square)
print(df)

Yields below output.


    A  B  C
0   9  5  7
1   4  4  6
2  25  8  9

12. Using NumPy.square() Method

You can also do the same without using apply() function and directly using Numpy.

 
# Using numpy.square() and [] operator
df['A'] = np.square(df['A'])
print(df)
 

Yields same output as above.

13. Multiple columns Using NumPy.square() and Lambda Function

Apply a lambda function to multiple columns in DataFrame using Dataframe apply(), lambda, and Numpy functions.


# Apply function NumPy.square() to square the values of two rows 
'A'and'B
df2 = df.apply(lambda x: np.square(x) if x.name in ['A','B'] else x)
print(df2)

Yields below output.


    A   B  C
0   9  25  7
1   4  16  6
2  25  64  9

Happy Learning !!

Conclusion

In this article, you have learned how to apply a function to a single column, multiple and all columns of pandas DataFrame using apply(), transform() and NumPy.square(), map(), transform() and assign() methods.

Happy Learning !!

Also Read

References

NNK

SparkByExamples.com is a Big Data and Spark examples community page, all examples are simple and easy to understand and well tested in our development environment Read more ..

Leave a Reply