Pandas apply() Function to Single & Multiple Column(s)

Using pandas.DataFrame.apply() method you can execute a function to a single column, all and list of multiple columns (two or more). In this article, I will cover how to apply() a function on values of a selected single, multiple, all columns. For example, let’s say we have three columns and would like to apply a function on a single column without touching other two columns and return a DataFrame with three columns.

1. Quick Examples of pandas Apply Function to a Column

If you are in a hurry, below are some of the quick examples of how to apply a function to a single and multiple columns (two or more) in pandas DataFrame.


# Below are quick examples
# Using Dataframe.apply() to apply function add column
def add_3(x):
   return x+3
df2 = df.apply(add_3)

# Using apply function single column
def add_4(x):
   return x+4
df["B"] = df["B"].apply(add_4)

# Apply to multiple columns
df[['A','B']] = df[['A','B']].apply(add_3)

# apply a lambda function to each column
df2 = df.apply(lambda x : x + 10)

# Using Dataframe.apply() and lambda function
df["A"] = df["A"].apply(lambda x: x-2)

# Using Dataframe.apply() & [] operator
df['A'] = df['A'].apply(np.square)

# Using numpy.square() and [] operator
df['A'] = np.square(df['A'])

# Apply function NumPy.square() to square the values of two rows 
#'A'and'B
df2 = df.apply(lambda x: np.square(x) if x.name in ['A','B'] else x)

# Apply function single column using transform() 
def add_2(x):
    return x+2
df = df.transform(add_2)

# Using DataFrame.map() to Single Column
df['A'] = df['A'].map(lambda A: A/2.)

# Using DataFrame.assign() and Lambda
df2 = df.assign(B=lambda df: df.B/2)

2. pandas.DataFrame.apply() Function Syntax

If you are a learner let’s see the syntax of apply() method and executing some examples of how to apply it on a single column, multiple, and all columns. Our DataFrame contains column names A, B, and C.

Below is a syntax of pandas.DataFrame.apply()


DataFrame.apply(func, axis=0, raw=False, result_type=None, args=(), **kwargs)

Let’s create a sample DataFrame to work with some examples.


import pandas as pd
import numpy as np
data = [(3,5,7), (2,4,6),(5,8,9)]
df = pd.DataFrame(data, columns = ['A','B','C'])
print(df)

Yields below output.


   A  B  C
0  3  5  7
1  2  4  6
2  5  8  9

3. Pandas Apply Function to Single Column

We will create a function add_3() which adds value 3 column value and use this on apply() function. To apply it to a single column, qualify the column name using df["col_name"]. The below example applies a function to a column B.


# Using apply function single column
def add_4(x):
   return x+4
df["B"] = df["B"].apply(add_4)
print(df)

Yields below output. This applies the function to every row in DataFrame for a specified column.


   A   B  C
0  3   9  7
1  2   8  6
2  5  12  9

4. Pandas Apply Function to All Columns

In some cases we would want to apply a function on all pandas columns, you can do this using apply() function. Here the add_3() function will be applied to all DataFrame columns.


# Using Dataframe.apply() to apply function add column
def add_3(x):
   return x+3
df2 = df.apply(add_3)
print(df2)

Yields below output.


   A   B   C
0  6   8  10
1  5   7   9
2  8  11  12

5. Pandas Apply Function to Multiple List of Columns

Similarly using apply() method, you can apply a function on a selected multiple list of columns. In this case, the function will apply to only selected two columns without touching the rest of the columns.


# apply() function on selected list of multiple columns
df = pd.DataFrame(data, columns = ['A','B','C'])
df[['A','B']] = df[['A','B']].apply(add_3)
print(df)

Yields below output


   A   B  C
0  6   8  7
1  5   7  6
2  8  11  9

6. Apply Lambda Function to Each Column

You can also apply a lambda expression using the apply() method, the Below example, adds 10 to all column values.


# apply a lambda function to each column
df2 = df.apply(lambda x : x + 10)
print(df2)

Yields below output.


    A   B   C
0  13  15  17
1  12  14  16
2  15  18  19

7. Apply Lambda Function to Single Column

You can apply the lambda function for a single column in the DataFrame. The following example subtracts every cell value by 2 for column A – df["A"]=df["A"].apply(lambda x:x-2).


# Using Dataframe.apply() and lambda function
df["A"] = df["A"].apply(lambda x: x-2)
print(df)

Yields below output.


   A  B  C
0  1  5  7
1  0  4  6
2  3  8  9

Similarly, you can also apply the Lambda function to all & multiple columns in pandas, I will leave this to you to explore.

8. Using pandas.DataFrame.transform() to Apply Function Column

Using DataFrame.apply() method & lambda functions the resultant DataFrame can be any number of columns whereas with transform() function the resulting DataFrame must have the same length as the input DataFrame.


# Using DataFrame.transform() 
def add_2(x):
    return x+2
df = df.transform(add_2)
print(df)

Yields below output.


   A   B   C
0  5   7   9
1  4   6   8
2  7  10  11

9. Using pandas.DataFrame.map() to Single Column

Here is another alternative using map() method.


# Using DataFrame.map() to Single Column
df['A'] = df['A'].map(lambda A: A/2.)
print(df)

Yields below output.


     A  B  C
0  1.5  5  7
1  1.0  4  6
2  2.5  8  9

10. DataFrame.assign() to Apply Lambda Function

You can also try assign()


# Using DataFrame.assign() and Lambda
df2 = df.assign(B=lambda df: df.B/2)
print(df2)

Yields below output.


   A    B  C
0  3  2.5  7
1  2  2.0  6
2  5  4.0  9

11. Using Numpy function on single Column

Use df['A']=df['A'].apply(np.square) to select the column from DataFrame as series using the [] operator and apply NumPy.square() method.


# Using Dataframe.apply() & [] operator
df['A'] = df['A'].apply(np.square)
print(df)

Yields below output.


    A  B  C
0   9  5  7
1   4  4  6
2  25  8  9

12. Using NumPy.square() Method

You can also do the same without using apply() function and directly using Numpy.

 
# Using numpy.square() and [] operator
df['A'] = np.square(df['A'])
print(df)
 

Yields same output as above.

13. Multiple columns Using NumPy.square() and Lambda Function

Apply a lambda function to multiple columns in DataFrame using Dataframe apply(), lambda, and Numpy functions.


# Apply function NumPy.square() to square the values of two rows 
'A'and'B
df2 = df.apply(lambda x: np.square(x) if x.name in ['A','B'] else x)
print(df2)

Yields below output.


    A   B  C
0   9  25  7
1   4  16  6
2  25  64  9

Conclusion

In this article, you have learned how to apply a function to a single column, all and multiple columns (two or more) of pandas DataFrame using apply(), transform() and NumPy.square(), map(), transform() and assign() methods.

Happy Learning !!

Also Read

References

NNK

SparkByExamples.com is a Big Data and Spark examples community page, all examples are simple and easy to understand and well tested in our development environment Read more ..

Leave a Reply

You are currently viewing Pandas apply() Function to Single & Multiple Column(s)