Site icon Spark By {Examples}

Pandas Convert Multiple Columns To DateTime Type

Naveen Nelamali
Pandas Convert Multiple DateTime

Let’s see different ways to convert multiple columns from string, integer, and object to datetime (date & time) type using pandas.to_datetime(), DataFrame.apply() & astype() functions. Usually, we get Data & time from the sources in different formats and in different data types, by using these functions you can convert them to a data time type datetime64[ns] of pandas.

In this article, I will explain how to change multiple columns to datetime type datetime64[ns] with examples.

1. Quick Examples of Convert Multiple Columns To DateTime

If you are in a hurry, below are some quick examples of how to convert multiple columns to DateTime in Pandas DataFrame.


# Below are quick examples

# Example 1: Use DataFrame.apply() to convert multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime)

# Example 2: Convert pandas multiple columns to Datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime, errors='coerce')

# Example 3: Convert multiple column to datetime using astype() method
df[['Inserted','Updated']] = df[['Inserted','Updated']].astype(datetime64[ns])

Now, let’s create a DataFrame with a few rows and columns, execute these examples, and validate the results. Our DataFrame contains column names Courses, Fee, Duration, Discount, Inserted and Updated.


# Create DataFrame
import pandas as pd
from datetime import datetime, timedelta
from pandas import DataFrame
df = DataFrame.from_dict(
      {'Courses':["Spark","Hadoop","pandas"],
     'Fee' :[20000,25000,30000],
     'Duration':['30days','40days','35days'],
     'Discount':[1000,2500,1500],
     'Inserted': ["10/02/2021, 10:39:24","09/12/2021, 08:09:24","01/22/2021, 10:29:14"],
     'Updated':  ["11/12/2021, 11:39:24","10/22/2021, 10:39:34","05/12/2021, 04:49:04"]},
    orient='index', 
    columns=['A', 'B', 'C']).T
print("Create DataFrame:\n", df)

Yields below output.

Pandas Convert Multiple DateTime

2. Use DataFrame.apply() to Convert String Multiple Columns to DateTime

You can use the DataFrame.apply() and pd.to_datetime() function to convert multiple columns to DataTime. apply() function applies function to each and every row and column of the DataFrame.


# Use DataFrame.apply() to convert multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime)
print(" After converting multiple columns to datetime:\n", df)

Yields below output.

Pandas Convert Multiple DateTime

Now let’s convert Inserted and Updated columns to DateTime type.


# Convert pandas multiple columns to Datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime, errors='coerce')
print(" After converting multiple columns to datetime:\n", df)

Yields the same output as above.

3. Convert Multiple Columns to DateTime Using astype() Method

Use series.astype() method to convert the multiple columns to date & time type. First, select all the columns you want to convert and use the astype() function with the type you want to convert as a param. This function is also used to convert data types (String to int e.t.c) in Pandas DataFrame 


# Convert multiple column to datetime using astype() method
df[['Inserted','Updated']] = df[['Inserted','Updated']].astype(datetime64[ns])
print(" After converting multiple columns to datetime:\n", df)

Yields the same output as above.

4. Complete Example – Convert Multiple Columns To DateTime


import pandas as pd
from datetime import datetime, timedelta
from pandas import DataFrame
df = DataFrame.from_dict(
      {'Courses':["Spark","Hadoop","pandas"],
     'Fee' :[20000,25000,30000],
     'Duration':['30days','40days','35days'],
     'Discount':[1000,2500,1500],
     'Inserted': ["10/02/2021, 10:39:24","09/12/2021, 08:09:24","01/22/2021, 10:29:14"],
     'Updated':  ["11/12/2021, 11:39:24","10/22/2021, 10:39:34","05/12/2021, 04:49:04"]},
    orient='index', 
    columns=['A', 'B', 'C']).T
print(df)

# Use DataFrame.apply() to convert multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime)
print(df)

# Convert pandas multiple columns to Datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime, errors='coerce')
print(df)

# Convert multiple column to datetime using astype() method
df[['Inserted','Updated']] = df[['Inserted','Updated']].astype(str)
print(df)

Conclusion

In this article, you have learned how to convert multiple columns of pandas DataFrame to DateTime using pandas.to_datetime(), astype() and DataFrame.apply() function. Using these you can convert data from string, object, and int columns to DateTime format.

Happy Learning !!

References

Exit mobile version