Pandas Convert Multiple Columns To DateTime Type

  • Post author:
  • Post category:Pandas
  • Post last modified:January 9, 2024
  • Reading time:9 mins read

Let’s see different ways to convert multiple columns from string, integer, and object to datetime (date & time) type using pandas.to_datetime(), DataFrame.apply() & astype() functions. Usually, we get Data & time from the sources in different formats and in different data types, by using these functions you can convert them to a data time type datetime64[ns] of pandas.

In this article, I will explain how to change multiple columns to datetime type datetime64[ns] with examples.

1. Quick Examples of Convert Multiple Columns To DateTime

If you are in a hurry, below are some quick examples of how to convert multiple columns to DateTime in Pandas DataFrame.


# Below are quick examples

# Example 1: Use DataFrame.apply() to convert multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime)

# Example 2: Convert pandas multiple columns to Datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime, errors='coerce')

# Example 3: Convert multiple column to datetime using astype() method
df[['Inserted','Updated']] = df[['Inserted','Updated']].astype(datetime64[ns])

Now, let’s create a DataFrame with a few rows and columns, execute these examples, and validate the results. Our DataFrame contains column names CoursesFeeDurationDiscount, Inserted and Updated.


# Create DataFrame
import pandas as pd
from datetime import datetime, timedelta
from pandas import DataFrame
df = DataFrame.from_dict(
      {'Courses':["Spark","Hadoop","pandas"],
     'Fee' :[20000,25000,30000],
     'Duration':['30days','40days','35days'],
     'Discount':[1000,2500,1500],
     'Inserted': ["10/02/2021, 10:39:24","09/12/2021, 08:09:24","01/22/2021, 10:29:14"],
     'Updated':  ["11/12/2021, 11:39:24","10/22/2021, 10:39:34","05/12/2021, 04:49:04"]},
    orient='index', 
    columns=['A', 'B', 'C']).T
print("Create DataFrame:\n", df)

Yields below output.

Pandas Convert Multiple DateTime

2. Use DataFrame.apply() to Convert String Multiple Columns to DateTime

You can use the DataFrame.apply() and pd.to_datetime() function to convert multiple columns to DataTime. apply() function applies function to each and every row and column of the DataFrame.


# Use DataFrame.apply() to convert multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime)
print(" After converting multiple columns to datetime:\n", df)

Yields below output.

Pandas Convert Multiple DateTime

Now let’s convert Inserted and Updated columns to DateTime type.


# Convert pandas multiple columns to Datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime, errors='coerce')
print(" After converting multiple columns to datetime:\n", df)

Yields the same output as above.

3. Convert Multiple Columns to DateTime Using astype() Method

Use series.astype() method to convert the multiple columns to date & time type. First, select all the columns you want to convert and use the astype() function with the type you want to convert as a param. This function is also used to convert data types (String to int e.t.c) in Pandas DataFrame


# Convert multiple column to datetime using astype() method
df[['Inserted','Updated']] = df[['Inserted','Updated']].astype(datetime64[ns])
print(" After converting multiple columns to datetime:\n", df)

Yields the same output as above.

4. Complete Example – Convert Multiple Columns To DateTime


import pandas as pd
from datetime import datetime, timedelta
from pandas import DataFrame
df = DataFrame.from_dict(
      {'Courses':["Spark","Hadoop","pandas"],
     'Fee' :[20000,25000,30000],
     'Duration':['30days','40days','35days'],
     'Discount':[1000,2500,1500],
     'Inserted': ["10/02/2021, 10:39:24","09/12/2021, 08:09:24","01/22/2021, 10:29:14"],
     'Updated':  ["11/12/2021, 11:39:24","10/22/2021, 10:39:34","05/12/2021, 04:49:04"]},
    orient='index', 
    columns=['A', 'B', 'C']).T
print(df)

# Use DataFrame.apply() to convert multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime)
print(df)

# Convert pandas multiple columns to Datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime, errors='coerce')
print(df)

# Convert multiple column to datetime using astype() method
df[['Inserted','Updated']] = df[['Inserted','Updated']].astype(str)
print(df)

Conclusion

In this article, you have learned how to convert multiple columns of pandas DataFrame to DateTime using pandas.to_datetime(), astype() and DataFrame.apply() function. Using these you can convert data from string, object, and int columns to DateTime format.

Happy Learning !!

References

Naveen (NNK)

Naveen (NNK) is a Data Engineer with 20+ years of experience in transforming data into actionable insights. Over the years, He has honed his expertise in designing, implementing, and maintaining data pipelines with frameworks like Apache Spark, PySpark, Pandas, R, Hive and Machine Learning. Naveen journey in the field of data engineering has been a continuous learning, innovation, and a strong commitment to data integrity. In this blog, he shares his experiences with the data as he come across. Follow Naveen @ @ LinkedIn

Leave a Reply