Pandas Convert Multiple Columns To DateTime Type

  • Post author:
  • Post category:Pandas
  • Post last modified:December 5, 2024
  • Reading time:13 mins read
You are currently viewing Pandas Convert Multiple Columns To DateTime Type

Let’s see different ways to convert multiple columns from string, integer, and object to DateTime (date & time) type using pandas.to_datetime(), DataFrame.apply() & astype() functions. Usually, we get Data & time from the sources in different formats and in different data types, by using these functions you can convert them to a data time type datetime64[ns] of pandas.

Advertisements

In this article, I will explain change multiple columns to datetime type datetime64[ns] with examples.

Key Points –

  • Use the pd.to_datetime() function to convert each selected column to DateTime type.
  • Specify the format parameter if the date format is non-standard.
  • Handle errors gracefully using the errors parameter to manage conversion issues.
  • Confirm successful conversion by checking the data types of the transformed columns.
  • Handle errors by setting the errors='coerce' parameter to convert invalid date entries to NaT.
  • Apply conversion to multiple columns using the apply() method on selected columns.

Quick Examples of Convert Multiple Columns To DateTime

Below are quick examples of converting multiple columns to DateTime in Pandas DataFrame.


# Quick examples of convert multiple columns to datetime

# Example 1: Use DataFrame.apply() 
# To convert multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime)

# Example 2: Convert pandas multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime, errors='coerce')

# Example 3: Convert multiple column to datetime 
# Using astype() method
df[['Inserted','Updated']] = df[['Inserted','Updated']].astype(datetime64[ns])

To run some examples of Pandas converting multiple columns to DateTime type, let’s create a Pandas DataFrame.


# Create DataFrame
import pandas as pd
from datetime import datetime, timedelta
from pandas import DataFrame
df = DataFrame.from_dict(
      {'Courses':["Spark","Hadoop","pandas"],
     'Fee' :[20000,25000,30000],
     'Duration':['30days','40days','35days'],
     'Discount':[1000,2500,1500],
     'Inserted': ["10/02/2021, 10:39:24","09/12/2021, 08:09:24","01/22/2021, 10:29:14"],
     'Updated':  ["11/12/2021, 11:39:24","10/22/2021, 10:39:34","05/12/2021, 04:49:04"]},
    orient='index', 
    columns=['A', 'B', 'C']).T
print("Create DataFrame:\n", df)

Yields below output.

Pandas Convert Multiple DateTime

Use DataFrame.apply() to Convert Multiple Columns to DateTime

You can use the DataFrame.apply() and pd.to_datetime() function to convert multiple columns to DataTime. apply() function applies function to each and every row and column of the DataFrame.


# Use DataFrame.apply() to convert multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime)
print(" After converting multiple columns to datetime:\n", df)

Yields below output.

Pandas Convert Multiple DateTime

Now let’s convert Inserted and Updated columns to DateTime type.


# Convert pandas multiple columns to Datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime, errors='coerce')
print(" After converting multiple columns to datetime:\n", df)

# Output:
# After converting multiple columns to datetime:
#    Courses    Fee Duration Discount            Inserted             Updated
# A   Spark  20000   30days     1000 2021-10-02 10:39:24 2021-11-12 11:39:24
# B  Hadoop  25000   40days     2500 2021-09-12 08:09:24 2021-10-22 10:39:34
# C  pandas  30000   35days     1500 2021-01-22 10:29:14 2021-05-12 04:49:04

Convert Multiple Columns to DateTime Using astype() Method

Use series.astype() function to convert the multiple columns to date & time type. First, select all the columns you want to convert and use the astype() function with the type you want to convert as a param. This function is also used to convert data types (String to int e.t.c) in Pandas DataFrame


# Convert multiple column to datetime using astype() method
df[['Inserted','Updated']] = df[['Inserted','Updated']].astype(datetime64[ns])
print(" After converting multiple columns to datetime:\n", df)

# Output:
# After converting multiple columns to datetime:
#    Courses    Fee Duration Discount            Inserted             Updated
# A   Spark  20000   30days     1000 2021-10-02 10:39:24 2021-11-12 11:39:24
# B  Hadoop  25000   40days     2500 2021-09-12 08:09:24 2021-10-22 10:39:34
# C  pandas  30000   35days     1500 2021-01-22 10:29:14 2021-05-12 04:49:04

Complete Example


import pandas as pd
from datetime import datetime, timedelta
from pandas import DataFrame
df = DataFrame.from_dict(
      {'Courses':["Spark","Hadoop","pandas"],
     'Fee' :[20000,25000,30000],
     'Duration':['30days','40days','35days'],
     'Discount':[1000,2500,1500],
     'Inserted': ["10/02/2021, 10:39:24","09/12/2021, 08:09:24","01/22/2021, 10:29:14"],
     'Updated':  ["11/12/2021, 11:39:24","10/22/2021, 10:39:34","05/12/2021, 04:49:04"]},
    orient='index', 
    columns=['A', 'B', 'C']).T
print(df)

# Use DataFrame.apply()
# To convert multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime)
print(df)

# Convert pandas multiple columns to Datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime, errors='coerce')
print(df)

# Convert multiple column to datetime 
# Using astype() method
df[['Inserted','Updated']] = df[['Inserted','Updated']].astype(str)
print(df)

FAQ on Pandas Convert Multiple Columns To DateTime Type

How can I convert multiple columns to datetime in a Pandas DataFrame?

You can convert multiple columns to datetime by using the pd.to_datetime() function with the apply() method.

What if the date columns are in different formats?

If the columns have different date formats, you can use the errors='coerce' argument in pd.to_datetime() to handle invalid parsing, and it will convert those entries to NaT (Not a Time) if parsing fails.

How can I convert columns with specific date formats?

If the date columns have a specific format (e.g., 'dd-mm-yyyy'), you can pass the format parameter in pd.to_datetime() to speed up the conversion.

Can I convert multiple columns in a loop?

You can loop through the columns and apply pd.to_datetime() individually.

How do I check if the conversion was successful?

To check if the conversion was successful, you can use the .dtypes attribute of the DataFrame. This will give you the data types of each column. If the conversion to datetime was successful, the corresponding columns will have the datetime64[ns] type.

Conclusion

In this article, you have learned to convert multiple columns of a Pandas DataFrame into DateTime format using pandas.to_datetime(), astype(), and DataFrame.apply() functions. These methods facilitate the conversion of data across string, object, and integer columns to DateTime format, enhancing data manipulation and analysis capabilities.

Happy Learning !!

References

Tags: , ,

Leave a Reply