Pandas Convert Multiple Columns To DateTime Type

  • Post author:
  • Post category:Pandas
  • Post last modified:May 30, 2024
  • Reading time:8 mins read
You are currently viewing Pandas Convert Multiple Columns To DateTime Type

Let’s see different ways to convert multiple columns from string, integer, and object to DateTime (date & time) type using pandas.to_datetime(), DataFrame.apply() & astype() functions. Usually, we get Data & time from the sources in different formats and in different data types, by using these functions you can convert them to a data time type datetime64[ns] of pandas.

Advertisements

In this article, I will explain change multiple columns to datetime type datetime64[ns] with examples.

Key Points –

  • Use the pd.to_datetime() function to convert each selected column to DateTime type.
  • Specify the format parameter if the date format is non-standard.
  • Handle errors gracefully using the errors parameter to manage conversion issues.
  • Confirm successful conversion by checking the data types of the transformed columns.

Quick Examples of Convert Multiple Columns To DateTime

Below are quick examples of converting multiple columns to DateTime in Pandas DataFrame.


# Quick examples of convert multiple columns to datetime

# Example 1: Use DataFrame.apply() 
# To convert multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime)

# Example 2: Convert pandas multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime, errors='coerce')

# Example 3: Convert multiple column to datetime 
# Using astype() method
df[['Inserted','Updated']] = df[['Inserted','Updated']].astype(datetime64[ns])

To run some examples of Pandas converting multiple columns to DateTime type, let’s create a Pandas DataFrame.


# Create DataFrame
import pandas as pd
from datetime import datetime, timedelta
from pandas import DataFrame
df = DataFrame.from_dict(
      {'Courses':["Spark","Hadoop","pandas"],
     'Fee' :[20000,25000,30000],
     'Duration':['30days','40days','35days'],
     'Discount':[1000,2500,1500],
     'Inserted': ["10/02/2021, 10:39:24","09/12/2021, 08:09:24","01/22/2021, 10:29:14"],
     'Updated':  ["11/12/2021, 11:39:24","10/22/2021, 10:39:34","05/12/2021, 04:49:04"]},
    orient='index', 
    columns=['A', 'B', 'C']).T
print("Create DataFrame:\n", df)

Yields below output.

Pandas Convert Multiple DateTime

Use DataFrame.apply() to Convert Multiple Columns to DateTime

You can use the DataFrame.apply() and pd.to_datetime() function to convert multiple columns to DataTime. apply() function applies function to each and every row and column of the DataFrame.


# Use DataFrame.apply() to convert multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime)
print(" After converting multiple columns to datetime:\n", df)

Yields below output.

Pandas Convert Multiple DateTime

Now let’s convert Inserted and Updated columns to DateTime type.


# Convert pandas multiple columns to Datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime, errors='coerce')
print(" After converting multiple columns to datetime:\n", df)

# Output:
# After converting multiple columns to datetime:
#    Courses    Fee Duration Discount            Inserted             Updated
# A   Spark  20000   30days     1000 2021-10-02 10:39:24 2021-11-12 11:39:24
# B  Hadoop  25000   40days     2500 2021-09-12 08:09:24 2021-10-22 10:39:34
# C  pandas  30000   35days     1500 2021-01-22 10:29:14 2021-05-12 04:49:04

Convert Multiple Columns to DateTime Using astype() Method

Use series.astype() function to convert the multiple columns to date & time type. First, select all the columns you want to convert and use the astype() function with the type you want to convert as a param. This function is also used to convert data types (String to int e.t.c) in Pandas DataFrame


# Convert multiple column to datetime using astype() method
df[['Inserted','Updated']] = df[['Inserted','Updated']].astype(datetime64[ns])
print(" After converting multiple columns to datetime:\n", df)

# Output:
# After converting multiple columns to datetime:
#    Courses    Fee Duration Discount            Inserted             Updated
# A   Spark  20000   30days     1000 2021-10-02 10:39:24 2021-11-12 11:39:24
# B  Hadoop  25000   40days     2500 2021-09-12 08:09:24 2021-10-22 10:39:34
# C  pandas  30000   35days     1500 2021-01-22 10:29:14 2021-05-12 04:49:04

Complete Example


import pandas as pd
from datetime import datetime, timedelta
from pandas import DataFrame
df = DataFrame.from_dict(
      {'Courses':["Spark","Hadoop","pandas"],
     'Fee' :[20000,25000,30000],
     'Duration':['30days','40days','35days'],
     'Discount':[1000,2500,1500],
     'Inserted': ["10/02/2021, 10:39:24","09/12/2021, 08:09:24","01/22/2021, 10:29:14"],
     'Updated':  ["11/12/2021, 11:39:24","10/22/2021, 10:39:34","05/12/2021, 04:49:04"]},
    orient='index', 
    columns=['A', 'B', 'C']).T
print(df)

# Use DataFrame.apply()
# To convert multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime)
print(df)

# Convert pandas multiple columns to Datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime, errors='coerce')
print(df)

# Convert multiple column to datetime 
# Using astype() method
df[['Inserted','Updated']] = df[['Inserted','Updated']].astype(str)
print(df)

Conclusion

In this article, you have learned to convert multiple columns of a Pandas DataFrame into DateTime format using pandas.to_datetime(), astype(), and DataFrame.apply() functions. These methods facilitate the conversion of data across string, object, and integer columns to DateTime format, enhancing data manipulation and analysis capabilities.

Happy Learning !!

References

Tags: , ,

Leave a Reply