Let’s see different ways to convert multiple columns from string, integer, and object to DateTime (date & time) type using pandas.to_datetime(), DataFrame.apply()
& astype() functions. Usually, we get Data & time from the sources in different formats and in different data types, by using these functions you can convert them to a data time type datetime64[ns]
of pandas.
In this article, I will explain change multiple columns to datetime type datetime64[ns] with examples.
Key Points –
- Use the
pd.to_datetime()
function to convert each selected column to DateTime type. - Specify the format parameter if the date format is non-standard.
- Handle errors gracefully using the errors parameter to manage conversion issues.
- Confirm successful conversion by checking the data types of the transformed columns.
- Handle errors by setting the
errors='coerce'
parameter to convert invalid date entries toNaT
. - Apply conversion to multiple columns using the
apply()
method on selected columns.
Quick Examples of Convert Multiple Columns To DateTime
Below are quick examples of converting multiple columns to DateTime in Pandas DataFrame.
# Quick examples of convert multiple columns to datetime
# Example 1: Use DataFrame.apply()
# To convert multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime)
# Example 2: Convert pandas multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime, errors='coerce')
# Example 3: Convert multiple column to datetime
# Using astype() method
df[['Inserted','Updated']] = df[['Inserted','Updated']].astype(datetime64[ns])
To run some examples of Pandas converting multiple columns to DateTime type, let’s create a Pandas DataFrame.
# Create DataFrame
import pandas as pd
from datetime import datetime, timedelta
from pandas import DataFrame
df = DataFrame.from_dict(
{'Courses':["Spark","Hadoop","pandas"],
'Fee' :[20000,25000,30000],
'Duration':['30days','40days','35days'],
'Discount':[1000,2500,1500],
'Inserted': ["10/02/2021, 10:39:24","09/12/2021, 08:09:24","01/22/2021, 10:29:14"],
'Updated': ["11/12/2021, 11:39:24","10/22/2021, 10:39:34","05/12/2021, 04:49:04"]},
orient='index',
columns=['A', 'B', 'C']).T
print("Create DataFrame:\n", df)
Yields below output.
Use DataFrame.apply() to Convert Multiple Columns to DateTime
You can use the DataFrame.apply()
and pd.to_datetime()
function to convert multiple columns to DataTime. apply() function applies function to each and every row and column of the DataFrame.
# Use DataFrame.apply() to convert multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime)
print(" After converting multiple columns to datetime:\n", df)
Yields below output.
Now let’s convert Inserted
and Updated
columns to DateTime type.
# Convert pandas multiple columns to Datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime, errors='coerce')
print(" After converting multiple columns to datetime:\n", df)
# Output:
# After converting multiple columns to datetime:
# Courses Fee Duration Discount Inserted Updated
# A Spark 20000 30days 1000 2021-10-02 10:39:24 2021-11-12 11:39:24
# B Hadoop 25000 40days 2500 2021-09-12 08:09:24 2021-10-22 10:39:34
# C pandas 30000 35days 1500 2021-01-22 10:29:14 2021-05-12 04:49:04
Convert Multiple Columns to DateTime Using astype() Method
Use series.astype()
function to convert the multiple columns to date & time type. First, select all the columns you want to convert and use the astype()
function with the type you want to convert as a param. This function is also used to convert data types (String to int e.t.c) in Pandas DataFrame
# Convert multiple column to datetime using astype() method
df[['Inserted','Updated']] = df[['Inserted','Updated']].astype(datetime64[ns])
print(" After converting multiple columns to datetime:\n", df)
# Output:
# After converting multiple columns to datetime:
# Courses Fee Duration Discount Inserted Updated
# A Spark 20000 30days 1000 2021-10-02 10:39:24 2021-11-12 11:39:24
# B Hadoop 25000 40days 2500 2021-09-12 08:09:24 2021-10-22 10:39:34
# C pandas 30000 35days 1500 2021-01-22 10:29:14 2021-05-12 04:49:04
Complete Example
import pandas as pd
from datetime import datetime, timedelta
from pandas import DataFrame
df = DataFrame.from_dict(
{'Courses':["Spark","Hadoop","pandas"],
'Fee' :[20000,25000,30000],
'Duration':['30days','40days','35days'],
'Discount':[1000,2500,1500],
'Inserted': ["10/02/2021, 10:39:24","09/12/2021, 08:09:24","01/22/2021, 10:29:14"],
'Updated': ["11/12/2021, 11:39:24","10/22/2021, 10:39:34","05/12/2021, 04:49:04"]},
orient='index',
columns=['A', 'B', 'C']).T
print(df)
# Use DataFrame.apply()
# To convert multiple columns to datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime)
print(df)
# Convert pandas multiple columns to Datetime
df[['Inserted','Updated']] = df[['Inserted','Updated']].apply(pd.to_datetime, errors='coerce')
print(df)
# Convert multiple column to datetime
# Using astype() method
df[['Inserted','Updated']] = df[['Inserted','Updated']].astype(str)
print(df)
FAQ on Pandas Convert Multiple Columns To DateTime Type
You can convert multiple columns to datetime
by using the pd.to_datetime()
function with the apply()
method.
If the columns have different date formats, you can use the errors='coerce'
argument in pd.to_datetime()
to handle invalid parsing, and it will convert those entries to NaT
(Not a Time) if parsing fails.
If the date columns have a specific format (e.g., 'dd-mm-yyyy'
), you can pass the format
parameter in pd.to_datetime()
to speed up the conversion.
You can loop through the columns and apply pd.to_datetime()
individually.
To check if the conversion was successful, you can use the .dtypes
attribute of the DataFrame. This will give you the data types of each column. If the conversion to datetime
was successful, the corresponding columns will have the datetime64[ns]
type.
Conclusion
In this article, you have learned to convert multiple columns of a Pandas DataFrame into DateTime format using pandas.to_datetime()
, astype()
, and DataFrame.apply()
functions. These methods facilitate the conversion of data across string, object, and integer columns to DateTime format, enhancing data manipulation and analysis capabilities.
Happy Learning !!
Related Articles
- Pandas Convert Column to Numpy Array
- Pandas DatetimeIndex Usage Explained
- Pandas Convert Dictionary to DataFrame
- Convert Pandas DatetimeIndex to String
- Pandas Convert Integer to Datetime Type
- Pandas Convert Datetime to Date Column
- Pandas Convert String Column To DateTime
- How to Print Pandas DataFrame without Index
- Check If a Column Exists in Pandas DataFrame
- Pandas Convert Boolean to String in DataFrame
- Pandas Convert Integer to String in DataFrame
- Pandas Convert Floats to Strings in DataFrame
- Convert Multiple Columns to String in Pandas DataFrame