Explain how to format DateTime in Pandas DataFrame. Pandas Library is best known for supporting a diverse type of data which also includes date and time. The Python Pandas Library has extensive support for DateTime datatype and provides core DateTime computation functionalities out of the box.
In some cases, the dates and times are not in the Pandas DateTime format that you wish, and you want to store, print, or visualize it in a different format inside a DataFrame. So here is a complete list of possibilities that you can do with a datetime datatype in your DataFrame.
Table of contents
Let’s say you want to store the date of birth of a particular person, we will take the date of birth of John (13/04/1999) for the sake of example, we store this date of birth in a DataFrame with type object. See the following code.
# Create Format Pandas Datetime
import pandas as pd
df = pd.DataFrame({'DOB': {0: '11/1/1999 ', 1: '22/12/2001 '},
'Name':{0:"Ali",1:"Zahid"}})
print(df.dtypes)
# Output:
# DOB object
# Name object
# dtype: object
Here, the date of birth (DOB) type is an string object, but we don’t want it to be that way, instead, we want to convert it to a DateTime object. In this case, we need to use pandas.to_datetime() function.
1. Pandas DateTime Format with pandas.to_datetime()
With pandas built-in function pd.to_datetime(), we can convert the date time string object into a datetime64 object. It will automatically change the default format of the string date time into the default format of datetime64.
# Pandas DateTime Format with pandas.to_datetime()
import pandas as pd
df = pd.DataFrame({'DOB': {0: '11/1/1999 ', 1: '22/12/2001 '},
'Name':{0:"Ali",1:"Zahid"}})
# converting date to dtype datetime64
pd.to_datetime(df.DOB)
# Output:
# 0 1999-11-01
# 1 2001-12-22
# Name: DOB, dtype: datetime64[ns]
The following picture shows how the pandas.to_datetime() change the format of DateTime in pandas.
2. Pandas Format DateTime from YYYY-MM-DD to DD-MM-YYYY
In the above example, you have seen how we can change the default format string into DateTime format. The default DateTime format for the datetime64 will be YYYY-MM-DD. In most cases, the attribute dayfirst attribute of to_datetime() will work but dayfirst=True
is not strict, but will prefer to parse with the day first.
To change the datetime format from YYYY-MM-DD to DD-MM-YYYY use the dt.strftime(‘%d-%m-%Y’) function. See the following Example.
# Pandas Format DateTime from YYYY-MM-DD to DD-MM-YYYY
import pandas as pd
df = pd.DataFrame({'DOB': {0: '11/1/1999 ', 1: '22/12/2001 '},
'Name':{0:"Ali",1:"Zahid"}})
df['DOB'] = pd.to_datetime(df.DOB)
print(df['DOB'])
new_style=df['DOB'].dt.strftime('%d-%m-%Y')
print(new_style)
# Output:
# 0 1999-11-01
# 1 2001-12-22
# Name: DOB, dtype: datetime64[ns]
# 0 01-11-1999
# 1 22-12-2001
# Name: DOB, dtype: object
Look at the following picture of how the program changes the datetime format in the pandas dataframe. Remember doing so we will lose the data type of the datetime64.
3. Pandas DateTime format with df.style.format
In the above example, the datatype of the DateTime was changed using the strftime() function. There are two ways of formatting DateTime.
- Changing the internal representation
- Changing only the format for displaying
The above method we used was changing the internal representation of the DateTime. With using df.style.format we will only change the style for the printing and displaying but the internal structure will be still the same.
This example will only work on Jupyter notebook and not elsewhere because it is related to the style of the dataframe, not the dataframe itself.
# Pandas DateTime format with df.style.format
import pandas as pd
df = pd.DataFrame({'DOB': {0: '11/1/1999 ', 1: '22/12/2001 '},
'Name':{0:"Ali",1:"Zahid"}})
df['DOB'] = pd.to_datetime(df.DOB)
print(df.dtypes)
df.style.format({"DOB": lambda t: t.strftime("%d-%m-%Y")})
# Output:
# DOB datetime64[ns]
# Name object
# dtype: object
# DOB Name
# 0 01-11-1999 Ali
# 1 22-12-2001 Zahid
In the above example, you can see the style of the date change while the dtype is still the same. This is just another way of formatting datetime in a DataFrame.
4. Change the format of time in pandas datetime
Until now, we have seen how to deal with the format of date in datetime in pandas. We can also change the format of time in pandas datetime. Let’s create a sample DataFrame first for dealing with the format of pandas datetime.
# Change the format of time in pandas datetime
import pandas as pd
dt_range= pd.date_range('2022-07-01', periods=5, freq='H')
df = pd.DataFrame({'DateWtime' : dt_range})
print(df)
# Output:
# DateWtime
# 0 2022-07-01 00:00:00
# 1 2022-07-01 01:00:00
# 2 2022-07-01 02:00:00
# 3 2022-07-01 03:00:00
# 4 2022-07-01 04:00:00
4.1 Change time format from HH:MM:SS to SS:MM:HH in Pandas
The default format of the datetime which is generated using the pd.date_range() function is HH:MM:SS. To change the time format use the dt.strftime() function.
See the following example to change datetime format in pandas.
# Change time format from HH:MM:SS to SS:MM:HH in Pandas
df['DateTime'] = df.Date.dt.strftime('%Y-%m-%d %S:%M:%H')
print(df['DateTime']
# Output:
# 0 2022-07-01 00:00:00
# 1 2022-07-01 00:00:01
# 2 2022-07-01 00:00:02
# 3 2022-07-01 00:00:03
# 4 2022-07-01 00:00:04
# Name: DateTime, dtype: object
5. Month to String Format in Pandas Datetime
Let’s say you want to make your datetime format more readable for the readers. In this case, you might want to change the date time format to a string name corresponding to that month. Let’s say for the 1st month you want to convert it to January and for the 12th month to be December.
See the following example of how we have changed the format of the month to a string name.
# Month to String Format in Pandas Datetime
import pandas as pd
dt_range= pd.date_range('2022-07-01', periods=5, freq='M')
df = pd.DataFrame({'Date' : dt_range})
new_df=df.Date.dt.strftime('%Y-%b-%d %S:%M:%H')
print(new_df)
# Output:
# 0 2022-Jul-31 00:00:00
# 1 2022-Aug-31 00:00:00
# 2 2022-Sep-30 00:00:00
# 3 2022-Oct-31 00:00:00
# 4 2022-Nov-30 00:00:00
# Name: Date, dtype: object
The following picture shows the before and after the status of the datetime.
6. Conclusion
In this article, you have learned the different ways to format pandas datetime. If you still face any problems please let us know in the comments.
Related Articles
- Select Pandas DataFrame Rows Between Two Dates
- Pandas DatetimeIndex Usage Explained
- Convert Pandas DatetimeIndex to String
- pandas Convert Datetime to Seconds
- Sort Pandas DataFrame by Date (Datetime)
- Pandas Extract Year from Datetime
- Pandas Get Day, Month and Year from DateTime
- Pandas Extract Month and Year from Datetime
- Pandas Convert Integer to Datetime Type
- Pandas Convert Datetime to Date Column
- Pandas Convert Date (datetime) to String Format
- Pandas Convert Multiple Columns To DateTime Type
- Pandas Convert Column To DateTime