Pandas Convert Integer to Datetime Type

  • Post author:
  • Post category:Pandas
  • Post last modified:September 20, 2024
  • Reading time:13 mins read
You are currently viewing Pandas Convert Integer to Datetime Type

In Pandas, you can convert an integer column representing dates or times into a DateTime type using the pd.to_datetime() function. To convert integer to DateTime type datetime64[ns] format by using pandas.to_datetime() and astype() methods. astype() method is used to cast from one type to another.

Advertisements

In these pandas DataFrame article, I will explain converting integer holding date & time to datetime format using the above-mentioned methods and also using DataFrame.apply() with lambda function.

Key Points –

  • Ensure the integer values represent valid date formats before conversion.
  • Use the format parameter in pd.to_datetime() to specify the date format explicitly for accurate conversion.
  • Be aware of different date formats, such as YYYYMMDD, YYMMDD, and others, to choose the appropriate conversion method.
  • The apply() method allows for more complex transformations, particularly useful for custom formats or additional processing.
  • Vectorized operations like pd.to_datetime() are generally faster than using apply() with a lambda function.
  • For large datasets, consider using optimized methods to enhance performance and reduce memory usage.

Quick Examples of Converting Integer to Datetime Format

Following are quick examples of how to convert integer column type to datetime format.


# Quick examples of convert integer to datetime format

# Example 1: Convert integers to datetime format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%Y%m%d')

# Example 2: Use pandas.to_datetime() and DataFrame.apply() with lambda function
df['InsertedDate'] = df['InsertedDate'].apply(lambda x: pd.to_datetime(str(x),format='%Y%m%d'))

# Example 3: Use series.astype() method 
# To convert integers to datetime format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'].astype(str), format='%Y%m%d')

# Example 4: Use pandas.to_datetime() 
# To convert integers to "yymmdd" format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%y%m%d')

# Example 5: Changing integer values to dates and time format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%Y%m%d%H%M%S')

Let’s generate a Pandas DataFrame to demonstrate several examples of converting integers into DateTime type within the context of Pandas.


# Create DataFrame
import pandas as pd
technologies = ({
    'Courses':["Spark","PySpark","Hadoop"],
    'Fee' :[22000.0,25000.0,23000.0],
    'InsertedDate':[20201124,20210210,20211215]
               })
df = pd.DataFrame(technologies)
print("Create Dataframe:\n", df)
print("----------------------------------")
print("Type of the columns:\n", df.dtypes)

Yields below output.

pandas convert integer datetime

As you see above, you can get the data types of all columns using df.dtypes. You can also get the same using df.infer_objects().dtypes.

Convert Integer to Datetime Format

In the below example, note that the data type for the ‘InsertedDate’ column is Integer. To convert it into Datetime, I use pandas.to_datetime(). This method takes a parm format to specify the format of the date you wanted to convert from. Here, the InsertedDate column has the date in the format yyyymmdd, 


# Convert integers to datetime format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%Y%m%d')
print("After converting integer to datetime:\n", df)
print("----------------------------------")
print("Type of the columns:\n", df.dtypes)

Yields below output.

pandas convert integer datetime

Use Series.apply() with Lambda Function

You can also use pandas.to_datetime() and DataFrame.apply() with lambda function to convert integer to datetime.


# Use pandas.to_datetime() and DataFrame.apply() with lambda function
df['InsertedDate'] = df['InsertedDate'].apply(lambda x: pd.to_datetime(str(x),format='%Y%m%d'))
print("After converting integer to datetime:\n", df)
print("----------------------------------")
print("Type of the columns:\n", df.dtypes)

Use DataFrame.astype() to Convert Integer to Datetime Format

DataFrame.astype() method is also used to convert integers to datetime format. The dtype of this datetime column would be datetime64[ns].


# Use series.astype() method to convert integers to datetime format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'].astype(str), format='%Y%m%d')
print("After converting integer to datetime:\n", df)
print("----------------------------------")
print("Type of the columns:\n", df.dtypes)

Use pandas.to_datetime() to Convert Integer to Date & Time Format

Let’s suppose that your integers contain both the date and time. In that case, the format should be specified is '%Y%m%d%H%M%S'.


# Use pandas.to_datetime() to Convert Integer to Date & Time Format
import pandas as pd
technologies = ({
    'Courses':["Spark","PySpark","Hadoop"],
    'Fee' :[22000.0,25000.0,23000.0],
    'InsertedDate':[20201124063015,20210210084021,20211215032511]
               })
df = pd.DataFrame(technologies)

# changing integer values to dates and time format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%Y%m%d%H%M%S')
print("After converting integer to datetime:\n", df)
print("----------------------------------")
print("Type of the columns:\n", df.dtypes)

Yields below output.


# Output:
# After converting integer to datetime:
   Courses      Fee        InsertedDate
0    Spark  22000.0 2020-11-24 06:30:15
1  PySpark  25000.0 2021-02-10 08:40:21
2   Hadoop  23000.0 2021-12-15 03:25:11
------------------------------------------
Type of the columns:
Courses                 object
Fee                    float64
InsertedDate    datetime64[ns]
dtype: object

Complete Example For Convert Integers to Datetime Format


# Example For Convert Integers to Datetime Format
import pandas as pd
technologies = ({
    'Courses':["Spark","PySpark","Hadoop"],
    'Fee' :[22000.0,25000.0,23000.0],
    'InsertedDate':[20201124,20210210,20211215]
               })
df = pd.DataFrame(technologies)
print(df)
print(df.dtypes)

# Convert integers to datetime format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%Y%m%d')
print(df)
print(df.dtypes)

# Use pandas.to_datetime() and DataFrame.apply() with lambda function
df['InsertedDate'] = df['InsertedDate'].apply(lambda x: pd.to_datetime(str(x),format='%Y%m%d'))
print(df)
print(df.dtypes)

# Use series.astype() method 
# To convert integers to datetime format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'].astype(str), format='%Y%m%d')
print(df)
print(df.dtypes)

# Use pandas.to_datetime() 
# To convert integers to "yymmdd" format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%y%m%d')
print(df)
print(df.dtypes)

# Changing integer values to dates and time format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%Y%m%d%H%M%S')
print(df)
print(df.dtypes)

FAQ on Pandas Convert Integer to Datetime Type

What types of integer formats can be converted to datetime?

Common formats include YYYYMMDD, YYMMDD, and YYYYMMDDHHMMSS. Ensure the integer values align with a valid date representation.

Is it faster to use apply() with a lambda function or pd.to_datetime() directly?

pd.to_datetime() is generally faster since it is vectorized. Use apply() for more complex transformations that require custom logic.

Can I convert a Series of integers in one go?

You can pass the entire Series to pd.to_datetime() for bulk conversion, ensuring the correct format is specified.

How can I convert back to integer format if needed?

You can use astype(int) or strftime() to format the datetime back into an integer representation, but be cautious about the resulting format.

How can I check if the conversion was successful?

After conversion, you can check for NaT values in the resulting datetime column to identify any failed conversions.

Conclusion

In conclusion, we have explored the conversion of integers to DateTime format using various methods in pandas, such as pandas.to_datetime(), astype(), and apply() with lambda functions. Through examples, we’ve illustrated how these approaches can be effectively implemented.

Happy Learning !!

Tags:

Leave a Reply