Pandas Convert Integer to Datetime Type

  • Post author:
  • Post category:Pandas
  • Post last modified:October 5, 2023

Pandas Convert DataFrame Column Type from Integer to datetime type datetime64[ns] format – You can convert the pandas DataFrame column type from integer to datetime format by using pandas.to_datetime() and DataFrame.astype() method. astype() method is used to cast from one type to another

In these pandas DataFrame article, I will explain how to convert integer holding date & time to datetime format using above mentioned methods and also using DataFrame.apply() with lambda function.

1. Quick Examples of Convert Integer to Datetime Format

If you are in a hurry, below are some quick examples of how to convert integer column type to datetime in pandas DataFrame.


# Below are some quick examples

# Convert integers to datetime format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%Y%m%d')

# Use pandas.to_datetime() and DataFrame.apply() with lambda function
df['InsertedDate'] = df['InsertedDate'].apply(lambda x: pd.to_datetime(str(x),format='%Y%m%d'))

# Use series.astype() method to convert integers to datetime format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'].astype(str), format='%Y%m%d')

# Use pandas.to_datetime() to convert integers to "yymmdd" format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%y%m%d')

# Changing integer values to dates and time format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%Y%m%d%H%M%S')

Now, let’s create a DataFrame with a few rows and columns, execute these examples and validate results. Our DataFrame contains column names Courses, Fee and InsertedDate.


# Create DataFrame
import pandas as pd
technologies = ({
    'Courses':["Spark","PySpark","Hadoop"],
    'Fee' :[22000.0,25000.0,23000.0],
    'InsertedDate':[20201124,20210210,20211215]
               })
df = pd.DataFrame(technologies)
print(df)
print(df.dtypes)

Yields below output.


# Output:
   Courses      Fee  InsertedDate
0    Spark  22000.0      20201124
1  PySpark  25000.0      20210210
2   Hadoop  23000.0      20211215
Courses          object
Fee             float64
InsertedDate      int64
dtype: object

As you see above, you can get the data types of all columns using df.dtypes. You can also get the same using df.infer_objects().dtypes.

2. Convert Integer to Datetime Format

In the below example, note that the data type for the ‘InsertedDate’ column is Integer. To convert it into Datetime, I use pandas.to_datetime(). This method takes a parm format to specify the format of the date you wanted to convert from. Here, the InsertedDate column has date in format yyyymmdd, 


# Convert integers to datetime format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%Y%m%d')
print(df)
print(df.dtypes)

Yields below output.


# Output:
   Courses      Fee InsertedDate
0    Spark  22000.0   2020-11-24
1  PySpark  25000.0   2021-02-10
2   Hadoop  23000.0   2021-12-15
Courses                 object
Fee                    float64
InsertedDate    datetime64[ns]
dtype: object

3. Use Series.apply() with Lambda Function

You can also use pandas.to_datetime() and DataFrame.apply() with lambda function to convert integer to datetime.


# Use pandas.to_datetime() and DataFrame.apply() with lambda function
df['InsertedDate'] = df['InsertedDate'].apply(lambda x: pd.to_datetime(str(x),format='%Y%m%d'))
print(df)
print(df.dtypes)

Yields same output as above.

4. Use DataFrame.astype() Method to Convert Integer to Datetime Format

DataFrame.astype() method is also used to convert integer to datetime formate. dtype of this date time coulumn would be datetime64[ns].


# Use series.astype() method to convert integers to datetime format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'].astype(str), format='%Y%m%d')
print(df)
print(df.dtypes)

Yields same output as above.

5. Use pandas.to_datetime() to Convert Integer to Date & Time Format

Let’s suppose that your integers contain both the date and time. In that case, the format should be specify is '%Y%m%d%H%M%S'.


# Use pandas.to_datetime() to Convert Integer to Date & Time Format
import pandas as pd
technologies = ({
    'Courses':["Spark","PySpark","Hadoop"],
    'Fee' :[22000.0,25000.0,23000.0],
    'InsertedDate':[20201124063015,20210210084021,20211215032511]
               })
df = pd.DataFrame(technologies)

# changing integer values to dates and time format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%Y%m%d%H%M%S')
print(df)
print(df.dtypes)

Yields below output.


# Output:
   Courses      Fee        InsertedDate
0    Spark  22000.0 2020-11-24 06:30:15
1  PySpark  25000.0 2021-02-10 08:40:21
2   Hadoop  23000.0 2021-12-15 03:25:11
Courses                 object
Fee                    float64
InsertedDate    datetime64[ns]
dtype: object

6. Complete Example For Convert Integers to Datetime Format


# Example For Convert Integers to Datetime Format
import pandas as pd
technologies = ({
    'Courses':["Spark","PySpark","Hadoop"],
    'Fee' :[22000.0,25000.0,23000.0],
    'InsertedDate':[20201124,20210210,20211215]
               })
df = pd.DataFrame(technologies)
print(df)
print(df.dtypes)

# Convert integers to datetime format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%Y%m%d')
print(df)
print(df.dtypes)

# Use pandas.to_datetime() and DataFrame.apply() with lambda function
df['InsertedDate'] = df['InsertedDate'].apply(lambda x: pd.to_datetime(str(x),format='%Y%m%d'))
print(df)
print(df.dtypes)

# Use series.astype() method to convert integers to datetime format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'].astype(str), format='%Y%m%d')
print(df)
print(df.dtypes)

# Use pandas.to_datetime() to convert integers to "yymmdd" format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%y%m%d')
print(df)
print(df.dtypes)

# Changing integer values to dates and time format
df['InsertedDate'] = pd.to_datetime(df['InsertedDate'], format='%Y%m%d%H%M%S')
print(df)
print(df.dtypes)

Conclusion

In this article, you have learned how to convert integer to datetime format by using pandas.to_datetime(), DataFrame.astype() and DataFrame.apply() with lambda function with examples.

Happy Learning !!

References

Tags: , ,

Naveen

I am a Data Engineer with 20+ years of experience in transforming data into actionable insights. Over the years, I have honed my expertise in designing, implementing, and maintaining data pipelines with frameworks like Apache Spark, PySpark, Pandas, R, Hive and Machine Learning. My journey in the field of data engineering has been a continuous learning, innovation, and a strong commitment to data integrity. I have started this SparkByExamples.com to share my experiences with the data as I come across. You can learn more about me at LinkedIn

Leave a Reply

You are currently viewing Pandas Convert Integer to Datetime Type