Pandas – How to Merge Series into DataFrame

  • Post author:
  • Post category:Pandas
  • Post last modified:October 5, 2023

Let’s say you already have a pandas DataFrame with few columns and you would like to add/merge Series as columns into existing DataFrame, this is certainly possible using pandas.Dataframe.merge() method.

I will explain with the examples in this article. first create a sample DataFrame and a few Series. You can also try by combining Multiple Series to create DataFrame.


import pandas as pd

technologies = ({
    'Courses':["Spark","PySpark","Hadoop"],
    'Fee' :[22000,25000,23000]
               })
df = pd.DataFrame(technologies)
print(df)

# Create Series
discount  = pd.Series([1000,2300,1000], name='Discount')


1. Merge Series into pandas DataFrame

Now let’s say you wanted to merge by adding Series object discount to DataFrame df.


# Merge Series into DataFrame
df2=df.merge(discount,left_index=True, right_index=True)
print(df2)

Yields below output. It merges the Series with DataFrame on index.


# Output:
   Courses    Fee  Discount
0    Spark  22000      1000
1  PySpark  25000      2300
2   Hadoop  23000      1000

This also works if your rows are in different order, but in this case you should have custom indexes. I will leave this to you to explore.


# Rename Series before Merge
df2=df.merge(discount.rename('Course_Discount'),
             left_index=True, right_index=True)
print(df2)

Yields below output


# Output:
   Courses    Fee  Course_Discount
0    Spark  22000             1000
1  PySpark  25000             2300
2   Hadoop  23000             1000

2. Using Series.to_frame() & DataFrame.merge() Methods

You can also create a DataFrame from Series using Series.to_frame() and use it with DataFrame to merge.


# Merge by creating DataFrame from Series
df2=df.merge(discount.to_frame(), left_index=True, right_index=True)
print(df2)

Yields same output as in first example.

Conclusion

In this article I have explained how to merge/add series objects to existing pandas DataFrame as columns by using merge() method. also covered by creating a DataFrame from series using to_frame() and using on merge() method.

Happy Learning

Reference

Naveen

I am a Data Engineer with 20+ years of experience in transforming data into actionable insights. Over the years, I have honed my expertise in designing, implementing, and maintaining data pipelines with frameworks like Apache Spark, PySpark, Pandas, R, Hive and Machine Learning. My journey in the field of data engineering has been a continuous learning, innovation, and a strong commitment to data integrity. I have started this SparkByExamples.com to share my experiences with the data as I come across. You can learn more about me at LinkedIn

Leave a Reply

You are currently viewing Pandas – How to Merge Series into DataFrame