There are multiple ways to append two pandas DataFrames, In this article, I will explain how to append two or more pandas DataFrames by using several functions with examples.
In order to append two DataFrames you can use DataFrame.append() method. When you are appending two or more DataFrames, pass all DataFrames as a list to this method.
Alternatively, you can also use pandas.DataFrame.concat() to concatenate DataFrames which can also be used to append two DataFrames.
1. Quick Examples of Append Two DataFrames
If you are in a hurry, below are some quick examples of how to append two DataFrames in pandas.
# Below are some quick examples
# Append two DataFrames of same column
df3 = df1.append(df2)
# Append two DataFrames of different columns
df2 = df.append(df1)
# Using append() with ignore_index
df2 = df.append(df1, ignore_index=True)
# Appending three DataFrames
df3 = df.append([df1, df2], ignore_index=True)
2. Append Two DataFrames With the Same Columns
Let’s create a pandas DataFrame from Dict to explore this with an example.
# Create two DataFrames with same columns
import pandas as pd
df1 = pd.DataFrame({'Courses': ["Spark","PySpark","Python","pandas"],
'Fee' : [20000,25000,22000,24000]})
print("First DataFrame:\n", df1)
df2 = pd.DataFrame({'Courses': ["Pandas","Hadoop","Hyperion","Java"],
'Fee': [25000,25200,24500,24900],
'Duration': ['30days','35days','40days','45days']})
print("Second DataFrame:\n", df2)
Yields below output.
To append two DataFrames with the same columns in Padas, you can use the concat()
function. It appends the column with NaN on the result for rows where the same column does not exist.
# Append two DataFrames of same columns
# using append() function
df3 = df1.append(df2)
print("After appending DataFrames:\n", df3)
Using this method you can also append list as a row to the DataFrame.
3. Append Two DataFrames With the Different Columns
To append two DataFrames with different columns in Pandas, you can use the concat()
function. This function allows you to combine DataFrames along a specified axis (rows or columns), and it handles the alignment of columns with different names.
# Craete DataFrames of different columns
import pandas as pd
df1 = pd.DataFrame({'Courses': ["Spark","PySpark","Python","pandas"],
'Fee' : [20000,25000,22000,24000]})
print("First DataFrame:\n", df1)
df2 = pd.DataFrame({'discount': [2500,2520,2450,2490],
'Duration': ['30days','35days','40days','45days']})
print("Second DataFrame:\n", df2)
# Append two DataFrames of different columns
# using append() function
df3 = df1.append(df2)
print("After appending DataFrames:\n", df3)
Yields below output.
# Output:
First DataFrame:
Courses Fee
0 Spark 20000
1 PySpark 25000
2 Python 22000
3 pandas 24000
Second DataFrame:
discount Duration
0 2500 30days
1 2520 35days
2 2450 40days
3 2490 45days
After appending DataFrames:
Courses Fee discount Duration
0 Spark 20000.0 NaN NaN
1 PySpark 25000.0 NaN NaN
2 Python 22000.0 NaN NaN
3 pandas 24000.0 NaN NaN
0 NaN NaN 2500.0 30days
1 NaN NaN 2520.0 35days
2 NaN NaN 2450.0 40days
3 NaN NaN 2490.0 45days
4. Pandas Append Two DataFrames Ignore Index
Append by default merges all rows including indices. if you want to ignore the index on DataFrames, you can set the new index on the pandas DataFrame by using ignore_index=True
param.
# Using append() with ignore_index
df3 = df1.append(df2, ignore_index=True)
print(df2)
Yields below output.
# Output:
Courses Fee Duration
0 Spark 20000 NaN
1 PySpark 25000 NaN
2 Python 22000 NaN
3 pandas 24000 NaN
4 Pandas 25000 30days
5 Hadoop 25200 35days
6 Hyperion 24500 40days
7 Java 24900 45days
5. Append Three DataFrames
Similarly, If you have three DataFrames pass all these as a list to the append() method. Use ingore_index=True
param to reset the index on pandas DataFrame to start from zero.
# Create third DataFrame
df2 = pd.DataFrame({'Courses':['PHP','GO'],
'Duration':['30day','40days'],
'Fee':[10000,23000]})
# Appending multiple DataFrame
df3 = df.append([df1, df2], ignore_index=True)
print(df3)
Yields below output
# Output:
Courses Fee Duration
Courses Fee Duration
0 Spark 20000 NaN
1 PySpark 25000 NaN
2 Python 22000 NaN
3 pandas 24000 NaN
4 Pandas 25000 30days
5 Hadoop 25200 35days
6 Hyperion 24500 40days
7 Java 24900 45days
8 PHP 10000 30day
9 GO 23000 40days
6. Complete Example of Append Two DataFrames
import pandas as pd
# Complete Example of Append Two DataFrames
df = pd.DataFrame({'Courses': ["Spark","PySpark","Python","pandas"],
'Fee' : [20000,25000,22000,24000]})
df1 = pd.DataFrame({'Courses': ["Pandas","Hadoop","Hyperion","Java"],
'Fee': [25000,25200,24500,24900],
'Duration': ['30days','35days','40days','45days']})
# Using append() method
df3 = df1.append(df2)
print(df3)
# Using append() with ignore_index
df3 = df1.append(df2, ignore_index=True)
print(df2)
# Create third DataFrame
df2 = pd.DataFrame({'Courses':['PHP','GO'],
'Duration':['30day','40days'],
'Fee':[10000,23000]})
# Appending multiple DataFrame
df3 = df.append([df1, df2], ignore_index=True)
print(df3)
A. Appending two DataFrames in Pandas is a common operation used to combine two DataFrames vertically, stacking one on top of the other. This can be useful when you have data split across multiple DataFrames and want to consolidate them into a single DataFrame for analysis.
A. You can use the pd.concat()
function to append two DataFrames.
A. Yes, you can append DataFrames with different column names. By default, pd.concat()
will keep all columns from both DataFrames. However, if columns in one DataFrame do not exist in the other, the missing columns will be filled with NaN values.
A. You can reset the index of the resulting DataFrame using the reset_index()
method. For example: The drop=True
argument prevents the old index from being added as a new column in the resulting DataFrame.
A. Appending DataFrames (using pd.concat()
) stacks DataFrames vertically, combining rows. Merging DataFrames (using pd.merge()
) combines DataFrames based on common columns, aligning rows based on common values in those columns.
Conclusion
In this article, you have learned how to append two DataFrames by examples. Also learned how to append even three DataFrame by passing all DF’s you wanted to append as a list.
Related Articles
- Pandas Concat Two DataFrames Explained
- Pandas Combine Two DataFrames With Examples
- Pandas – Append a List as a Row to DataFrame
- How to Combine Two Series into pandas DataFrame
- How to Append Row to pandas DataFrame
- Pandas Append Rows & Columns to Empty DataFrame
- Pandas append() Usage by Examples
- How to Append Pandas Series?
- Append Pandas DataFrames Using for Loop
- How to Merge Series into Pandas DataFrame
- Pandas Merge DataFrames on Index
- Pandas Merge Two DataFrames
- Pandas Merge DataFrames Explained Examples
- Pandas join two DataFrames
- Split the column of DataFrame into two columns
- Split Pandas DataFrame by column value
- Split pandas DataFrame