How to Append Row to pandas DataFrame

  • Post author:
  • Post category:Pandas
  • Post last modified:November 7, 2023
  • Reading time:15 mins read

You can append one row or multiple rows to an existing pandas DataFrame in several ways, one way would be creating a list or dict with the details and appending it to DataFrame.

You can append a row to DataFrame by using append(), pandas.concat(), and loc[]. In this article, I will explain how to append a Python list, dict (dictionary) as a row to Pandas DataFrame, which ideally inserts a new row(s) to the DataFrame with elements specified by a list and dict.

1. Quick Examples of Appending Row to DataFrame

If you are in a hurry, below are some quick examples of how to append a row to Pandas DataFrame.


# Below are some quick examples

# Example 1: Append Row to DataFrame
list_row = ["Hyperion", 27000, "60days", 2000]
df.loc[len(df)] = list_row

# Example 2: Insert Dict to the dataframe using DataFrame.append()
new_row = {'Courses':'Hyperion', 'Fee':24000, 'Duration':'55days', 'Discount':1800}
df2 = df.append(new_row, ignore_index=True)

# Example 3: Append new row to specifig index name
df2 = df.append(pd.DataFrame([new_row],index=['7'],columns=df.columns))

# Example 4: Append row to the DataFrame
df2 = df.append(pd.Series(new_row, index=df.columns, name='7'))

# Example 5: Using pandas.concat() to append a row
new_row = pd.DataFrame({'Courses':'Hyperion', 'Fee':24000, 'Duration':'55days', 'Discount':1800}, index=[0])
df2 = pd.concat([new_row,df.loc[:]]).reset_index(drop=True)

# Example 6: Append specific row/index name using DataFrame.loc[]
df.loc['7', :] = ['Hive',25000,'45days',2000]

# Example 7: Append row in DataFrame using DataFrame.loc[]
df.loc['7'] = ['Hive',25000,'45days',2000]

Let’s create a pandas DataFrame from Dict with a few rows and columns and execute some examples to learn how to insert rows. Our DataFrame contains column names Courses, Fee, Duration, and Discount.


# Create DataFrame
import pandas as pd
technologies = ({
    'Courses':["Spark","Hadoop","pandas","Java","Pyspark"],
    'Fee' :[20000,25000,30000,22000,26000],
    'Duration':['30days','40days','35days','60days','50days'],
    'Discount':[1000,2500,1500,1200,3000]
               })
df = pd.DataFrame(technologies)
print("Create DataFrame:\n", df)

Yields below output.

pandas append row dataframe

2. Append Dict as Row to DataFrame

You can append a new row to the existing DataFrame from the dictionary. First, create a dictionary, and then apply the append() function, this method is required to pass ignore_index=True in order to append dict as a row to DataFrame, not using this will get you an error.


# Insert row to the dataframe using DataFrame.append()
new_row = {'Courses':'Hyperion', 'Fee':24000, 'Duration':'55days', 'Discount':1800}
df2 = df.append(new_row, ignore_index=True)
print("After appending a row to DataFrame:\n", df2)

Yields below output.

pandas append row dataframe

3. Append List to DataFrame

If you have a list and want to append it to DataFrame use loc[] attribute. For more similar examples, refer to how to append a list as a row to pandas DataFrame.


# New list to append Row to DataFrame
list = ["Hyperion", 27000, "60days", 2000]
df.loc[len(df)] = list
print("After appending a row to DataFrame:\n", df)

The output same as the above.

4. Append Row at the Specific Index Name

Note that in this example, first we are creating a new DataFrame/Series and append this to another DataFrame at the specified index.


# Append new row by specifig index name
df2 = df.append(pd.DataFrame([new_row],index=['5'],columns=df.columns))
print("After appending a row to DataFrame:\n", df)

# Append row to the DataFrame using append()
df2 = df.append(pd.Series(new_row, index=df.columns, name='5'))
print("After appending a row to DataFrame:\n", df)

Yields below output for both examples.


# Output:
# After appending a row to DataFrame:
    Courses    Fee Duration  Discount
0     Spark  20000   30days      1000
1    Hadoop  25000   40days      2500
2    pandas  30000   35days      1500
3      Java  22000   60days      1200
4   Pyspark  26000   50days      3000
5  Hyperion  24000   55days      1800

5. Use concat() Function to Append

Use pd.concat([new_row,df.loc[:]]).reset_index(drop=True) to append the row to the first position of the DataFrame as the Index starts from zero.


# Using pandas.concat() to append a row
new_row = pd.DataFrame({'Courses':'Hyperion', 'Fee':24000, 'Duration':'55days', 'Discount':1800}, index=[0])
df2 = pd.concat([new_row,df.loc[:]]).reset_index(drop=True)
print("After appending a row to DataFrame:\n", df)

Yields below output.


# Output:
# After appending a row to DataFrame:
    Courses    Fee Duration  Discount
0  Hyperion  24000   55days      1800
1     Spark  20000   30days      1000
2    Hadoop  25000   40days      2500
3    pandas  30000   35days      1500
4      Java  22000   60days      1200
5   Pyspark  26000   50days      3000

6. Use DataFrame.loc[] to Append Row

By using df.loc[Index_label] = new_row you can append a list as a row to the DataFrame at a specified index.


# Append specific row/index name using DataFrame.loc[]
df.loc['5', :] = ['Hive',25000,'45days',2000]
print("After appending a row to DataFrame:\n", df)

# Append row in DataFrame using DataFrame.loc[]
df.loc['5'] = ['Hive',25000,'45days',2000]
print("After appending a row to DataFrame:\n", df)

Yields below output.


# Output
# After appending a row to DataFrame:
         Courses      Fee Duration  Discount
0          Spark  20000.0   30days    1000.0
1         Hadoop  25000.0   40days    2500.0
2         pandas  30000.0   35days    1500.0
3           Java  22000.0   60days    1200.0
4        Pyspark  26000.0   50days    3000.0
7           Hive  25000.0   45days    2000.0

7. Complete Example of Append Row to DataFrame


import pandas as pd
technologies = ({
    'Courses':["Spark","Hadoop","pandas","Java","Pyspark"],
    'Fee' :[20000,25000,30000,22000,26000],
    'Duration':['30days','40days','35days','60days','50days'],
    'Discount':[1000,2500,1500,1200,3000]
               })
df = pd.DataFrame(technologies)
print(df)

# Insert row to the dataframe using DataFrame.append()
df = pd.DataFrame(technologies)
new_row = {'Courses':'Hyperion', 'Fee':24000, 'Duration':'55days', 'Discount':1800}
df2 = df.append(new_row, ignore_index=True)
print(df2)

# Append new row to specifig index name
new_row = {'Courses':'Oracle','Fee':25000,'Duration':'65days','Discount':2800}
df2 = df.append(pd.DataFrame([new_row],index=['Index'],columns=df.columns))
print(df2)

# Append row to the DataFrame
df2 = df.append(pd.Series(new_row, index=df.columns, name='Index'))
print(df2)

# Using pandas.concat() to add a row
new_row = pd.DataFrame({'Courses':'Hyperion', 'Fee':24000, 'Duration':'55days', 'Discount':1800}, index=[0])
df2 = pd.concat([new_row,df.loc[:]]).reset_index(drop=True)
print (df2)

# Add specific row/index name using DataFrame.loc[]
df.loc['Index5', :] = ['Hive',25000,'45days',2000]
print(df)

# Add row in DataFrame using DataFrame.loc[]
df.loc['Index5'] = ['Hive',25000,'45days',2000]
print(df)

Frequently Asked Questions on Append Row to DataFrame

How do I append a row to a Pandas DataFrame?

To append a row to a Pandas DataFrame, you can use the append() method. For example, df = df.append(new_row, ignore_index=True).

What is the ignore_index parameter in the append() method?

The ignore_index parameter is a boolean flag. When it is set to True, it will reindex the DataFrame after appending the row, so the index will be consecutive numbers starting from 0. If it is set to False, the index of the appended row will be retained.

Is append() the most efficient way to add rows to a DataFrame?

The append method is simple to use, but it may not be the most efficient for adding multiple rows to a DataFrame. If you need to append multiple rows, it’s often more efficient to create a list of rows as DataFrames and then concatenate them using pd.concat.

How do I append a row to a specific position in a DataFrame?

Pandas does not provide a direct method to append a row to a specific position in a DataFrame. To insert a row at a specific location, you can create a new DataFrame with the desired order and then concatenate it.

How do I append a row with a custom index?

To append a row with a custom index, you can set the index of the row DataFrame before appending it to the main DataFrame. For example, new_row = pd.DataFrame({'A': 1, 'B': 2}, index=[custom_index]) df = df.append(new_row)

Conclusion

In this article, you have learned how to append a row to DataFrame using loc[], concat(), and append() methods. Using these you can append a row from list/dict at any position/index.

Happy Learning !!

References

Naveen (NNK)

Naveen (NNK) is a Data Engineer with 20+ years of experience in transforming data into actionable insights. Over the years, He has honed his expertise in designing, implementing, and maintaining data pipelines with frameworks like Apache Spark, PySpark, Pandas, R, Hive and Machine Learning. Naveen journey in the field of data engineering has been a continuous learning, innovation, and a strong commitment to data integrity. In this blog, he shares his experiences with the data as he come across. Follow Naveen @ @ LinkedIn

Leave a Reply