Append Pandas DataFrames Using for Loop

By using Python for loop you can append rows or columns to Pandas DataFrames. You can append rows to DataFrame by using append(), pandas.concat(), and loc[]. In this article, I will explain how to append rows or columns to pandas DataFrame using a for loop and with the help of the above functions.

Quick Examples of Append to DataFrame Using For Loop

If you are in a hurry, below are some quick examples of appending pandas DataFrames using Python for loop.


# Quick examples of append to DataFrame using for loop

# Example 1: Append rows within a for loop
for i in range(1,4):        
    df.loc[len(df)] = i *1 

# Example 2: Append values to DataFrame
for i in range(1,4):        
    df[i] = i *1 

# Example 3: Append rows within for loop
# Create empty DataFrame 
df = pd.DataFrame(columns = ['c1', 'c2', 'c3'])  
for i in range(5):                                 
    df.loc[len(df)] = i * 5

# Example 4: Append DataFrame using for loop
# Create a List
list1 = ['Python','PySpark', 'Pandas', 'NumPy']
# Create an empty list
list2 = []
# Create new values using for loop
for value in list1:
    df_values = value
# Append df_values to llist2
    list2.append(df_values)
# create DataFrame using for loop
df = pd.DataFrame(list2, columns=['Course'],index=['I1','I2','I3','I4'])
 
# Example 5: Append DataFrames using Dictionary
# Create a DataFrame
df = pd.DataFrame({'Squares':[4, 9, 16, 25 ],
            'Cubes':[8, 27, 64, 125]})
for i in range(6,10):
    df=df.append({'Squares': i**2, 'Cubes': i**3}, ignore_index=True)

Let’s create a pandas DataFrame from Python Dictionary with a few rows and columns and execute some examples to learn how to insert rows. Our DataFrame contains column names Courses, Fee, Duration, and Discount.


# Create DataFrame
import pandas as pd
technologies = ({
    'Courses':["Spark","Hadoop","pandas","Java","PySpark"],
    'Fee' :[20000,25000,30000,22000,26000],
    'Duration':['30days','40days','35days','60days','50days'],
    'Discount':[1000,2500,1500,1200,3000]
               })
df = pd.DataFrame(technologies)
print("Create DataFrame:\n", df)

Yields below output.

Append Pandas DataFrames using For Loop

You can use a for loop to append a range of values at the end of our DataFrame. The following example shows how to add the row with the same values to DataFrame for each iteration. Let’s append rows to a pandas DataFrame within a loop.


# Append rows within for loop
for i in range(1,4):        
    df.loc[len(df)] = i *1 
print("After appending the rows to DataFrame:\n", df)

Yields below output.

Alternatively, using a for loop we can add a range of values as a column of DataFrame. We will get the values of the new columns at each iteration.


# Append values to DataFrame
for i in range(1,4):        
    df[i] = i *1 
print("After appending the columns to DataFrame:\n", df)

Yields below output.


# Output:
# After appending the columns to DataFrame:
  Courses    Fee Duration  Discount  1  2  3
0    Spark  20000   30days      1000  1  2  3
1   Hadoop  25000   40days      2500  1  2  3
2   pandas  30000   35days      1500  1  2  3
3     Java  22000   60days      1200  1  2  3
4  PySpark  26000   50days      3000  1  2  3

Append Rows to Empty DataFrame in a For Loop

Let’s see how to append rows to an empty DataFrame using a for loop, first let’s create an empty DataFrame.


# Create empty DataFrame 
df = pd.DataFrame(columns = ['c1', 'c2', 'c3'])  
print(df)  

# Output:
# Empty DataFrame
# Columns: [c1, c2, c3]
# Index: []

Use a for loop to append new rows to our empty DataFrame.


 # Append rows within for loop
for i in range(5):                                 
    df.loc[len(df)] = i * 5
print(df)

Yields below output.


# Output:
   c1  c2  c3
0   0   0   0
1   5   5   5
2  10  10  10
3  15  15  15
4  20  20  20

Another way to Append DataFrame within a for loop

This is another way in which I want to append DataFrames within a loop. To append first create a DataFrame, using a dictionary and concatenate them into a single DataFrame within a for a loop. This process is faster than appending new rows to the DataFrame after each step, as you are not constructing a new DataFrame on every iteration.


# Create a DataFrame
df = pd.DataFrame({'Squares':[4, 9, 16, 25 ],
            'Cubes':[8, 27, 64, 125]})
for i in range(6, 10):
    df=df.append({'Squares': i**2, 'Cubes': i**3}, ignore_index=True)
print(df)

Yields below output.


# Output:
   Squares  Cubes
0        4      8
1        9     27
2       16     64
3       25    125
4       36    216
5       49    343
6       64    512
7       81    729

FAQ on Append Pandas DataFrames Using for Loop

Can I append DataFrames in a loop?Y

You can append DataFrames iteratively in a for loop, but it is not the most efficient way for large datasets.

How does appending in a loop work?

In each iteration, a new DataFrame or row is added to an existing DataFrame using the append() method or pd.concat().

What are the performance implications of using a loop for appending?

Appending DataFrames repeatedly in a loop can be slow because each append() or concat() operation creates a new DataFrame. This leads to increased memory usage and computational overhead.

Does the append() method modify the original DataFrame?

The append() method returns a new DataFrame. The original DataFrame remains unchanged unless reassigned.

Can I append rows directly in the loop?

You can append rows by creating a new DataFrame for each row or by using a dictionary that matches the DataFrame columns.

Conclusion

In this article, I have explained how to append pandas DataFrames by using for loop with the help of append() function.

Quick Examples of Append to DataFrame Using For Loop

Append Pandas DataFrames using For Loop

Append Rows to Empty DataFrame in a For Loop

Another way to Append DataFrame within a for loop

FAQ on Append Pandas DataFrames Using for Loop

Conclusion

Related Articles