Use from_dict(), from_records(), json_normalize() methods to convert list of dictionaries (dict) to pandas DataFrame. Dict is a type in python to hold key-value pairs. Key is used as a column name and value is used for column value when we convert dict to DataFrame. When a key is not found for some dicts and it exists on other dicts, it creates a DataFrame with NaN
for non-existing keys.
In this article, we will see how to convert a list of dictionaries (dict) to a pandas DataFrame using pd.DataFrame()
, pd.DataFrame.from_dict()
, and pd.DataFrame.from_records()
. Each dictionary in the list can have similar or different keys but different values.
1. Quick Examples of Convert a List of Dictionaries to a DataFrame
If you are in a hurry, below are some quick examples of how to convert a list of dictionaries (dict) to a Pandas DataFrame.
# Below are quick examples
# Create a list of dictionary objects
technologies=[{'Courses':'Spark','Duration':'30days','Discount':1000}
{'Courses':'python','Fee':25000,'Courses_Fee':'Spark'},
{'Fee':30000,'Duration':'35days','Duration_Discount':'10days'}]
# Convert a List of dictionaries using from_records() method.
df = pd.DataFrame.from_records(technologies)
# Set Custom index by using index parameter.
df = pd.DataFrame.from_records(technologies, index=['1', '2', '3'])
# Convert a List of Dictionaries by from_dict method.
df = pd.DataFrame.from_dict(data)
# Dictionary orientations of column.
df=pd.DataFrame.from_dict(technologies, orient='columns')
# Convert a list of dictionaries using json_normalize().
df=pd.json_normalize(technologies)
1. Create a DataFrame from List of Dict
If you have a list of dictionaries (dict), it is easy to create a DataFrame by using the DataFrame constructor. For more examples refer to how to create a pandas DataFrame with examples.
# Create a DataFrame from list of Dictionaries with default indexes.
import pandas as pd
# List of dict object
technologies=[{'Courses':'Spark','Duration':'30days','Discount':1000}
{'Courses':'python','Fee':25000,'Courses_Fee':'Spark'},
{'Fee':30000,'Duration':'35days','Duration_Discount':'10days'}]
# Create DataFrame from list of dic object
df=pd.DataFrame(technologies)
print(df)
Yields below output. Note that when a key is not found for some dicts and it exists on other dicts, it creates a DataFrame with NaN
for non-existing keys. In case you would like to change the NaN values refer to How to replace NaN/None values with empty String.
# Output:
Courses Duration Discount Fee Courses_Fee Duration_Discount
0 Spark 30days 1000.0 NaN NaN NaN
1 python NaN NaN 25000.0 Spark NaN
2 NaN 35days NaN 30000.0 NaN 10days
2. Using from_records() Method Convert a List of Dictionaries
The from_records()
method is used to convert a list of dictionaries to DataFrame. It can also be used to convert structured or record ndarray
to DataFrame and is used to create a DataFrame from a structured ndarray, sequence of tuples
or dicts
, or from another DataFrame.
# Convert a List of dictionaries using from_records() method.
df = pd.DataFrame.from_records(technologies)
print(df)
Yields same output as above.
3. Set Custom Index by Using Index Parameter
In this section, let’s see how to set a custom index by using index
parameter. If you already have a DataFrame, you can set the Index to the DataFrame by using df.index
.
# Set Custom index by using index parameter.
df = pd.DataFrame.from_records(technologies,index=['1', '2', '3'])
print(df)
Yields below output.
# Output:
Courses Duration Discount Fee Courses_Fee Duration_Discount
1 Spark 30days 1000.0 NaN NaN NaN
2 python NaN NaN 25000.0 Spark NaN
3 NaN 35days NaN 30000.0 NaN 10days
4. Convert a List of Dictionaries by Using from_dict() Method
Use pd.DataFrame.from_dict()
to transform a list of dictionaries to pandas DatFrame. This function is used to construct DataFrame from dict of array-like or dicts.
# Convert a List of Dictionaries by from_dict method.
df = pd.DataFrame.from_dict(technologies)
print(df)
Yields below output.
# Output:
Courses Duration Discount Fee Courses_Fee Duration_Discount
0 Spark 30days 1000.0 NaN NaN NaN
1 python NaN NaN 25000.0 Spark NaN
2 NaN 35days NaN 30000.0 NaN 10days
5. Dictionary Orientations of Columns and Index
There are two primary types of dictionary orientations which are called Columns
and index
. It is to make the distinction between the different types of dictionary orientations with the orient='columns'
.
# Dictionary orientations of column.
df=pd.DataFrame.from_dict(technologies, orient='columns')
print(df)
Yields same output as above.
6. Convert a List of Dictionaries by Using json_normalize()
If we want to convert an object to a JSON
string, we have to note that NaN’s
and None
will be converted to null and datetime objects will be converted to UNIX
timestamps. json_normalize()
function works with lists of dictionaries (dict).
# Convert a list of dictionaries using json_normalize.
df=pd.json_normalize(technologies)
print(df)
Yields same output as above.
7. Complete Examples to Convert a List of Dictionaries of Pandas DataFrame
# Create a DataFrame from list of Dictionaries with default indexes.
import pandas as pd
technologies=[{'Courses':'Spark','Duration':'30days','Discount':1000},{'Courses':'python','Fee':25000,'Courses_Fee':'Spark'},{'Fee':30000,'Duration':'35days','Duration_Discount':'10days'}]
df=pd.DataFrame(technologies)
print(df)
# Convert a List of dictionaries using from_records() method.
df = pd.DataFrame.from_records(technologies)
# Set Custom index by using index parameter.
df = pd.DataFrame.from_records(technologies, index=['1', '2', '3'])
# Convert a List of Dictionaries by from_dict method.
df = pd.DataFrame.from_dict(data)
# Dictionary orientations of column.
df=pd.DataFrame.from_dict(technologies, orient='columns')
# Convert a list of dictionaries using json_normalize.
df=pd.json_normalize(technologies)
Conclusion
In this article, you have learned about how to convert a list of dictionaries to pandas DataFrame by from_record(), from_dict(), json_normalize() with the examples.
Related Articles
- Pandas Empty DataFrame with Specific Column Types
- Retrieve Number of Columns From Pandas DataFrame
- Create Pandas DataFrame With Working Examples
- Rename Specific Columns in Pandas
- Pandas Create Empty DataFrame
- Pandas Convert Column to Int in DataFrame
- Pandas Convert JSON to DataFrame
- Pandas Convert Integer to Datetime Type
- Pandas Convert Datetime to Date Column
- pandas convert column to numpy array
- Convert Pandas Column to Lowercase
- Pandas Convert String to Integer
- How to Convert Pandas DataFrame to List?
- Pandas – Convert DataFrame to Dictionary (Dict)
- Pandas Series.isin() Function
- Pandas.Series.combine()
- Pandas Rolling Sum