Use from_dict(), from_records(), json_normalize() methods to convert list of dictionaries (dict) to pandas DataFrame. Dict is a type in python to hold key-value pairs. Key is used as a column name and value is used for column value when we convert dict to DataFrame. When a key is not found for some dicts and it exists on other dicts, it creates a DataFrame with NaN for non-existing keys.
In this article, we will see how to convert a list of dictionaries (dict) to a pandas DataFrame using pd.DataFrame(), pd.DataFrame.from_dict(), and pd.DataFrame.from_records(). Each dictionary in the list can have similar or different keys but different values.
1. Quick Examples of Convert a List of Dictionaries to a DataFrame
If you are in a hurry, below are some quick examples of how to convert a list of dictionaries (dict) to a Pandas DataFrame.
# Below are quick examples
# Create a list of dictionary objects
technologies=[{'Courses':'Spark','Duration':'30days','Discount':1000}
{'Courses':'python','Fee':25000,'Courses_Fee':'Spark'},
{'Fee':30000,'Duration':'35days','Duration_Discount':'10days'}]
# Convert a List of dictionaries using from_records() method.
df = pd.DataFrame.from_records(technologies)
# Set Custom index by using index parameter.
df = pd.DataFrame.from_records(technologies, index=['1', '2', '3'])
# Convert a List of Dictionaries by from_dict method.
df = pd.DataFrame.from_dict(data)
# Dictionary orientations of column.
df=pd.DataFrame.from_dict(technologies, orient='columns')
# Convert a list of dictionaries using json_normalize().
df=pd.json_normalize(technologies)
1. Create a DataFrame from List of Dict
If you have a list of dictionaries (dict), it is easy to create a DataFrame by using the DataFrame constructor. For more examples refer to how to create a pandas DataFrame with examples.
# Create a DataFrame from list of Dictionaries with default indexes.
import pandas as pd
# List of dict object
technologies=[{'Courses':'Spark','Duration':'30days','Discount':1000}
{'Courses':'python','Fee':25000,'Courses_Fee':'Spark'},
{'Fee':30000,'Duration':'35days','Duration_Discount':'10days'}]
# Create DataFrame from list of dic object
df=pd.DataFrame(technologies)
print(df)
Yields below output. Note that when a key is not found for some dicts and it exists on other dicts, it creates a DataFrame with NaN for non-existing keys. In case you would like to change the NaN values refer to How to replace NaN/None values with empty String.
# Output:
Courses Duration Discount Fee Courses_Fee Duration_Discount
0 Spark 30days 1000.0 NaN NaN NaN
1 python NaN NaN 25000.0 Spark NaN
2 NaN 35days NaN 30000.0 NaN 10days
2. Using from_records() Method Convert a List of Dictionaries
The from_records() method is used to convert a list of dictionaries to DataFrame. It can also be used to convert structured or record ndarray to DataFrame and is used to create a DataFrame from a structured ndarray, sequence of tuples or dicts, or from another DataFrame.
# Convert a List of dictionaries using from_records() method.
df = pd.DataFrame.from_records(technologies)
print(df)
Yields same output as above.
3. Set Custom Index by Using Index Parameter
In this section, let’s see how to set custom index by using index parameter. If you already have a DataFrame, you can set the Index to the DataFrame by using df.index.
# Set Custom index by using index parameter.
df = pd.DataFrame.from_records(technologies,index=['1', '2', '3'])
print(df)
Yields below output.
# Output:
Courses Duration Discount Fee Courses_Fee Duration_Discount
1 Spark 30days 1000.0 NaN NaN NaN
2 python NaN NaN 25000.0 Spark NaN
3 NaN 35days NaN 30000.0 NaN 10days
4. Convert a List of Dictionaries by Using from_dict() Method
Use pd.DataFrame.from_dict() to transform a list of dictionaries to pandas DatFrame. This function is used to construct DataFrame from dict of array-like or dicts.
# Convert a List of Dictionaries by from_dict method.
df = pd.DataFrame.from_dict(technologies)
print(df)
Yields below output.
# Output:
Courses Duration Discount Fee Courses_Fee Duration_Discount
0 Spark 30days 1000.0 NaN NaN NaN
1 python NaN NaN 25000.0 Spark NaN
2 NaN 35days NaN 30000.0 NaN 10days
5. Dictionary Orientations of Columns and Index
There are two primary types of dictionary orientations which are called Columns and index. It is to make the distinction between the different types of dictionary orientations with the orient='columns'.
# Dictionary orientations of column.
df=pd.DataFrame.from_dict(technologies, orient='columns')
print(df)
Yields same output as above.
6. Convert a List of Dictionaries by Using json_normalize()
If we want to convert an object to a JSON string, we have to note that NaN’s and None will be converted to null and datetime objects will be converted to UNIX timestamps. json_normalize() function works with lists of dictionaries (dict).
# Convert a list of dictionaries using json_normalize.
df=pd.json_normalize(technologies)
print(df)
Yields same output as above.
7. Complete Examples to Convert a List of Dictionaries of Pandas DataFrame
# Create a DataFrame from list of Dictionaries with default indexes.
import pandas as pd
technologies=[{'Courses':'Spark','Duration':'30days','Discount':1000},{'Courses':'python','Fee':25000,'Courses_Fee':'Spark'},{'Fee':30000,'Duration':'35days','Duration_Discount':'10days'}]
df=pd.DataFrame(technologies)
print(df)
# Convert a List of dictionaries using from_records() method.
df = pd.DataFrame.from_records(technologies)
# Set Custom index by using index parameter.
df = pd.DataFrame.from_records(technologies, index=['1', '2', '3'])
# Convert a List of Dictionaries by from_dict method.
df = pd.DataFrame.from_dict(data)
# Dictionary orientations of column.
df=pd.DataFrame.from_dict(technologies, orient='columns')
# Convert a list of dictionaries using json_normalize.
df=pd.json_normalize(technologies)
Conclusion
In this article, you have learned about how to convert a list of dictionaries to pandas DataFrame by from_record(), from_dict(), json_normalize() with the examples.
Related Articles
- Pandas Empty DataFrame with Specific Column Types
- Retrieve Number of Columns From Pandas DataFrame
- Create Pandas DataFrame With Working Examples
- Rename Specific Columns in Pandas
- Pandas Create Empty DataFrame
- Pandas Convert Column to Int in DataFrame
- Pandas Convert JSON to DataFrame
- Pandas Convert Integer to Datetime Type
- Pandas Convert Datetime to Date Column
