MongoDB Combine data from Multiple Collections

To combine data from multiple collections in MongoDB, we can use the aggregation framework. The aggregation framework provides a powerful set of operators to manipulate data and perform complex data transformations. In the following article, we will examine the various methods for combining data from multiple collections.

1. Using the $lookup Aggregation to Combine Multiple Collections in MongoDB

You can use the MongoDB aggregate() function with $lookup stage to combine documents from multiple collections. $lookup is a aggregation pipeline operator to combine the data. The $lookup stage is used to perform join MongoDB documents from multiple collections in a single query. This stage performs a left outer join between the specified collection and the documents in the pipeline.

In the following query, we have the $lookup operator which joins the student collection with the course collection based on the _id field in the student collection and the c_id field in the course collection.

Then, the result of the query will be a new array of documents that include all the fields from the student collection, as well as an additional field called Details from the course collection.


# Use $lookup to combine data from multiple collections
db.student.aggregate([
  {
    $lookup: {
      from: "course",
      localField: "_id",
      foreignField: "c_id",
      as: "Details"
    }
  }
])

The output displayed the collections are merged into a single collection by executing the above $lookup query.

2. Using the $unwind aggregation stage to combine data in MongoDB

We can also use the $lookup and $unwind aggregation pipeline stages together to combine the collections into a single collection. Generally, the output of a $lookup query can be flattened using the $unwind stage. This can be beneficial if we want to combine data from different collections and then conduct further aggregation on the output set.

Here’s an example of this, we first performed the $lookup stage which joins the data from the course to the student collection based on the matching fields and stores the combined data as a separate document Data. Next, the $unwind stage transforms the array of combined data into a Data document.


# Usage of $lookup and $unwind aggregation pipeline
db.student.aggregate([
    { $lookup:
        {
           from: 'course',
           localField: '_id',
           foreignField: 'c_id',
           as: 'Data'
        }
    },
    {
       $unwind: '$Data'
    }
]).pretty();

The output displayed the combined collections here. We are able to observe the differences between using the $unwind stage and not using it.

3. Using the $unionWith aggregation stage to combine data

In addition to the aforementioned aggregation frameworks to add collections, we have another stage $unionWith. The $unionWith operator is used to combine the documents of two or more collections into a single result set. This operator is particularly useful when we need to combine data from different collections with different schemas.

Moreover, the $unionWith method does not require any matching condition but it combines all documents from the specified collections into a single result set.

In the following query, we use the $unionWith operator to merge the data from the course collection. We use the coll parameter to specify the name of the collection course to merge and the pipeline parameter to define the query pipeline for the course collection. Inside the pipeline parameter, we use the $project operator to rename the c_id field to courseID and select the desired output fields.


# Usage of $unionWith aggregation stage 
db.student.aggregate([
  {
    $unionWith: {
      coll: "course",
      pipeline: [
        {
          $project: {
            _id: 0,
            courseId: "$c_id",
            courseTitle: 1
          }
        }
      ]
    }
  }
])

The output displays documents from the “course” collection attached to the “student” collection. The $unionWith stage contains the fields that are specified in the projection stage of the pipeline.

4. Conclusion

In conclusion, the ability to combine data from different collections in MongoDB is an important feature that enables us to obtain data from various sources and enhance the performance of our queries. The $lookup, $unwind and $unionWith operators are useful tools that we can use to accomplish this goal.