Populating MongoDB schema

We'll be creating only the backend of the application for understanding how to post data in to a MongoDB schema that references another schema.

TL;DR

How can we get the following JSON data with the user schema referencing the todo schema?

{
  "todo": [
    {
      "_id": "61023642610b8d4ce4f56f81",
      "title": "test-title-1",
      "description": "test-description-1",
      "__v": 0
    },
    {
      "_id": "6102365b610b8d4ce4f56f84",
      "title": "test-title-2",
      "description": "test-description-2",
      "__v": 0
    }
  ],
  "_id": "6102361f610b8d4ce4f56f7f",
  "name": "test-user",
  "__v": 0
}

User Model

Todo Model

Here, the User schema is referencing the Todo schema. To get the JSON data with the todo data we need to do the following

While creating the todo data we need to add the ObjectId of the new todo to the todo array of the User. At this stage the data will look something like this.

{
  "todo": ["61023642610b8d4ce4f56f81", "6102365b610b8d4ce4f56f84"],
  "_id": "6102361f610b8d4ce4f56f7f",
  "name": "test-user",
  "__v": 0
}

To get the data of the todo created by the user we will reference the Todo table using the populate method which will get the data of the todo.

It is like joining two tables in SQL where User table references the Todo table using the primary key of the Todo table. Here, the primary key of the Todo table is the ObjectId.

Initialize project

Initialize our backend using npm and install necessary packages.
Set up a MongoDB database.
Set up server using Node and Express.
Create a database schema to define a Todo.
Set up API routes to create user and todo and read user and todo.
Testing our API routes using Insomnia.

Install

VS Code or any other editor
Latest version of Node.js
Insomnia or Postman
Prettier VS code extension to format the code

1. Initializing our project

Create a new folder and name it anything that you like and then open the folder in VS code and run the following code from the command prompt.

npm init -y

After running this command you will find a package.json if the folder.

2. Setting up package.json

i. Install the following dependencies

Run the following commands in the terminal to install the dependencies

npm i cors dotenv express mongoose

cors: allows cross-origin api calls
dotenv: needed to access data from .env files
express: web application framework for node.js
mongoose: It is needed to define the database schema and connecting to mongoDB

ii. Install the following development dependencies

Now install the following development dependencies, -D is used to install the development dependencies.

npm i -D nodemon

After installing the dependencies the package.json folder should look as follows.

// package.json
{
  "name": "mongodb-schema-populate-blog",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/mritunjaysaha/mongodb-schema-populate-blog.git"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "bugs": {
    "url": "https://github.com/mritunjaysaha/mongodb-schema-populate-blog/issues"
  },
  "homepage": "https://github.com/mritunjaysaha/mongodb-schema-populate-blog#readme",
  "dependencies": {
    "dotenv": "^10.0.0",
    "express": "^4.17.1",
    "mongoose": "^5.13.3"
  },
  "devDependencies": {
    "nodemon": "^2.0.12"
  }
}

iii. change the `main` entry point to `server.js`

Now, create a server.js file and a .env. The server.js file will be the entry point of the server and the .env file will contain the MONGO_URI. We also have to make the following changes in the package.json

//package.json
{
  "name": "mongodb-schema-populate-blog",
  "version": "1.0.0",
  "description": "",
  "main": "server.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/mritunjaysaha/mongodb-schema-populate-blog.git"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "bugs": {
    "url": "https://github.com/mritunjaysaha/mongodb-schema-populate-blog/issues"
  },
  "homepage": "https://github.com/mritunjaysaha/mongodb-schema-populate-blog#readme",
  "dependencies": {
    "dotenv": "^10.0.0",
    "express": "^4.17.1",
    "mongoose": "^5.13.3"
  },
  "devDependencies": {
    "nodemon": "^2.0.12"
  }
}

Now, create the following folders

config: Inside the config folder, create a file named db.js. This file will contain the required code for connecting to the MongoDB database.
controllers: The controllers folder will contain the files which will have the methods for the end points to communicate with the database.
models: The models folder, will contain the files which will define the MongoDB schema
routers: The routers folder will contain the files with the endpoints.

At this stage the file structure should look as follows

.
├── config
│   └── db.js
├── controllers
│   └── user.js
├── models
│   ├── todo.js
│   └── user.js
├── node_modules
├── routes
│   └── user.js
├── .env
├── server.js
├── package-lock.json
└── package.json

iv. Change the `scripts` to the following

"scripts": {
  "start":"node server.js",
  "dev":"nodemon server.js"
}

The package.json file should look as follows

{
  "name": "mern-todo",
  "version": "1.0.0",
  "description": "",
  "main": "server.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "node server.js", //added
    "dev": "nodemon server.js" //added
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "config": "^3.3.6",
    "cors": "^2.8.5",
    "dotenv": "^10.0.0",
    "express": "^4.17.1",
    "mongoose": "^5.13.2"
  },
  "devDependencies": {
    "nodemon": "^2.0.12"
  }
}

v. Setting up server

We will do the following to setup the server

Import express
Initialize our app using express()
Set up a get method for the endpoint http://localhost:8000 using app.get()
Set the PORT to 8000 for our server to run
Have our app to listen to PORT using app.listen()

.
├── config
│   └── db.js
├── controllers
│   └── user.js
├── models
│   ├── todo.js
│   └── user.js
├── node_modules
├── routes
│   └── user.js
├── .env
├── server.js <-- we are here
├── package-lock.json
└── package.json

The code will look as follows

And start the server using nodemon using the following code. Make sure you are running the following command from the project directory.

npm run dev

If the server has started successfully then it should show the following message in the terminal

[nodemon] 2.0.11
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node server.js`
server is running on http://localhost:8000

You can also open http://localhost:8000 on your browser.

vi. Getting the `MONGO URI` from mongoDB

To connect to the database we will need the link for the mongoDB collection.

Log in to mongoDB
Create a new project
Build a cluster
Select cloud provider
Create cluster
wait for the cluster to be created.
Click on connect
click on allow access from anywhere. Then Add IP address

Create a database user. You'll need the username and password for the MongoDB URI.
Click on the Choose a connection method
Click on Connect your application
Select the following driver and version
Copy the mongodb+srv and paste it in the .env file

vii. Setting up `.env` file

//.env
MONGO_URI = mongodb+srv://<username>:<password>@cluster0.owmij.mongodb.net

Replace the <username> and <password> with your database username and password which you will set in step 9.

viii. Connecting to database

.
├── config
│   └── db.js <-- we are here
├── controllers
│   └── user.js
├── models
│   ├── todo.js
│   └── user.js
├── node_modules
├── routes
│   └── user.js
├── .env
├── server.js
├── package-lock.json
└── package.json

Now, open the db.js file which is in the config folder and add the following changes.

Import mongoose
Import MONGO_URI from .env
Define the connectDB methof for connecting to the database
Export the connectDB method to be called in server.js

Add the following changes in the server.js file.

Import dotenv
Import connectDB method from config/db.js
Call the connectDB method.

Let us make the the following changes in server.js

Save the changes it will restart the server or use the command npm run dev. The terminal should show a message of MongoDB is connected which we have added in the db.js under the try block.

ix. Defining database schema

Create a todo.js file in the models folder. We will define the database schema in this file.

.
├── config
│   └── db.js
├── controllers
│   └── user.js
├── models
│   ├── todo.js <-- we are here
│   └── user.js
├── node_modules
├── routes
│   └── user.js
├── .env
├── server.js
├── package-lock.json
└── package.json

Import mongoose
Create a Schema called TodoSchema
We will add two fields for our todo; title and description
Type of title will be String and it is a mandatory field
Type of description will be String and it is not a mandatory field
Export the model

The code will look as follows

Create a schema for the user using the above steps.

After making the changes, the user model will look something like this

x. Defining the controllers

.
├── config
│   └── db.js
├── controllers
│   └── user.js <-- we are here
├── models

│   └── todo.js
├── node_modules
├── routes
│   └── user.js
├── .env
├── server.js
├── package-lock.json
└── package.json

Import Todo and User schemas
Define createUser method will create a new user
Define createTodo method will do the following
- create a new todo
- save the todo
- use the userId to find the user
- update the todo array with the ObjectId of the new todo
Define getUser to get the user details. The output of this method we can see that todo consists of some random value which is the ObjectId of the todo that the user has created. We cannot figure out what the todo contains.

{
  "todo": ["61023642610b8d4ce4f56f81", "6102365b610b8d4ce4f56f84"],
  "_id": "6102361f610b8d4ce4f56f7f",
  "name": "test-user",
  "__v": 0
}

Define getAllTodo method we will use the userId to find the user and then use the populate method to reference the todo with the ObjectId from the Todo table. The exec method is used to check for errors and return the populated data.

{
  "todo": [
    {
      "_id": "61023642610b8d4ce4f56f81",
      "title": "test-title-1",
      "description": "test-description-1",
      "__v": 0
    },
    {
      "_id": "6102365b610b8d4ce4f56f84",
      "title": "test-title-2",
      "description": "test-description-2",
      "__v": 0
    }
  ],
  "_id": "6102361f610b8d4ce4f56f7f",
  "name": "test-user",
  "__v": 0
}

xi. Defining the end points

.
├── config
│   └── db.js
├── controllers
│   └── user.js
├── models

│   └── todo.js
├── node_modules
├── routes
│   └── user.js <-- we are here
├── .env
├── server.js
├── package-lock.json
└── package.json

We will define the end points to create users and todo and to read them.

Import express
Import all the methods from controllers
Initialize router
Define a POST method to create a user
Define a POST method to create a todo and save it in the user
Define a GET method to read user data
Define a GET method to read user data and todo data

After making the above changes the code will look something like this

xii. Adding the routes end points in the `server.js`

.
├── config
│   └── db.js
├── controllers
│   └── todo.js
├── models
│   └── todo.js
├── node_modules
├── routes
│   └── todo.js
├── .env
├── server.js <-- we are here
├── package-lock.json
└── package.json

The final part of completing the backend is to add the endpoints to the server.js file.