Fetching data from mongo via express, build object, and send to React

Question

I am currently stuck in asynchronous hell. In my React, I have a page /menu, that would load data from my mongo instance via expressjs api.

In my database, called menu, i have collections which represent a meal-type eg "breakfast", "lunch" etc. In those collections, the documents for every item looks like this bread collection example:

{
  _id: 2398jcs9dn2f9f,
  name: "Ciabatta",
  desc: "Italian bread",
  imageURI: "image01.jpg",
  reviews: []
}

This is my api that would be called when the page loads

exports.getAllFoods = (req, res, next) => {
    const db = mongoose.connection

    const allCollections = {}

    try {
        db.db.listCollections().toArray((err, collections) => {
            collections.forEach((k) => {
                allCollections[k.name] = []
            })

            Object.keys(allCollections).map(k => {
                let Meal = mongoose.model(k, MealSchema)
            
                meal = Meal.find((err, docs) => {
                    allCollections[k] = docs
                    console.log(allCollections)
                })
            })
            res.send(allCollections)
        })
    } catch (error) {
        console.log(error)
        res.send('unable to get all collections')
    }
}

The last output of the console.log(allCollections) produces this:

{ snacks:
   [ { review: [],
       tags: [],
       _id: 5fcec3fc4bc5d81917c9c1fe,
       name: 'Simosa',
       description: 'Indian food',
       imageURI: 'image02.jpg',
       __v: 0 } ],
  breads:
   [ { review: [],
       tags: [],
       _id: 5fcec41a4bc5d81917c9c1ff,
       name: 'Ciabatta',
       description: 'Italian bread',
       imageURI: 'image02.jpg',
       __v: 0 } ],
}

This is exactly what I need, but I am stuck in figuring out how to send to React. What am I to do to send the above json? The res.send(allCollections) gives me this:

{
    "snacks": [],
    "breads": [],
    "drinks": []
}

I understand why the above is being sent, but I dont know what I need to do to address it.

This is my React on page load

useEffect(() => {
        axios
        .get('http://localhost:8888/api/allFoods')
        .then((res) => {
            setMealTypes(res.data)
        })
        .catch((err) => [
            console.log(err)
        ])
    }, [])

Ultimately, I need the json outputted in console as I wanted to loop through that data and use the key as a title, and then list the values from the value array eg

<div>
  <h2>Breads</h2>
  <img src=image01.jpg/>
  <h3>Ciabatta</h3>
  <p>Italian bread</p>
  ...
</div> 
...

I'd appreciate any help, and any docs I should read to help and improve my javascript understandings

Penryn · Accepted Answer · 2020-12-09 15:41:29Z

I'd prefer to solve this using async/await and Promise.all, replacing most callbacks.

Because you're calling the DB when you're iterating through an array, you have the most annoying callback situation: how do you issue a bunch of async things and then get the results after? You'll need something else to ensure all callbacks are called before sending the results.

Async/await means we can declare a function is async, and await the results of an async operation. async/await is annoying in JS because it abstracts away callbacks and is actually creating a Promise underneath. Complicating things further, async/await doesn't solve issuing multiple async functions, so again we have to rely on this fancy Promise.all() function combined with map-ing the desired input array to async functions.

Original:

Object.keys(allCollections).map(k => {
  let Meal = mongoose.model(k, MealSchema)
  meal = Meal.find((err, docs) => {
    allCollections[k] = docs
    console.log(allCollections)
  })
});

Suggested async/await:

await Promise.all(Object.keys(allCollections).map(async k => {
  let Meal = mongoose.model(k, MealSchema)
  let docs = await Meal.find();
  allCollections[k] = docs;
  console.log(allCollections);
}));

Another advantage is error handling. If any errors happen in the callback of the original example, they won't be caught in this try/catch block. async/await handles errors like you'd expect, and errors will end up in the catch block.

...
      // Now that we have awaited all async calls above, this should be executed _after_ the async calls instead of before them.
      res.send(allCollections);
    })
  } catch (error) {
    console.log(error)
    res.send('unable to get all collections')
  }
}

Technically Promise.all() returns an array of results, but we can ignore that since you're formatting an Object anyway.

There is plenty of room to optimize this further. I might write the whole function as something like:

exports.getAllFoods = async (req, res, next) => {
  const db = mongoose.connection.db;

  try {
    let collections = await db.listCollections().toArray();

    let allCollections = {};
    collections.forEach((k) => {
      allCollections[k.name] = [];
    })

    // For each collection key name, find docs from the database
    // await completion of this block before proceeding to the next block
    await Promise.all(Object.keys(allCollections).map(async k => {
      let Meal = mongoose.model(k, MealSchema)
      let docs = await Meal.find();
      allCollections[k] = docs;
    }));

    // allCollections should be populated if no errors occurred
    console.log(allCollections);
    res.send(allCollections);
  } catch (error) {
    console.log(error)
    res.send('unable to get all collections')
  }
}

Completely untested.

You might find these links more helpful than my explanation:

https://javascript.info/async-await

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all

https://medium.com/dailyjs/the-pitfalls-of-async-await-in-array-loops-cf9cf713bfeb

Greatly appreciated. This solved my issue. As you said, this allowed all the callbacks to complete before sending my req back. Many many thanks. I am going to go over those resources to better understand async/await

Hubert Hilczer · Accepted Answer · 2020-12-08 20:07:11Z

0

I hope this will help you : You need to first use the stringify method before sending the collections from the express api and then use JSON.parse on the React front end to restore the object. PS: can you do a console.log(allCollections) one line above res.send(allCollections)?

answered Dec 8, 2020 at 20:07

Hubert Hilczer

955 bronze badges

Comments

Brewha · Accepted Answer · 2020-12-08 22:13:12Z

0

You need to send it to the front-end in a JSON format.

replace res.send(allCollections) with res.json(allCollections)

answered Dec 8, 2020 at 22:13

Brewha

662 silver badges4 bronze badges

Collectives™ on Stack Overflow

Fetching data from mongo via express, build object, and send to React

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related