Node.js: Receive chained data while saving to database asynchronously

Question

I'm using sockets in JS (Node.js) to communicate with a remote host. After I connected to the remote host, the remote host sends me an array of data. When I receive the data, I need to parse the data and store them in my database (using Sequelize). The entries are chained together. So, entry 1 must exist before I can write entry 2 to my database.

The problem is that saving to the database is asynchronous while receiving the data is sychronous. Let's say I receive the first 10 database entries in the "data"-event of the Socket, I parse the first entry and save the entry to the database. Saving to the database is asynchronous and so it's possible that while saving the first entry to the database, the "data"-event happens again (receiving entry 11 till 20). So I parse again and save entry 11 to my database. But when saving entry 11, the previous 10 aren't saved...

How to solve this issue? I already thought about the blukCreate()-function but this doesn't solve the problem because I need any "event" or "point" where I can say something like "wait for async operation before receiving next data". Any idea how to solve this problem?

Jonas Wilms · Accepted Answer · 2019-07-28 09:00:09Z

1

You could use a non-blocking lock:

   const createLock = () => {   
     let chain = Promise.resolve();
     return (task) => {
       const result = chain.then(() => task());
       chain = result.catch(() => {});
       return result;
     };
  };

That way you can use the lock like this:

  const useDB = createLock();

  const addEntry = entry => useDB(async () => {
    await db.add(entry); 
  });

   addEntry(0);
   addEntry(1); // will only be added after 0 is done

answered Jul 28, 2019 at 9:00

Jonas Wilms

139k20 gold badges164 silver badges164 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Sergey Suslov · Accepted Answer · 2019-07-28 08:38:53Z

0

saving is async but you can use promise or await to wait for db to write your data

const transaction = await db.sequelize.transaction()
try {
  for (let data of dataset) {
    await db.YourModel.create(data, {transaction})
  }
} catch(e) {
  transaction.rollback()
  return
}
transaction.commit()

answered Jul 28, 2019 at 8:38

Sergey Suslov

1292 bronze badges

Collectives™ on Stack Overflow

Node.js: Receive chained data while saving to database asynchronously

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related