Sending binary data via WebSocket in NodeJS/ESP32 - how to recognize binary and text messages

Question

I'm working on a CCTV camera with ESP32-Cam and NodeJS server. The way I want to do it is to use WebSocket client on the ESP and send binary data with captured frames to NodeJS server, which then will forward it to connected users. I can't use socket.io, so I'm using plain websockets (on node 'ws' module).
What I'm struggling about is that while sending binary data I can't tell what kind of message it is, meaning if it is stream, thumbnail, or a plain json message.

My ESP32-Cam client code (C++):

void liveCam(uint8_t num){
    //capture a frame
    camera_fb_t * fb = esp_camera_fb_get();
    if (!fb) {
        Serial.println("Frame buffer could not be acquired");
        return;
    }

    webSocket.sendBIN(num, fb->buf, fb->len);

    //return the frame buffer back to be reused
    esp_camera_fb_return(fb);
}

And what I'm working on in Node:

this.socket.on("connection", (ws) => {
            ws.on("message", (msg) => {
                // recognize channel of msg and process it (ArrayBuffer / JSON String)
            })
        })

To send JSON I would use webSocket.sendTXT (using WebSocketsServer library) method on the ESP, just I don't know how to handle both binary and text messages on the server, and how to let the server recognize them (something like message tag, so binary stream would be "stream" while json would be "blink" or "fetch_temp", depending on usage). Without a binary stream I would do it just with json, so every message would have a "tag" and "message" property, but the image stream complicates things.

I had an idea to send a json text message before binary message, that would set the channel for the next message sent. Just no idea if it could work, if it wouldn't break if messages would come out of sync (is that possible?). Also, I need a solution as fast as possible (to stay at good framerate of the video, now clear binary stream is really neat!)

Thanks for any (good) ideas and solutions!

romkey · Accepted Answer · 2022-07-27 04:03:52Z

You're asking how to tell whether the data you send over a web socket is binary or JSON. Easy problem to solve. As you pointed out, just prefix the data with an indication of what's coming next. If you're worried about performance, don't send JSON as the prefix, send a single character - say, "B" to indicate the next message is binary or "J" to indicate that it's JSON.

And no, websocket messages cannot get out of sync unless your code makes a mistake. Websockets provide a reliable, ordered stream of bytes. Each byte is guaranteed to be received in order, exactly once, correctly, or you'll receive an error if the websocket isn't able to do this. "Correctly" is a bit of a stretch on an unencrypted connection - an unencrypted web socket will protect against simple network errors but it won't protect against intentional tampering by an attacker.

Collectives™ on Stack Overflow

Sending binary data via WebSocket in NodeJS/ESP32 - how to recognize binary and text messages

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related