Reading JSON from Arduino via Serial Missing Parts of Message

Question

I have a project (Digital Dashboard for a Motorhome) in Pi that reads JSON data from an Arduino board and sends it via Serial. The Pi gets the JSON data, makes some adjustments and then sends it to an webapp (fronted of the digital dashboard). I'm getting a lot of the data correct but as you will see below on the log I'm also getting randomly missing parts of the JSON string, which makes the gauges not super "realtime".

Why could this be happening?

Raspberry Pi Python code:

@webiopi.macro
def getSerialArduinoData():
    data = ""
    
    # empty input buffer before starting processing
    while (serial.available() > 0):
        serial.readString()

    webiopi.sleep(1)

    serial.writeString("d") #send command "d" to request data from Arduino

    if (serial.available() > 0):
        data = serial.readString()        # read available data as string
        print("")
        print(data)
        print()
    
    return data

Arduino Code:

void loop() {
   if (Serial.available() > 0) {
    input = Serial.read();
    switch (input) {
      case 'd':
        // call all data here
        // Get the current millis.
        unsigned long currentMillis = millis();
        //Initiate Json Document
        JsonDocument docJson;
        //Add Time to Json
        docJson["timeMillis"] = currentMillis;
        // Read all data
        readAirTanks(docJson);
        readFuelLevel(docJson);
        readEngineOilPressure(docJson);
        // Only read external Temperature and Humidity if module is connected.
        if (adafruitSensorAHTX0_OK) {
          docJson["externalTempHumidity"] = true;
          readExternalTempHumidity(docJson);
        } else {
          docJson["externalTempHumidity"] = false;
        }
        // Only read internal Temperature, Humidity and Pressure if module is connected.
        if (adafruitSensorMS8607_OK) {
          docJson["internalTempHumidityPressure"] = true;
          readInternalTempHumidityPressure(docJson);
        } else {
          docJson["internalTempHumidityPressure"] = false;
        }

        // Only read J1708 data if module is connected to vehicle.    
        if (Vehicle_OK) {
          docJson["ecmData"] = true;
          readJ1708Data(docJson);
        } else {
          docJson["ecmData"] = false;
        }
        // Send Json over Serial.
        serializeJson(docJson, Serial);
        // Add a newline character to mark the end of the JSON object
        Serial.println();
        break;

      default:
        break;
    }
  }
}

Pi log sample:

{"timeMillis":289391,"primaryAirTank":120,"secondaryAirTank":120,"fuelLevel":49,"engineOilPressure":56,"externalTempHumidity":true,"externalTemperature":86,"externalHumidity":62,"internalTempHumidityPressure":true,"internalTemperature":71,"internalHumidity":62,"internalPressure":993,"ecmData":true,"vehicleSpeed":0,"engineLoad":1,"coolantTemperature":134,"batteryVoltage":24.91321,"rpm":999,"intakeManifoldAirTemp":96}

{"timeMillis":290478,"primaryAirTank":120,"secondaryAirTank":120,"fuelLevel":51,"engineOilPressure":57,"externalTempHumidity":true,"externalTemperature":86,"externalHumidity":62,"internalTempHumidityPressure":true,"internalTemperature":71,"internalHumidity":63,"internalPressure":993,"ecmData":true,"vehicleSpeed":0,"engineLoad":1,"coolantTemperature":134,"batteryVoltage":25.17189,"rpm":1001,

{"timeMillis":291527,"primaryAirTank":120,"secondaryAirTank":120,"fuelLevel":50,"engineOilPressure":57,"externalTempHumidity":true,"externalTemperature":86,"externalHumidity":62,"internalTempHumidityPressure":true,"internalTemperature":71,"internalHumidity":63,"internalPressure":993,"ecmData":true,"vehicleSpeed":0,"engineLoad":1,"coolantTemperature":134,"batteryVoltage":25.4047,"rpm":1002,"intakeManifoldAirTemp":96}

{"timeMillis":292570,"primaryAirTank":120,"secondaryAirTank":120,"fuelLevel":50,"engineOilPressure":57,"externalTempHumidity":true,"externalTemperature":86,"externalHumidity":63,"internalTempHumidityPressure":true,"internalTemperature":71,"internalHumidity":63,"internalPressure":993,"ecmData":true,"vehicleSpeed":0,"engineLoad":1,"coolantTemperature":134,"batteryVoltage":25.61423,"rpm":999,"intakeMani

On the Raspberry Pi, type sudo apt install picocom && picocom -b 9600 /dev/ttyACM0 (with 9600 and ttyACM0 replaced with the appropriate port and baud rate). Then hit d. If you see the full JSON strings, then your problem is on the Python side. Exit picocom with Ctrl-A Ctrl-Q. — Edgar Bonet
– Edgar Bonet, Commented Jul 3 at 19:08
Thank you for the reply! I will try some of these code and report back. — Honeybadger22
– Honeybadger22, Commented Jul 4 at 22:24
please don't edit the solution into the question. I rolled it back. use the Answer box to add the solution of the problem — Juraj
– Juraj ♦, Commented Jul 6 at 7:01
Thank you @Juraj I made the correction and posted as an answer. — Honeybadger22
– Honeybadger22, Commented Jul 7 at 15:14

Honeybadger22 · Accepted Answer · 2025-07-07 15:13:02Z

1

For anyone else looking for this in the future the change I had to make was in the Python side, using pyserial instead of the version provided by the webiopi.

This is the final code:

import serial

# Configure the serial port (replace with your port and baud rate)
ser = serial.Serial('/dev/ttyACM0', 9600, timeout=1)

@webiopi.macro
def getSerialArduinoData():
    json = ""

    webiopi.sleep(1)

    ser.write(b'd') #send command "d" to request data from Arduino

    data_bytes = ser.read_until(b'\n') # Read until the newline character

    json = data_bytes.decode('utf-8') # Decode the bytes to a string
    json = json.strip() # Remove the delimiter from the end
    return json

answered Jul 7 at 15:13

Honeybadger22

313 bronze badges

Did you check why serial.readString() fails? I doubt it has something to do with the interface implementation as such, but more with some property like timeout. Are you aware that serial transmission with 9600 bps takes some time?

the busybee
– the busybee

2025-07-08 17:14:59 +00:00
Commented Jul 8 at 17:14

Add a comment |

Stack Exchange Network

Reading JSON from Arduino via Serial Missing Parts of Message

1 Answer 1

Your Answer

Hot Network Questions

Reading JSON from Arduino via Serial Missing Parts of Message

1 Answer 1

Your Answer

Sign up or log in

Post as a guest

Related

Hot Network Questions