socket.recv() response varies in length

Question

I have a normal TCP client connection which I use to retrieve data from a climate sensor (SCD30). In theory, it does what it is supposed to do, but I always get different numbers of bytes.

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as tcp:
 try:
  tcp.settimeout(self.timeout)
  tcp.connect((self.ip, self.port))
  tcp.settimeout(None)
  tcp.sendall(b"\x61\x03\x00\x20\x00\x01\x8c\x60")
  data = tcp.recv(self.bytes)
  tcp.close()

The correct response is:

b'a\x03\x02\x03B\xb8\x8d'
Bytes: 7

what else I received:

b'a'
Bytes: 1

b'a\x03'
Bytes: 2

b'\x02\x03B\xb8\x8da\x03\x02\x03B\xb8\x8d'
Bytes: 12

b'\x03\x02\x03B\xb8\x8d'
Bytes: 6

That's normal behaviour of recv. There are tons of materials explaining this. Just doing recv is plain wrong, you need a protocol that tells where the content ends and a loop. — freakish
– freakish, Commented Apr 1 at 6:39
there is no end character. i only know it has to be 7 bytes. i had already tried a loop, but it doesn't work because the number of bytes returned always varies. either the loop hangs or there is garbage on the line — Microscoop
– Microscoop, Commented Apr 1 at 6:53
If you know that it is always 7 bytes then you need to loop until you read 7 bytes. That's all. — freakish
– freakish, Commented Apr 1 at 7:43
yeah, of course. but sometimes there are only 2 bytes. it is either a timeout error or our “network security” is destroying the data packets. — Microscoop
– Microscoop, Commented Apr 1 at 7:52
i could put the entire send-receive functionality in a loop... will try this — Microscoop
– Microscoop, Commented Apr 1 at 7:55

Remy Lebeau · Accepted Answer · 2025-04-02 14:50:30Z

2

TCP is a byte stream, it has no concept of message boundaries. As such, recv() can return fewer bytes than requested. By default, it returns an arbitrary number of available bytes up to the buffer size you specify.

So, you will need to call recv() in a loop until you have received all of the bytes you are expecting, in this case 7 bytes total, eg:

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as tcp:
try:
  tcp.settimeout(self.timeout)
  tcp.connect((self.ip, self.port))
  tcp.settimeout(None)
  tcp.sendall(b"\x61\x03\x00\x20\x00\x01\x8c\x60")
  data = []
  while len(data) < 7:
    bytesRead = tcp.recv(7-len(data))
    if len(bytesRead) = 0:
      break
    data += bytesRead
  tcp.close()

Alternatively, if you are running your code on a 'Nix-based system, you can use the MSG_WAITALL flag when calling recv() instead of using a loop, eg:

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as tcp:
try:
  tcp.settimeout(self.timeout)
  tcp.connect((self.ip, self.port))
  tcp.settimeout(None)
  tcp.sendall(b"\x61\x03\x00\x20\x00\x01\x8c\x60")
  data = tcp.recv(7, socket.MSG_WAITALL)
  tcp.close()

edited Apr 2 at 14:50

answered Apr 1 at 19:47

Remy Lebeau

609k36 gold badges516 silver badges875 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Microscoop Apr 2 at 6:21

thanks for help, that was my first thought too, but it doesnt work. at first iteration i got 1Byte, at second iteration there are no bytes available. if i restart i got 7bytes plus the last missing 6. i dont know how that is possible

Microscoop Apr 2 at 6:45

However, I think that your solution is usually the right way to go

Microscoop · Accepted Answer · 2025-04-01 08:17:31Z

-2

ok, that works great, in 100% of all cases so far

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as tcp:
            try:
                tcp.settimeout(self.timeout)
                tcp.connect((self.ip, self.port))
                tcp.settimeout(None)
                i = 0
                data = []
                while len(data) != 7 or i > 10:
                    tcp.sendall(b"\x61\x03\x00\x20\x00\x01\x8c\x60")
                    time.sleep(0.05)
                    data = tcp.recv(self.bytes)
                    i += 1
                tcp.close()

50ms waiting time has solved the problem. The 10-fold white loop is only for additional security

answered Apr 1 at 8:17

Microscoop

155 bronze badges

3 Comments

user207421 Apr 1 at 9:11

The `sleep() is just literally a waste of time. What you have to do is concatenate into your receive buffer until you've received all the data you need, and preserve anything left over for next time.

Microscoop Apr 1 at 9:19

how do i do this? and of course, i hate sleep()

Remy Lebeau Apr 1 at 19:48

@Microscoop Why are you re-sending the command on each loop iteration? You should send the command one time, and then read in a loop until all expected bytes have arrived. Also, you are not preserving previously read bytes when calling recv() again to get more bytes

Collectives™ on Stack Overflow

socket.recv() response varies in length

2 Answers 2

2 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related