Go conversion between struct and byte array

Question

I am writing a client - server application in Go. I want to perform C-like type casting in Go.

E.g. in Go

type packet struct {
    opcode uint16
    data [1024]byte
}

var pkt1 packet
...
n, raddr, err := conn.ReadFromUDP(pkt1)  // error here

Also I want to perform C-like memcpy(), which will allow me to directly map the network byte stream received to a struct.

e.g. with above received pkt1

type file_info struct {
    file_size uint32       // 4 bytes
    file_name [1020]byte
}

var file file_info
if (pkt1.opcode == WRITE) {
    memcpy(&file, pkt1.data, 1024)
}

I'd recommend trying to write it in go first. You just wouldn't do anything like this in go. Among other things, go doesn't have casting. uint also isn't 4 bytes. Conn.Read takes a []byte which is effectively a smart, sized pointer into a backing array. You will have a much better time just writing this in go. — Dustin
– Dustin, Commented Oct 15, 2014 at 3:55
Have you already looked at built-in binary serialization with the encoding/gob package? — dyoo
– dyoo, Commented Oct 16, 2014 at 7:47

Ainar-G · Accepted Answer · 2014-10-15 09:16:54Z

20

unsafe.Pointer is, well, unsafe, and you don't actually need it here. Use encoding/binary package instead:

// Create a struct and write it.
t := T{A: 0xEEFFEEFF, B: 3.14}
buf := &bytes.Buffer{}
err := binary.Write(buf, binary.BigEndian, t)
if err != nil {
    panic(err)
}
fmt.Println(buf.Bytes())

// Read into an empty struct.
t = T{}
err = binary.Read(buf, binary.BigEndian, &t)
if err != nil {
    panic(err)
}
fmt.Printf("%x %f", t.A, t.B)

Playground

As you can see, it handles sizes and endianness quite neatly.

answered Oct 15, 2014 at 9:16

Ainar-G

36.5k14 gold badges103 silver badges125 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

deshkor · Accepted Answer · 2017-06-19 22:58:10Z

5

I've had the same problem and I solved it by using the "encoding/binary" package. Here's an example:

package main

import (
  "bytes"
  "fmt"
  "encoding/binary"
)

func main() {
  p := fmt.Println
  b := []byte{43, 1, 0}

  myStruct := MyStruct{}
  err := binary.Read(bytes.NewBuffer(b[:]), binary.BigEndian, &myStruct)

  if err != nil {
    panic(err)
  }

  p(myStruct)
}

type MyStruct struct {
  Num uint8
  Num2 uint16
}

Here's the running example: https://play.golang.org/p/Q3LjaAWDMh

answered Jun 19, 2017 at 22:58

deshkor

511 silver badge2 bronze badges

Comments

Shriganesh Shintre · Accepted Answer · 2014-10-21 22:13:20Z

Thank you for answers and I am sure they work perfectly. But in my case I was more interested in parsing the []byte buffer received as network packet. I used following method to parse the buffer.

var data []byte // holds the network packet received
opcode := binary.BigEndian.Uint16(data) // this will get first 2 bytes to be interpreted as uint16 number
raw_data := data[2:len(data)] // this will copy rest of the raw data in to raw_data byte stream

While constructing a []byte stream from a struct, you can use following method

type packet struct {
    opcode uint16
    blk_no uint16
    data   string
}
pkt := packet{opcode: 2, blk_no: 1, data: "testing"}
var buf []byte = make([]byte, 50) // make sure the data string is less than 46 bytes
offset := 0
binary.BigEndian.PutUint16(buf[offset:], pkt.opcode)
offset = offset + 2
binary.BigEndian.PutUint16(buf[offset:], pkt.blk_no)
offset = offset + 2
bytes_copied := copy(buf[offset:], pkt.data)

I hope this gives general idea about how to convert []byte stream to struct and struct back to []byte stream.

OneOfOne · Accepted Answer · 2014-10-15 00:16:13Z

You'd have to use unsafe, also uint is 8 bytes on 64bit systems, you have to use uint32 if you want 4 bytes.

It's ugly, unsafe and you have to handle endianess yourself.

type packet struct {
    opcode uint16
    data   [1022]byte
}

type file_info struct {
    file_size uint32     // 4 bytes
    file_name [1018]byte //this struct has to fit in packet.data
}

func makeData() []byte {
    fi := file_info{file_size: 1 << 20}
    copy(fi.file_name[:], []byte("test.x64"))
    p := packet{
        opcode: 1,
        data:   *(*[1022]byte)(unsafe.Pointer(&fi)),
    }
    mem := *(*[1022]byte)(unsafe.Pointer(&p))
    return mem[:]
}

func main() {
    data := makeData()
    fmt.Println(data)
    p := (*packet)(unsafe.Pointer(&data[0]))
    if p.opcode == 1 {
        fi := (*file_info)(unsafe.Pointer(&p.data[0]))
        fmt.Println(fi.file_size, string(fi.file_name[:8]))
    }
}

play

Collectives™ on Stack Overflow

Go conversion between struct and byte array

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related