Parsing binary data in Ruby

Question

We're doing a major rewrite of a project previously written in C into Ruby. We have a bunch of C structures, written as C typedefs:

struct my_struct {
    uint32_t foo;
    uint8_t bar;
    char baz[80];
}

Is there a quick way to load them all up in Ruby? For example, is there some way to convert these definitions into something that resembles code like

@foo = io.read(4).unpack('V')[0]
@bar = io.read(1).unpack('C')[0]
@baz = io.read(80)

There are literally tons of it, I'd rather not convert them by hand...

Can you describe where this data is stored and what format it will be in? It sounds like it might be stored as binary in a file but without more information on delimiters and file structure its hard to give a full answer. — Tyler Ferraro
– Tyler Ferraro, Commented Mar 11, 2016 at 20:30
@TylerFerraro The data is either sent across the network (and thus resides in memory when it's time to parse it) or read/written in local disc files. — Maigio
– Maigio, Commented Mar 11, 2016 at 21:19
@theTinMan I've described what I've done so far — I'm convert these parsers by hand, i.e. rewriting C data structures in a way I've shown above. Of course, I've searched Google, etc, done all the basic scouting, but found no answer. Please advice how can I improve the question? — Maigio
– Maigio, Commented Mar 11, 2016 at 21:21
This doesn't answer your question but I wanted to mention that the code you've given could be more simply expressed as @foo, @bar, @baz = io.read(85).unpack('VCA*'). — Jordan Running
– Jordan Running, Commented Mar 11, 2016 at 22:46
This is not very quick but I would first use cast, then iterate over the nodes to print the BinData lines... — rdupz
– rdupz, Commented Mar 24, 2016 at 9:25

Alex · Accepted Answer · 2016-03-24 08:52:11Z

2

Use BinData.

class MyStruct < BinData::Record
    endian :little
    uint32 :foo
    uint8  :bar
    string :baz, read_length: 80
end

It's pretty much a 1:1 mapping between C structs and BinData Records. It should be easy to write a conversion script.

answered Mar 24, 2016 at 8:52

Alex

211 bronze badge

Sign up to request clarification or add additional context in comments.

Comments

Daniel · Accepted Answer · 2016-03-12 02:09:18Z

0

If you have tons of them, you may want to write a quick program to parse the C code for you :)

Then the parsing code could create the corresponding ruby code from the C code.

answered Mar 12, 2016 at 2:09

Daniel

7,2027 gold badges46 silver badges51 bronze badges

2 Comments

Maigio Over a year ago

I'm not very good at programming parsers, it's always been a major space in my CS knowledge, so I'd rather not. If I must, could you recommend some good books on writing parsers then?

Daniel Over a year ago

I wouldn't go all the way in writing a C level parser, just write enough to recognize a struct { } and convert it. The goal here is to write throw-away code that will convert something for us once. So, it doens't matter how terrible it is... to some point :)

Collectives™ on Stack Overflow

Parsing binary data in Ruby

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related