1

I am writing a program that works on two proto messages, I need to process the byte[] sent from different sources which sends either foo message or bar message. Since I cannot figure out which message it belongs to, I used Any Class (comes along with protobuf) to parse the byte array and find which class it belongs to, but met with a compile time error. Is there any other method that I can use to detect if I add more proto message classes in future?

//Foo.proto

syntax = "proto3";

option java_outer_classname = "FooProto";

message Foo {
    int32 a = 1;
}

and the second proto

//Bar.proto
syntax = "proto3";

option java_outer_classname = "BarProto";

message Bar {
    int32 b = 1;
}

Code:

Any anyEvent = Any.parseFrom(protoBytes);
if (any.is(Foo.class)
{
  Foo foo = any.unpack(Foo.class);
  ...
} else {
  Bar bar = any.unpack(Bar.class);
  ...
}

Error in if statement while trying to invoke any.is() :

The method is(Class< T>) in the type Any is not applicable for the arguments (Class< Foo>)

Improve this question
0

1 Answer 1

Reset to default
3

Any doesn't mean "any"; it means "a type serialized via Any". If you didn't store it with Any: you can't decode it via Any.

The key point here is that protobuf does not include type metadata in a message payload. If you have a BLOB, you usually can't know what the message type is. Any solves that by encoding the message type in a wrapper message, but you won't have that here.

If your design is to have an API that accepts two different non-Any message types without prior knowledge of which it is: you probably have a bad design. Because that doesn't work with protobuf. On the wire, there is literally no difference between a Foo with a=42 and a Bar with b=42; the payloads are identical:

Foo with a=42 is the bytes 08 2A; Bar with b=42 is the bytes 08 2A. The 08 means "field 1, encoded as a varint", the 2A is a varint with raw value 42.

A better design might be a wrapper message specific to your scenario:

message RootMessage {
  oneof test_oneof {
     Foo foo = 1;
     Bar bar = 2;
  }
}

This adds a wrapper layer similar to how Any works, but much more efficiently - it just knows how to distinguish between your known types as an integer, rather than having to handle every possible type (as a rooted type name).

Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

Thanks for the info about Any, The problem is the foo message is produced from a source already and the design cannot be changed. Is there any other solution for it?
@Santosh is it just the two types as shown? In that scenario I might add an enum field (field 2) to Bar with a default of 0=Foo,1=Bar, and include that value with 1=Bar on the new producer - decode everything as a Bar, but if the enum field says Foo, interpret it as a Foo (just read the b and pretend it was an a). This only works because the field type is identical, though; if Foo has an int32 a=1 and Bar has a string b=1, it won't work.
@santosh the key thing to remember is that field names don't matter in the binary protocol; they aren't sent, and aren't used to decode; only the numbers matter (and their field types)
The data types defined are different from what I mentioned in the question, the field types wont match making it hard to identify the class it belongs to.
@Santhosh yup, that makes it very hard; your best bet, then, may be to use the "reader" API to inspect the payloads and see what fields and wire-types it has, to see if you can manually identify which is which; ultimately, protobuf isn't well-suited to what you're trying to do here - it wants to know the target type in advance
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.