[nanomsg] Re: Framing - separation of multiple parts of the message
- From: Paul Colomiets <paul@xxxxxxxxxxxxxx>
- To: "nanomsg@xxxxxxxxxxxxx" <nanomsg@xxxxxxxxxxxxx>
- Date: Fri, 22 Nov 2013 20:28:43 +0200
Hi Steve,
On Fri, Nov 22, 2013 at 7:28 PM, Steve Atkins <steve@xxxxxxxxxxx> wrote:
>
> On Nov 22, 2013, at 9:19 AM, Paul Colomiets <paul@xxxxxxxxxxxxxx> wrote:
>
>> Hi Martin,
>>
>> On Fri, Nov 22, 2013 at 7:03 PM, Martin Sustrik <sustrik@xxxxxxxxxx> wrote:
>>>
>>> It would be a nice weekend project to implement 0MQ-style multi-part
>>> messages on top of nanomsg.
>>>
>>
>> I wouldn't encourage anybody to do the work. For the following reasons:
>>
>> 1. Multipart messages are too attractive to use for coarse-grained
>> protocol. But at the end of the day they suck.
>>
>> 2. I've used to think that's multipart are nice, because even simplest
>> C client can make use of them without complex dependencies. But full
>> msgpack parser take a few hours to implement (in C), and it's nice,
>> well known format, that is going to be standardized by IETF
>> (hopefully). It's even easier to implement subset of msgpack.
>>
>> 3. That "project" would need bindings to every language. And there are
>> already bindings for each and every language for msgpack (or put your
>> favourite serialization format name here).
>
> One place where it'd be useful is where you want to use it for routing
> information. Cracking open a big protobuf or msgpack payload just to find out
> where it should be sent to is a fair bit of overhead.
>
The subset of msgpack that parses a string of up to 32 bytes looks like:
if(*c & 0xE0 == 0xA0) {
memcpy(routing_data, c+1, *c & ~0xE0);
}
I think it's simple enough to use for routing (And it's not that hard
if you need more than 32 bytes too). You don't need full parser for
the purpose.
> The extreme of that is pub/sub - you can't send packed messages over
> pub/sub and use subscriptions without adding some sort of basic
> framing (or getting *way* too chummy with the packing format in use).
>
Yes. But you need different formats based on how do you want to match.
If you want full match for topic, you might be better served with
length-prefixed data, like above. But if you want prefix matches to
work, you need to put a delimiter at the end of the topic (or in some
cases you might end up without a delimiter at all).
Anyway if we ever need the topics to be matched by nanomsg, e.g. in
subscription matching plugin, it's easy enough to embed an msgpack
decoder. E.g. the one in nanoconfig is less than 500 lines of code, it
parses everything except strings > 4 Gb, and does no memory
allocations. So should be good enough for the purpose. (And I believe
we eventually embed nanoconfig anyway :) )
--
Paul
Other related posts:
