[nanomsg] Re: end-to-end security

  • From: Garrett D'Amore <garrett@xxxxxxxxxx>
  • To: Drew Crawford <drew@xxxxxxxxxxxxxxxxxx>, nanomsg@xxxxxxxxxxxxx
  • Date: Tue, 11 Mar 2014 15:56:28 -0700

Thanks for the references.  I’ll look at them with an open mind.  I’m not at 
all in love with TLS.  I just have a natural skepticism towards unreviewed 
crypto — there are far too many people pushing snakeoil in this space — and 
probably the vast majority of them are doing so out of just plain ignorance or 
laziness rather than any malicious intent.  Often times in the name of 
“expediency”.  (Expediency is what created the abortion that was WEP.)

It sounds like your intended direction is to insert a kind of “shim” in the 
protocol between the core of nanomsg, and the application.  That should be easy 
enough to do, whether as an unbundled library, or as a modification.  I think 
I’d prefer to see a mechanism by which this could be plugged into nanomsg 
optionally, rather than builtin to core, sort of like an “encode” or “encrypt” 
and “decode/decrypt” callbacks that could be registered on the socket.  That 
said others on the list have indicated a preference for avoidance of expansion 
of nanomsg’s API.

Reading the caffeine-crypto README, it seems like you’ve at least considered 
the things I’m most worried about, so I feel better.  That said, the use of 
DJB’s algorithms (crypto) concerns me a little bit, mostly because I’ve not 
heard of them, so I’m not sure how comprehensive the review and cryptanalysis 
of them has been.  That said, DJB can hardly be considered one of your usual 
snake-oil salesmen.  The other thing is that use of his crypto automatically 
precludes use in applications that involve federal funding —only NIST-approved 
algorithms are permitted by FIPS (specifically 140-2).  So I’d prefer not to 
see the crypto algorithms “hard-coded”.  (This goes for all of them — 
asymmetric, symmetric, and hash.)

The libcurve approach doesn’t seem entirely insane either — but I’ve hardly had 
time to dive into it.  I’d feel better if the wire protocol were documented 
somewhere easily.

Garrett D'Amore
Sent with Airmail

On March 11, 2014 at 2:46:37 PM, Drew Crawford (drew@xxxxxxxxxxxxxxxxxx) wrote:

What’s the latency problem?  

The caffeinecrypto readme includes an extended discussion on this point, see 

If its key negotiation (basically the SSL handshake), then your protocol is 
going to have either the same problem, or you’re going to have weaknesses by 
not covering all the cases that TLS does.  Strength crypto (key agreement) and 
mutual authentication kind of play at odds with latency.  

I opened with “TLS isn’t a good fit for my problem” and your response is that 
whatever I am planning to does not address TLS usecases.  Yes.  That’s the 
*point*.  The *point* is to make different tradeoffs than TLS does.  My whole 
premise is that some applications may be satisfied to trade one thing for 
another.  If you are interested in what kinds of things people want to trade 
off and why, the links below are a good jumping off point.
Maybe you should start with an application layer on top of nanomsg, then post 
it.  If it makes sense, I suppose people can consider something more tightly 
bound to nanomsg itself.  

So, that has been done, at least twice that I know of, in the past year.

Here was my approach: https://github.com/drewcrawford/caffeine-crypto, although 
in its present form it isn’t completely portable.

Also there was Pieter Hintjen’s libcuve: https://github.com/zeromq/libcurve

There are probably even more approaches that I don’t know about, but those are 
some actual, go-poke-and-prod-at-them implementations.

Just as a disclaimer, there are some issues with caffeinecrypto that I would 
want to revisit (and there are some things I would change about libcurve as 
well) so I’m not out here saying that they are bulletproof and good-to-merge as 
they sit.  What I am saying is that the general approach of doing a 
Bernstein-derived authentication handshake is solid, is deployed in stable 
software, is a proven solution for this problem, and something in this search 
space is appropriate to be merged.

These approaches are different in some details, but they are both deliberately 
not guarantee-for-guarantee TLS replacements, they are both based on real 
cryptographic work by real cryptographers, they are both used in message-queue 
systems today and they are both motivated by annoyance with TLS.

A libcurve-derived system is what eventually made it into ZeroMQ.  Whether that 
was the right move is up for debate, but I think it is notable that zeromq 
considered TLS for years and instead launched an entire project to avoid it.  
Again I’m not saying TLS is bad, I’m saying it’s bad for me, and for a lot of 
other people, and something should be done in core, and I am willing to do it.

The problem is that there aren’t any *good* lightweight C libraries for TLS.  
OpenSSL is kind of all there is, for better or worse.

Worth pointing out here that both of the examples I’ve linked to are 
MIT-licenseable, so there is no licensing problem with moving them to core.

I want to see the protocol.  How will you prevent replay attacks?  What is your 
key management scheme.  You *can’t* just blithely encrypt using a pre-shared 

I think all the “how can anyone be secure if not with TLS” type questions can 
be settled one way or another fairly quickly by consulting the two concrete 
examples I’ve linked.  Every point you’ve raised so far is addressed there.  
Whether that treatment is convincing is a different question, but from my 
keyboard, if the discussion about replay attacks etc. on neither of those 
projects is satisfactory, then no protocol can meet the burden except for TLS.

There are, of course, some actual weaknesses with Bernstein-derived 
authentication schemes.  One is that there is no chain-of-trust, so certificate 
pinning is used instead.  In my view that decision in particular is a feature, 
not a bug, but I understand that intelligent people disagree about such 
matters.  It is with that kind of issue that I appeal to “different tradeoffs 
are appropriate for different applications” type arguments.

On Mar 11, 2014, at 3:40 PM, Garrett D'Amore <garrett@xxxxxxxxxx> wrote:

On March 11, 2014 at 1:06:32 PM, Drew Crawford (drew@xxxxxxxxxxxxxxxxxx) wrote:

I feel the need to say, that it is not my intention to argue that my proposal 
is the very best proposal for everybody and should be adopted to the exclusion 
of every other proposal.  I think this is a place where zeromq has stumbled, by 
taking a very long time to study all the options and eventually producing a 
system pretty similar to one I suggested and landed privately ages ago.

My view is that there is no one-size-fits-all solution.  There are tradeoffs in 
security, performance, automatic-ness, etc., that make sense (or not) for 
different classes of applications.

Really, my intention is to argue that my proposal is a local maxima for a 
certain class of applications, and the user could choose from different 
side-by-side schemes that might be better for different types of applications, 
as the community has motivation to produce them.  Here, someone (me!) has the 
motivation to produce one scheme.  Some encryption is better than no 
encryption, and it makes the road easier, not harder, to land an alternative 
solution in the future.

At the same time I agree that there are some standards for what should be 
considered “merge-grade”.  Things that are obviously broken should not be 
considered, nor things where a better scheme already handles the case in every 
dimension.  But, for reasons that will become clear, I think it is much more 
dangerous to do nothing than to accept a competent patch that is limited in 

Which is why using a protocol that has already been through this — like TLS — 
is so attractive.

I concede that TLS is both safer and quite a lot simpler than implementing what 
I’m proposing.  However for reasons you can find on my post to this list 
several months ago, it’s not a good fit for my class of applications (there’s a 
latency problem).  I support a TLS-based scheme if someone is volunteering to 
write one side-by-side but at the moment I don’t have a use case for it.

What’s the latency problem?  If its key negotiation (basically the SSL 
handshake), then your protocol is going to have either the same problem, or 
you’re going to have weaknesses by not covering all the cases that TLS does.  
Strength crypto (key agreement) and mutual authentication kind of play at odds 
with latency.  That said, if you can separate the assymmetric key negotiation 
from the  symmetric encryption data path, then once the connection is 
established you’re golden.  TLS already does that if you reuse the connection.  
If you don’t reuse, you’re hosed.

 Compared with layering on top of secure transport, using a secure transport 
seems *lots* easier.

It is most certainly easier, for things like TLS and for certain network 
topologies.  However, the easy solution is not a good fit for my usecase.  I’ve 
described the architecture problems in zeromq’s transport-layer solution that 
impede meaningful work getting done by new contributors, and nanomsg will 
inherit some of those problems. Transport layer also complicates the multihop 
case, which is very important for my set of problems.  None of this is to say 
that transport-layer is bad all the time; I would not oppose such a scheme.  
It’s mostly that I do not particularly want to tackle these problems, while 
there is in the alternative a solution that fits my situation better.

It sounds to me like you might have an application specification problem.  
Maybe you should start with an application layer on top of nanomsg, then post 
it.  If it makes sense, I suppose people can consider something more tightly 
bound to nanomsg itself.  

For me, I’d like to see a protocol level description of what you’re talking 
about.  I want to ensure that the *protocol* is sound, and secure.  While I 
don’t necessarily consider myself fully qualified as a reviewer there, I do 
consider myself *more* qualified than myriads of fools that I’ve seen try to 
implement cryptography/security naively.  I expect I can spot some of the more 
gaping holes (replay attacks, man-in-the-middle weaknesses, etc.)

Really though, the situation is this.  I’ve volunteered to land an end-to-end 
scheme soon even though it is hard.  To my knowledge, nobody has volunteered to 
land a transport scheme in any timeframe even though it is easy.  I suggest, 
given these facts, that it does not really matter whether some solution is easy 
or hard.  What matters is what patches can be written and what patches can be 

I’m planning on implementing TLS in my Go library, on top of nanomsg.  I’ll 
probably wind up adding it to a fork of nanomsg itself as well.  However the 
crosslinkage between OpenSSL (GnuTLS is out due to licensing) is possibly going 
to preclude it from being mainstream.  The problem is that there aren’t any 
*good* lightweight C libraries for TLS.  OpenSSL is kind of all there is, for 
better or worse.  And I don’t feel like implementing TLS by hand myself. :-)

So far, an end-to-end scheme can be written, and it is an open question whether 
or not it can be merged.  You are saying that a transport scheme can be merged, 
but I have not yet seen any evidence that one will be written.  

Its not hard to write one.  nanomsg has a pretty good abstraction layer.  As I 
said, I’m implementing nanomsg’s SP protocols in Go, and once I do, adding TLS 
support should be a matter of at most an hour or so.  Its more effort in C 
because you have to deal with OpenSSL *cough*, but that shouldn’t deter a 
serious effort given need.  Probably instead of an hour or two its more like a 
day or two.  I expect the Go implementation of what I’m talking is more than a 
few days away, but also less than a month, from being ready for folks to start 
looking at and playing with.  It will be wire compatible with nanomsg, but will 
add tls and websocket (both ws and wss) transports (experimental).

And again to emphasize, these two paths are not necessarily mutually 
exclusive—there is also the possibility of both schemes, or of no scheme at 
all.  The latter is I think the worst of all worlds, and I am concerned that 
nanomsg will follow zeromq’s lead in that regard.

Of course.  But you can of course implement end-to-end security in your 
application.  At some level, if you’re concerned, that’s the cleanest solution. 
 But it does require that *you* know what you’re doing.  If you don’t… well 
then your stuff wouldn’t be appropriate merging either.

I’d like to see something merged; I recall about 2 years ago a colleague of 
mine wanted to implement his own cryptographic layer on top of pub/sub sockets 
(zmq, before zmq added SSL).  It was swiss cheese full of holes.  I tried to 
convince him to skip it, and just use IPsec instead. Ultimately that’s what we 
did, although I had to rip out all his crummy attempts at crypto security.  It 
cost me a lot of extra time, and wasted energy trying to get him to understand 
that “encryption != security”.

It would have been better to have a solid implementation that folks like him 
could just *use*, instead of having everyone invent their own shoddy security 

I would also like to make sure that any new protocol support is designed in a 
way that makes it possible to support the other protocol topologies (bus, 
pub/sub, etc.)  I think that’s a bit non-trivial, as you then are talking about 
multicast topologies.

I have thought about the pub/sub problem in a little bit of depth.  The problem 
is that there is quite a cornucopia of different solutions:

Pre-sharing a session key
Time-based or N-many-messages-based key rotation
Bootstrapping keys from a REQ/REP key server
Exploit subscriptions / unsubscription commands to rotate keys
Just encrypting the message N times

Each of these schemes has pros and cons and might make sense for a particular 
class of applications.  There is no one-size-fits-all solution.

I am confident that someone with sufficient real-world motivation to tackle the 
pub/sub problem would find one of these ideas compelling for their use case, or 
be able to generate better ones.  However I think it would be unwise to make 
decisions for PUB/SUB without someone who has a real use-case at the table 
making comments.  I also think it is unwise to delay a solution for REQ/REP 
until the question of what other sockets will do has been sorted out, 
particularly if nobody with a clear need for securing those sockets has in 
several months inserted themselves into the conversation to describe their 

I think it would be unwise to build into the core a solution that only works 
for REQ/REP.  I’d rather see this properly designed, than something cobbled 
together that won’t support the main patterns.   As I indicated, the one case 
I’ve seen for this in the real world was actually PUB/SUB, and it was badly 
botched by the person who tried to solve it.

Using a PKI, the problem of PUB/SUB is much more easily solveable, although it 
only addresses the security of the transport itself, and not end-end 

The trouble with waiting for consensus for other use cases is that you never 
really get all the people in the room at the same time.  When you turn away a 
concrete REQ/REP proposal due to lack of addressing PUB/SUB, that 
user/contributor finds some other way to solve his problem and doesn’t become a 
user.  And then perhaps a sufficiently-motivated PUB/SUB user does finally 
appear, but by then the REQ/REP implementor has moved on, so the PUBSUB 
proposal is now rejected due to lack of REQ/REP, and around and around it goes. 
 And this is basically the story of how libzmq went without any encryption at 
all for 3 major releases, although there was plenty of bikeshedding throughout.

I’m not talking about bikeshedding.  Indeed, most commenters probably aren’t 
qualified to judge a security protocol.  But I am saying that it would be 
*insane* to add a security layer to nanomsg without a review of the protocol 
itself, ideally from qualified security experts.   Adding a busted security 
layer to nanomsg would IMO be far worse than adding none at all.  (I cite WEP 
as an example here.)

If you want to push something quickly and can’t wait for such a review, I think 
you should proceed with application layer work.

They finally converged on a solution, but the fact that I am here volunteering 
to write patches for nanomsg is evidence that even after thinking about it for 
3 years they did not come up with something that works for everybody.  And I am 
convinced that waiting for a solution that pleases everybody is waiting for 
Godot.  No such solution will ever present itself, because it does not exist.

You’d also have to figure out some questions such as — devices — if you’re 
going to route requests, you need to be able to look at least at the envelope.  
That requires that at least envelopes are unencrypted.   

What I am proposing is more-or-less applying the encryption immediately after 
the call to nn_send, to nn_send’s argument.  So this doesn’t touch anything 
below that layer.  Specifically, it wouldn’t encrypt the envelope.

I want to see the protocol.  How will you prevent replay attacks?  What is your 
key management scheme.  You *can’t* just blithely encrypt using a pre-shared 
key and be secure.  You may gain *secrecy*, but that is *all* you will gain.

- Garrett

Other related posts: