You were mentioning that you were planning on adding statistics gathering code;
have you considered adding code that gathers performance statistics as well?
As end users we could send you bug reports that include those statistics, which
would let you know if it’s a problem or not.
Thanks,
Cem Karan
-----Original Message-----
From: nanomsg-bounce@xxxxxxxxxxxxx [mailto:nanomsg-bounce@xxxxxxxxxxxxx] On ;
Behalf Of Garrett D'Amore
Sent: Thursday, January 05, 2017 1:37 PM
To: nanomsg@xxxxxxxxxxxxx
Subject: [nanomsg] Re: [Non-DoD Source] status update libnng
All active links contained in this email were disabled. Please verify the
identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address to a
Web browser.
________________________________
I could certainly make it an array. :-)
If it seems to be universal that nobody needs more than a few hundred of
these things, I’ll switch to an array.
I wasn’t so much worried about list traversal times as I was about the O(n)
vs O(log(n)) thing though. If we know that O(n) is good
enough, then certainly an array is probably going to be superior. One thing
about an array, is that if I keep it sorted, then subscribe
operations will become more expensive, since I’ll have to shuffle on average
half the items in the array. I’m assuming subscription
changes occur less frequently than messages arrive. (I’d really like to know
if anyone has an application where changes to subscriptions —
subscribe/unsubscribe — is a performance sensitive operation.)
- Garrett
On Thu, Jan 5, 2017 at 10:23 AM, Daniel C <danielcccc@xxxxxxxxx <
Caution-mailto:danielcccc@xxxxxxxxx ;> > wrote:
I agree with Jason; an array would be faster. N definitely less than 10
or 20 in my case.
Dan
On Thu, Jan 5, 2017 at 7:58 AM, Jason E. Aten <j.e.aten@xxxxxxxxx <
Caution-mailto:j.e.aten@xxxxxxxxx ;> > wrote:
For a small N < 10, I would expect an array to be much faster
than a linked list.
My typical use is for small N, about 3.
On Thu, Jan 5, 2017 at 4:52 AM, Garrett D'Amore
<garrett@xxxxxxxxxx < Caution-mailto:garrett@xxxxxxxxxx ;> > wrote:
TCP works now. OMG, close() vs. accept() is such a
non-portable nightmare. I almost went down the path of
pthread cancellation. Thankfully, that crisis is now averted. I never
realized just how broken the use of UNIX file descriptors in the face of
close() and multiple threads really are. But its working now . Thank
goodness for shutdown(2) and dup2().
There’s a test suite for TCP and inproc, and I’ve
started PUB/SUB. Sub is actually done, untested.
I give 50/50 odds on me finishing up the remaining
protocols and IPC before the weekend.
What won’t happen before the weekend is writing the
eventing framework (necessary for file descriptor based
notifications for example), or statistics framework. And Windows will
probably happen next week. I’ve designed this such that I think I
will be able to crank out Windows stuff super fast.
One question about PUB/SUB. I’ve for now used a simple
sorted linked list for subscriptions. My instinct is that
for the vast vast majority of folks, this is not only sufficient, but
probably superior to a patricia tree; I think most of us only maintain
around a dozen or so subscriptions on any subscriber at any given time. That
said, I’m keen to hear of actual uses of nanomsg where the
patricia tree makes a real difference — cases with over several hundred
active subscriptions on a given end node. (Note that this is only
on the the end node; a server can service many many clients — thousands —
each having dozens of subscriptions — with zero negative
impact in my design.)
To be clear, the linked list means that filtering
messages is now an O(n) instead of O(log(n)) operation (where n is
the number of active subscriptions); I suspect that for small values of n,
the linked list approach I’ve taken is faster. I have implemented
this way for now for expediency, but I’m open to pulling in a patricia tree
if there is need. (I didn’t yank Martin’s old code because I want
to make sure I thoroughly understand it before I bring it into the new code
base — I haven’t had time to be certain of that yet.
On Wed, Jan 4, 2017 at 2:18 AM, Garrett D'Amore
<garrett@xxxxxxxxxx < Caution-mailto:garrett@xxxxxxxxxx ;>
wrote:
I’ve just committed the initial swag at TCP.
Turns out it was bit more code than anticipated, as I want to
support TCPv4 and TCPv6, and properly support different platforms that might
have different ways to resolve names, or handle low level
TCP details. This should make it really fast to write the winsock code later.
I’ve also tried to abstract the details so that
transports that want to build on top of TCP (e.g. websocket
or TLS) can do so easily.
Totally untested, but you can look at the last
commit if you want to see what I’m up to. The main missing
thing (besides testing that it actually works) is support for some of the
richer TCP options — e.g. KEEPALIVEs etc. I am disabling Nagle by
default, because I think 99% of nanomsg users have no business wanting Nagle
turned on. (And I’ve taken care to use writev to avoid
splitting writes up across separate packets when possible. This is something
that sadly isn’t possible using Go….)
On Tue, Jan 3, 2017 at 10:54 AM, Karan, Cem F
CIV USARMY RDECOM ARL (US)
<cem.f.karan.civ@xxxxxxxx < Caution-mailto:cem.f.karan.civ@xxxxxxxx ;> > wrote:
Got it, my brain glitched and forgot
that inproc is a thing; for some reason I thought you were
building out your own TCP stack, which would have made me... concerned. ;)
Thanks,
Cem Karan
> -----Original Message-----
> From: nanomsg-bounce@xxxxxxxxxxxxx <
Caution-mailto:nanomsg-bounce@xxxxxxxxxxxxx ;>
[Caution-mailto:nanomsg-bounce@xxxxxxxxxxxxx ;<
Caution-mailto:nanomsg-bounce@xxxxxxxxxxxxx ;> ] On Behalf Of Garrett D'Amore
> Sent: Tuesday, January 03, 2017 1:23
PM
> To: nanomsg@xxxxxxxxxxxxx <
Caution-mailto:nanomsg@xxxxxxxxxxxxx ;>
> Subject: [nanomsg] Re: [Non-DoD
Source] status update libnng
>
> All active links contained in this
email were disabled. Please verify the identity of the sender,
and confirm the authenticity of all links
> contained within the message prior to
copying and pasting the address to a Web browser.
>
>
> ________________________________
>
>
>
> I’m about 24 hours from having TCP
working. Right now the TCP transport is not written,
though I’ve written a fair bit of it locally.
>
> The REQ/REP pattern is working now,
as is PAIR, and inproc is working quit well and totally
thread safe.
>
> As far as lockless algorithms go —
I’m a big believer in mutexes. I need mutexes and condition
variables to make things work well, and in
> my experience if they are used
sensibly (with minimal contention), they do not significantly
impact performance.
>
> For some things lockless data
structures are superior, but you have to have use cases that work
for them; with mutexes and condvars I
> don’t have to think about this; they
Just Work.
>
> It may be useful to look at
alternative designs for that in the future, but I’m only going to do
that once I’ve demonstrated a clear need or
> benefit (or maybe someone else will
contribute at that point). Until then, I need to focus on
getting the code working with a sensible
> design and avoid prematurely
optimizing things.
>
> - Garrett
>
>
> On Tue, Jan 3, 2017 at 6:45 AM,
Karan, Cem F CIV USARMY RDECOM ARL (US)
<cem.f.karan.civ@xxxxxxxx < Caution-mailto:cem.f.karan.civ@xxxxxxxx ;> <
Caution-
>
Caution-mailto:cem.f.karan.civ@xxxxxxxx ;<
Caution-mailto:cem.f.karan.civ@xxxxxxxx ;> > >
wrote:
>
>
> > -----Original Message-----
> > From:
nanomsg-bounce@xxxxxxxxxxxxx < Caution-mailto:nanomsg-bounce@xxxxxxxxxxxxx ;>
< Caution-Caution-mailto:nanomsg-bounce@xxxxxxxxxxxxx ;<
Caution-mailto:nanomsg-bounce@xxxxxxxxxxxxx ;> > [Caution-Caution-
mailto:nanomsg- ;< Caution-mailto:nanomsg- ;>
> bounce@xxxxxxxxxxxxx <
Caution-mailto:bounce@xxxxxxxxxxxxx ;> < Caution-Caution-
mailto:nanomsg-bounce@xxxxxxxxxxxxx ;<
Caution-mailto:nanomsg-bounce@xxxxxxxxxxxxx ;> > ] On Behalf Of Garrett
D'Amore
> > Sent: Monday, December 26,
2016 2:46 AM
> > To: nanomsg@xxxxxxxxxxxxx <
Caution-mailto:nanomsg@xxxxxxxxxxxxx ;> < Caution-Caution-
mailto:nanomsg@xxxxxxxxxxxxx ;< Caution-mailto:nanomsg@xxxxxxxxxxxxx ;> >
> > Subject: [Non-DoD Source]
[nanomsg] status update libnng
> >
> > Just a brief Christmas
(western) update…
> >
> > libnng is now managing
connections. There’s a lot more to this than you’d think — and a
lot of details that I’ve written under
> the hood.
> >
> > I expect that by New Years
I’ll have it in a functional state for TCP for POSIX systems, for
at least the primary patterns /
> protocols. TBH,
> > I’m really only a couple of
hours from that point.
> >
> > Note that the current code
base is pretty much crash-immune with respect to things that
caused grief in libnanomsg — e.g.
> ENOMEM
> > errors will not impact
libnng. The whole approach has been from a “correct first”, so I
expect we will have fewer problems once
> we
> > actually start using this.
> >
> > Things are moving apace…
quite quickly actually.
> >
> > If anyone is so inclined to
review or give feedback on teh existing code, I’m now at the
point to receive it, understanding that
> there are
> > still large swathes still
unimplemented. You can’t use it for TCP for example. But at this
point I’d rather solicit feedback early
> rather than
> > late. Don’t tell me what’s
missing, but if you see problems with what’s already there,
please *do* let me know.
> >
> > I’m going on a ski trip for
the next several days, but will probably work in the evenings
getting the thing to a state where other
> folks can
> > start playing with for actual
experimentation.
> >
> > I’m really looking forward to
benchmarking this thing. I think we’re gonna blow the doors
off libnanomsg, at least for platforms
> that have
> > non-crappy pthreads. :-)
> >
> > Merry Christmas everyone! :-)
>
> Thank you for all your hard
work!
>
> BTW, when you say that you
can't use it for TCP, what exactly do you mean?
>
> Thanks,
> Cem Karan
>
>
>
Attachment:
smime.p7s
Description: S/MIME cryptographic signature
