So another transport layer built on top of UDP then. :-)
I still need lower level UDP for other reasons.
On Tue, Jul 18, 2017 at 12:52 PM Jason E. Aten <j.e.aten@xxxxxxxxx> wrote:
For UDP based stuff, I would suggest checking out the KCP protocol. It has
mature libraries for C and Go:
https://github.com/skywind3000/kcp/blob/master/README.en.md
https://github.com/xtaci/kcp-go
KCP is a fast and reliable protocol that can achieve the transmissioneffect of a reduction of the average latency by 30% to 40% and reduction of
the maximum delay by a factor of three, at the cost of 10% to 20% more
bandwidth wasted than TCP.
For example, an rpc lib called rpcx for Go is based on kcp, and is more
performant by far than most other popular rpc frameworks. (
https://github.com/smallnest/rpcx in particular
https://github.com/smallnest/rpcx/blob/master/_documents/throughput.png)
On Tue, Jul 18, 2017 at 1:25 PM, Garrett D'Amore <garrett@xxxxxxxxxx>
wrote:
Ok, that makes a *lot* more sense to me. So we could probably invent an
SCTP transport for nanomsg, for sure (or more likely for libnng). The
challenge *might* be that getting at raw IP is probably one of the less
portable things you can deal with. That said I’ve yet to take a long hard
look at it.
I’m about to start adding low-level UDP stuff (needed for another
transport), and this makes me wonder if instead of SCTP maybe QUIC has some
merit?
I’m pretty sure now that quic-go exists, I could build a QUIC
implementation of mangos pretty quickly.
It would probably a little harder to do the same in C, since the only
non-go implementation I’m aware of is inside Chromium…
On Tue, Jul 18, 2017 at 12:16 PM omid shahraki <dreamstechgroup@xxxxxxxxx>
wrote:
Thanks for confusion clarification.
That is right.
On Jul 18, 2017 23:40, "George Walker" <georgewalkeriv@xxxxxxxxx> wrote:
I think I see a point of confusion:
"for that reason we have implemented a SCTP over TCP/IP Raw to see
performance."
Reading between the lines (and noting that SCTP over TCP doesn't seem
to provide any advantages over SCTP over IP), I think Omid is probably
talking about a userspace SCTP implementation over raw sockets (which are
IP, but forgo TCP).
George
Sent from my iPhone
On Jul 18, 2017, at 3:01 PM, omid shahraki <dreamstechgroup@xxxxxxxxx>
wrote:
Thanks for quick reply. I do not want to interrupt your work progress
and focusing on development as it is so important right now. But, just to
make some adjustments and clarifications:
Concerning SCTP, there are limitations by TCP as you already know, such
as message framing, endpoint interface selection, associations, chunk
multiplexing and 4 way handshaking against attacks, just a few to mention,
which are in great importance. As you said, such a transport could be
developed, certainly and we hope you have plan for it.
Concerning SCTP over TCP ( actually protocol stack) , as far as I know
this protocol has not been supported on all OSes and for that reason we
have implemented a SCTP over TCP/IP Raw to see performance. Overally, it is
acceptable and we would like to see it in your great library.
Concerning exceptions, it is not a C stuff as you said. By exceptions,
I mean an exception handling engine which could take the role of C++
exceptions handling in C. In every project, one of the important things
that has to be taken into consideration from early steps, is exception
handling.
By having a portable exception handling engine, we would disable
exception handling in C++ either and use our memory optimized and high
performance exception handling engine which is portable and supported on
OSes which do not have exception handling mechanism.
This is just to make it more clear if there were doubtful stuff leading
to misunderstanding.
Thanks for your time and efforts.
------------------------------------------------------------------
On Jul 18, 2017 19:37, "Garrett D'Amore" <garrett@xxxxxxxxxx> wrote:
Thanks for your interest in libnng. The AIO design is intended to be
portable to pretty much anything; that said there is a lower level
portability layer, so some work will be necessary to adjust when porting to
new systems.
Today nanomsg does not support SCTP at all. Such a transport could be
developed, certainly. It would be good to understand the design goals for
such a thing. I am not sure I see any clear benefit to running SCTP over
TCP for example.
Nanomsg and libnng are intended to operate with their own sockets as
they have their own handshaking protocol. I’ve been approached about
breaking the AIO framework out to support other kinds of sockets as a
general purpose async I/O framework, and I may do that. (Others will point
to libuv, libev, etc. I had specific reasons for not using any of those
frameworks, although these days I wonder if I shouldn’t have just used
libuv.
Exceptions are a C++ thing, and as libnng and libnanomsg are written in
C, they don’t use or generate exceptions.
- Garrett
On Tue, Jul 18, 2017 at 6:24 AM omid shahraki <
dreamstechgroup@xxxxxxxxx> wrote:
Hi Garrett:
It is really nice to see such an effort and faithful representation
which is appreciated and could result in development of products and
services for the sake of society welfare. We also appreciate your sponsor,
Capitar IT Group, and any individual / enterprise entity in supporting
nanomsg in advance.
We, AMS Inc. Enterprise Systems and Solutions, are thinking about
nanomsg as one of the candidates for NebuNox, Capital Markets Platform,
which has been developing and the ROMAN sub-system is going to be
presented
in 2018 Insurance, Stock and Banking Fair in Iran.
We would like to know if:
1- nanomsg AIO is going to be portable to different operating systems
without any changes and modifications (portable reactor/proactor pattern
with concurrency models)?
2- nanomsg is going to support SCTP protocol even on OSes not
currently supporting that (SCTP over TCP)?
3- nanomsg fully supports standard sockets besides its specialized
ones?
4- nanomsg supports exception handling by its core?
Regards,
Omid Shahraki
On Jul 18, 2017 02:49, "Garrett D'Amore" <garrett@xxxxxxxxxx> wrote:
Hi nanomsg fans,
I have some excellent news to report. I’ve finally gotten to the
point that I feel the major rework of the guts of libnng’s core are ready
to be merged into master. This replaces the entire thread-based
architecture with a fully asynchronous model, using either IO completion
ports or non-blocking I/O and poll (and in the future kqueue/epoll, etc.)
as appropriate for the platform.
Accordingly, I’ve merged the asyncpolled branch into master. (The
pre-merge state was saved as a tag called “threaded”, in case anyone
wants
to compare.)
The code now passes all tests consistently, including a new
“scalability” test that opens up 2000 sockets (and does a REQ/REP
exchange
on them) without too much difficulty.
None of this work would have been possible without the very generous
sponsorship of Capitar IT Group; they’ve sponsored this work, and are
funding further enhancements of libnng with an intent to invest further
in
the nanomsg community. Please join me in thanking them for their
sponsorship and commitment to open source development!
There’s a bit of performance work still to be done, as there are many
opportunities to enhance the stack, and there are still some big
unimplemented areas — for example WebSockets are not implemented, and a
number of particular option setting things are missing as are statistics
reporting. Only the performance stuff should have a noticeable impact on
the core (and that won’t change any APIs, although I may expose some a
very
nice asynchronous, callback driven, API to make life easier — and
faster! —
for folks integrating into 3rd party event loops and such). As such, I
feel good about recommending folks start playing around with this; I’m
ready to begin soliciting feedback, and encourage folks to begin filing
issues against the nng bug tracker. If folks want to contribute other
things in the form of PRs, that’s welcome at this point too.
I do ask for some more patience as I work on the next steps to
further enhance performance and do some level of bug fixing such as
fixing
some small memory leaks.
Thank you all for your patience during this rather long development
process!
- Garrett
