Some of you may have noticed that github and email have been quiet lately.
This is for a couple of reasons. Based on other findings with testing, I
found that scalability was severely limited on some platforms with
threading, and that has caused me to rethink the architecture of libnng.
At the same time, I was burning out a bit and needed a break to think about
other things. There was a couple of weeks of reduced activity as a result.
I’m pleased to say I’m back on task now. I’ve been working in an offline
branch which I will be pushing momentarily. This branch is currently
horribly broken, since it represents a work in progress and I have had to
make breaking changes.
That said, I’m pleased that so far things are coming together. I’ve got a
callback-driven form of the pipeline protocol with the inproc transport
working quite well. I have a temporary emulation of the callback form of
POSIX transport using threads (until I can get a little more time to write
up a poll/kqueue/epoll variant).
Fundamentally, the big architectural hurdle is behind me now. The new
library is built on top of a generic “aio” structure, which represents an
I/O operation of some form, along with a callback routine to call when that
is complete. These routines will be executed by a relatively small number
of I/O threads performing the poll/kqueue work on POSIX, or using the I/O
completion port callbacks on Windows. For my internal message queues, the
completions are called directly, although I use a global taskq (with a pool
of threads) to avoid deadlocks.
More on this as I have it — but progress is again being made. :-)
- Garrett
