I actually hadn’t thought to expose a separate shutdown entry point — even
though I do this internally to cope with these problems. But now that
you’ve pointed out the obvious to me, I’ll add an nng_shutdown() entry
point. It will do everything nng_close() does, *except* that it won’t
actually free underlying memory, so that the socket object will remain
valid, although operations on it from that point forward will return
NNG_ECLOSED.
The EINTR problem is not an issue for me really. I always assume that if
close() fails, I can’t really do anything else with the descriptor. This
may mean the descriptor leaks, but shame on the implementation for having a
close that is permitted to fail. (Thinking on it, I’m not 100% sure that
I’ve never implemented an interruptible close in driver space myself — the
intention here is usually to permit an application to fully exit without
blocking forever, although since the kernel is going to close the file
after wards, it just means that exit(2) will probably block….)
The real ugliness with close() is that you can’t safely use it in
multithreaded applications to terminate outstanding operations, because the
descriptor can be reassigned to another file. dup2() saves the day here,
since it does both a close and atomically reclaims the descriptor safely.
This still isn’t enough for accept() on Linux, which only seems to be
interrupted by shutdown(), but shutdown() does not interrupt accept() on
other systems (macOS). What a friggin’ mess.
(There’s another close() related bug in the macOS tty driver, which I’ve
had to work around in another driver of mine — close() in that case does
not wake a blocking read against the driver! The only solution I was able
to come up with for that one was to poll the character device periodically
by injecting a timeout via termios. Longer term I might replace this with
a poll() loop and non-blocking I/O. Really close() is just one giant
mess. What’s needed is an analog for shutdown() that interrupts any
blocked threads in the kernel — signals *could* be used for this, and
pthread_cancel() also looks like it *could* work, if they weren’t both
steaming piles of brokenness themselves.)
- Garrett
On Thu, Jan 5, 2017 at 4:08 AM, Jan Bramkamp <crest@xxxxxxxxx> wrote:
On 05/01/2017 11:39, Garrett D'Amore wrote:
The things that nanomsg shines at are:
a) Simple RFCs (wire protocol implementations are easy to write, unlike
ZeroMQ)
b) Thread-safe from the get go, just like UNIX file descriptors.
(Sadly, with the same intrinsic race challenges with close though..)
c) C code, so no C++ nonsense.
d) Liberal licensing
Did you consider exposing a two stage shutdown+close API? The first stage
performs a clean shutdown without releasing resources. This stage would be
idempotent. The second stage just release resources and never fail?
The close(2) vs. EINTR problem doesn't exist on Linux and *BSD for
sockets. Some networks file systems (e.g. AFS) flush their per file write
cache on close(2). Because of this you can get write errors like ENOSPC on
close(2). An other problem is that some device drivers to a lot in their
close(2) handler e.g. wind back a tape. In those cases ignoring the
close(2) system call return value can lead to data loss.
While POSIX allows implementors to be stupid neither Linux nor *BSD are
that stupid. A close(2) call *always* closes the socket/pipe file
descriptor and the application must not retry the close(2) call.
