[openbeosstorage] Re: Registrar-Based Notification Mechanism

> Ingo Weinhold <bonefish@xxxxxxxxxxxxxxx> wrote:
> > > Just keep in mind that having the node monitor mechanism for the 
> > > kernel
> > > itself might also be a very nice idea, and that shouldn't be 
> > > dropped so
> > > easily. With a kernel delivering all those nice things, why 
> > > shouldn't
> > > it be able to benefit itself from these=3D3F
> > Yes, I briefly thought about that too, but didn't follow that idea. 
> > One of
> > the problems is, how the kernel entities should be notified. For 
> > userland
> > application this is trival, since you have messenging, but for the
> > kernel...=3F
> 
> You could also have messaging in the kernel as well, but I think 
> probably the best way would be to be able to specify a notification 
> hook that will be called when the event occurs.

Agreed.

> > Discriminating active and passive entities -- i.e. those that run 
> > an 
> > own
> > thread (e.g. daemons) and those that don't (e.g. file systems) -- I
> > could imagine different approaches:
> > 
> > 1) Event queues: Each subscriber has a queue the events are pushed 
> > into.
> > An event queue may have a counter semaphore released with each 
> > event
> > pushed into it. So the (active) entity can wait for events.
> > 
> > 2) Ports: The event notifications are written to a port. This 
> > doesn't 
> > work
> > very well with passive entities, I suspect.
> 
> They could, of course, spawn a thread for that purpose.

That makes them a little less passive, doesn't it? ;-)

> > 3) A combination of 1) and 2): The subscriber can optionally 
> > provide 
> > a
> > port to which only a `there is a new event in the queue' message is
> > written.
> > 
> > 4) Callbacks: The subscribers supply a function to be called when 
> > events
> > occur.
> > 
> > Certainly 4) is the most flexible of the approaches. The others 
> > could 
> > even
> > be implemented on top of it without performance issues. Of course, 
> > it
> > could be misused by subscribers by implementing time consuming 
> > callback
> > functions which will hit the performance of the entity issuing the 
> > events
> > -- one would need to be very careful.
> 
> That's the standard problem in the kernel anyway :-))
> 
> > BTW, when using callbacks the mechanism for userland notifications 
> > could
> > hook in as just another subscriber (at the cost of one memcpy, I 
> > think).
> 
> Even without the memcpy, I think - the notify=5Flistener() function 
> would 
> just call the hook function with the same parameters.
> And that one could then write the message to the registrar (using 
> shared memory).

The additional memcpy() I was referring to, has the following origin: 
Without a notification mechanism for the kernel, the entity at which 
the event occurred would have `allocated' ring buffer memory and 
written the notification data directly into it, while with the hook 
approach, it first has to write everything in a structure (on the stack 
certainly) passed to the hooks, and then it is memcpy()d into the ring 
buffer. Not that a memcpy() is extraordinarily expensive. ;-)

> > If there is the desire to unify the interfaces of the different 
> > watching
> > services (node monitoring, mounting, disk=5Fscanner stuff,...) a 
> > bit, I
> > could imagine something like this:
> > 
> > struct notification=5Fsubscriber {
> >   bool (*notify)(const notification=5Fsubscriber*,
> >                  const notification=5Fevent*);
> >   void *service=5Fparams;
> > };
> > 
> > Where, e.g. in case of the node watching service, service=5Fparams 
> > could
> > point to:
> > 
> > struct node=5Fwatching=5Fparameters {
> >   ino=5Ft      node;
> >   uint32     event=5Fmask;
> >   ...
> > };
> > 
> > The subscription/unsubscription functions would look like:
> > 
> > status=5Ft xyz=5Fsubscribe(const notification=5Fsubscriber *
> > subscriber);
> > status=5Ft xyz=5Funsubscribe(const notification=5Fsubscriber *
> > subscriber);
> > 
> > One could even provide:
> > 
> > status=5Ft subscribe(uint32 service,
> >                    const notification=5Fsubscriber *subscriber);
> > status=5Ft unsubscribe(uint32 service,
> >                      const notification=5Fsubscriber *subscriber);
> > 
> > Anyway, for sake of performance optimization the structure holding 
> > the
> > subscribers for a certain service would need to be managed by the 
> > service
> > itself.
> 
> Something like this could be nice, although I would also be fine with 
> a 
> less generic approach, i.e. a parameter list like the 
> send=5Fnotification() call.

Yep. Actually for a moment I had the even scarier idea, to provide the 
facilities to add arbitrary notification services, that can for 
instance be provided by modules. Then one could subscribe to a service, 
one doesn't even know about. But that does also mean, that one doesn't 
know the parameters needed by the service or the structure of the 
notification. I suspect when trying to turn that into something useful, 
one would end up with a whole component concept and had invented Java 
Beans for the kernel. ;-)

[...]
> > > Anyway, how does the Registrar know about the target of the 
> > > message
> > > =3D3F
> > > The kernel knows this, but the registrar (currently) cannot. The 
> > > only
> > > way would be double house-keeping, I think.
> > Partially. All the kernel needs to know is whether there is anyone 
> > at 
> > all
> > listening for an event. Only the registrar would know the concrete
> > targets.
> 
> Right. I will think about a possible hook-scheme implementation in 
> the 
> kernel - the changes to the current mechanisms would probably be very 
> small (and worth the effort).

Great!

> > > > Possible negative effects:
> > > > * Higher `event occurence -> arriving of notification' latency.
> > > Yes, there are more context switchs needed to deliver the 
> > > message, 
> > > I
> > > don't consider this a big problem though, as those notifications 
> > > aren't
> > > that time critical, are they=3D3F
> > That's what I think too.
> 
> BTW with the hook model discussed above, we could even "stick" in 
> different message delivering mechanisms, and have the possibility to 
> compare them directly (memory overhead, etc.) - and it would also 
> enable us to "design" the system as needed (i.e. different mechanisms 
> for embedded devices or large servers or whatever [not that I am 
> thinking "large server" already ;-))]).

That makes sense.

> > > That might be a bigger problem, but nowadays not sooo important,
> > > although we should prevent doing too many things to let the 
> > > kernel 
> > > run
> > > with a low amount of memory.
> > Well, yes with some reasonable policy, the mechanism I proposed 
> > shouldn't
> > eat up that much memory. Some for the ring buffer (I bet 10 KB 
> > would 
> > be
> > sufficient) and some for pending messages. But the latter should be 
> > rather
> > harmless when some sanity limits are set.
> 
> Probably - one notification will have a maximum of about 280 bytes 
> (256 
> bytes for the filename, and the rest for additional data).
> So a 12 kB buffer could hold a minimum of 43 messages - should be 
> okay, 
> but more wouldn't hurt as well (but we can play with those values 
> later).

Exactly.

> [...]
> > > In that case, why should the subscription functions in the 
> > > libroot.so
> > > then=3D3F
> > Er, personally I'd rather want them in libbe. And now I think about 
> > it,
> > actually only open=5Flive=5Fquery() (or whatever the exact name 
> > was) is 
> > not.
> 
> For whatever reason - I have never understood that call. It's not 
> really usable from C anyway. I could only use it to check if the BeOS 
> kernel would send BMessages directly ;-)

Its existence allowed us to already completely implement BQuery. So I'm 
not exactly ungrateful that it exists. ;-)

> [...]
> > > Perhaps we should have a libroot function where a team can 
> > > register 
> > > to
> > > provide that service, and once it goes down (i.e. crashes) that
> > > position is free in the kernel again, and the kernel knows that 
> > > it
> > > doesn't have to care about maintaining those notifications 
> > > anymore
> > > (therefore, the buffer won't overflow).
> > That is pretty much, what I intended anyway. As I wrote, the 
> > registrar
> > tells the kernel via a syscall when it is ready. Or more precisely, 
> > it
> > will pass the IDs of the locking semaphore and the area for shared 
> > memory
> > to the kernel. In theory any application could do that. When going 
> > to 
> > die
> > the registrar should invoke another syscall.
> > 
> > Regarding crashes of the registrar, I thought, that it would be 
> > enough,
> > that the locking semaphore is deleted with its death. But you're 
> > right,
> > some special handling is required, since otherwise things can end 
> > up
> > really bad, if the registrar crashes just after the kernel has 
> > acquired
> > the lock.
> 
> Also, the shared memory area must be cloned in the kernel as well, as 
> it must be accessible from kernel context.
> And the queue doesn't have to be maintained anymore in the case there 
> is no Registrar.

Right.

[...]
> > > > BTW, there is quite some potential for optimization, e.g. by
> > > > flattening a
> > > > notification message very early (i.e. directly after setting it 
> > > > up)
> > > > and
> > > > writing the flattened message to the target ports instead of 
> > > > using
> > > > the
> > > > high-level API for sending BMessages, which would cause the 
> > > > message
> > > > to be
> > > > flattened each time it is tried to be sent.
> > > That's something I would do anyway, and not regard as an 
> > > optimization :
> > > -))
> > Well, right. Currently it can't be implemented though, since it 
> > needs 
> > some
> > support from BMessage. And considering the unfortunately slow 
> > BMessage
> > progress, I suspect, we will be done with implementing the 
> > notification
> > stuff before our BMessage is ready. :-(
> 
> Despite the slow progress (Erik!!!), what's preventing calling 

[I think, Erik is not this list, so... ;-)]

> BMessage::Flatten() yourself and using write=5Fport() manually=3F It 
> should 
> be possible to get the target port for the Registrar, right=3F

The problem is not the target port (i.e. looper), but the handler. This 
info is stored in the BMessage and must be set for each target it shall 
be delivered to. It should, of course, be possible to find out, where 
the info resides in the flattened message and set it each time, but I 
would prefer, if the BMessage implementation provided sending flattened 
messages to a given target.

> > > Hehe, sounds very nice overall.
> > > I would just like to have a way to keep the notification 
> > > mechanism
> > > available for kernel modules - now if you have an idea to provide
> > > this... :-))
> > I haven't thought much about that yet. Above is a spontaneous
> > brain-storming -- maybe some more concrete ideas will form when 
> > thinking a
> > bit more about it.
> 
> Well, I think it's already the best idea for that problem, but please 
> go on, have a better one again ;-P

I didn't promise anything, did I? ;-)

> Of course, the registrar maintaining kernel functions must reside in 
> the kernel, and be activated via a syscall. We could place that stuff 
> into a kernel module that is loaded on demand of the registrar, and 
> registers its service on init.

That sounds good.

CU, Ingo

Other related posts: