[openbeosnetteam] Re: Kernel move... [LONG]

OK, final word on this from me for numerous reasons...

> David wrote:
> > Philippe, why? Please give me reasons that make sense...
>
> Sory, I was in the process to last night, but kind of shift-focus'ed
> since ;-)
>
> Before I continue, let me say that this kernel move thread is needed,
> as any new big architecture/design decision,
> it need to be understand why and where it comes, by all the team.
> So, we may have different point of view there, but it's not a flame
> war, don't take offense.
> ;-)

Never said it was, though it's starting to look like a religious argument...

BTW, if parts of this sound sarcastic they're not meant in a nasty way :)

> > The functions that move into the kernel will never require a change
> of
> > kernel to support a new protocol, so the argument that we'd have to
> > rerelease the kernel isn't valid
>
> Not until we've to change the way valid protocols (path where modules
> lives, modules api, whatever else...)
> are detected/known to a kernel living socket() call.
> And socket() is not the only socket-specif code here. We could ask
> ourself same questions for
> recv(), recvfrom(), send(), sendto(), etc... because, yes, if we aggree
> (which I don't today) to move socket() into the
> kernel, why not for simplicity, move all others sockets calls there
> too?

That was/is the plan...

> > I can't understand why you'd want to
> > have to mess with ioctl's and such like when we could simply have a
> syscall
> > that keeps things much simpler.
>
> Sorry, ioctl() doesn't looks any more *messy* than a "syscall" to me.
> In fact, ioctl() is far more in OS-agnostic/common knowledge, because
> of his POSIX root, when
> syscall() come from Unixes... and can't be found in every OSes.
> Many developers out there knows very well the open() / read() / write()
> / ioctl() / close() calls, even when they are coming from
> very differents platforms. We've to think about the learning curve for
> future new comers to OBOS code too as we see more
> and more *evidence* than OBOS could, eventually, succeed! ;-)
>
> And I failed to see what a syscall() on a socket can do that a ioctl()
> can't, since socket
> *are* file descriptors too.

You do realise what we are talking about when we use the term syscall right?
Or do you mean sysctl()? sysctl is strange as it doesn't ever need an fd to
operate.

> > I guess this is where I'm not understanding your argument. The socket
> call
> > has been part of the kernel on unix for almost as long as sockets
> have
> > existed
>
> AFAIK, we've a sockets-based working network stack, without any socket
> calls living in
> kernel code. In kernelland modules, yep, but kernel don't even knows
> what we're doing of the fd he give back from the stack driver, which is
> really fine because he don't make any assertion...

Up to a point. We have a lot of mess due to the way we do it.

> Beside this, [Open]BeOS is not UNIX...

Never said it was.

> > it's inclusion makes perfect sense when you think of the low
> > level tinkering it has access to. Look at the (no offence meant here)
> nasty
> > code that Philippe had to add for accept in libnet. Having accept in
> the
> > kernel removes that mess and makes life much more straightforward.
>
> Uh? What nasty code? The GET_COOKIE trick? Oh.
> But if we move to kernel, we'll have to introduce same trick in the vfs
> layer, because we need to
> bind a fd with a the new accepted socket, and someone need to create
> this fd somewhere.
> Having libnet accept() doing this fd creation seem far better to me,
> but whatever...

Yes, but that's a given. I'd rather have that sort of thing done at as low a
level as possible.

> > /dev/net/socket|stack was a hack that got us working, nothing more.
>
> Howard will not be happy about this statement, because BONE use the
> same *hack*.

:) Well, he reads this list from time to time so he can email me his
displeasure :) He may even agree with me? I won't put words in his mouth - I
have too much respect for him. :) (Go Howard!)

> > Now we have our own kernel code the kernel
> > should no longer be viewed as a "back box" (yes I know only too well
> they're
> > orange) but as a set of code we can modify to our needs. Let's take
> > advantage of that.
>
> So, let's take advantage of our new kernel advantage : dynamic
> kernelland modules loading and unloading!
> ;-)

That's all the davnatge you see to having our own kernel. Damn.

> Internal kernel code will still be (and should!) a black box for many
> OBOS contributors, because you'll need
> guru skills to modify it, and we're no many to have the skills
> required. Me first.

So we should just close it off from further improvement? How sad.

> On a more conceptual point of view, I rather line a kernel code only
> focusing on
> providing really low, *kernely* support, when all the rest live or in
> userland or in kernelland modules/drivers. That way the kernel code
> stay clean and, maybe more important even, small, easy to read and
> understand.
> And stable!

Hmmm, well interesting argument as I consider the socket() call and the
other assocaited calls to be low level *kernely* calls.

> Current forked kernel code have a good stability level, but it will
> decrease eventually soon as soon as
> the kernel team will tweak him here and there. I'm confident this level
> will increase again then, but if we add
> the sockets syscalls integration in the basket, it'll not help.

Oh, so in your world we just ignore any improvements we can achieve by
better integration as it may harm the stability. hey, let's just have an
immediate feature freeze and never touch the kernel again. We'll add
everything as kernel modules (I mean they can't harm the kernel stability
can they???) and that'll be very great!

> NewOS/forked OBOS kernel code already support both drivers and modules,
> why not use these features to
> expand the kernel services *without* modifying it...

Think I've been over this before.

> Modular kernel is a great thing now days, and NewOS/OBOS kernel offer
> the file descriptors, drivers and kernel modules we really only need to
> expand it to add network service.

Yes, but again it needs to be done in a resonable way. If we have a slightly
larger kernel and 5 fewer kernel modules is that a good trade off? maybe we
have a factionally smaller kernel and 10 extra modules? Is that a good trade
off? The problem with your argument is that it's difficult to draw the lines
and the arguments aren't as simple or as straightforward as you or Nathan
are making them out to be.

> For example, how to really fully restart the network stack without
> reboot if some things live in the kernel (note that I'm not talking
> about kernelland here, but kernel code), like mbufs, sockbufs, etc, get
> screwed somewhere?

To be honest, if the mbuf's get screwed then it's game over anyway. Bad
example. A better one is what happensif we detect that the routing tables
have been corrupted? Given that they'll live in a sepeprate module they can
be unloaded and restarted just as per our existing code.

The things that go into the kernel proper are things that work and don't
change. By having the sockbuf code and the code that deals with sockets (not
the protocol stuff, just the sockets) we put code that is effectively
"static" into a place where it shouldn't be changed much. We essentially
lock it away and simply use it. The modules we create are then simpler and
more straightforward and can be smaller and better "targetted" at their
needs and uses. This seems like an excellent idea to me and has more appeal
than than the ideas your putting forward.


> BTW, why bother changing our current architecture which work *AND* was
> the one BONE choose to use too, even if Howard could have made it fully
> in kernel_intel if he want to... but he do not, for modular reason I
> guess.

Well, again, Howard can comment if he wants to.

> Anyway, I agree that we should use any more generic memory pools/hash
> tables the new kernel provide, that right.
>
> Last but not least, if we move the socket() call into kernel, we'll
> kill any hope to release a network stack replacement
> for BeOS... and no one here could tell nor when nor if OBOS R1 will
> ever come. In this (unwanted) case, all these (yours!) net team efforts
> will be wasted? No, please, no.

That has NEVER been a desire of the project. I realise certain people here
have wanted it, but that's never been a target of my work. If people really
feel strangly about it they can fork and finish things themselves. I really
don't mind people doing that but would caution them to the effort involved.
select still doesn't work correctly despite all your efforts (amazing though
the work you've put in is) and fixing that will take some time I suspect.
Also PPP and the related protocols still need to be finished.

> > Given that much of the networking code will be in a seperate module
> anyway I
> > don't see the problem, but the advantages that I see are enormous.
>
> It's where I failed to see which ones?

Well, I guess maybe I haven't explained what I have in mind, but then as
it's still early days and I'd rather code and do other stuff than bash out
emails (despite the length of this one) then I guess we're at an impasse :(

david

Other related posts: