I've been thinking about the overall design of the stack for the past few days, and not to take away from the net_buffers discussion but I just wanted to get this down and see what you think of it.
Since Axel hinted at the possibility of using C++ internally, I thought why not make the whole thing OO? AFAIK this hasn't been done before, and there may be a good reason that I don't know about.
The basic purpose of a network stack is to move information from user space to hardware space and back. Generally it's not a 1:1 copy; the data is modified according to the protocols in use, and the direction the packet is going. These protocols can be linked together such that the output of one becomes the input of another.
These protocols all inherit from a basic Protocol class, and apply their own header/transformation to the packet. In the case of receive, they invert this transformation.
For example: UDP - attaches/removes udp header to/from packets IP - ditto Ethernet - ditto IPsec - performs encryption/encapsulation on the packet
I'm considering Ethernet a Protocol instead of an interface because it has to apply the Ethernet frame to the packet before it gets sent out to the device.
As far as the buffer design goes, I initially thought of simple arbitrary size buffers arranged in a linked list. A Packet object could be used to manage the buffers and other metadata associated with the transfer. Variable size buffers are more complex to deal with and make more work for the memory allocator, but really any of the other schemes discussed could be used.
http://home.rochester.rr.com/sg3web/haiku/doodle.png is a drawing I made when brainstorming this idea. It shows an example configuration of various Protocol objects and how they might work together.