[interfacekit] Re: BAppFileInfo test

Sorry about the slow response; spent most of my free time fixing bugs. =) I just did a check-in yesterday (or was it the day before?) that fixed a number of bugs in the flattening code.

I am bit worried, though, about the flatten/unflatten code - it seems not only complicated, but also very slow; there is one copy of the data due to the design, and there is yet another one that I don't really understand (says to align the data in memory).

I've been getting the kinks in the algorithm worked out (here's to hoping I've finally done it), and haven't really paid a lot of attention to performance thus far, so I'm not surprised that it's slow. The byte-aligning copy was something I got from an incomplete chunk of code someone mailed me. It makes sense to me from a certain perspective: if you've flattened a struct in which byte alignment might pose a problem, just pointing to the raw data isn't necessarily going to work. However, since the data then gets copied into the internal structures, it shouldn't be necessary. I'll take a look at eliminating that copy.

And while I honestly don't know enough about the 'FOB1' format, so I don't know if this could be done only a little bit better, or a whole lot :)

I've been working on some documentation on the format lately; once that's a bit more coherent, I'll check it in. The short answer is yes, it could be better. The long answer is that the format seems to have been designed to save space whenever possible, by allowing various fields to be altered in size or left out altogether. That makes it more difficult to manage the message as an flattened-in-place buffer. A more uniform format -- which I understand FOB2 provides -- would be much easier to understand and work with this way.

Which brings me to my next point: the flattening/unflattening code seems to be very static and hidden beyond several layers. While there is only FOB1 in R5, we may want to support either FOB2 or something self-made, too. Ingo and I also discussed the possibility of introducing a simple messaging system in the kernel that could be translated to a BMessage automatically (and probably vice-versa).
What I would have liked is a "native" format that don't need any copy or new when unflattening data. When I send a 1 MB message with the current implementation, it's not only very memory heavy, but also very slow. Messages are also often redirected or forwarded, and thrown away after a read-only usage - the implementation should reflect that. Another thing we should do is to use shared areas for messages above a certain size, and be able to work with those directly. too.
Do you have a good idea about that?
(of course, that's not the most important thing right now, it's more important to get it to work flawlessly, and the take care about these things)

Yes; premature optimization and all that. ;) The architectural idea was to isolate the actual twiddling of bits in storage as much as possible from the rest of the BMessage logic. I think some refactoring could make the boundaries much cleaner and more conducive to plugging in other formats.

I have also thought about using shared areas for messages, and that was part of my motivation in using STL containers -- by providing a new allocator class that did it's allocation from a shared area, the existing code would work without modification. I like the idea of having a format that allows you to just point at the buffer and be ready to work (i.e., not have to copy data for read-only operations), but I haven't gotten very far in thinking about it.

I didn't have a look at that code, but I guess that some refactoring of your code should be enough to meet the above mentioned points (note, that doesn't mean that you should get rid of the template madness, if you want to keep it :)). 

That's the idea. =)

There is always plenty of stuff to be done, but it's always a problem when there is one component a lot of other things depend on. BMessage really plays a central role, so I would definitely argue that it would be nice to have more than one person that can work on it. 

Agreed.

Having said that, I'm okay if you want to keep maintainership of BMessage, but it would probably nice to have a little more "overview" description in the source files, ie. a description of how it all connects. Just so that someone can fix a problem directly when it scratchs the surface.
Things like "what's the BDataBuffer class and why it's there", same for BMessageBody, a short comment on why Flatten()/Unflatten() should be implemented there, etc.

The FOB1 document I have in the works should help; I'll see if I can't get some more documentation on the code happening.

e



Other related posts: