[openbeos] Re: BeOS design flaws
- From: "Zenja Solaja" <zenja@xxxxxxxxxxxxxxxx>
- To: openbeos@xxxxxxxxxxxxx
- Date: Thu, 06 Sep 2001 18:29:18 EST
Great and educational post. I just hope that everyone realises that
every large software project suffers from similar architectural flaws -
what seemed like clever engineering at the initial design/prototype
phase ends up growing to an unmaintainable mess. Such is the fact of
the profession we're in. Great engineers working on other projects
also curse the state into which the code they work with has grown to -
don't fool yourselves into believing that other projects are any
better. Almost every project is a house of cards. And here we see one
of the major advantages of a true modularised system (in our case,
micro-kernel based system). If the initial design is solid, and each
module is a self-sustained unit, then its easy to replace one component
with a newer one. You sacrifice performance in order to gain
maintainability, which pays off in the long run.
So BeFS has bugs. Nothing new or unexpected. As long as the
architecture is modular, BeFS can be thrown out and replaced with
something better. If the architecture isn't modular (and for BFS I
expect it to be tied to the kernel), then you have to fix these
problems. BeInc couldn't risk breaking binary compatibility and had
the burden of economies to worry about. With OpenBeOS, these problems
dont exist. The choice of one file system over another is academic -
as long as the initial philosophy stays the same. As our (and
society's) experience grows, we implement better ideas and designs and
throw out the old. Microsoft went from Fat16 to Fat32 to NTFS. Whats
stopping us from going form BeFS to XFS in the future? (this is not a
question, just an illustratrative point).
Try and remember what initially brought us to a new OS from a company
called BeInc. It wasn't the applications, or hardware support or games
available which made us adopt a new OS. It was a realisation of the
following philosophy:
"Screw legacy, lets do whats right."
Basically, throw out the old-and-near-the-end-of-its-lifespan code and
start fresh, and build on the experience you've acummulated. The
initial version of OpenBeOS will also be crap. But as we gain
experience, we will redo the individual modules (kits) and with each
iteration create a better OS. And that, my friends, is the BeOS
philosophy.
>
>> > I wanna have a word with you. ;-) I have read many of your
>> > comments over the past weeks on different sites and lists, many
>> > of them critical of the "BeOS design", like for instance "Oh, you
>> > mean, there *was* a design? Damn, they could have told me."
>>
>> Hehe...
>
>Maybe Be Inc. payed him to sink BeOS...
>I don't really think so, I would understand M$ doing that, but Be...
>Anyway.
>After "Diana, her true story" ... :^)
>
>
>> > Now you know that we are trying to license the BeOS from Palm,
>> > and that we intend upgrading it and relaunch it worldwide.
>> >
>> > As a BeOS insider, I would like to know from you whether we are
>> > fooling ourselves, and whether the BeOS is too complicated,
>> > limited or deficient in order to succeed with our plans.
>>
>> I think that all you're likely to achieve will be to polish things
>> a little bit and to release "yet anoher version of BeOS that
>> doesn't really solve people's real problems". I could name many
>Yeah but it would help it stay on the market (was it there already
???)
>
>> examples, but I'll focus on one specific case : bfs, the filesystem.
>> There's no questioning that the features that are unique to
>> bfs have to be kept. Too much BeOS code relies heavily on it,
>> and bfs is one of the reasons why people like BeOS. bfs, as far
>> as I know, suffers from 6 major types of problems :
>At least 6 < $\infty$ :)))
>
>> -it has a serious limitation in the size of files that it can
handle.
>> (with the default settings, it's 33.75GB in the best case, but
>> can (in theory) sink as low as about 1GB, or possibly even
>> lower). This is not only a theoretical limitation, we had had
>> reports at Be of people hitting that limitation. It is inherent to
>> the layout of the filesystem, and cannot be lifted without
>> changing the on-disk format of the filesystem, which would
>> make it incompatible).
>Hmmm hopefully DivX files are generally under 800M :)))
>More seriously, that means that anyone wanting to deal with raw video
will
>have to make serious tests to see it the fs can handle that.
>
>> -the indexing code is known for being buggy, such that an index
>> can sometimes get corrupted. This can lead to data loss or full
>> system crashes. Nobody ever found that bug, and it has been
>> hitting Be engineers at least once a year per engineer. There isn't
>> a tool to fix an index when it is known for being broken, and not
>> even one to actually verify the indexes.
>> -the way journaling is implemented is so primitive that, in some
>> cases, the integrity of the filesystem meta-data is maintained at
>> the cost of explicitly corrupting files. There are real-world
>That's weird... You mean for inst. when it doubt on a file it will use
a block
>without checking if the file still use it ?
>
>> scenarios where this happens, I was myself hit twice by those
>> cases. Fixing that problems requires to massively overhaul the
>> interface between the filesystems and the disk cache, inside the
>> kernel.
>Well Linux has had several major redesigns of the block-cache AFAIK.
>So it's doable (didn't say easy).
>
>> -there are some types of crashes such that the space that was
>> allocated to a file is not freed again after the file is deleted and
>> seems to disappear forerer. bfs has a tool to reclaim that space
>> later, but it is a very slow tool that makes the machine entirely
>> unusable while it is running. There is a possible real fix for it,
>Hugh, SCANDISK ??? :-(
>It'd be far better to correct the fs code IMO (still didn't say
easier)
>
>> but nobody ever took the time to actually implement it.
>> -there are several aspects where bfs creates lots of fragmentation
>> (contrary to what some people believe). Those aspect cause
>> some perceivable slowdown.There is no available defragmenter.
>Hmmm, porting DEFRAG today :D
>That really is a problem too, I didn't know of all those, btw we all
thought
>BeOS was rock solid, and now we discover that though it worked for us
there
>are lots of flaws...
>
>> -the BeOS kernel is such that using too many big bfs filesystems
>> can cause some serious instability (there were some reported
>> cases where 150GB would take the whole system down).
>I still don't have any cash to buy such a hard disk btw :)))
>But if we want to get into the server market...
>Is it a layout problem or a memory problem ?
>I mean is it the size of the vars that hold the structure, or more
some
>tables that have their sizes proportionnal to the filesystem expand
and then
>fills up some memory area ?
>The second case would involve more coding to fix.
>
>> (and I can think of other problems, but I'll stop here).
>>
>> The actualy details don't really matter. bfs has bugs that need
>> fixing,
>> and has (as far as I can remember) tens of thousands of lines of
>> code. My point is, I am not certain that diving into that code
>> would necessarily take less time than building a new filesystem,
>> functionally equivalent to bfs, but simpler to maintain and without
>> those limtations. Sure, there might be a performance hit, but I'm
>> pretty sure that that the performance benefit of being able to
>> defragment a filesystem would outweigh the losses that a simpler
>> implementation might create.
>Yep, or maybe we could port one of those Linux filesystems (XFS,
ext3...)
>and hack^H^H^H^Hadd in attribute support ?
>Anyway M$ did such a jump from FAT to FAT32, though they had a tool to
convert
>the filesystem... that might be harder to write in case we go from bfs
to xfs
>or other. (Amiga also has had several default filesystems in its
history)
>
>
>>
>> > If you think it is, why did you work for so long at Be, Inc.? ;)
>>
>> Because it took me a long time to learn about all those
>> problems. Because working at Be was (mostly) a pleasurable
>> experience, in spite of the technical flaws of the products.
>> Because, as a foreigner in the US, Be was holding me with
>> a nice pair of golden handcuffs.
>Hehe
>>
>> > If you think it isn't, what do you think needs to be done to
>> > rectify it?
>>
>> Well, the deep issues that BeOS has would have to get eventually
>> fixed. Those issues need to be found (which usually means trying
>> to get non-BeOS developers to write applications and taking note
>> of all their gripes, or trying to port existing applications, or
even
>> to
>> write new applications, each time taking note of all the issues that
>> come along). Then, for each of those problems, you'll need to get
>> together the whole group of engineers working on the OS, and
>> figure out how to solve the problem, and what the consequences
>> will be on existing code, both in other parts of the OS and for
>> existing applications (a number of the fixes are already known for
>> breaking quite some existing code).
>Here I begin to think even if some criticised the openbeos project
because we
>would reinvent the whell, it might prove useful in some extend.
>Could we get our hands on the bug database at Be ???
>
>
>>
>> > Would you be willing to help us fix the flaws (if possible) in
>> > case we get the license?
>>
>> I don't really think so. I'll (hopefully) soon start at my new job,
>> where I'll be doing operating systems as well. I probably won't
>> feel like doing more of the same thing as I come back home in
>> the evening. And there might even be a conflict of interest there.
>Been hired at M$ ? ;)
>
>> > You have filled in the survey on my argo-navis.com site, and I am
>> > very much aware of your comments and expertise, but I'd really
>> > like to have your inside scoop, because if we are fooling
>> > ourselves, then I'd like to know that before approaching Palm,
>> > because I have no intention to beat what may be a dead horse.
>>
>> Well, in the short term, you might be able to get the BeIA
>> development platform in its current state, beat the hell out of
>> it, fix the most blatant bugs that you'll find, try to polish it a
>> bit, and release it as "R6". It's not even clear to me whether
>> all the features that people are buzzing about are actually there.
>> (I think that OpenGL is extremely limited). After that, you'll
>> get in a situation where the learning curve to actually re-architect
>> parts of the OS (and especially the kernel, that needs some
>> serious massaging) will be quite steep, such that it'll take quite
>> some time before the major changes that are actually needed
>> really get done.
>It still would help us catch ou breathe.
>
>> > Or.... is BeOS flawed, but so are others, and these flaws aren't
>> > significant enough to turn it into a commercial success? You tell
>> > me. Will you? :)
>>
>> Well, if you think you can make it fly, go for it. I think that
it'll
>> be a while before writing a BeOS application is as easy as writing
>> a Windows or MacOS application. As long as an alternative
>Hugh ? doubtful here...
>
>> platform is harder to write code for than a mainstream platform
>> and doesn't offer any really significant technical advantage, it'll
be
>> hard to take off.
>>
>> ------------
>>
>> I'm afraid I have to tell you that the story is not quite over.
>>
>> For professional audio, one more limitation you'll find is that the
>> kernel creates some high latencies when using the kernel-level
>> message-passing system, which the current Media Kit is built on
>> top of. Any attempt to use that API will result in glitches, sooner
>> or later.
>>
>> We could go in detail in pretty much any aspect of BeOS, and
>> see something that is inherently wring in the architecture of that
>> specific component.
>>
>> > If there is anything else, pls. let me know. As I said before,
>> > I'd like to make this work and believe we can, but if the
>> > foundation is more "fragile" than I thought it would be, then we
>> > obvioulsy have to rethink our marketing, development planning and
>> > such like.
>>
>> I believe that, indeed, BeOS is like a fortress built on a beach.
>> People looking at it from far away think that it's impossible to
>> take, while people working inside spend their time dealing with
>> the sand that the tide removes, and anything new that gets build
>> is done on top of sand.
>So it's true when they say 'from the ground up', except the ground
here is not
>a solid granit rock :-(
>
>>
>> I'm afraid that I am somewhat the bearer of bad news for the
>> BeOS community in general, and for you more specifically, but
>> I feel that I also have a duty of "due diligence".
>>
>> That being said, I still hope that something great can be done,
>> but I believe that it might involve a great deal more re-writing
>> that you might have initially thought.
>Nothing impossible for those who belive :)
>I think the openbeos coders will be of a great help in here.
>
>Fran=E7ois.
>
Other related posts:
