FreeLists / openbeos / [openbeos] Re: new filesystem design

["Alexander G. M. Smith" <agmsmith@xxxxxxxxxxxxxxxxxxx>]

A repost - my reply didn't seem to appear last time and I don't
see it in the digest, but if it does show up twice, I apologise
in advance.

- Alex


Kobi Lurie <k_lurie@xxxxxxxxxxxxxx> wrote on Tue, 05 Feb 2002 18:48:23 +0200:
> files   folder list (virtual)
> attributes are everything
> a file is assigned an ID.

Sounds a bit like ReiserFS where a "storage key" is assigned to
each file's actual contents.  Names are just a way of finding that
storage key.  Now that you've thought of it, you may find the
http://namesys.com/whitepaper.html document interesting and
more meaningful than it would be to ordinary people.

For that matter, Unix and BeOS have inodes / vnodes which are
numbers that uniquely identify a file or directory.  In Unix
they don't store the name of the file with the file, instead
directories are simply a list of names and inode numbers.  If
you look up a file name, it's really just searching through the
directory and then using the corresponding inode to really do
the file operations.  If you want to add an existing file to
a second directory, you use the link command (ln) to add a
hard link in that directory to the file (basically it just
adds another name and the inode number to the directory).
Since multiple directories can have names for the file (not
necessarily the same name), the file's inode data area has
a count of the number of references to it.  That way the file
only gets deleted when the number of references falls to zero.

> name, path and almost everything else is an attribute.
> a file can have multiple filenames and paths (which
> represent a copied/moved file)

Usually you want a copy of the file so that you can change it
without affecting the original.  Multiple names are already
done with symbolic links and hard links (BFS doesn't implement
hard links but it's available in the API).

COW (copy on write) could be useful for saving work when
copying a file (the copy uses the same disk blocks as the
original, except that when you write to it, it will switch
to using a new disk block for the part you are writing to).

> folders are not files.
> there is a special ID for the file which contains all the
> folder data.  there is also a special id for trash, which
> can be deleted, but its id and path is recreated 
> whenever we delete a file.

So folders are basically just a container of file ID numbers?  Or
why bother with folders at all, just sort through the path name
attributes.  A folder would be a range of path names, folder C
would conceptually be all things with path names "A/B/C/*".

> there should be a util that reorganizes data according to
> best blocksize for max. speed. though best seats at beginning
> of drive (to avoid drive seek time) are reserved for 
> frequently accessed files.

A packer or defragmenter or disk optimizer in other words.  BFS
is lacking one (other than the trick of copying all files to a
freshly formatted partition).

> (sort by times read, defrag by this order - file is created
> with the appropriate block size. files that need to move closer
> or further, according to how much they're accessed, will 
> move with their block.

That implies storing the usuage statistics somewhere, perhaps in
attributes.  This is similar to the last accessed time attribute
which most people turn off in Unix since it slows things down by
doing extra updates to the time info whenever a program reads a file.

> (the filesystem has many block sizes.)

ReiserFS goes better than that and packs files together so that
several files could share a disk block.  It also has interesting
ways of placing related things nearby.

> at delete time trash is recreated if it doesn't exist, and
> the path is deleted from the file.

You don't need a trash container if you are using attributes
for file paths.  Instead, it is the list of all files without
a file path attribute.

[Examples Cut]

> limitations and downsides:
> if we have 2 million files, going thru them(iterating)
> for every open folder will take too much time. [...]

Yes, but if the list is sorted then it won't be too bad.
On the other hand, updating the list would be slow if only
one program at a time could add new files.  BFS has the
advantage of letting multiple changes happen simultaneously
(so long as they aren't in the same directory).

> we can make the kernel load lower IDs first, so we can make
> important/system/boot files load first, [...]

A special case of defragmentation.  The 3DO game console did
this for later versions of their CD-ROMs so it would boot up
faster.  Made quite a big difference.

> added bonus: quick check of empty directories.

If you get rid of directories and just have a pile of files
with path names, empty directories don't exist.

> [...] The trash works by simply iterating thru the files
> looking at empty path attribute.

Yup.

Anyway, keep on thinking and have a look at some of the other
file system info out there.  It's annoying when people tell you
that your new idea is old (happens to me a lot, like that recent
directory-is-a-file fiasco on the ReiserFS mailing list), but
if you keep it up, you'll get to know the field and come up
with truely new stuff.

- Alex