[haiku-development] Re: gsoc2010: File systems

  • From: "Axel Dörfler" <axeld@xxxxxxxxxxxxxxxx>
  • To: haiku-development@xxxxxxxxxxxxx
  • Date: Thu, 01 Apr 2010 10:29:43 +0200

Janito Ferreira Filho <jvffprog@xxxxxxxxxxx> wrote:
> does the block cache and file cache operate in a similar fashion to 
> the
> Linux buffer and page cache? If so, the unification would then be to 
> use
> the file cache to manage the VM structures and the block cache would
> be based on the file cache?

I'm afraid I'm not familiar enough with Linux to answer that one.
How the unification works is still a bit blurry to me, as I haven't 
given it too much thought yet; maybe Ingo has done more in this regard 
already ;-)

> I still need to look into how the Haiku caches operate. Could you 
> point
> out where in source tree I can find them? I'll also read the BFS code 
> to
> see how it uses them.

The sources are in src/system/kernel/cache/ and there you would want to 
look for block_cache.cpp, and file_cache.cpp.

> Would it be possible to add hooks to specific file cache operations 
> so that
> the file system can do it's logging? Or maybe a better way would be 
> to do
> the initial implementation without using the caches (or rather, the 
> file cache)?

Not using the file cache would be quite annoying, though (for 
performance reasons, but also as currently only files with a file cache 
can be mmap'ed).
IOW I would either leave out using the log for file data (I believe 
this is an option in ext3 as well), or not implement logging for now 
(ie. only replay the log if necessary, and keep it that way).

> Sorry for asking about something posted a few posts earlier, but I 
> would like
> see if I understood it correctly: implementing write-support without 
> logging,
> and then adding logging support would (possibly) require a rewrite of
> write-support. Is this because without logging support I have to 
> check if a
> transaction is legal, and with logging I can just journal the 
> transaction, and
> commit it only there aren't any errors, without a previous 
> verification?

Rewriting might not fit, but you would need to restructure it a lot. 
For example, if you don't use transactions, you will have to make sure 
for yourself that if something fails at some point, you will recover 
the file system structure again. With transactions, you can just abort 
the transaction, and all changes are reversed for you automatically 
(you just have to make sure your in-memory structures are updated as 
well).

> I read about the ext3 file system in the internet (and a little on 
> the books I
> mentioned), and I think I have a reasonable understanding. What's 
> missing for
> me is the data structures existent in the journal (eg. how a commit 
> is indicated
> inside a transaction). I'll download the Linux source code to see the
> implementation and see if I can learn all that's missing from there.

It's usually the best way to dig in once you have an overview about how 
it should work. And in many cases (especially with Haiku), it's the 
only way there is, due to lack of (up-to-date) documentation.

> Is anyone willing to mentor such a project? Should I discuss it before 
> I start
> writing the application?

I would if I knew if I had the time to mentor this year. I don't know 
yet, though. I don't know if there will be someone to mentor this 
project, but this isn't unusual at this point - we see what kind of 
applications we have, and then we'll try to map our mentors to those in 
a way that makes sense.
I don't think we had to reject a good application because we didn't 
have a mentor for it yet. Usually, the number of slots we get is what 
limits the projects we can mentor.

Anyway, if you feel uncomfortable with just one application, you can 
write as many applications as you want - it's just a good pile of work 
to come up with a good one.

> As for the application, I'm think about giving priority to file meta-
> data operations
> (like file creation, deletion, renaming, truncation, etc.) and 
> possibly xattr support
> before doing actual file data operations. Is this a reasonable plan?

Usually, the thing you should start with is the allocator for new blocks 
(and freeing blocks), because this will be needed for almost all 
operations.
With file data, you would just allow file data to be changed at known 
positions, and then later would allow the file to grow/shrink.

Bye,
   Axel.


Other related posts: