[haiku-development] Re: Do optional packages need to include everything they do?

  • From: "Michael Lotz" <mmlr@xxxxxxxx>
  • To: haiku-development@xxxxxxxxxxxxx
  • Date: Wed, 02 Sep 2009 03:24:16

Hi Ingo

> You may have misunderstood how the I/O scheduling is supposed to 
> work. An 
> IOScheduler is only catering for a single hardware device. It 
> serializes 
> and reorders all I/O requests for that device to minimize the average 
> seek 
> time. Using one in file systems is really not intended and I don't 
> see what 
> good it would do.

Yes, I see how it works now. At first my problem was that I didn't see 
how to get the necessary information out of the request directly. I 
therefore ended up thinking I'd have to set up an IOScheduler to break 
the request into IOOperations that I would then handle. Completely 
wrong track...

> For the write overlay, I would expect the io() hook to work like 
> this:
> * A write request should be handled completely and synchronously by 
> the 
> hook. All data is just copied.
> * A read requests, if the file has not been written to yet, shall be 
> passed 
> straight to the underlying layer.
> * A read request, if the file has already been written to, needs to 
> be 
> processed by the hook:
>   - The request might not intersect with the modified data range, so 
> it can
>     be passed straight to the underlying layer.
>   - The request might be fully covered by modified data, so it can be
>     handled completely (synchronously), by copying out the respective 
> data.
>   - The request might be partially covered by modified data, in which 
> case
>     the covered ranges need to be served synchronously and sub-
> requests need
>     to be created for the not-covered ranges and be forwarded to the
>     underlying layer. An optimization is possible when the beginning 
> or the
>     end of the range is covered: then that part could be handled
>     synchronously, the request be adjusted and passed on to the 
> underlying
>     layer.

Understood and implemented pretty much exactly that way in r32900. It 
really was the same for read/write and read_pages/write_pages. They all 
use the same backend now, just the method of filling buffers depends on 
the type of hook called (IORequest::CopyData() for IO, direct memcpy 

> I don't know how the write overlay is implemented, but to keep things 
> simple it could just fully cache a file when first written to, to 
> avoid the 
> somewhat unhandy partially covered reads. That's seriously suboptimal 
> when 
> only small parts of large files are changed, but I don't think that's 
> a 
> particularly common case for the installer CD boot (probably not even 
> for a 
> live CD).

It turned out not to be a problem at all. I've already implemented 
reading in the write_overlay to linearly fill buffers, alternating 
between write_buffers and filling gaps using the underlaying 
filesystem. This concept seems pretty optimal to me and now also works 
for IO by creating subrequests for the gaps that are then handed over 
to the underlaying filesystem.

> Note, that when creating sub-requests for an IORequest you might need 
> to 
> associate an iteration and/or finished cookie and hook with the 
> request. 
> You'll have to replace the original cookie, but save it first and 
> later 
> call the original hooks, in such a case. For reference: 
> do_iterative_fd_io() (in vfs_request_io.cpp) does exactly that.

I didn't do that, and it seems to work as expected, so I assumed it 
wasn't necessary in how I've implemented it. It's possible that I'm 
missing something though.

> > The attribute_overlay always just passes through the requests, as 
> > it is
> > not concerned with file data btw, so it is not a problem there.
> I'm not quite sure, if that's correct. The io() hook operates on 
> nodes. 
> Some file systems, I believe including BFS, expose attributes as 
> nodes to 
> the VFS/file cache. IIRC you designed the attribute overlay also to 
> be used 
> as write overlay for file systems that actually do support attributes 
> (but 
> are read-only), so the io() hook can be called for nodes that are in 
> fact 
> attributes. I believe this happens only when requested by a 
> {read,write}_attr() file system hook, so this is maybe not a problem, 
> but 
> not knowing the implementations of the overlays I couldn't say for 
> sure.

So far I haven't encountered that situation, but I see what you are 
getting at. How would the attribute_overlay be able to tell if a node 
is in fact an attribute though?


Other related posts: