[openbeosstorage] Re: Logging/RAID

Tyler Dauwalder <tyler@xxxxxxxxxxxxx> wrote:
> > > > If the boot partition is not affected by the changes, the whole
> > > > process
> > > > can be simplified by logging to a file on the boot partition. 
> > > > For 
> > > > 2.
> > > > there are some more options. E.g. resizing a partition doesn't 
> > > > need
> > > > external logging, if the FS is journalled anyway.
> > >
> > > That last part I'm not so sure about. I for some reason imagined 
> > > we
> > > would be doing complete logging (i.e. meta data AND user data).
> > 
> > I'm not sure, if I understand, what you mean by meta data and user 
> > data.
> > User data = FS contents, i.e. dirs and files, while
> > Meta data = partition layout and the like?
> 
> No, I meant:
> meta data = file and dir inodes, etc
> user data = actual file data
> 
> IIRC, only the former is journalled by BFS.

Oh, is that true?! Didn't know that.

> > Resizing a partition has two phases, resizing the partition 
> > (changing 
> > the
> > partition map) and resizing the FS. In this order when the 
> > partition 
> > is
> > enlarged, the other way around when being shrunk. If the FS 
> > supports
> > journalling, the FS should be able to use it's own log for the FS
> > resizing. That may even be easier to implement for the FS add-on
> > implementor.
> 
> Yes. I'm just concerned about whether or not the data is also 
> journaled during 
> a resize (perhaps this is an unnecessary concern...). There's always 
> the 
> chance that, say, kernel_intel would need to be relocated during a 
> resize, and 
> should the power be cut halfway, part of the data could be lost.

That must not happen, and the FS has to take care, that it doesn't. I 
think, this shouldn't be extraordinarily hard, since the FS basically 
needs to move blocks, that is, it can copy them first, and update the 
info, which blocks the file consists of thereafter. It would log the 
transaction before it starts to carried it out, so that it can be 
replayed, when being interrupted.

[...]
> > > There also could be a need for:
> > >
> > > 4. A place to log filesystem changes during repartitioning ops.
> > >
> > > But I really like the idea of leaving that problem to the fs 
> > > itself.
> > 
> > If a FS itself doesn't support journalling (e.g. FAT), the system 
> > must
> > provide it with a space for a resizing (or moving in case of 
> > absolute
> > addressing) log.
> 
> So here I wonder about the repititious overhead of implementing 
> resizing 
> journalling for any non-journaled filesystem. Would it be possible to 
> provide 
> a generic API for use in these instances that would work in the 
> manner of 
> "here are a bunch of disk block writes, add them to the log and don't 
> remove 
> them until all of them have actually occurred", rather than simply 
> presenting 
> each fs with a chunk of disk space for a log and saying "here, 
> implement your 
> own journaling system".

Is see no problem in letting the FS ask for what kind of service it 
needs. None, if it uses it's own journal, a contiguous chunk of disk 
space (or callbacks for writing and reading blocks), if it wants to 
implement logging itself, or a complete logging service provided by the 
system.

> > > So, thoughts, corrections?
> > 
> > What might be tricky, if the job log is dynamically allocated when 
> > needed
> > -- beside the alloction, which might be difficult itself -- is 
> > parallel
> > processing of jobs. They are by definition working in disjoint 
> > branches of
> > the partition hierarchy, but they might compete for space for their 
> > job
> > log. Or even worse, the one job's log might get into the way of the
> > other's actions (e.g. if placed in a spare region where a new 
> > partition
> > shall be created).
> 
> Plus, there would need to be a way to tie all the job logs together 
> so they 
> all could be found in the event of an interruption. Perhaps it would 
> be easier 
> to have just one job log, with all parallel job queues logged to it, 
> anyway 
> (for ease of reinitiating after an interruption). Separate logs could 
> be 
> allocated by each process as necessary for external management logs 
> (i.e. 
> during partition resizing, FAT resizing, etc).

A central place where all the existing jobs are entered is needed in 
any case. Whether it should also be the common place for logging, well, 
I don't know -- I don't think, I understand the whole thing good enough 
to make some qualified statement at this time. :-)

> As to the dynamic allocation, if previously empty partition area 
> space is 
> consumed for the log, perhaps the partition system should create a 
> temporary 
> BPartition typed as a "temporary partition management job log" or 
> something to 
> that degree to officially hold the space until the job is done.

That would be possible, yes.

CU, Ingo

Other related posts: