[openbeosstorage] Re: Logging/RAID

Already a bit older, but I still wanted to add something :-)

Tyler Dauwalder <tyler@xxxxxxxxxxxxx> wrote:
> No, I meant:
> meta data = file and dir inodes, etc
> user data = actual file data
> IIRC, only the former is journalled by BFS.

Exactly.

> Yes. I'm just concerned about whether or not the data is also 
> journaled during 
> a resize (perhaps this is an unnecessary concern...). There's always 
> the 
> chance that, say, kernel_intel would need to be relocated during a 
> resize, and 
> should the power be cut halfway, part of the data could be lost.

Right, the fs implementation need to make sure that this won't happen. 
The worst case would be that we also need to copy the kernel to that 
spare partition.
Anyway, under normal circumstances (at least for BFS), it should be 
possible, to make it safe:
1) allocating a chunk of data at the new location, and copy it over 
from the old location
2) updating the old pointer to the new location
3) remove old data
-> crashs would have no effect between the items

For BFS, this would have to (and could) be done with every 
allocation_group.

> > If a FS itself doesn't support journalling (e.g. FAT), the system 
> > must
> > provide it with a space for a resizing (or moving in case of 
> > absolute
> > addressing) log.
> So here I wonder about the repititious overhead of implementing 
> resizing 
> journalling for any non-journaled filesystem. Would it be possible to 
> provide 
> a generic API for use in these instances that would work in the 
> manner of 
> "here are a bunch of disk block writes, add them to the log and don't 
> remove 
> them until all of them have actually occurred", rather than simply 
> presenting 
> each fs with a chunk of disk space for a log and saying "here, 
> implement your 
> own journaling system".

Using the approach from above, that shouldn't be too hard. I would only 
offer a log area to the fs - whatever it will do with it is up to its 
implementation.


> > What might be tricky, if the job log is dynamically allocated when 
> > needed -- beside the alloction, which might be difficult itself -- 
> > is 
> > parallel processing of jobs. They are by definition working in 
> > disjoint 
> > branches of the partition hierarchy, but they might compete for 
> > space for their job log. Or even worse, the one job's log might get 
> > into the way of the other's actions (e.g. if placed in a spare 
> > region 
> > where a new partition shall be created).
> Plus, there would need to be a way to tie all the job logs together 
> so they 
> all could be found in the event of an interruption. Perhaps it would 
> be easier 
> to have just one job log, with all parallel job queues logged to it, 
> anyway 
> (for ease of reinitiating after an interruption). Separate logs could 
> be 
> allocated by each process as necessary for external management logs 
> (i.e. 
> during partition resizing, FAT resizing, etc).
> 
> As to the dynamic allocation, if previously empty partition area 
> space is 
> consumed for the log, perhaps the partition system should create a 
> temporary 
> BPartition typed as a "temporary partition management job log" or 
> something to 
> that degree to officially hold the space until the job is done.

And there it is again, the spare partition :-))
If the log data is placed on another (unaffected) partition, that 
partition has to be locked against any changes. Or move that log to 
another partition, or ... :-)

Adios...
   Axel.

Other related posts: