On Mon, 28 Mar 2011 08:10:11 +1300, Ankur Sethi <get.me.ankur@xxxxxxxxx> wrote:
In any case, I'm looking at "modern" file systems for ideas. I'll hopefully have a reasonable design doc in a day or two. I'm especially interested in how other file systems handle attributes. Almost every FS I've looked at supports extended attributes, but I'm not sure how many of them actually index them for fast querying.
I think NTFS is doing something like this, right?IMHO the index should be in userland. In this way you can have add-ons for different types of indices. For example, you might want to update the fulltext indexer for some new supported file format, doing that in the fs is quit dangerous and not very handy. By the way I rewrote the index server (its in the tree now), the server supports add-ons and is able to catch up its work from a time when it was not running. In this way you are able to stop the indexer for a while, e.g when compiling the haiku source and start it again later. This could speed up io critical stuff!
The main problem is how does the index stay in sync with the file system. The current index server does not handle file modifications very effective this needs some more work. Furthermore, when a file is deleted we just get a node but in theory we need the path of the deleted file...
One solution for the sync problem is to log every operation in the fs. The indexer would just need to look into this log to see whats happened recently/ when it was not running.
In the ideal case you would also be able to index other none bfs volumes without a log or any indices. This needs some other strategies. Furthermore, a persistent attribute overlay for this volumes would be needed. Think skyos already did this!
What's (IMHO) also missing is a more powerful node monitor. For example, the index server is interested in all changes on the fs. Tracker is interested in all changes in one directory. For this use-cases the current node monitor is not very handy, though.
What do you think to work on some of this topics?
Regarding changing the file system interface, that is a necessity and absolutely desired. E.g. we understand now that queries should not be implemented in the file system, but rather moved to a userland service.I don't understand: do you mean it would be better to parse the query in a userland process and then pass it on to the filesystem[2]? Or do you want to entirely move metadata handling to the userland?
Attributes in files system but index in userland.
By the way an attribute array is missing in bfs!
Don't hesitate to ask if you need more details.
Regards,
Clemens
