Metadata/database filesystem

What do people think on here about a metadata/database filesystem? Is there any plans or thoughts of plans for Haiku to use it, maybe as default?

IMHO, it would free us from the maintenance and pain associated with the standard folder-based system we all use. Here’s an article I wrote years ago which shows the numerous advantages over the old way:

"Towards A Single Folder Filesystem"
http://www.skytopia.com/project/articles/filesystem.html

Have you looked into the properties on the BFS used in Haiku ? its very much like a metadata/database file system.

I actually prefer this nicely written description of BFS on Haiku :slight_smile:

Hi, if I’m being honest, I’ve only just found out that Haiku is available for download! (been occasionally visiting the site, but never saw the news story on Slashdot unfortunately). I’ve just read the page at: http://www.haiku-os.org/about and that mentions it too. Smashing stuff :slight_smile:

I’m really hoping the implementation is similar to the one I have on my site above. Is searching a million files’ metadata instantaneous (i.e. less than 0.1 seconds)? Is there a single ‘window’ which lists them all until filtered? Are programs set up by default to allow saving of metadata by the user (and is some automatically generated metadata added)? I would appreciate any feedback about the article - I really think the database system is the way to go. In the meantime, I’ll set about installing Haiku on CD to try out.

Any thoughts on a Haiku emulator for Windows so that people can try it out more quickly, or would that ruin the lovely speed?

Haiku works fine in virtualbox and VMware play on windows, although if you own a multi-core processor it would be a shame not to use the real hardware.

Some of what your suggesting is hardware limited. disk acess speeds and what not. Unless you loaded the metadata to the ram at system boot, even then you would still incur acess penaltys. I will say this. Haik u search is dumb fast. Faster then anything I have tried and I have a media drive loaded with music “50,000 songs” and it chomps through those files in a few seconds.

the OS can only work within the capability of the hardware.

[quote=twinbee]Hi, if I’m being honest, I’ve only just found out that Haiku is available for download! (been occasionally visiting the site, but never saw the news story on Slashdot unfortunately). I’ve just read the page at: http://www.haiku-os.org/about and that mentions it too. Smashing stuff :slight_smile:

I’m really hoping the implementation is similar to the one I have on my site above. Is searching a million files’ metadata instantaneous (i.e. less than 0.1 seconds)? Is there a single ‘window’ which lists them all until filtered? Are programs set up by default to allow saving of metadata by the user (and is some automatically generated metadata added)? I would appreciate any feedback about the article - I really think the database system is the way to go. In the meantime, I’ll set about installing Haiku on CD to try out.

Any thoughts on a Haiku emulator for Windows so that people can try it out more quickly, or would that ruin the lovely speed?[/quote]

Without being nasty, your article is just a bit late(10 years), regarding the BeOS capabilities and its marvelous filesystem: BFS.

Metadata are (already) used extensively in all the system. It’s a fundamental part of what motivated the rewrite of BeOS (aka Haiku).

BFS is a great substitute for: searchable MP3’s tags, Windows’registry stuff or things like that…
You can query an audio file by author, title or album, or find the appropriate application to open a specific file type.
Even filenames extensions are obsolete, since a file have a mimetype attribute.

BeOS has not had the commercial success he deserves!

Welcome in Haiku !

Sorry about the thread necro. (Well… not really…)

What would it take to patch Haiku’s filesystem to use MongoDB or some other NoSQL database? That is a question that I wonder if it would make a mark on the 21st century.

FIrst, what it would bring to the table from the perspective of a power user:

  • Directory sharding to allow processing of massive data to be spread over multiple CPU cores when processing multiple records.
  • Automated backups via replica sets over the Internet.
  • Multiuser support via web authentication
  • Cloud-based filesystems made simple
  • It might even allow the use of an existing host kernel to add to the portability and 64-bit support.

Second, my suggested way of implementing it if using MongoDB:

  • The MongoDB way to implement a tree structure such as directories involves storing the path of the file/record in an array data structure. The query pipeline system can figure out the rest from there.
  • Mimetypes will be another tree structure to indicate inheritance chain of OOP.
  • MongoDB is hash-table based rather than SQL tables so that unneeded fields need not take up space nor are they limited to a fixed length.

My perceived downsides:

  • It would need a flat filesystem or partition underneath to implement the MongoDB infrastructure.
  • If implemented on top o a POSIX OS kernel instead of the current one, it may inherit some clunkiness.

It would perhaps be better to start a new thread.
Anyway, the main filesystem will probably not change, but Haiku has a modular kernel and also an userland filesystem solution. This means new filesystems can be added without limits to your creativity.
For example, we have a “google” filesystem, which turns queries into Google searches. There were plans for an IMAP filesystem at one point. Good candidates would also be online file storage systems like Dropbox. And, you could try to plug the filesystem interface onto many things, as you see fit.