Database as a file system

Was already looking for where it is planned (and discussed). Sadly didn’t find much here or any meta tickets or similar. Maybe you can point me to correct discussion/ticket?

Thank you for your work. :slight_smile:

There is not much discussion, just a short wiki page about limitations of BFS: FutureHaiku/BFSIssues – Haiku

Certainly this could benefit from more details. Personally I don’t know enough about BFS structure to understand most of it.

2 Likes

Don’t get me wrong. People are doing a good job working on different topics. But this specific one seems to be one nobody is actually doing something but everyone having an opinion on how not to do it. :wink:

I’m just not sure of this is not just everyone blocking everyone from moving.

If the goal is to just improve what is there: Specific tickets should be opened and someone should work on them one by one. This would however in no way interact with what this thread is about.

If the goal is to make something that is new: Then people must accept it to break with old ideas and maybe not matching such lists at all. I mean half of that list is about specific data structures that would not be present in a new design to begin with.

Or maybe both is wanted or needed, but then the teams need to figure out each their own topics in their own threads.

Maybe it’s also about people fear that having something new will remove what is there already. History has shown: no, the pile is just getting bigger. :wink:

Everyone have a good day!

1 Like

I was thinking about a few things recently (in different contexts). And I think I stumbled over two things that could improve database-ness a good bit with very little code changes:

The first would be to support POSIX O_EXEC and O_SEARCH, as well as Linux’s O_PATH. They are all basically aliases (POSIX specifically/explicitly allows to use the same value for all of them). If set in open() (and friends) they open the file for filesystem functions but no data functions. This allows them to be used e.g. for *at() calls without the need to get them to the point you can actually do I/O. Plus the topic of permissions is slightly different.

In Haiku this is already a thing. Haiku however uses the classic model of O_RDONLY with O_RDONLY=0. What I would do here is to add the constants tu the corresponding header and reserve a bit for it. Also check in the kernel about that bit and kernel-land alias that to O_RDONLY (which is a superset of O_EXEC/O_SEARCH/O_PATH, therefore a safe kernel land aliasing).

This change will also improve general POSIX and Linux compatibility with likely less than 10 lines of code change.

The second change would be to support data sets that are not classic files in the filesystem. Those files would be always zero bytes (just like for example sockets, fifos, device files, …). The sole purpose is to use them to store extended attributes plus keeping the inode alive (with the inode number being the primary index for the filesystem basically, so making them a true inode makes them first class citizens, just like extended attributes and stuff).

This is already “the Haiku way”, see e.g. Person files.

This would mean to define a new inode type (within S_IFMT, which might need to be adjusted). It would break the binary compatibility towards old code (but not towards newer code). In the most simple case the kernel would only need to check that no write is ever done to such an inode. Any filesystem checking would check that they are 0 byte (and if not allow to convert them to a plain file or delete the data).

Creation could be done via mknod() (and friends). Also there could be an ioctl() or similar to convert an existing 0 byte plain file to the new type.

I think that those two changes together would bring the most improvement at the lowest work right now. And specifically the first one would clearly also be good as it would improve POSIX compatibility at basically no cost at all.

What is still missing is proper references. Maybe at some point an extended metadata could be referencing another inode. I don’t think that would be much of the problem. However it might need full hardlink support first.

As always, happy to discuss here or on IRC. Also happy to provide/discuss specific patches.

3 Likes

I am not really sure what the second change is intended to achieve over what can already be done with a zero sized file with attributes?

References are something @grexe is working on I believe. **ANNOUNCEMENT** SEN + Personal Knowledge Graphs Book featuring Haiku

1 Like

Hi everyone! I’m new here. Here is my problem: I do some research and have downloaded a huge amount of docs. As I explore these, read some of them, I reorganise and add new folders. I also rename the files with suffix codes indicating whether I have explored, or (partially) read, etc…What bugs me is to BACKUP after having moved files. I don’t want to run a daily full backup and wear out my storage drives (am I making a fuss here?). I also have an excel file for planning my study. I’m thinking of reorganizing and renaming files in virtual (database) filesystem instead of continuously changing the os filesystems. Of course, when the content of a file changes, it has to be backed up. What is the simplest solution for me? I use windows (7) and have done some Python in the past. Thanks!

A bit offtopic, but still.

In exactly your use case I would keep the original files intact and rather create symlinks to each document I touch. Then I can rename these links, re-arrange them in whatever folder structure etc. Symlinks do not take (much) space on disk. In case of Windows, for the same purpose there are *.lnk files, which can be sorted in similar manner.

1 Like

Thanks a lot! I have found this guide (below) and download links. I will let you know how it goes. https://www.howtogeek.com/16226/complete-guide-to-symbolic-links-symlinks-on-windows-or-linux/

I tried Link Shell Extension to create hard link clones of folders (Ihave a >30000-file folder to hard link). Unfortunately, hard links don’t seem to work as expected: when I modify a docx, the link is broken and the “other file” is unchanged (except for txt files). Also file browsers indicate these files are actual copies, not simple 0-byte pointing links.

You need softlinks for that,not hardlinks.
Softlinks will do exactly what you expect,they’re just a pointer to the actual file.

Softlinks show only (their) names. I need to see the size, date, etc. of the target files. [Every time I create a clone with symbolic links, explorer “stops working”. But the clone is created.]

Your problem seems to be more with MS Office and its files. In Windows 7, a hard link shows the size/dates as you want, and a soft link doesn´t.

But Office will save your modifications on a temp file, delete the original one, then rename the temp file to the old name. In this, the “linkage” is broken.

If what you want is just a way to backup those files, a full backup will be easier, and work better. No need to worry about damaging the destination storage, but if you really insist, a normal hard disk is pretty cheap nowadays.