I hope you are doing well. My name is Mohamed Emad and I am very interested in the XFS write support idea for Haiku. I find this topic exciting both technically and in terms of its impact on Haiku’s usability. I am highly motivated to work seriously on this idea and contribute meaningful progress.
I have already started learning more about XFS and Haiku’s filesystem internals. Some of the resources I’ve been studying include:
I want to prepare myself well so I can make valuable contributions. Could you please advise me on what skills, knowledge or tasks I should focus on to become a strong candidate for this idea? Are there specific parts of the codebase or early steps you recommend I start working on?
Thank you very much for your time and guidance. I’m looking forward to contributing!
You are welcome !.. especially with your interest !!
Just after 2 mins you posted here and I opened your fresh thread - I realized just right now ! - so you can imagine how this topic interest me regarding available FSs on Haiku !..
Have a great coding journey with XFS or else, here , contributing to our project …
I have recently merged several patches from @Mashijams who worked on xfs in a previous Google Summer of Code. I think he can give you an idea of the current state of things better than I could.
It looks like xfs can now be mounted read-only in Haiku but there are still issues in some cases. Did you already set up a development environment where you are able to test this? For exampl building Haiku from Linux and then running it in a virtual machine?
The next steps would be testing that, fixing issues with the read-only mode if there are some, maybe resume work on the next big step which I think is adding caching support to the filesystem. Have a look at the existing work-in-progress changes for xfs and their respective comment: https://review.haiku-os.org/q/status:open+-is:wip+xfs
I’m not a developer, so I cannot be your mentor, but otherwise I can suggest some stuff to read and person to contact with besides the Haiku developers who may have been your mentors.
First, you can read the previous GSoC trainee’s blogposts from
GSoC2022 - XFS filesystem support may @Mashijams (Raghav Sharma) can help you
first reading his status reports from the past and helping out to extend actual XFS reading capabilities, directory functions to writable as well, and extending the filesystem size too.
As he worked on it in the past and even provided some patches
those were merged in master branch - aka nightly version - even recently (2014-02-14, 2014-02-07, 2014-02-14).
Also it would be profitable if these new filesystem stuffs would be available in Haiku’s Drivesetup application and mounting service – at least on nightly.
Thank you for the clarification. I’ll reach out to @Mashijams to better understand the current state of the XFS driver and the recent changes.
At the moment, I’m still in the knowledge-gathering phase. I tend to build a solid conceptual understanding first so that when I start implementing, I can move more efficiently and avoid unnecessary mistakes.
I’ve been using Linux for around five years and I’m currently on Arch, so setting up the Haiku build environment on Linux and running it in a VM shouldn’t be an issue. That will be my next concrete step once I finish mapping out the current XFS driver structure and understanding its integration points.
Thank you for outlining the next steps, that helps a lot.
Thank you for the detailed pointers, I really appreciate it.
I’ll go through the previous GSoC 2022 blog posts and status reports to better understand the progression of the XFS work. That should give me a clearer view of what was already achieved and what remains incomplete.
The idea of improving integration with DriveSetup and the mounting service is also very interesting. I’ll keep that in mind while exploring the current driver behavior on nightly builds.
Thanks again for taking the time to share these resources.
I hope you’re doing well. I’m currently exploring the XFS write support idea and was advised to reach out to you since you previously worked on the driver.
I would really appreciate your help in understanding the current state of the implementation. In particular, I’d like to better understand:
1- The overall architecture of the XFS driver in Haiku
2- What each main folder/module in the driver is responsible for
3- Which parts are considered stable vs incomplete
4- What libraries or kernel subsystems the driver depends on
5- Known limitations or edge cases that still need work
6- Any design decisions or challenges you encountered during your GSoC
I’m currently studying the XFS on-disk structures, and I want to build a solid mental model before attempting contributions.
If you have any recommended starting points in the codebase or small tasks to begin with, I’d be very grateful.
Yepp, that is rather recommended to see how errors/workarounds happened and Raghav’s solutions also,
but for a short, crisp overview of XFS status - it’s better to read the Haiku site about it as you wrote above + maybe the 4 patches as additions.
(I added them as links to them here, from the Gerrit’s Log view. Unfortunately since the old Gerrit site was often assaulted by bots a new Gerrit site view was setup such way where
–> there is nohrev – Haiku revision number of the affected patch – info, so not known for first sight which Haiku version contains the affected patch
and
–> search function that previously enabled to find all the patches which relates to a specific pattern like ‘xfs’ in Log view for example does not exist anymore.)
That is not Gerrit. Gerrit is only the code review tool at review.haiku-os.org.
The old log browser was run by cgit, which has now been replaced by gitore. That saves a lot of work for the server, but unfortunately gitore is a bit newer and missing several of the functionalities. If there’s a Rust developer around willing to help with that, your help is welcome!
ACK for cgit/gitore.
For me that was the ‘Gerrit’, as of course sometimes I clicked on links in logfiles that opened at real Gerrit reviews, so I know the Gerrit as well. I had not the exact naming of cgit service -
Sorry for any misleading definition above …
It would be good if gitore would know what cgit was capable, so
I wanted to give you a quick update. At the moment, I’m intensively studying filesystem concepts, architecture, design, and implementation from multiple sources to build a very strong theoretical foundation before diving deeper into the xfs driver work.
My goal is to reach a high level of understanding so that when I start implementing or debugging, I can reason about the system properly rather than approaching it superficially.
I’m working through the material systematically and aiming to finish it as soon as possible, then move directly into hands-on testing and contributing to the current read-only issues and caching work.
@PulkoMandy
I’ve also read the previous contributor’s blog posts, including the final report and project updates, along with reviewing some of the patches he submitted.
At this point, I think that I have a solid general understanding of the project and what has been accomplished so far. I’ve also started forming some initial intuitions about the possible timeline and development plan moving forward.
Buring the GSoC application period (which is now) you will have to prepare your application proposal as well as do a code contribution. GSoC mentors and other developers are here to help you. You can share a draft of your application and we can comment on it and help you improve it, note which areas need further research, what seems too ambitious (or not enough, but usually things are too ambitious). Likewise for the code contribution, you can submit work in progress changes when you need help.
For XFS read support is mostly complete with some gaps and bugs.
I fixed some basic functionality bugs already (My patches shared above) and working on other important gaps like directory block fragmentation, cache, and then of course write support.
Now to answer your specific questions.
The overall architecture of the XFS driver in Haiku
The entry point is going to be kernel_interface.cpp file, it contains driver hooks that is expected to be implemented by all functional file systems in Haiku.
Check this out: The Haiku Book: File System Modules its documented good and you can understand some Haiku specific terms such as Volume, vfs_hooks, etc…
2. What each main folder/module in the driver is responsible for
File name itself will give you an initial good idea I guess, still if you have questions you can ask here.
Which parts are considered stable vs incomplete
Well let’s divide on the basis of functionality.
Directories:
ShortForm → very stable and performant.
Block → very stable and performant.
Leaf → stable.
Node → Unstable(?) and slow.
Files:
Extents based → stable but slow.
B+Tree based → stable but slow.
For Symlinks and Attributes assume all types are stable and performing good enough except maybe node attributes..
What libraries or kernel subsystems the driver depends on
Don’t have to worry about this all system required headers are already present in system_dependencies.h header file.
We have to implement vfs_hooks for functionality we need (for example: vfs_create_dir()) and go from there,
Known limitations or edge cases that still need work
Extents fragmentation currently doesn’t work.
What I meant is suppose we are reading a Directory block which is made up of 4 FS disk blocks, our code is currently hardcoded to read continuous 4 FS disk blocks but it could be fragmented in multiple extent entries.
Need to add support for Block and File Cache while improving performance.
I restarted working on XFS recently and maintain my commits here: GitHub - sleipbyte/haiku at xfs
Some I have already sent for review, some are WIP, and some I have yet to send.
Any design decisions or challenges you encountered during your GSoC
Oh well I struggled to even read data from disk and validate in the beginning
But anyways my mentor @PulkoMandy helped and I was able to figure out majority of stuff.
That’s good to know! We should keep it in mind if there is a GSoC project about this, to avoid creating conflicts with two different work-in-progress versions of the code.
Thank you for the clarification. I’ll start working on my application proposal within the next couple of days and share a draft with you for feedback as soon as I have a solid version ready.
Today, I’m also going to start setting up my development environment. If I run into any issues that I can’t resolve on my own, I’ll definitely reach out and ask for your help.
Thank you for taking the time to explain all these details about the project this was incredibly helpful and gave me a much clearer picture of the overall architecture, the challenges involved and what to expect during the development process.
I really appreciate you sharing your experience and insights from your contribution.
Sorry for not being active over the past four days. I had several university assignments and was spending almost the entire day on campus.
In the meantime, I watched a few conference talks about xfs, the problems it solves, and details about its implementation. I’ve also been reading a lot of related resources beside the xfs source code. I now feel that I have a solid foundational understanding of how the file system works.
At this point, I’d like to move on to implementation and testing. I was wondering if you could guide me or recommend any resources regarding the Haiku build process, especially from the perspective of an Arch Linux user.
I’ve also recently connected to the IRC channel and am currently working on my GSoC proposal. I’ll send you a draft version as soon as possible.
No problem with that, I’m myself in the middle of helping my soon-former roommate repaint her new appartment as well as packing all my stuff to move to a new appartment myself, so things have been quite busy here the last few weeks.
That’s what happens when juggling contributions to an open source project with other commitments, and we pretty much all are in that situation here in Haiku.
This will get you set up and tell you how to build jam and gcc and configure your Haiku build.
Then you should prepare for the two ways of testing the XFS filesystem: building xfs_shell (that you can then run in Linux) and running the actual filesystem in Haiku (probably in a virtual machine). I think the linked documentation above, combined with the documentation at Welcome to Haiku internals’s documentation! — Haiku internals documentation should provide everything you need. But if you hit a part of it that’s unclear, or you’re not sure what to do next at some step, feel free to ask
Make sure you have a good setup with a quick edit/compile/test cycle if possible. That will be a good investment of time that will quickly repay itself in the next steps of your work!
project … 
