[XFS] Global Structures completed and tested

Priyanshu · April 11, 2024, 11:03am

I have tried to work with cache and saw the suggestion in the discussion https://review.haiku-os.org/c/haiku/+/5765. Here you suggest replacing the read_pos call with the cache_get(); I have done the same and implemented the Cache_get_etc() for reading the direct offset in case of reading inodes. Now I want to replace the read_pos call in Inode::GetFromDisk where we fill the buffer. I have replaced read_pos with this but it’s giving segmentation faults can you suggest where I’m making a mistake or how to correctly replace them with Cache, Any references?

	if (Cache->SetTo(block) == nullptr) {
		ERROR("Inode::Inode(): IO Error");
		return B_IO_ERROR;
	}

	const uint8* block_data = Cache->Block();
	memcpy(fBuffer, block_data + offset* len, len);

PulkoMandy · April 11, 2024, 11:25am

It’s hard to debug or review just 6 lines of code from a much bigger piece of software.

Please send your changes on Gerrit, so it can be reviewed in full.

Still here are some things to check:

That Cache is not null at that point and is pointing to a valid cache object
That offset * len does not go past the end of the block you got from the cache. If the offset or len is too large, you may be reading past the data that the cache can provide you?

Priyanshu · April 11, 2024, 11:49am

Thank you for the Suggestion. Sorry for this silly mistake, I have resolved the error now I’m thinking of Changing the inode’s read_pos and then submitting a change regarding this and I still have to work with Cache_get_etc.

Priyanshu · April 14, 2024, 6:23pm

Hii all,
I have tried to implement Block Cache and they are working fine inside the xfs_shell now I want to test them inside the haiku itself. I have compiled haiku with the xfs in the image definition and created the image definition using :

dd if=/dev/zero of=fs.img bs=1 count=0 seek=5G

after this the /sbin/mkfs.xfs fs.img isn’t working and while trying to mount this image on the mounting point which is the testing folder, I’m getting this.

@Mashijams Can you help me with this issue?

KitsunePrefecture · April 14, 2024, 6:49pm

Ahoy @Priyanshu ,

I’m not a developer, but seems your Haiku install is corrupted - the ladybird on the ‘VBOX HARDDISK’ indicades it-

Try checkfs -c /boot in Terminal - it may show the error

without -c switch the checkfs command attempt to fix your bfs partition.
I would do a reboot after that.
If it had not fixed, you need a new install of Haiku in the VirtualBox or if you thinked about it and you have a spare copy of your virtual drive … then you can make a copy about it onto the original one … overwritten.
It is useful to copy the virtual drive time-to-time – creating a “snapshot” about it manually as then you can restore a healthy status of your drive without too much struggle to keep the needed at your hands healthy and reliable.

Ah of course, the copy of the virtual drive (the file) should be done while the affected guest OS is shutted down, so the drive does not changing.

Priyanshu · April 14, 2024, 7:24pm

the checkfs -c /boot isn’t giving any error I have also run it without -c as you suggested but it’s still providing the same result. I’ll try to follow up with the other suggestions as well.

KitsunePrefecture · April 14, 2024, 8:09pm

Well, it is not like message

"You have problem here and here "

BUT

you have to look at what reports the checkfs- in details

    15870 nodes checked,
    0 blocks not allocated,     <-- OK
    0 blocks already set,       <-- OK
    0 blocks could be freed  <-- if it is not '0' ,

then your BFS should be cleaned

using checkfs without -c , so not check-only what it means exactly.

e.g.
… after a ‘Force reboot’ on Team monitor panel {CTRL+ALT+DEL}

that skip housekeeping _
write back caches content
and close things as regularly happens …
that ALWAYS write out things from RAM to DISK area before shutdown or close app(s).
However force NEEDED as not applicable the regular way of handling

    direct block runs               16827 (17.02 GiB)
    indirect block runs             1167 (in 48 array blocks, 62.34 GiB)
    double indirect block runs      0 (in 0 array blocks, 0 bájt)

–>Sum the first 2 lines should be equal with FS size which already allocated so size which reported in df command output as FREE minus the TOTAL

→ and double indirect block runs should be zero as once someone warned me, … I assume that means inodes are OK as well, there is no cross-linked and other issues there

79.34 GB, so cca 80GB

~> df -h
Mount Type Total Free Flags Device
/boot bfs 119.0 GiB 39.5 GiB QAM-P-W /dev/disk/usb/0/0/1

119 - 39.5 = 79.5 , so cca. 80GB

It equals.

~> bfs_clean_boot.sh 

   Executing -   checkfs -c /boot   - to find possible/real BFS filesystem erros of installed Haiku operating system ... 

        15870 nodes checked,
        0 blocks not allocated,
        0 blocks already set,
        0 blocks could be freed

        files           13630
        directories     1415
        attributes      459
        attr. dirs      319
        indices         47

        direct block runs               16827 (17.02 GiB)
        indirect block runs             1167 (in 48 array blocks, 62.34 GiB)
        double indirect block runs      0 (in 0 array blocks, 0 bájt)

   Executing -   checkfs /boot  -  to attempt to fix (possible or existing) filesystem errors ...  

        15870 nodes checked,
        0 blocks not allocated,
        0 blocks already set,
        0 blocks could be freed

        files           13630
        directories     1415
        attributes      459
        attr. dirs      319
        indices         47

        direct block runs               16827 (17.02 GiB)
        indirect block runs             1167 (in 48 array blocks, 62.34 GiB)
        double indirect block runs      0 (in 0 array blocks, 0 bájt)

   Re-executing -   checkfs -c /boot  -  to validate fix attempt result(s) 

        15870 nodes checked,
        0 blocks not allocated,
        0 blocks already set,
        0 blocks could be freed

        files           13630
        directories     1415
        attributes      459
        attr. dirs      319
        indices         47

        direct block runs               16827 (17.02 GiB)
        indirect block runs             1167 (in 48 array blocks, 62.34 GiB)
        double indirect block runs      0 (in 0 array blocks, 0 bájt)

 
 BFS filesystem check / possible fix / re-check -- DONE. Bye-bye !.. 

~> df -h 
 Mount             Type      Total     Free      Flags   Device
----------------- --------- --------- --------- ------- ------------------------
/boot             bfs       119.0 GiB  39.5 GiB QAM-P-W /dev/disk/usb/0/0/1
/boot/system      packagefs   4.0 KiB   4.0 KiB QAM-P-- 
/boot/home/config packagefs   4.0 KiB   4.0 KiB QAM-P-- 
/boot/system/var/shared_memory 
                  ramfs             0  28.9 GiB QAM-PRW 
~>

I suggested to reboot whether the ladybird bug disappears from yout system disk icon or remains after checkfs.

Me had not - I had to reinstall Haiku 64 bit
But I HAD further error message in checkfs output that had not disappeared to me.

You can read some additional about it in this threads of post HERE

KitsunePrefecture · April 14, 2024, 8:22pm

Of course your mentors can give better advice than me ,
for your corrupted Haiku install
and
the affected issue you are facing with
after your corruption solved.
I just indicated that waste of time and effort IF your Haiku itself is not healthy

Priyanshu · April 15, 2024, 4:00am

thank you @KitsunePrefecture for this suggestion I’ll try to look into it

PulkoMandy · April 15, 2024, 5:48pm

What???

Please stop guessing and confusing other people with your guesses. It leads in a completely wrong direction, and it wastes time for everyone (for you to write the explanation, for us to read and understand it, for Priyanshu to try to find a problem that doesn’t exist, and then for us to try to set things back on the right course…)

The ladybird just indicates that this is a development build. There is nothing wrong with it.

@Priyanshu I recommend you to ignore what @KitsunePrefecture wrote…

Here are some useful things to check instead:

The filesystem can only return an error code to the mount command. In this case, EINVAL or “invalid argument”. This isn’t very helpful, of course, but you will usually find more information in the syslog (/var/log/syslog). So, you can look here. Of course, this will be more useful if your filesystem is built with tracing enabled. For that, you can configure it if you are building Haiku from the build/jam/UserBuildConfig file:

    SetConfigVar DEBUG : HAIKU_TOP src add-ons kernel file_systems xfs : 1 : global ;

(as usual in jamfiles, be careful about not removing any of the whitespace, otherwise it won’t work). After doing that and building your Haiku image, you will get all the debugging and tracing from the XFS driver. Make sure the driver is properly installed as well (I think it should be in /boot/system/add-ons/kernel/file_systems)

You can also check your filesystem image using the “file” command. This uses a database of known patterns to identify files, including filesystem images. Does it think that that the XFS image is a valid one or does it just say something like “data”?

Priyanshu · April 15, 2024, 7:50pm

it’s showing data here . can you suggest what I should do now?

Priyanshu · April 15, 2024, 8:00pm

How can I verify it?

PulkoMandy · April 16, 2024, 7:02am

Using the ls command to show that the driver is in that directory.

Check more closely the content of the file, for example with an hex editor. Is it empty? Does it contain some data? Does the data look like a valid XFS filesystem (for example, can you find the superblock magic number)?

Also try what I suggested: build XFS in debug mode so you get all the traces, and check the syslog in /var/log/syslog after running the mount command. Do you see any logs from the XFS driver there? What does it say?

Priyanshu · April 16, 2024, 12:25pm

Finally had the error actually the filesystem image wasn’t a valid one. I looked into the /var/log/syslog with the debug enabled and it Traces the error which is Invalid SuperBlock.I have tried some experimenting within my Linux environment.

priyanshu@Bobo:~/haikuos/haiku/src/tests/add-ons/kernel/file_systems/xfs$ dd if=/dev/zero of=fs.img bs=1 count=0 seek=5G
0+0 records in
0+0 records out
0 bytes copied, 0.000348005 s, 0.0 kB/s
priyanshu@Bobo:~/haikuos/haiku/src/tests/add-ons/kernel/file_systems/xfs$ file fs.img
fs.img: data
priyanshu@Bobo:~/haikuos/haiku/src/tests/add-ons/kernel/file_systems/xfs$ sudo mkfs.xfs fs.img
[sudo] password for priyanshu: 
meta-data=fs.img                 isize=512    agcount=4, agsize=327680 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=1    bigtime=0 inobtcount=0
data     =                       bsize=4096   blocks=1310720, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
priyanshu@Bobo:~/haikuos/haiku/src/tests/add-ons/kernel/file_systems/xfs$ file fs.img
fs.img: SGI XFS filesystem data (blksz 4096, inosz 512, v2 dirs)

file created with dd command is a Data file but after implementing the mkfs.xfs command it has converted into a Valid XFS fs. But Haiku doesn’t have mkfs.xfs and xfsprogs packages. So can you Suggest to me What can be done here? here is the Syslog file

PulkoMandy · April 16, 2024, 1:30pm

Yes, the xfs tools have not yet been ported to Haiku. So you have to create the image in Linux, and then copy it to Haiku. Or you can format an USB drive as XFS in Linux, and then connect it to the Haiku machine.

If you run Haiku in a virtual machine you can also create the image in Linux and then attach it as a disk to the VM.

Priyanshu · April 18, 2024, 10:51am

I have attached the xfs image as a disk while running Haiku on the virtual machine and after lots of attempts successfully mounting it to the testing point. There is a little anomaly in the behavior.

I can’t see any GUI representation of my Directories in the testing point which is the testing directory in my case.
While listing the Directories from Terminal using ls command I got up into Kernel Debug mode and the reason is the BlocK no seems to take a negative value in the block_cache_get() method (probably due to endianness) but it’s working fine inside xfs_shell can you tell me what should be done here.
After exiting from kernel Debug mode all the 1000 directories are listed properly, How?
Relevant Screenshots from the issue-

inavlid_cahe1024×768 16.5 KB

PulkoMandy · April 18, 2024, 11:27am

I’m not sure what you mean. The “img” directory seems to be there on the desktop in your screenshot?

You have to implement some hooks to make it appear as a “disk” rather than a directory (to return the free and used space, that kind of things), I think at least the “read filesystem stats” hooks. Here is a very simple example: websearchfs.c « websearchfs « file_systems « kernel « add-ons « src - haiku - Haiku's main repository but you can find more complete ones in other filesystems.

block_cache_get will return an error code, depending on how it is handled in NodeDirectory::FillBuffer, execution can continue normally, handle the error, and still manage to access the disk in some other ways.

You have to check the code in NodeDirectory::FillBuffer carefully to see how it ends up passing invalid values to block_cache_get. In the kernel debugger you can use the “syslog” command to see the last entries in the syslog, just before the crash, this should give you some idea of where the problem happened. Of course, if needed you can add even more logs to trace it better.

khallebal · April 18, 2024, 7:33pm

That’s a known issue from the previous GSoc, currently the directories don’t show up in tracker, the author created a ticket for it, something you might consider fixing if time permits?;

Priyanshu · April 22, 2024, 6:39pm

I have made changes but I can’t seem to locate the errors I’m thinking of pushing a patch regarding this and getting your suggestion there on my work. Sounds Good?

marcoapc · April 28, 2024, 8:15pm

Sounds are always good when work is posted and can be reviewed.

Seeing the progress of the work, as well as tips from the devs, helps a lot.

https://review.haiku-os.org/c/haiku/+/7628