Haiku packages, for long time and still now, raised protests among the older and newer users. I’m honest with you, at the beginning I was frustrated, but in the end I realised that it is a very cool feature. So take this as a “congrats” for having the courage of integrating this feature in haiku against oppositions. You know better than me the gains and the losses of this approach so I don’t spend other words about this topic.
I don’t remember when it happened, but in one of my hardware tests a kdl flushed syslog data to bash_history file.
Then I thought how can i check filesystem for integrity? Searching some info around, it seems BFS has only metadata checks, no data integrity checks.
The next thought was: hey! Hpkgs are actually mounted as filesystem. And they are read only! So, why not check data integrity with an hash?
As a proof of concept (with the help of AI) I built and integrated a system where haiku compilation hpkgs and software downloaded hpkgs files saves the checksum in a bfs attribute. So whenever I wan I can run a fsck.hpkgs to check data integrity.
I think something similar happens in Haikudepot or pkgman when they look for checksums parity to the repo_data, but this is a conceptually different approach.
it checks data integrity, not repo corrispondence.
a change of repository does not mean a change in actual installation data/update
it’s not a security approach, as anyone can change the checksum attribute. It’s an integrity check.
a mismatch between the saved checksum and the data integrity (sha256sum) raises a suggestion for automatic fix (if available) in the case of repo downloaded software, or asks a manual step for reinstalling the corrupted package.
I wanna discuss this with you and accept any criticism about this.
What do you think about this feature?
if you want to implement it then you have to make a seperate programme because haiku does not allow llm/ai content at all as part of the os. otherwise you can programme it from scratch.
But actually that’s out of my scope. Though the procedures maybe are related. My idea is strictly about detecting accidental corruption, not fixing intentional damage. This because my solution can be easily bypassed by removing the checksum attribute from the package or changed along with the package data to prove it’s sanity. This is the reason I specifically said it is not a security feature, just an integrity check.
The point was nyanshield uses rpkg-rs which is a native rust implementation of the hpkg format. It would be able to detect errors or corruption within the hpkg archive
That makes me wonder if there would be a way to add this to BFS instead?
This is an interesting problem because files can be modified piece-wise (you can truncate, append, rewrite a few bytes in the middle, …). So, would the hash need to be block-per-block? Would that be a limitation, except for the fact that it would be a lot of hashes to store?
Or could we make it optional and enabled only for some files? Should such files be read only then?
Hmm… I think it might be more practical chunk by chunk (despite the overhead in the payload) because the reader is more likely to consume it in that way; there isn’t (or shouldn’t be) a point in time where a reader consumes the entire payload.
The chunks are optionally compressed with zstd or zlib. From a quick look, zstd does have a checksum mechanism built in so potentially this may be the best way to achieve this at the chunk level without actual changes to the format?
