I also think the current package management system works well for all the reasons @PulkoMandy gave, but I think you could also add appdir support.
For example, I have mainly just native applications with few dependencies installed (probably not as pure as @PulkoMandy has though ). If I want to do something with Libreoffice, I can install the hpkg though HaikuDepot and it will pull in loads of dependency hpkgs as well. These all get mounted on boot.
If we had an appdir support as an additional feature, for things like Libreoffice, we could just use the appdir version. I wouldnāt care too much if the libraries it uses get security holes - Iām behind a firewall, itās not a network centric application.
but I donāt understand what youāre missing in Haiku and its package system for this to happen. If someone provides an hpkg file that has an app and all its dependencies in /boot/system/apps, and a symlink to it in the deskbar menu, does that not work well enough?
If someone provides a zip that extracts in /boot/apps/, is that not enough?
Ok, it is a few extra clicks to enter the folder and click on the executable, but if thatās the only problem, I think it can be solved without making much changes.
Several of the developers have used BeOS (of course), as well as AmigaOS and Mac OS. There was an extensive review and discussion of all the existing options, with people who knew in some detail about each of them (both developers and users of Haiku). And then an attempt to design something that would work for all the identified uses. Being able to easily install a standalone app with no dependencies and no package manager was one of the requirements, and that was taken into account in the original design, and it was implemented, and it works.
It just happens that the Linux-style option is the most popular one. I donāt know why that is, but I donāt think it is because of technical limitations of the package system, or because the people designing the package system did not understand the other ways to do things or didnāt care about them.
Ah right, I see what youāre saying - a āfatā hpkg with all the dependencies? Yes that would be good. I wonder if haikuporter could be extended to produce one of these?
Commercial/independent software providers can already do āfatā bundles for Haiku.
If you are a small/independent developer, and your project is not overly complicated in term of dependencies, it is possible to create āfatā hpkg including external libraries. There are GUI apps in SoftwareDepot to assist you in your package creation (add you library binaries directly instead of declaring them as dependencies) if you do not want to do everything manually.
If you want to scale this and are looking for automation, you could even use the available/official Haiku/HaikuPorts docker containers to create you own haikuport & associated repository : haikuās Profile | Docker Hub. Create your recipes in a way that dependencies get compiled and included in the final package instead of being added as a dependency. You can then push that to your own hosted repository.
I see no added value for these fat hpkg in the context of the official HaikuPorts, but at least people wanting to do/test this can do it
All you need is haikuporter and the package_repo tool running on Haiku. Why make this complicated by bringing in Docker (and Linux with it)?
And you can even do it without haikuporter, by invoking the package tool directly to create your package, since you donāt need dependency management and chroot for building.
All you need is haikuporter and the package_repo tool running on Haiku. Why make this complicated by bringing in Docker (and Linux with it)?
My comment was more about giving the two extremes (from manual hpkg creation to full build factory running on a linux server).
I agree that haikuporter and the package_repo tool is probably the best compromise for most users.
If you are willing to use āfatā hpkgs why not link the application statically instead, it will give you the same disadvantages (no security updates) but without the size inflation: all uneedrd symbols in dependencies are not included.
If you are thorough enough this can give you a single ELF file with everything in it, on haiku embeding ressources in ELF files is also easier to acomplish (though not with ported software, and sadly without a proper elf section)
Hi @nephele This looks like a bug or some specific problem. I am running Haiku (32 bits) on a laptop with only 2 GB or RAM (an old Dell 11z) and I can run LibreOffice & Calligra without issues (well, they work a bit slow, but works).
In fact, with only 2 GB, the only application that really struggles to run is the Ephiphany browser.
That I am sure is part of the issues from the time yesā¦ I am pretty sure binaries from back then are still broken though that assumed .config would be writable though right?
Most users arenāt on the ML and STILL arenātā¦ its mostly developer discussion obviously. Even some developers arenāt on the ML because they find MLs annoying to use etcā¦ and only interact via other means.
For other issues public polls in a front page blog post were doneā¦ again provide the link to said poll for this issue there just isnāt one. Iām not blaming anyoneā¦ but I that is how it went down and it frankly meant that the design of packgefs suffered due to much less public scrutiny and critique.
At that time we had no forum. So, where else would we reach people? Ah, yes, blog posts. What about weekly reports on the blog? Sure we had that, you can read them here: package management | Haiku Project in several runs, in 2011, 2013, and 2015, before the switch to package manager Haiku was finally done in 2017 (and also announced on the blog).
And, of course, this was AFTER we ran the R1 feature poll, of which results are archived here: FutureHaiku/Features ā Haiku (proposal #31). 59% of users voted it as a āmust haveā. Only 48% of developers did (14 out of 29). So, if anything, this shows the users wanted package management more than the developers did.
So that makes you wrong on two levels:
We did a poll
The user did decide that package management was a requirement for R1, while the devs didnāt agree
The users were listened to against the advice of the devs
There is no .config on Haiku nor on BeOS and there never was. I guess you meen /boot/home/config? Several parts of this were made writable (config/settings, config/cache). Some others are not (config/lib and config/add-ons, but Iām not sure why you would want an app to write there).
Can you find any broken binaries (from BeOS)? I am happy to investigate and provide fixes. But a vague āold binaries from back thenā does not cut it. I am not interested in hypothetical problems that no existing binary ever hitsā¦
So, do you have even one example of an application that is broken by this? Just one example. It canāt be that hard to find one?
Yes, neither did the devs. The original plan was to have the underlying BFS and the packaged files mesh together transparently. it caused two problems:
It was confusing, as you never knew if a file came from a package or from a standalone file, leading to hard to debug situations
It was slow
The second one could maybe be fixed, but the first one couldnāt. And so a better solution was designed. It was done quite late in the implementation. Yet, it was well tested and found to not cause too much problems (despite what you insist saying without willing or being able to offer any example of apps that were actually broken).
I think those are both fair, reasons to do somethingā¦ and think the direction taken was ok, its just the implementation of that as non-packaged, rather than defaulting to read / write and having a read only packaged folder for those specific things ā¦ is very confusing to date as well as user hostile. What really should exist is that packages should be essentially the first snapshot of a copy on write filesystemā¦ ala ZFS.
In fact I think that is something ZFS itself actually can do already,ā¦ but would probably suffer scalability issues in heavy systems. But as a proof of concept I think it would work out the boxā¦ that a copy on write filesystem with per package read only snapshots would work.
What we have now isnāt what Iād call a solutionā¦ as it creates more problems than it solved. Note Iām not promoting ZFSā¦ just pointing out that it does things in a way that would provide an example of how things could be done. Haiku doesnāt really need all the heavy features of that, but it could adopt the copy on write aspectā¦ to retain performance with many combined filesystems just like ZFS can. The cost you pay for this is disk spaceā¦ which in todays systems I think is acceptable for the use case.
Where things went awryā¦ I think is the assumption that all this had to be done via what were equivalent to union mounts on Linux, which is very limited, but that isnāt what ZFS does at all to achieve this feature.
Switching to ZFS would also require removing queries, which will break a lot more things, including making Haiku itself not bootable at all. Does not seem like a great solution to me.
Also, snapshots are far from doing what we need. Letās imagine the following scenario:
You install package A. A snapshot with A is created
You install package B. A snapshot with packages A and B is created
Now you want to uninstall package A. There is no way to revert to a snapshot containing only B, because there is no such thing.
And thatās just a simple case with packages. Now letās see with allowing writes to packages.
You install package A. Your ābaseā package snapshot is updated to include A.
You edit some files from package A, this is stored as a new snapshot
Now you uninstall package A. What happens then? The āchangesā from your edit now apply to some files that donāt exist anymore.
You notice that you made a mistake, and reinstall package A. Do your edits reappear on top of it? Are they gone?
Another example:
You install package A.
You edit some files that are inside this package
Now you update package A, changing the initial version of one of the files you edited. Do your changes get erased? Do you get a merge conflict or something like that?
Really I donāt see how this is user friendly. The current solution where there are places that can be written directly, and places that canāt, is much easier to wrap my head around. Especially now that Tracker clearly shows it by greying out the background in system folders that canāt be written to. This is very easy to understand for users: āsystem folders are read-only, donāt touch them, if you want to add some things use the non-packaged folderā. I donāt think you can describe in one simple sentence like this what would happen with a ZFS snapshot based system?
What we have now isnāt what Iād call a solutionā¦ as it creates more problems than it solved.
@cb88 What are those problems, exactly? Just wondering, because being able to design something requires that you know what the actual problem is first. Telling people what you think is easy all of the time and then not being able to do those things yourself sounds more like you donāt actually know what youāre talking about. If you did, then you would be able to implement all of your āeasyā suggestions yourself.
I see this all the time in dev circles, of taking for granted that existing solutions were well designed or that they will work for other situations. Good designers donāt just import something else like ZFS or Linuxās graphics stack or wayland just because itās popular or used on other systems, they define the problem first, and only then do they come up with solutions that solve the problem, is easy on devs and/or users (although, devs tend to overcomplicate things for the exact reasons given above, so users are more important, imo), and has decent performance to power ratio.
A great example of taking designs for granted as good is Unix and Linux, which are, imho, terribly designed and have had many criticisms early on that have largely been ignored and forgotten. The people who hate unix hate it for a reason - it was designed for teletypes where every character printed cost something, and was then shoehorned into being something itās not. It also didnāt actually follow unix philosophy because of the obsession with ambiguous character streams over structured data, which has led to one of the biggest blunders in computing - unix shells and their POSIX āstandardizationsā.
The impression I get is that BeOS/Haiku is a user operating system first, hence why it has a full desktop setup. Using complicated enterprise filesystems that requires a bunch of tuning does not make sense for this type of system.
an implementation based on zfs would be feasible where each package gets extracted to one zfs āsliceā and thise get union mounted.
But this still has problems with rw data (basically everything pulkomandy pointed out above) since it is not clear what package has to be overwritten as such. As a result the OS that does use thus way, FreeBSD, uses a competent traditional package manager. packagefs like magick just doesnāt work with ājust cowā
FreeBSD boot enviroments are also an interesting solution to the same problem, but quite complex and hard to grasp compared to packagefs
edit: the normal advice on FreeBSD is also to not alter anything that pkg touches, instead using /usr/local
FreeBSD also deliberately has a flag for file immutability to prevent tempering, even by root (for example the kernel file)
I cautioned you not to take this as an actual suggestion to use ZFSā¦ the point was not to suggest actually using ZFS AT ALL. But pointing out that the claims made in favor of packagefs being designed the way it is, were baseless because multiple existing FS have already solved those problems. Packagefs ended up the way it is probably due to being corralled into thinking about he problem in a certain way namely unioning of FSā¦ which is not a requirement to implement the features required of package management.
Unioning means all the unions can be resolved simultaneouslyā¦ but that isnāt actually a requirement of any package management system ever. Copy on write filesystems solve this problem with a single unionless file tree, with versioned snapshots with the data being copy on write on diskā¦ its fast, its performant and it allows filesystem versioning required to do rollbacks exactly like we have in Haiku today.
I NEVER SUGGESTED SWITCHING TO ZFSā¦ GOT IT!? I how many times do I have to reiterate that it was being used as an example of how things have already been made to work in another FS that disproves that packagefs had to be designed the way it was.
The problems are today, that BFS is old, and is a dated design with some design flawsā¦ Pulkomandy also brings up no filesystem level searches but several Haiku developers have already stated that this is undesirable anyway and should be moved to a database. Most developers from what I can tell see BFS queries as a detriment rather than a boon because it fails the test of a filesystem doing one thing and doing it wellā¦ BFS is a rather slow FS because of this.
If I must format my idea as a suggestion (I guess because people feel that suggestions must be shot down or something, rather than discussing the pros and cons of them) I will say this.
Copy on write with snapshoting. (no unions)
No built in search in the on disk format (use a database instead like every other modern system it was a mistake to build this into the filesystem as it over complicates things, no change to the end user use of Haiku though because that functionality would get emulated by the operating sytem) Queries remain tightly integrated but are not directly tied to the on disk block format.
ZFS like bitrot resistance (block level checksum would be nice, with optional copy of data for single volume bitrot recovery). It should just be an FS though and not also try to be a raid, if that ever gets implemented do so separately. It should be able to redistribute itās blocks if it is ever resized or its on disk layout changed.
Ultimately must keep resource usage low to implement these features. With a copy on write FS the main thing you spend extraā¦ is disk space, not ram or CPU performance necessarily.
With separate query database you also get the option to optimize for speed (more in ram) or low resource usage similar to BFS where its mostly just left on disk in an easy to query format (just not directly in the filesystem data structures).
). If I want to do something with Libreoffice, I can install the hpkg though HaikuDepot and it will pull in loads of dependency hpkgs as well. These all get mounted on boot.