Haiku’s initial goal was to more or less clone BeOS the 1980s-style operating system written in the 1990s by people already nostalgic for a past era. For the most part we don’t see exciting modern conveniences for Haiku such as sophisticated GPU features, and support for modern hardware features is lacking. If it’s your contention that it’s not a retro-computer system, what is it?
Just porting and maintaining all the dependencies without a proper infrastructure (recipe files, and checks to make sure they actually work) would be a lifetime of work. With haikuporter, it became possible.
Haikuporter needs running the build in a chroot where the packages can be mounted/installed. Before package management, the prototype versions of haikuporter relied on packages already installed on your main install, there was no way to uninstall or downgrade something, and dependencies were not checked, so most package did not actually declare their dependencies and you’d have to figure it out manually everytime you wanted to build something.
I don’t know how they achieved it for Windows, but surely there is some “package management” in a broad sense somewhere to build all the dependencies and package them together with the LibreOffice installer. And Windows also does some quite complex work to handle shared DLLs, with the WinSxS directory and its hardlinks. This is a source of confusion for Windows users, as it looks like that directory is using up a lot of space, but in fact it isn’t, because it’s hardlinks to files stored elsewhere.
So, it is very different, maybe we can call it “dependency management” instead of “package management” in Windows case, but still, you need something like this, to handle a large number of dependencies, keep them up to date, etc.
The developer of Insite Constructor reappeared after many years to do a fix before (Zeta era) so might still have the source code around.
Great programme for the time, the big problem with using it years later was that it couldn’t highlight custom CSS classes - can’t remember if it was at all; or only if they were defined in a shared file.
I bought it so late I wonder if I was the last person to pay for it!
2 Likes
No centralized package management and dependency resolution solution is needed to build LibreOffice. It can be some ad-hoc solution per project, in simplest form a copies of all dependencies and patchsets. Some existing dependency management be used like Nix/Pacman/etc. Some build systems such as Meson provide its own dependency management that allow for example building GTK application with GLib, GTK, GStreamer etc. by using only meson setup; ninja install.
PS: I am not saying that HaikuPorter and package management is not needed, it makes life easier, but it is not strictly required to port and build complex software.
Yes, sure, it can be, but the people who tried to port OpenOffice that way in BeOS days did not succeed because it was too much work for them. And my experience with other projects tend to agree with that: haikuporter (and packagefs, since it builds upon it) is very helpful in managing that complexity. Doing this manually for each project is not sustainable if there is just a small team of people porting software to Haiku (maybe it is if it is handled by each project own developers? but even that, I’m not so sure…)
So, yes, of course it is not strictly needed. But it shifts things from “a lifetime of work” to “a few months of work”, and, I think we all have better things to do with our lifetime than just managing to port LibreOffice 
Would a more traditional package manager also work? Yes, certainly. For a time there was pkgsrc for Haiku and there was also TiltOS, a fork of Haiku adding its own package manager (called box) as well as an X server, that one had GIMP running in X11 on Haiku long before we got there with the new package management system. These things would not happen without a package manager, because people would spend way too much time doing a misstep in a “make install” and breaking their Haiku install (ask me how I know). Personally I am a LOT more happy with the safety net of a mostly immutable /system directory and the ability to revert back to an older package state.
To me it sounds a bit like (to take an obviously exagerated example) “why do you put shielding on your nuclear reactors? that’s a waste of concrete, if you know how to operate your reactor, it won’t explode!”. Yes, I am very willing to pay the price for the extra concrete in that situation. And likewise, although in the case of a crashed Haiku system it is just some lost configuration and a few hours wasted to reinstall everything, personally I think the safety net of the package manager and the ability to test packaged software in haikuporter without touching my main install is quite useful. Even if the cost is something like 1% of the RAM on my computer.
Would I be happier if Haiku could run on 128MB of RAM? I don’t know. I don’t have any machine that would benefit from that. We’re talking about 25+ years old hardware here. Sure, it would be nice, but would it really be useful? For me it is not interesting to spend my time on it. But if someone else does, cool.
Likewise, if someone actually puts in the work to make it easier to build an Haiku image without packagefs, sure, why not. I will also not put the work on it myself because I like working with the safety net. The past experiments with this (I think the most recent published one was by s40in?) shows that it is not so hard (you can just copy over the files from an existing booted Haiku install into a new BFS partition, I think?), yet no one offered to add that to Haiku buildsystem as a default option. It probably wouldn’t help with simplifying our buildsystem, which is one of the other things I’d prefer to spend my time on. Anyways, it shows that making a non-packaged Haiku is very possible, so if there are some people really interested in that, there’s a way to do it. The outcome of packagefs could very well have been “cool thing, but no one wants to use it”. Pretty much the opposite happened: old BeOS software was repackaged, existing Haiku apps were also repackaged, and a lot of software got ported. I think it is a lot more succesful than anyone expected, and I don’t think this required a lot of push from Haiku developers against users. Most people just jumped in, and a few other didn’t agree and… could have left, but are still around arguing about it on the forum a whole 10 years later. I don’t know what to make of that, honestly.
4 Likes
I made a collection of Bash scripts to produce unpacked Haiku image and quickly update only selected files. It it primary used for kernel code testing. Updating system from source with packagefs is painfully slow.
Script to make unpacked Haiku image from regular one (UnpackPackages.sh):
#!sh
set -e
if [ -d /HaikuSrcVolume ]
then
unmount -f /HaikuSrcVolume || true
rmdir /HaikuSrcVolume
fi
if [ -d /HaikuDstVolume ]
then
unmount -f /HaikuDstVolume || true
rmdir /HaikuDstVolume
fi
echo "Mounting"
mkdir /HaikuSrcVolume
mount -t bfs /Haiku/data/packages/haiku-riscv-vm/generated.riscv64/haiku-nightly.image /HaikuSrcVolume
mkdir /HaikuDstVolume
mount -t bfs /Haiku/data/packages/haiku-riscv-vm/generated.riscv64/haiku-nightly-test.image /HaikuDstVolume
echo "Mounting PackageFS"
mount -t "packagefs" -p "packages /HaikuSrcVolume/system/packages; type system" /HaikuSrcVolume/system
echo "Clean up system directory"
rm -rf /HaikuDstVolume/system
echo "Write system directory"
mkdir -p /HaikuDstVolume/system
(cd /HaikuSrcVolume/system ; find . -mindepth 1 -maxdepth 1 -not -name "packages" -not -name "package-links" -exec cp -ar '{}' '/HaikuDstVolume/system/' ';')
echo "Unmount"
unmount -f /HaikuSrcVolume/system
unmount -f /HaikuSrcVolume
rmdir /HaikuSrcVolume
unmount -f /HaikuDstVolume
rmdir /HaikuDstVolume
echo "Done"
2 Likes
I’ve often wondered if you could adapt haikuporter to produce appdirs - all dependencies included, with some tweaks to Tracker to hide that they are appdirs.
I’m not sure it would help with machines with limited resources, since multiple instances of the same shared libraries would be in memory, but interesting as an alternative to having a lot of hpkgs mounted for performance reasons.
Chris
Package management is good but nothing beats the flexibility of application bundles.
Nothing is stopping you from extracting hpkg instead of mounting them.
However note that this may be inconsistent for attributes on dirs etc.
There is also no uninstall routine this way obviously : )
Well, I’m talking about something a bit more ambitious than that - appdirs/application directories (RISCOS) or application bundles (macOS):
The uninstall routine is to delete the dir/bundle 
. Would need some OS underpinnings - i.e. Tracker showing the application directory as an application and not navigable (unless you hold down a modifier key or something when double-clicking), mime association as well?
Adapting haikuporter to build such a bundle should be possible, seeing as it already does the dependency resolution and mounts them all etc. in the work dir.
extracting to an app specific dir works fine too.
I don’t see the point to add app bundles though, as in support in tracker.
The Binary can already, ideally, be visible in deskbar with a querry… (It’s not now, but it really should :D)
I recall in the alpha4 days, a 32-bit build didn’t boot with any less than around 93MB, and was basically unusable with that little. Getting from that to being usable in 64MB (so 30% less!) is probably more or less impossible. Haiku is simply architected in a way that presumes you have a lot more than 64MB of RAM, and that’s without taking packagefs into account.
I think there may be a bit of missing history in the exchange between @cb88 and @pulkomandy. It’s my vague recollection that, in the early days after the package management merge in 2013, the “LegacyPackageInstaller” (then just called “PackageInstaller”) application didn’t properly handle creating missing paths, or redirecting Haiku-packaged applications that tried to install to /boot/system. A lot of the allegations of “breaking applications” came from .pkg apps that couldn’t be installed due to these problems.
But all those problems, once they were actually reported instead of made as vague accusations, got fixed (most of them in February 2014, it appears.) LegacyPackageInstaller now handles creating /boot/beos or putting things in /system/non-packaged (or ~/config/non-packaged) automatically, and .pkg from both BeOS and older Haiku versions work just fine now.
2 Likes
appdirs were considered during the development of the package system. They were not used because of the library duplication, which we already knew was a problem for two reasons: the one you mention (waste of disk space and memory) and also because when there is a fix in a library, every application using it needs to be rebuilt and re-shipped. Not only this is a waste of resources (the builders will run a lot more, and also people will ahve to download the same library several dozen times), it is also a problem for quickly releasing security updates.
That’s why we decided to introduce a proper dependency sharing system, but at the same time, to make it possible to manage packages without any special tools (you can just drag and drop them into /system/packages), retaining some of the simplicity of these systems (at least for BeOS apps that do not come with so many dependencies).
If I’m not wrong there was no release of Haiku between 2013 (package management merge) and 2018. So, these problems only ever existed in nightly builds. I’m going to say that makes them irrelevant…
2 Likes
The downside is that faulty updates breaks functionality.
Is there support for manually stating which library to use? Let’s say I use two applications that use the same shared library. Application 1 works flawlessly but Application 2 has some problems requiring the library to be updated. After updating the library, Application 2 works flawlessly but Application 1 is now broken. In this case I would like to manually override the library version for Application 1 so it uses an older working version while Application 2 uses the newer updated library.
Ah, I never understood that kind of thinking. Nextstep/Openstep used app dirs/bundles already, even on less capable hardware before Haiku. If the users had been asked back then, I think this decision would be different. For me, this solution is simply reflecting the “Linux way of doing things”, most likely because the devs didn’t know anything else.
And the arguments regarding waste of space and memory are even less understandable from today’s point of view, since the entire container world is delivering some kind of “app bundles” (even stripped down OS bundles) and is considered as some kind of “holy grail of software distribution”.
And commercial software is always coming with an own set of libraries, because you don’t want to rely on a package management which is not under your control and that can change the environment for your software in an instant.
But, hey, I think the discussion is mood. Haiku has gone that way, there is no reason to fight it all over again.
Wipe everything and unpack again.
There are litterally hundreds of messages on the mailing list where everyone was involved. The users were asked. We did not hide the developers in a basement to make decisions between themselves. ALL discussions in Haiku are public.
Also, the developers have a great and diverse background. We looked at RISCOS bundles. We looked at AmigaOS use of “assigns” to create virtual directories. We looked at Apple .dmg files. We looked at Linux as well, of course. We looked at the current practices in BeOS and Haiku.
So, saying “the devs didn’t know anything else” is just insulting.
The recent security vulnerabilities found in libwebp, that requires an update to every app using it, including all Electron apps, all web browsers, etc, is showing otherwise.
Yes, and this usage is ALSO allowed in Haiku. You have even TWO choices for achieving it:
- You can provide a package that installs your app in /boot/system/app, with all its libraries
- You can decide to not use an HPKG file, install your app as plain files in /boot/apps as it was done in BeOS, and that will also fully work without any problems.
These use cases were all taken into account in the design of the package system. What was very surprising to us, is that, indeed, the USERS of the system decided to go full-on with haikuports, which uses the package system mostly in the Linux way. We expected that Haikuware would start offering downloads as hpkg files. Instead, the owner of the website decided to burn it down and threaten anyone who wanted to make an archive of it with copyright infringement. We expected that people writing apps would distribute them by themselves, on their own websites, either as .zip files as they always did, as hpkg files, or as additional repositories you could add to Haiku.
None of this happened (well, not really, there were and still are some alternative package repositories).
This is not related to a design limitation of the package system. It is made to support BOTH linux-style dependencies (how haikuports uses it) and also self-contained packages. But no one uses the latter. I don’t see how you can blame the Haiku developers for this. The users (and people making packages) have decided to not use this feature, and then pretend it doesn’t exist and it’s the fault of the developers. So, instead of complaining, just start making self-contained application packages that install in /boot/system/apps, or if you don’t like the way packagefs handles that, start making apps that install in /boot/apps as a plain old zip file. That still works, that is still supported and was never removed from the system.
5 Likes
I like an idea of installing each package in separate directory. Duplicating contents is not needed, symlinks can be used instead. Symlinks also allow fine-grained dependency management, for example using specific version of library of avoiding file name conflicts. Symlinks can be automatically generated by PackageFS based on dependency information. In Linux world GoboLinux do something similar.
Linux-like idea of exacting everything in one place “junkyard” is bad idea because it easily trigger collisions.
1 Like
First, I am not against the package system of today, I settled with that years ago. But I still do understand people which have problems with that.
It’s a different thing looking at some concepts or try to use these and live with that. Only working with different concepts will provide you with some insights.
But I don’t want to sound insulting here.
And what exactly has this to do with “holy grail” containers? How would it help to do a fix in a package system? Only because you test and provide a new library doesn’t mean that you didn’t have to test the app using it. So at the end, it is still the issue of the app provider to check and fix that.
I could also argue that, if a dependency package management is so great, why do people invent Flatpak, AppImage, Snap…? It’s (at least for me) a question of providing software to a user, not a question of maintaining it.
But there is no “one fits all” solution, I just wanted to point out that in my opinion memory and space are no argument here…
Point taken.
Maybe it is just because of having only “Linuxoid Haikuports” that the other features of the package system are forgotten…