You might want to look at previous talks for hints… I had one at FOSDEM in 2016 (hmm no video, IIRC the recording was broken), but likely there are more up-to-date things around.
The package metadata that is stored in the repository and allows to do things like “pkgman install cmd:foo” without having to know which package contains cmd:foo. This makes implementing command-not-found on Haiku trivial (it does not need to check which package the command is in)
Likewise, dependencies for libraries are handled by lib:foo instead of directly on package names, allowing to reorganize (split, merge, …) or replace packages (for example use libjpeg-turbo instead of libjpeg) easily
Maybe add a note on how to create recipes (some quick guide or link on the howto)? .PackageInfo contains almost the same information as a reccipe does if you do it manually.
I know, this is only a draft, but after you added 2 more pages for creating a package in Haiku, you ended with pages 14/13 and 15/13 (still counts total pages as 13).
There are several possible directions to dig further into I can think of, not sure how far you want to go
A bit more history of how we decided to create this package manager:
The initial goal was to allow updates to the OS itself (without reinstalling it all). Before this, BeOS “packages” were either a zip file to extract anywhere (sometimes containing some manual instructions to copy files in various places) and sometimes Windows-style installer wizards
One goal of the package system is to allow applications to be easily redistributed, even offline. As an example, a few years ago I was in a train trip and someone asked me if I happened to have VLC on my laptop so he could copy it and watch a DVD. On Windows, this is no problem, you probably have the installer .exe on your download directory somewhere and can copy it. But try to figure out how to do this on Linux? With Haiku apps, there are usually few dependencies, and so a Windows-like solution should be possible. That’s how we came up with the idea of not extracting the packages at all
On the technical side:
Mention use of libsolv (also used by SUSE I think) for resolving dependencies
Maybe give a bit more info about the hpkg file format: how the data is split in fixed size blocks (a few kilobytes) and each block is compressed separately, allowing reasonably cheap random access to the data (unfortunately most compression formats are not designed for this, and operate better on a long stream rather than smaller independant blocks)
More “cool features” (you already gave a good overview of the most important ones I think):
The ability to identify where any file comes from by looking at the SYS:PACKAGE_FILE filesystem attribute (using Tracker “Get Info” or listattr in Terminal). No need for a separate database to keep track of this
The possibility to reuse the packages in a separate mount point, to create something similar to a python virtualenv, but in a generalized non-python-specific way (used by haikuports to build packages in a controlled environment)
Nice work.
On the Metadata in Packages page, I found bullet point number three a little awkward and hard to understand.
Other packages, and the command line, can specify dependencies on
by referring to the the library name or command without having to
specify which package exactly.
That’s because those slides at the end are intended to be an appendix, to be covered if I don’t run out of time, or if there are questions relating to those items.
So to regurgitate: because the packages are on the disk, it’s easy to provide said packages to someone else in an off-line environment, and all dependencies are guaranteed to be present on the host machine?
What should be mentioned about libsolv in regards to Haiku?
Great info to random access to the archive, I’ve never heard of this before.
Could you elaborate by what is meant by reusing the packages in a separate mount point? Do you mean like a chroot/jail environment?
Not part of your post, but how much space to the different versions take up? I imagine someone will ask this, that having an aging Haiku installation will take more and more disk space as the packages are updated.
How could someone remove old packages, say from the beginning of the Haiku installation, for example, to save some space?
It means you dont need to know the name of the “fancynamedpackage” to install it, it is enough if you know for example the name of the command you need.
So instead of
pkgman install imagemagick7
You can say
pkgman install cmd:convert
Comfy, heh?
So to regurgitate: because the packages are on the disk, it’s easy to provide said packages to someone else in an off-line environment, and all dependencies are guaranteed to be present on the host machine?
Yes. That is the main point of free/libre software, yet the typical Linux way makes it complicated, and centralizes the software distribution in the hands of distros.
What should be mentioned about libsolv in regards to Haiku?
Nothing special, just mention that we didn’t reinvent the wheel on everything and the Linux package managers do get some things right
Great info to random access to the archive, I’ve never heard of this before.
Could you elaborate by what is meant by reusing the packages in a separate mount point? Do you mean like a chroot/jail environment?
Yes, in haikuporter this is used with a chroot, but it could also be used without one, just like python does with virtualenv. The idea is to mount a packagefs in some directory, and then adjust some variables (PATH, LIBRARY_PATH and ADDON_PATH mainly) to make the mounted packages “visible” in a shell session. So you could keep the environment for a specific project this way.
You can find the package backups in /system/packages/administrative/state-* (the directory name is a timestamp of when the state was created). A new state directory is created for each operation (install, uninstall, …). The directory contains a list of activated packages in a text file, and the packages necessary to rebuild this state from the previous (or next? I don’t remember which way this works) one. If you only ever add new packages, this takes almost no space (just the text file with the package list). If you remove packages, they are kept there so no space is freed. If you update packages, this is the worst case since both the old (in the state directory) and new (in the active /system/packages) versions are kept.
You can delete the states when you don’t need them anymore. Currently that’s not automated, but some 3rd party app can help with cleaning them up. We may add this to the OS someday if we can decide what the right rules are for automatic deletion. I think you can only safely delete the oldest states, if you leave gaps between the current state and an older one, you may end up missing some data to reactivate that older package.
Edit: There was an interesting talk just before I spoke about Haiku on a project called Distri (video), which is a package manager which mounts files into a read-only space to save installation time and reduce package complexity. Sound familiar?
Thank you! I was thinking of a follow-up for next year’s PackagingCon:
Haiku ports
porter
recipes
repo infrastructure
Cross compiling
Also, wasn’t Haiku support upstreamed in gcc? I thought I had read something about that on Phoronix but can’t find it now. That could also be an interesting thing to mention.
Are there other conferences where we think a Haiku talk would be appropriate and well-received? I’d be happy to put more talks together and tell more people about our awesome OS
Haiku support was merged in binutils but not in gcc yet. First we need to get Haiku fully built with gcc11, so we have a working version of our patches for the latest gcc version