Package management improvement ideas

I think this is something that is not very clear in the documentation.

So the packagefs driver sits between the packages themselves and the kernel FS layer, and is responsible for how/where packages appear in the filesystem. I think I prefer the terms activate/deactivate because it is otherwise confused with actual mounting/unmounting of loop images.

I wonder how this trades off vs a seekable (like squashfs) overlayfs/unionfs type implementation and a package daemon that mounts/unmounts loop package images. It seems like there could be some duplication of effort but I can see that the current approach may allow you to more easily do certain things that dont fit into the normal filesystem paradigm.

Now I don’t see what distinction you’re making. packagefs is essentially “an unionfs that mounts/unmounts loop package images”. That’s pretty much all it does. The packages are all visible in the same mount point (packagefs are mounted by default in /system or ~/config, or in haikuports chroots, you could mount more of them in different places if needed) and added to the packagefs/unionfs.

There is an additional thing re-exposing the package data in the “activated-packages” directory but I don’t really understand what it’s useful for in its current form.

1 Like

The difference is only between:

  1. Mounting individual packages as individual loop filesystem images directly to the kernel.
  2. “Activating” multiple packages via a single packagefs filesystem driver mount, which then presents a single filesystem to the system at its mount point, which contains all activated packages in whatever way the packagefs driver chooses.

I don’t really understand why the latter approach was chosen in favour of the former. Maybe because the former would have required adding support for union/overlay type filesystems to the kernel fs layer?

All the packages share a common directory (/system) and put their files there. So you can have all your libraries in the same directory (/system/lib).

The kernel by itself wouldn’t be able to do that. You need something that can track which file belogs to which package, because it is not trivially deducted from the path as it is when you have separate mount points. Also, queries can easily be restricted to a single filesystem, but searching a single package with queries is not so useful, while searching a whole packagefs is.

Or do you mean mounting each package in a completely separate directory? That would man having your libraries and binaries scathered in many directories, which wouldn’t be convenient in the general case (there could be some uses for it).

This makes sense now, thank you for explaining. I can see that it’s too use-case specific to have a general API for in the kernel really.

Data from root packages is globally accessible (drivers, add-ons etc.). Some additional mechanisms can exist.

You need to declare needed library packages to compile each project by using (4). This will also simplify building from sources a lot because build and runtime dependencies are explicitly declared and can be automatically installed before building/running.

2 Likes

Regarding conflicting packages in PackageFS context, the question is the following: If package A provides /system/lib/libXYZ.so and package B provides different /system/lib/libXYZ.so, which one is visible:

  • For A?
  • For B?
  • For some other package C?

If I understand correctly the point (3), there is no proposal to have conflicting packages per se in PackageFS, only conflicting dependencies. In this case, the answers would be:

  • For A, the accessible version would be dependence of A (let it be C),
  • For B, the accessible version would be dependence of B (let it be D),
  • For all others, there will not be accessible version of /system/lib/libXYZ.so.

Is this correct?

The same package can be both a stand-alone package (installed for itself) or a dependence of some other package (installed together with it). In case I understand the point (3) correctly, PackageFS should know which package is installed manually and which is installed as dependence. Moreover, PackageFS should keep track of which dependence version is installed for which package. But what should happen when one of these different dependent packages is manually installed. Say, one now installs package C. In this case, should its version of /system/lib/libXYZ.so be also visible for B? Or if one manually installs package E, which provides /system/lib/libXYZ.so different from both C and D, will it be visible only for all the rest of packages except A and B? Or if one uses pkgman to choose the right package that provides /system/lib/libXYZ.so (and A, B, C, D and E are not installed), which package should pkgman choose: C, D or E? What can be said about circular dependencies in bootstrapping process? Say, package A version ax depends on package B version bx, which depends on package A version ay, which depends on package B version by and so on.

These are just several use-cases on surface. I am sure the real usage of the feature (3) would expose much more. Even worse is when package maintainers over-use this feature for some fancy effects, so this can potentially lead to totally unexpected results.

Mechanism of visible package files is following:

  1. Executable outside of package:

mappedPackages = rootPackages + Dependencies(rootPackages)

  1. Executable inside package:

mappedPackages = package + Dependencies(package) + rootPackages + Dependencies(rootPackages)

rootPackages + Dependencies(rootPackages) have lower priority then package + Dependencies(package).

Dependencies(packageList) is a function that recursively resolve all dependencies for provided package list.

Package management work in functional style, root package list and repository package lists are the only state, the rest is calculated in predictable way. User alter root packages list or update repository to change installed/active package state.

It sounds Nix-ish, though I am not an expert on Nix, but I have used it some. Nix is gaining quite a following and I think moving Haiku packages to be more Nix-like is a good idea from both a technical and marketing perspective. They are already very close in many ways.

4 Likes

I have read an article ( Wesley Aptekar-Cassels | The Curse of NixOS ) about nix some weeks ago, it was featured at osnews if i remember correctly. According to the article nix “almost” solves the problem u*ix and its advocates made but introducing extra complexity while doing so. As it tries to solve problems with additional complexity i consider it harmful and i dont think it is a good idea to introduce any of those ideas to Haiku.
U*ixen per definition can’t be radical, so they cant solve their own problems, but they are always up to propagate their ideas to other systems to chain them down to stop them from being innovative. I would just leave u*ix to die the death it deserve.

4 Likes

I know you hate Unix and I tend to agree it is holding back computing but I think you are too quick to dismiss Nix. The article shows a lot of the good points of Nix which are not just solving problems unique to Unix. The problems it solves are generic to any operating system, this implementation just tends to be made for Unix systems.

The Haiku package system already solves a lot of these issues and X512 is proposing some things which make it act more like Nix in a good way. It would be nice to have different versions of the same package/application/library, to be able to have an isolated environment in some cases, and to not necessarily have every package pollute the global namespace.

If we can implement this in a clean way I think it would be good. It is certainly worth experimenting with, which I assume X512 will do.

4 Likes

The ability to have multiple versions of the same application would be a very welcome change. It’s an important feature for regression testing and other reasons. We take this feature for granted on Windows, but in the open-source world it is often seen as “impossible”.

2 Likes

True, i try to be radical in computing because all the nonsense i have to face during my daily job. Most of those problems doesn’t even have any sane, explainable or logical reason, not even a tradition, they are just simple limitations. Like admins needs to allow you to register to a fileshare, but you have to wait 24 hours to be able to register after that.
I don’t know who is responsible for limitations like this, but i assume, they are humans like me.
Now the IT teams most of the time are considered untouchable and the simple people like me have almost no way to fix or workaround this (hence shadow-IT exists).
So if i am unable to workaround things like this a sane-ish idea would be to try to make limitations like this impossible. Therefore eliminating dinosaurs like u*ix is a sane idea.

Most of the problems in the computing world - from my POV - is basically the result of humans trying to solve problems the laziest ways. Therefore nix is an example how NOT to attempt to solve things. Each and every other opinion would make sure Haiku gets chained down like the u*ixes.

Sorry, but this can’t be true. The problem lies either in the programming languages, the programming schemes and ways and methods, the linking, basically the big-scale infrastructure, in the operating system which allows this ways and the developer culture whih promotes this ways and the people, who are unable to learn anything new or different.

It is not only about the packaging system, but also about the declaring dependecies and the way of packaging things. I don’t know all the BS u*ixes came up, but i consider the Haiku way the most sane way.

It would be nice to have different versions of the same package/application/library
[/quote]I don’t think we need to introduce any nixism for this.

I think the package management could and should allow to ad-hoc create a chroot for this. Possibly a bit easier than it was done with HaikuPorter.

2 Likes