Some package management ideas

This is just a collection of ideas I would like the haiku package manager to have.

The list of all installed packages would be stored in a database, such as sqlite (being fast, constantly developed and suitable for storing this kind of information).

Now when a person downloads and installs a package the database of installed software is updated to store information on the files installed, much like the box package manager does. In addition, for each file belonging to a package and written to the file system there are a set of mandatory file attributes that every package has to store on installation. This is not something that a bundle/package creator has to worry about, it is handled completely transparently by the package manager that writes the attributes to the fs.

For an example every file belonging to a package that that gets installed via the package manager would have mandatory attributes such as Package ( this file belongs to this package ), version and a hash sum. Other mandatory attributes could be added, the aforementioned are just examples.

This approach has some advantages. For example if you are browsing a file system and wonder what some file is or what package it belongs to or just about any information that is present in the form of mandatory attributes, you can just issue a simple “lsattr file.so” command to see all available information. This could also prove to be a powerful feature when using the “query” command. You don’t necessarily need separate tools to find information on Packages/Files. Instead you are essentially just using the advanced features that the haikufs has. In other words Programs are not just a collection of zipped files you unpack under your home directory and forget. They become part of the file system and OS. You can install software under a common folder that is available to everyone using the system or users belonging to some user group or just you. The person installing the software can decide as far as his/hers privileges on the system allow. The point of this idea is “coherency”.

Another thing is that if the package database gets accidentally wiped or corrupt or a file gets accidentally deleted, you can still have a working system but you don’t have any means to uninstall software or know what is installed on your system.
a package recovery tool that scans the whole file system looking for files and checking their attributes can rebuild the database from scratch and “restore” your system to a working state. The hash sum is something that I’m not sure whether it’s needed. It could be used in cases like file corruption and virus infections (though a virus can easily bypass this).

For system updates IMHO multiuser capabilities with access permissions would be needed. The package manager in charge of installing .pkg “bundles” would not have access to files belonging to the System/Core package. This just to ensure that a badly created / malicious package wouldn’t overwrite important system files.
The system “updater” responsible for updating and extending the core functionality of the OS would instead run with full administrator rights.

For Packages that want to update themselves (and I personally prefer this method having used both windows and various linux distributions and like the simplicity of just having a “Remove Program” option instead of a centralized solution) there might be a need for an API for the package manager to handle these cases so that all file attributes (hashes, versions, all additional attributes) are updated to the filesystem and database accordingly. a Package can of course bypass this but I would consider that “lazy programming”.

I’d like to make a suggestion. Focus on separating requirements and implementation. Write out detailed requirements first and foremost. Only when completed should you consider how you would implement it.

With that in mind, how would you write out requirements for package management without implementation details?

I should add that I think some sort of package management solution for Haiku is a good idea. I’m just giving suggestions on how to clearly articulate your ideas. Please proceed!

I don’t understand the need of a sql database as you say that you want sign the files using attributes: the filesystem IS the database :slight_smile:

For the rest I think your idea is a more “Haikuish” way of doing a package manager…

I’ll try to explain more specifically the idea I presented in my first post

a package bundle:

As an example I’ll use a stripped down version of gnu coreutils package.
The .pkg file would contain:

pre-script
description.html
icon.svg
/system/bin/arch
/system/bin/basename
/system/bin/cat
/system/bin/chgrp
/system/bin/chmod
/system/bin/chown
/system/bin/cp
/system/bin/date
/system/bin/dd
/system/bin/df
/system/bin/echo
post-script
checksum

When you double click the bundle file the package manager unzips the file and shows all
information that is supplied with the package like the package description and file list
like for example the GDebi installer does.

If you choose to install the package, the package manager that is part of the core os
first checks that the package database is in a working state. This could be done by calculating
the checksum of the database and comparing that to the checksum value stored as it’s attribute
(I don’t know if sqlite already does this internally better…)

If the database is not corrupted the pre-script is run (this can do stuff like present you with
a welcome screen, ask you to accept the license of the software, query for settings or validation
codes for proprietary software). After this the package manager validates that the same version
of the software isn’t already installed and if it is, it asks if you want to reinstall it
(for example if it happens to be broken and you are reinstalling).
Package manager first writes the information in the database and marks a “dirty” bit that gets
cleared when the package is successfully installed. It then writes every file to the location
specified in the bundle and adds the mandatory attributes to the filesystem.
From command line it would be something like “addattr Package coreutils cp” for every file found in the
bundle. This is of course done with haiku api…
when the files are in place the post-scrip is executed and once finished the dirty bit is cleared.

Package Manager:

Package Manager can be found from tracker and it gives you the list of software installed
to the system in the upper view. Below is a tabbed view that shows you a description of the
package / file list etc.

The Window would look very much alike to that found on windows, but with more details.
You essentially need only three buttons in the Window itself:
Uninstall, Check Database and Close.
Highlight a Program and click Uninstall and the files are first removed from disk and then
from db. Check forces a scan of the whole system if no Program is highlighted or a per-package
scan if one is (you can either do a short scan that only checks if files are in place or
a long scan that detects corrupted files by comparing checksums. If packages are
found to be broken, they are marked accordingly in the package manager.
You only need to reinstall the bundle to fix your system and this seems to be the only way in a
non-centralized package management system.

Now, if you occasionally use the Terminal for some work and need to know what package
a file belongs to its relatively easy:

listattr -l join

If you want to find all files belonging to a package you would do something like

query “((name=”*")&&(Package==“coreutils”))"

The above is a bit awkward to use so it would be better that the Package attribute would
be indexed as well.
Then all you would need to do is:

query Package=“coreutils”

The mandatory attributes + the sqlite package database would bring a sort of double bookkeeping
mechanism to the system. As I mentioned earlier the database file could be lost or broken but
it could be restored from the attributes stored in the filesystem. A file corruption would
be easily detectable also just by comparing the checksums. In the case of missing file(s)
you would of course detect that when you launch the program and the package manager scan
would also detect that a file is listed in the database but not found from disk.

One other reason for mandatory file attributes is pure convenience. If I use
Windows or linux, most times I simply can’t know just by seeing the name of the
file to which package it belongs to.

@fano

That’s a good point. The main need for the sql database is performance. If all files
belonging to a package are allowed to be installed only to predefined locations, uninstalling
software is quite painless. However if packages can be installed anywhere on the filesystem
the package manager has write permissions to, you would have to check almost the whole
fs to be sure there are no files left. When everyone is using high speed ssd disks your point
would become valid.

Heck, If that was the case I would want the sys_open() syscall to add an attribute
to every file that gets opened telling which program created it.
Of course only on haiku filesystem that supports arbitrary attributes.

lsattr -l somefile.forgottosaveesuffix

Text “CreatedBy” application/x-vnd.vim

The above would IMHO be very convenient.

[quote=danho222]
@fano

That’s a good point. The main need for the sql database is performance. If all files
belonging to a package are allowed to be installed only to predefined locations, uninstalling
software is quite painless. However if packages can be installed anywhere on the filesystem
the package manager has write permissions to, you would have to check almost the whole
fs to be sure there are no files left. When everyone is using high speed ssd disks your point
would become valid.

Heck, If that was the case I would want the sys_open() syscall to add an attribute
to every file that gets opened telling which program created it.
Of course only on haiku filesystem that supports arbitrary attributes.

lsattr -l somefile.forgottosaveesuffix

Text “CreatedBy” application/x-vnd.vim

The above would IMHO be very convenient.[/quote]

OK I understand the point: performance thing thinking of the Linux way of “installing” application is right… a package can sparse its file in /bin, /usr/bin/, /usr/share/pixmap/sarcazzo, /etc, /usr/local/lib…

I hope the Haiku-way will continue to be different: the application can be installed only in 2 places:

  1. /boot/common/yourApp (systemwide application, usable by all user)
  2. /boot/users/fano/yourApp (usable only by the "fano" user)

All application should have this subdir:

  1. conf for all the configuration thing (if it is systemwide configuration per user can be in /boot/users/username/yourApp/conf)
  2. lib library used only by application
  3. data for pixmamp and so on...

So… there is no way application sparse all files on the filesystem!
You can say well but ported GNU/Linux application rely on that illogical file hierarcy!
Well they must be ported in the good way… it should never exist /etc in Haiku…

Your idea to use a “CreatedBy” attribute and a limited number of dirs in which to install application can avoid the need of sql database… I don’t like the idea of have a database as an obligated dependency, as we have a db like filesystem -:slight_smile: