Folder Distinctions for Development

Development OS
1

Something I don’t see is explanation of the folders here in more conceptual depth:

https://www.haiku-os.org/docs/api/FindDirectory_8h.html#aa10135f62098fc92d18c653a1fff9c0e

Are “packaged” directories for the USER, for example, “files installed when selected a package for installation from the HaikuDepot”? If so, does that mean “non-packaged” are “things script-based installation of software initiated by the user outside of the Depot which aren’t in a ‘.hpkg’ or similar?”

The closest one gets–but which doesn’t answer this (or other conceptual topics) is:

I’d like to ask this (and more) questions as I am working through some scripts and tools (that ease terminal development across platforms) that I figure I should also get working on Haiku (why not, it has Bash!) while at it, when I a not really blessed with a plethora of Be or C++ experience and it doesn’t appear these concepts (per searching so far) are really detailed for anyone yet.

2

The packaged directory is where the files from the hpkgs mounted via packagefs go, these are readonly;
The (writable) non-packaged directory are where all programs not installed via hpkg go, such as from make install and BeOS programs.

3

I guess those are all executable scripts, so if you want to try those out those should go to ~/config/non-packaged/bin for the system to be picked up (got a few in there myself).
If your sources contain some Makefile or tools in the order of autotools/meson/cmake … you can always use the PREFIX option to point it there, for package creation the easiest way (for me) is to create a recipe for it and package the scripts/tools to be installed in /system/bin.

4

Yes.

There is a distinction of “system” (/boot/systtm) and “user” (“/boot/home/config”) directories. Both ofthese have packaged and non-packaged versions.

Currently HaikuDepot always installs things to the system directory. The user is free to put their own packages in their own directory manually, and that will override the system packages (due to user directories being searched first in the PATH and LIBRARY_PATH)

The non-packaged directories can be written to directly (no need for an hpkg file) and allow to override their respective packaged directories

2 Likes
5

Thanks guys. I appreciate it, given the sparse explainers at The Haiku Book: FindDirectory.h File Reference

(I’m sharing these links, by the way, to help aggregate work/infosources for others down the line who might be googling.)

I noticed at https://github.com/haiku/haiku/blob/master/data/system/boot/SetupEnvironment that the comments are a bit dated, but it was helpful towards working on mapping things e.g. Haiku equivalents to XDG directories.

Do these still look reasonable to you,

XDG_CONFIG_HOME : finddir B_USER_SETTINGS_DIRECTORY
XDG_DATA_HOME : finddir B_USER_NONPACKAGED_DATA_DIRECTORY
XDG_CACHE_HOME : finddir B_USER_CACHE_DIRECTORY

If so, as I’m tooling things I don’t want to wind-up setting some kind of bad precedent. Here is some necessary background thinking:

Currently for scripts and tools that need “state” that is local-only (that is, it’s for that machine, not for being backed-up later) I have, on other Linux/Unice boxen a variable

XDG_STATE_HOME

set to

$HOME/.local/state

What would be a good candidate locations and/or Haiku-native directory variables (like those in the documentation above like B_USER_[more]) to use as an equivalent, to keep things Haiku-native? I’m thinking I want to keep things human-friendly. On Other nixes the location breakdown

“$HOME/.local” is a good signal of “yours, things for local-only”,

and “state” is also obvious

hence the $HOME/.local/state I use on other Nixes.

I also noticed that Haiku does already have some “lib” locations in non-packaged, but I am curious: what is a variable that points there? (i.e. so I can follow the documented advise to use variables rather than locations.)

6

there’s a related ticket: #18257 (XDG_STATE_HOME environment variable should be initialized) – Haiku

7

Note that on Haiku .something is avoided whenever it is for a file or directory.

8

You can use the PathFinder API and findpaths command to find all of the possible paths. For example findpaths B_FIND_PATH_LIB_DIRECTORY or to find only existing directories findpaths -e B_FIND_PATH_LIB_DIRECTORY. If you want to use the FindDirectory constants then you’ll probably want B_SYSTEM_NONPACKAGED_LIB_DIRECTORY or B_USER_NONPACKAGED_LIB_DIRECTORY. You can see the full list of constants by running finddir -l or findpaths -l for the PathFinder constants

9

Hi, 8ln. Are you reffering to the documentation you linked at the start of your message?

I’ve recently improved it so I would be interested to find out what might still be lacking in it.

Edit: looking at the ticket davidkaroley linked I don’t think there is any good equivalent for the state directory on Haiku currently. The “this machine only” part is a bit confusing to me, what data do you have, that is not a cache, that has to be persisted but should not be backed up?

10

@nephele –

(1) the github code shows “defaults to” comments under the XDG that show values that are basically for Linux rather than Haiku, possibly from an older state of Haiku? (I consider “mere comments” as being valuable parts of code/documentation!)

(2) I think of “cache” as “data you operate on and then discard” OR “data that has to be persisted and rapidly accessed”, but “state” as “data used by application” that may need to persist (e.g. such as across runs of a script) but which isn’t relevant to backup. The XDG documentation (e.g. XDG Base Directory Specification) also calls the XDG_CACHE_HOME as “non-essential” data–but per-machine I already know of scenarios [cache-like data] IS essential to keep, per the given setup/installation, but would not be if you re-install.

One example might be keeping connection information for networking (in mind just because of a habit of tinker-breaking networking things, particularly on the systemd-intrusion into all-things already-were-working on Linux without documenting any of it for us that they did so): scripts may need information about networking state from prior runs to make decisions, but that is relevant only to the given machine. If this data were in the location for “cache” then things that make decisions about what to discard/eliminate (e.g. cleanup) might just wipe it out and leave scripts without data to make decisions. There are multiple kinds of state worth considering, but it’s a kind of intermediate between persistent data (e.g. traditionally logged in /var) and cache (that really would traditionally go in /tmp), “log, but not that long, and then if you still need it, then add to log before you change it.”

(Worth noting: depending on what you’re doing it may also then break down to even finer-grained levels, but the per-machine cutoff is a useful threshhold and I can sub-dir , sub-identify, label-in-data-file, etc., to get that finer granularity rather than relying on XDG vars and filling-up memory at initialization of BASH and its vars.

I’m also contemplating (from the work of “helloSystem”) the reality of applications that move with removable devices…but not that much. In the era of having to secure things, mobile (not just phones but laptops), package managers, stores, walled gardens, and administered-by-IT, “portable apps” aren’t really that important/useful/etc. for most people.)

EDITED: changed "this is not correct, where per-machine the “state” above to “[cache-like data]”

11

As I’m a new user “can only reply to 2 users” to…

Thanks @davidkaroly. I’ll add that link to my TODO: watch [url] list of things.
Thanks @Lrrr. Already proving useful with some experimentation. :smiley:

e.g. so far in my little mapping script

XDG_STATE_HOME

has gone from set to “$HOME/.local/state”

to

=“finddir B_USER_VAR_DIRECTORY”

[EDIT: fix – aligning to use of a user-var at the ticket about state and with thinking by nephele now that I know the variable to use.]

12

They are not outdated, they refer to the XDG spec that defined these variables, basically if we did not specify those then that is where ported linux apps would try to store their stuff, in these dot directories we do not want.

As for peristent like data we have a constant that is supposed to be the same as /var in linux and one for cache. If that is not enough you could make a case that we should add a new Variable for this. (in that case feel free to open a bugtracker ticket so we can discuss the semantics in more detail)

As for portable apps, most native applications in haiku are portable, usually having no dependencies above the OS and containing all needed content/data in their application image. : )

13

Gotcha - -I thought they were outdated in the sense they must be referring to Haiku locations (when the Haiku locations show /boot at the beginning of locations but the locations in-code comments do not). Just isn’t clear to a newcomer!

I didn’t literally mean “/var” though – “/var” was an old thing from the naive days where everyone on a shared Unix was trustedp[1] (the kind of thing Brian Kernighan talks about when asked about the design of Plan9 and if it is/can be secured and he openly says those were more innocent times when they were naive). Thus on e.g. OpenBSD and others not using /var and instead finding other ways… like not having users write to system locations outside of their own $HOME!

Right now on systems and configs where people are really getting into customization and tooling “state” is already an XDG var, regardless whether it’s in-spec or not: there was need, the XDG standards weren’t keeping up, so people began to code things with XDG_STATE_HOME=“$HOME/.local/state” assumed/hardcoded/in mind and it’s already sticky/prevalent in the scene where people are coding to Linux rather than coding to e.g. Gnome or KDE or other desktops or metaphors etc.

The portability thing is interesting. Are apps saving data in the packages DL’d via Haiku Depot once mounted, sorta “contaminating” packages (i.e. they shouldn’t be shared to install on other users’ systems), or in system-specific dirs, or ‘it depends’?

Just sharing/asking as I want anything I write as a shim to play nice in-ecosystem for other folks (assuming it’s ever in a state to share!)…

[1] speaking of which, anyone ever pay attention to the damage possible with people assuming “/tmp” is safe on systems (including modern Linux)!?

14

There is great context/info for this sort of thing/shim here – Environment Variables Behavior On Haiku - #24 by phw

Might be worth doing some C/C++'ing near-future to harmonize between shell-shim and other.

I noticed @nephele isn’t a huge shell fan. Don’t blame him, but “trade offs” – lots of flexibility and ability to glue things, and reasons that are similar to why the Web went “use all text”, even if the trade-offs are sometimes hard.

For sanity and a Bash-upgrade bath, check out:

https://www.oilshell.org/

The guy behind that project is doing a Herculean work, creating both

a “real” language, to underly the shell, and

a variant that works like Shell

– both as Bash (to be implemented as Bash-compatible, where they mean “we are running real build scripts for existing systems and checking they’re 100% compat., and if they aren’t we’re not done yet”), and

– and to include options to selectively-break Bash scripts to allow leverage of the “real” language features (e.g. including types).

Far, far better ideas actually exist already as implemented shells (from those which have something like “secret” as a data type which include considerations as far as “people might stream things so hide secrets on the shell in case of OBS”, to things like ksh93 and mksh…) but not that right now allow efforts to leverage all the existing work in largely-Bash -targeting Shell. He also happens to constantly find and report the bugs in Bash (and in others, e.g. reports vulns in the ksh shipped with OpenBSD, which is funny since OpenBSD keeps their shell simpler than e.g. the ksh93’s circulating, for auditing etc.)