That makes sense, thanks for the details. Indeed the issue happens when launching QuickLaunch via shortcut. So the environment is fine when starting the app via Tracker.
So actually we have multiple levels of this when I get this right? Tracker does provide a basic environment, but inside Terminal additional environment variables are defined (e g. XDG_*). Some ported applications might need those, but it should not affect native applications.
But on a lower level there is even a difference between the basic environment that is available via Tracker and no environment when launched via Shortcuts. And this in some cases even affects core applications (e.g. Terminal starting bash with a different starting directory).
Why should Tracker in particular handle this? Why isn’t it possible to simply use BRoster::Launch? Does that API need to be extended to allow to specify a context in which the apps should be run?
Also, why does input_server gets a different environment in the first place? Isn’t it part of the user session just like app_server and Tracker?
And in any case, a variable like LD_LIBRARY_PATH sounds like it should always be set, no matter what you do. Why and how is it getting not set in this specific case? That will break a lot of things. If it’s missing from input_server environment, how does input_server even manage to load itself? Wouldn’t it fail to find libbe?
According to X512 the input_server is not currently part of the user session but is launched by app_server. Apparently app_server gets a much more minimal environment, and input_server inherits that, and therefore we get the issue with Shortcuts. This appears to be somewhat of a regression from when the launch_daemon was introduced compared to the previous Bootscript.
I don’t know how difficult it would be to have input_server be part of the user session. Thinking about it I don’t think that makes sense because if there was some sort of login screen that would need the input_server. So probably fixing how Shortcuts work is in fact the “correct” fix.
Keep in mind I’m just trying to figure this stuff out, and unfortunately I haven’t had a chance to try to fix it. I was just trying to help since Axel seems too busy to help and apparently no one else is interested in trying to fix this. But I also don’t have much time right now either.
I need to verify whether or not variables defined in the bash profile show up in the launch_daemon environment or not, I am currently unsure. From some of the bugs I’ve seen they may not be set up.
To give a bit of background the environment is just a block of memory in each running program, and when one program launches another the environment of the parent is copied into the child. I think this is handled by the kernel, or maybe it is the runtime_loader. Someone else can chime in, but it doesn’t really matter for this.
But to directly answer your question yes there are currently two ways variables will be undefined when you might want them:
This issue with input_server and Shortcuts.
Things from the shell profile that are defined in Terminal but not Tracker, at least I think so.
For 1 I think we need to adjust show Shortcuts launches things, or find some other fix. For 2 we should try to move any variables needed by applications from the shell profile into the SetupEnvironment script that the launch_daemon runs. You can also define a UserSetupEnvironment if there are particular variables you might need as a user, and maybe in this case that means LD_LIBRARY_PATH: haiku/data/config/boot/UserSetupEnvironment.sample at master · haiku/haiku · GitHub
But even with that fix for now we still have the Shortcuts problem.
Yes, actually it should be LIBRARY_PATH in Python, not sure why LD_LIBRARY_PATH is used there? That is linux specific. In that case, could the fix in Python be as simple as using the correct variable?
Sorry for the confusion, that was my bad, that’s just my Linux wired brain kicking in while typing. Of course it is LIBRARY_PATH, and the problem is LIBRARY_PATH not being set when launching from QuickLaunch. I updated my posting above.
Anyway, back to topic: LIBRARY_PATH really should be set, and I don’t fully understand how this works at all in other cases.
EDIT: And I also just confirmed experimentally what was already suspected: When I launch QuickLaunch from tracker everything works, only from Shortcuts things fail due to missing environment.
I created some crude BeOS app that just displays all environment variables. The result is interesting. The screenshot below shows on the left the app started from Tracker, and on the right the same app started via Shortcuts:
There are exactly two environment variables being available when launched via Shortcuts, SAFEMODE (what’s that?) and LC_TYPE.
SAFEMODE indicates if you are in safe mode (selected from the boot menu). Various system components use it to decide if they should load add-ons, use a default configuration, etc.
So, clearly a large part of the environment is unexpectedly lost. Even if that was the “system” environment from launch_daemon, I expect that most of the variables should still be there? You know, things like PATH, LIBRARY_PATH, ADDON_PATH sound quite essential to running anything.
Ideally the environment would be the same. But at a minimum to my opinion LIBRARY_PATH, ADDON_PATH, HOME, PATH, PWD and all of LC_* should be set.
Also tested what is additionally set when running from bash (on a pretty standard install, I don’t have any custom environment variables). A lot is very terminal specific, such as TERM, HISTFILESIZE and LS_COLORS. But then there are the XDG_* variables to indicate various file locations. I think those don’t need to and probably don’t should be set by default, as they are a Freedesktop thing and Haiku has it’s own way to handle this. But it might be a good idea to have a compatibility mode for ported applications, e.g. by setting an attribute on the application. Haiku could then fill the XDG_* variables for this application with values from find_directory.
Afaik we already use find_directory to set the xdg variables, while haiku native apps do not need these at all they are still much easier to use in ported apps than patching them, i would suspect that a specific compt mode would be more complex than simply having those env vars normally, even if not as pretty.
Would probably be ok as well, I think above had been some concerns that this might give wrong incentives to application developers.
The point is that currently those variables are not set at all (except when running from bash). If an application could just set a flag, e.g. B_XDG_COMPAT or something, and that would trigger the variables being available that would be a solution that can be easily applied to ported applications.
The environment isn’t unexpectedly lost, it just isn’t set up. It doesn’t spring from nothing, clearly whatever is not provided by the kernel or launch_daemon doesn’t exist. I have not tracked it down but SAFEMODE and for some reason LC_TYPE are set up early enough by something that they are in the system launch_daemon and therefore are inherited by input_server and Shortcuts.
As for why anything works I feel like environment variables are a very Unix thing that maybe BeOS and Haiku don’t need quite as badly as Linux and friends. Though it would be interesting to find out the details in this case.
Either way, as I have said many times we now know what the problem is. I feel like maybe @axeld could provide some insight here and suggest what approach should be taken. Either the system launch_daemon also needs to run SetupEnvironment (except it makes less sense to set up USER, GROUP and HOME) or Shortcuts needs to tell Tracker to launch things and not use BRoster. Clearly everything seems to run fine except Shortcuts and things it launches, so I don’t know if it makes sense to re-architect launch_daemon when there is a simpler solution. Shortcuts already delegates opening folders to Tracker so it won’t even complicate the code that much. But I suppose we can argue over the details when I have a change on Gerrit, which won’t be for a few weeks at least as I have a trip coming up.
I don’t understand why you insist on going through Tracker here. Why Tracker? It looks like just picking one random app that is likely to be running. And if, for example, you wanted to kill Tracker for some time, do some experiments, and have a shortcut to bring it back, well, you can’t anymore.
I think it makes more sense to make BRoster::Launch work properly. Currently it inherits the environment of the parent app, and in this case (at least), that is not what we want. So we should fix it. Not just for the specific case of Shortcuts, but in a general way. Maybe it needs to talk to the launch daemon and get the correct environment from there? Or maybe, rather than forwarding launch requests to Tracker, we should forward them to launch_daemon? That makes a bit more sense to me than Tracker, at least.
X512 mentioned it, and when I looked at the Shortcuts input_server add-on it was already messaging Tracker to open directories. Obviously this ties Shortcuts to Tracker but Tracker is already really intertwined in the system. This is a bigger issue but long term if we wanted to make it easier to swap out Tracker we would need some way to override things using some sort of protocol or provider layer, maybe like Android or iOS have.
Tracker not running could be handled when sending the message and it could fallback to BRoster.
I can get behind this. I don’t know enough about launch_daemon yet to know how to specify which launch_daemon to talk to, because this needs to be the user session one, though I think there is some infrastructure already for this. Maybe Axel already had some plans for this, I can recall something he had written about launch_daemon and the roster but I don’t remember the details. One thing to avoid is a weird circular dependency between them. Edit: though in this case you are talking about the BRoster code running in the application, so maybe there are less issues there related to the actual roster and launch_daemon servers. I still think Axel had thoughts about these two…
Anyhow, I get your perspective, we at least understand the issue and I think we can figure out a good solution. I will probably experiment and see what looks good and works.
I agree that BRoster::Launch() should work properly. If you look at /system/data/launch/system, you see that it doesn’t setup any environment.
It just starts the system servers, and ‘autologin’, which will then start a new session via BLaunchRoster::StartSession().
Then the interesting stuff happens: the user session is created, and /system/data/user_launch/user is evaluated. The “desktop” target will initialize the environment via SetupEnvironment.
Therefore, if you let a system server launch a user application, it misses any environment.
Tracker is responsible for handling file open requests and selecting compatible applications depending on MIME type. It can also launch applications if opened item it an application.
In the case of opening files I would have expected the requests to go through registrar, rather than Tracker, for example? But it would create the same problem, I guess, because registrar is in the “system” session?
