Read/Write App Data In HKPG

Hello

I have an application that is written using c++ and the wxWidgets toolkit. Several years ago I was able to port it to Haiku via wxQt, although it was quite buggy. I put it on the back burner until recently when I installed beta4. Since then both gtk and wxGtk have been ported, making my app build and run pretty well with very few changes.

For the anyhow of it. I decided to try building an installable package via the makefile. Using package, rc, and resattr along with a .PackageInfo file, I got it to generate a HKPG that installed perfectly, including Deskbar integration and a vector app icon.

I installed it and it ran perfectly until I tried to make changes to the application database which I had stored in data/ in the package. Turns out no rw for the /system/data/ folder. So I decided to try putting the appā€™s data folder into /system/settings/.

Copying the app into an apps/ subfolder and app data into a data/ subfolder resulted in those files being installed to /system/apps/ and /system/data/ when the package was activated via HaikuDepot. However this did not seem to work with settings no matter what I tried.

Under Linux my app looks for ./Data_Folder_Name. If itā€™s run from a folder, it looks in that folder. Otherwise it looks in ~. Under Windows it only looks in the same folder. I can set rw permissions for the data folder via the installer. Under macOS it gets included in the .app and gets copied to ~ on the first run.

After tinkering a little, I discovered that Haiku also looks in the local folder or home folder. However Iā€™m at a loss on how to get that folder to where it needs to be via the package. I guess I could probably install it to /system/data then copy to ~ at first startup. But Iā€™d like to try to install directly from the package if possible.

Then too, it seems ā€œdotā€ application data folders are kinda frowned on in Haiku. I have not found a clear explanation on where exactly applications are supposed to store their rw resources. I can very easily modify my app to look in specific folders, but I like to know where the acceptable place would be and how to get the data there.

Thanks in advance

mill-j

2 Likes

@mill-j
Haikuā€™s packages are read-only. The contents cannot be changed. The storage of settings (dotfiles in Linux) go in ~/config/ subfolder designated to your application (I think itā€™s ~/config/settings/appname/). Thereā€™s another subfolder in there for application datafiles as well.

I hope that gets you started.

3 Likes

You are looking for post-install scripts. Check Packaging Policy ā€” Haiku internals documentation and the openssh recipe haikuports/net-misc/openssh at master Ā· haikuports/haikuports Ā· GitHub for more info.

3 Likes

Hereā€™s an example on how I did this for CudaText (credits for the script to @roiredxsoto ) :slight_smile:

Youā€™d have to make a rule in your app to search for the data there for Haiku then too. :slight_smile:

Post-install scripts are one possibility.

ETA: For a better solution, see my comment below.

Another, maybe better, solution is to have the package install to the data folder - I suppose using ~/config/data/yourapp/ would be better than /system/data/yourapp/ in case Haiku becomes multi-user in the future.
Then your app checks on startup if there a user database at ~/config/settings/yourapp/ and if there isnā€™t copy it from ~/config/data/yourapp/. That way you have a way to recover, should users delete the database file in their settings folder.

Hi!

Haiku has an api exactly for this purpose to querry paths for such things.

https://www.haiku-os.org/docs/api/FindDirectory_8h.html

You probably want The Haiku Book: FindDirectory.h File Reference

2 Likes

There should be a difference in user data and system data, the latter comes with the app and is mostly installed in $dataDir (this is ā€œmostlyā€ not required to be write able), the user data is something the app should look for and should point to ~/config/non-packaged/data, thereā€™s also user config files, these should go to ~/config/settings/appName. My 2 cents :slight_smile:

1 Like

Right. Itā€™s early, I wasnā€™t thinking straightā€¦ :slight_smile:
The ā€œemptyā€ database file coming from the package is fine in /system/data/.

Or better The Haiku Book: BPathFinder Class Reference.

Letā€™s try that again. :slight_smile:

  • Have the package put the ā€œemptyā€ database into /system/data/yourapp/
  • Use BPathFinder to load your database with B_FIND_PATH_DATA_DIRECTORY.
    Itā€™ll find a database in /boot/home/config/non-packaged/data/yourapp/ first, if not there it gets it from /system/data/yourapp/.
  • Save your userā€™s database in /boot/home/config/non-packaged/data/yourapp/.

Thereā€™s the findpaths command available to experiment in Terminal.

3 Likes

Iā€™m wondering if boot/home/config/non-packaged/data/yourapp/ is the right choice. In theory
non-packaged directory should be used for, wellā€¦ non-packaged software, shouldnā€™t it?
For packaged applications I donā€™t find it coherent. I would go for ~/config/settings/appName.
On a side note, I think the non-packaged directories require some clarity on when they should be used and the different recommendations provided above are maybe a sign of this unclarity?
IMHO, non-packaged should be used only for applications installed manually or by any other mean but a hpkg. non-packaged/data should be used for data, assets, and other resources that come with the software. Any other user data (preferences, project-related files, etc.) should go into ~/config/settings/appName. Any thoughts?

1 Like

Iā€™ve documented settings as only having settings, I like the idea of ā€œjustā€ copying settings to a new install.

to me non-packaged just means ā€œnot in a packageā€ Maybe that is wrong? should we have a rw persistent data dir?

If I understand that correctly, I tend to agree. However, in case of an update the script should take care of any existing setting and not only not overwriting it but possibly migrating it to a new format, if applicable.

I can just provide my own take on this which is: this is the place for a piece of software (applications, add-ons, drivers, and more) that is not installed via the package manager.
I donā€™t think that non-packaged means anything that is not inside a package.
I see it as a mean for a user to install something that does not come with a hpkg, or if they want to customize the flags or the icon of a packaged app by installing it manually or more often a way for developers to quickly test their own software.
Does it make sense?

I think we can agree that thereā€™s some confusion and things arenā€™t always easily separated between what should go into ā€œsettingsā€ and what into ā€œdataā€.

Generally, I like the idea of an app having one folder for writable files, e.g. settings and databases etc. So, ~config/settings/ seems to be fine.

In this case, I just thought itā€™d be elegant to use B_FIND_PATH_DATA_DIRECTORY as itā€™d load the user modified file and automatically fall back to the package-provided ā€˜emptyā€™ database.

WRT ā€˜non-packagedā€™, I think in past discussions there was the opinion, that itā€™s just called that way for the lack of a better name. It wouldnā€™t be needed, if we had ā€˜transparently mergedā€™ folders, i.e. one folder that shows e.g. a merged ~/config/data/ and ~/config/non-packaged/data/ (though IMO that could also be problematic, because thereā€™s be no distinction whatā€™s writable and whatā€™s not).

Meaning, ā€˜non-packagedā€™ can just as well be used by applications that are themselves packaged.
I think. Others may have a better understanding of it allā€¦

Try my tutorial (BeSly knowledge base)about porting compiled software, because here we have most times problems with files they need to placed at other places as the package himself.

Porting a compiled program to Haiku
Haiku Package - Tips and Tricks

Although it is used mostly for settings, thereā€™s also the possibility to make a packaged file writable. Hopefully, it seems that thereā€™s no abusive use of it yet.
From a user point of view, I like the idea to keep user data and settings separated. Sometimes, old settings can be incompatible with the new version specially if you waited long before upgrading. In these cases, you may prefer a fresh start. Thatā€™s almost never the case with user data or an importation thing is provided.

Thanks everybody. It appears that installing app data to /system/data/appname and copying it to /home/config/settings/appname on the first run is the way to go.

1 Like

donā€™t hardcode that path, use the apis mentioned above

1 Like

Only one problem. Iā€™m using wxGTK. I can modify two lines of code and Iā€™m good. Using another API would require linking against just another library that doesnā€™t exist on any other platform.

I donā€™t see /system/data changing anytime soon. And if Haiku becomes multi-user, it should still be fine since the app uses the working directory (currently home folder) as a starting point to find config/settings/.

This is already wrong, apps can already be installed to other paths and are expected to work.

If linking to the lib is too much work you can use the commandline tool humdinger mentioned instead and parse itā€™s output.

Hardcoding directories is definetely not a proper port and will break quite easily.

Edit: also the library you have to link against is already linked to your application anyway.

1 Like

Could you elaborate on how applications get installed to different locations? Also would the working directory change if installed elsewhere?

There are some other possible locations to install , but that doesnĀ“t mean those are the recommended ones.

Same as MS directory conventions, from the times of Windows 3.x onwards : there was the ā€œofficialā€ way, mentioned in the documentation, and then there are the million of different ways developers chose to use ā€œjust becauseā€. Or in linux, same thing. And that brings us to our current chaotic situation ā€¦

1 Like