pkgman is just an userland frontend that tells the package filesystem what to do. If you remove the filesystem, pkgman wouldn’t work anymore.
Anyway, this way of working has the agreement of all Haiku devs and a large part of the community. And, you did not make any proof of your statements for reduced performance. So please bring in the benchmarks, and we’ll see what can be done. Otherwise you are only spreading FUD, and that won’t help anyone.
There are known ways to improve performance already, one of them is not compressing the packages using zlib, and either leaving them uncompressed (already possible) or switching to a more CPU-efficient compression like zstd (this is work in progress). As you can see, we are aware of the issues, however we don’t think throwing away the whole package filesystem is the right solution.
If you break things with the package filesystem, it’s super easy to go to the boot menu and boot an older state of your system. This means installing a package is not a very risky operation (it is very easy to remove it). That would be impossible without the package filesystem, and it is great to have a safety net like that.
You say that installing some packages can break everything, and yes, that is true and it happened to me in the early days (now people understand better how to make good packages). However, it also happened to me, many many more times, on old versions of Haiku and also on BeOS, so back then I became very reluctant to installing new software - I always feared it would break something and I would have to do a clean install.
Package management actually improved the situation a lot, in that aspect and also in others. For example, it is only thanks to it that we got Qt running reliably. Yes, it was broken for some time recently. But in just a few weeks, all libraries and applications were rebuilt and everything was up and running again. Imagine what would have happened if we did not have recipes and a reliable tool to build them. People would still be messing around, trying to figure out who compiled this or that lib for the last time and where they stored their patches and why their readme with compilation instructions doesn’t work anymore.