[GSOC 2023] Alternate proposal - Non-native binary execution

The locale kit is nothing like adding a libOS layer to the kernel. The entire point of that aspect of the comment was there is low availability of kernel / driver developers for Haiku. Anyone going after such a goal would have to find someone even more rare than because they’d also have to be interested in the topic. You are reading my comment out of context.

The locale kit is almost entirely useland and mostly file IO and string processing is it not? Which is easier to find a mentor for.

You have a link to that? Lots of OSs inpmenent some kind of misc binary handing. But not too many implement libOS / Unikernel support which is apparently what this is more akin to.

I wasn’t aware of FreeBSD doing anything like that. They do have userland compat layers but I’m pretty sure those are not implemented like that.

Please Google (or Bing) the facts before you post…

This is the first thing I got from Bing. It said:

In effect, there is a Linux kernel in the FreeBSD kernel. The various underlying functions that implement all of the services provided by the kernel are identical to both the FreeBSD system call table entries, and the Linux system call table entries: file system operations, virtual memory operations, signal delivery, and System V IPC. The only difference is that FreeBSD binaries get the FreeBSD glue functions, and Linux binaries get the Linux glue functions. The FreeBSD glue functions are statically linked into the kernel, and the Linux glue functions can be statically linked, or they can be accessed via a kernel module.

https://wiki.freebsd.org/Linuxulator

Yeah that’s not a libOS though… is one kernel that implements both types of syscalls. Linux does this also, its done right inline in the kernel code.

When posting this, I did expect the topic to attract a little bit of debate of “building more Haiku-native solutions” and “keeping Haiku simple and consistent” vs “port more apps and bring more compatibility”.

But I don’t expect this to become so hot and political…

1 Like

Please also read the link I attached about the picoprocesses.

They are not the old, abandoned “subsystems” implemented in the early days of Windows NT to support OS/2 and POSIX PE binaries. They’re a new thing that works a lot like FreeBSD’s kernel driver, and they power Windows’s WSL1.

They aren’t new at all. They have been around since NT was written, the picoprocess aspect may be a specific case of libOS though.

NT has always and continues to have libOS as the way NT’s OS personality is implemented. In the past it also had additional personalities for at least OS/2 and Unix, not sure if there were others. The hardware subsystem is also modular as you probably know with the HAL.

Apparently Windows 2k was when the OS/2 personality was dropped.

Whoever said they were “new”? And what’s all this about “libOS”? Being able to run foreign binaries doesn’t require a “libOS” and usually isn’t implemented with one…

Also, Linux supports part of this on the kernel side, with being able to register “binary handlers” so qemu-user can be run implicitly. There’s also ways of doing this without kernel support, too, as in a few projects that tried to implement a WSL1-style system on top of macOS.

We support that functionality with filetypes already (that is “run this app with this magic at file start”), I don’t think that’s what the proposal directly wants but rather foreign /ELF/ binaries, if I understood correctly.

If you read back through ignoring the several posts of nonsense from myself and others… that is also what I originally thought. But then OP pointed at NT’s picoprocesses… which are implemented as libOS… at least on NT. And that is why the discussion went in that direction because OP had a specific implementation in mind, eg multiple kernel APIs living in kernel land. Or libOS as an in kernel application.

WSL on Windows is implemented as a libOS / picoprocess.

What you and I originally though was the Linux binfmt_misc support… but that is apparently not what OP is talking about. binfmt_misc is as you stated just a kernel side handler to pass off a binary to some potentially arbitrary method of execution.

binfmt_misc is also good enough to implement, emulation of alternate platforms as well (since it would just recognize say an ARM binary on x86 and pass it off to QEMU pointed at the ARM userland)… but probably has tradeoffs with the NT like libOS method.

One example of a drawback is the binfmt_misc method… relies on QEMU to execute binaries from other platforms, and that only supports running Linux binaries on Linux etc… it doesn’t actually translate between OS personalities it translates between architectures only.

As I’ve said twice before, we already have binfmt_misc equivalent functionality. Nothing needs to be done here.

I tend to agree but OP isn’t talking about binfmt_misc…he already clarified this when he replied to me. He is talking about picoproceses/libOS. Which can accomplish similar things but different and is potentially better in some use cases, since its implemented natively as an OS module.

For example
Binfmt_misc + QEMU can do
x86 Linux binary > qemu translating syscalls and emulating the x86 instructions > arm syscalls on ARM CPU

libOS/picoprocesses
Do the same thing for Operating system APIs. In addition to syscalls. So for example the whole BeAPI could be a plugin on windows NT that’s totally possible. And it would not pass through any of the NT userspace API to do that it would be written on top of kernel the same as the NT apis. There can also be unikernel applications operating on top of the HAL instead of through the userland…

And so, if a GSoC student who has the skills in that area offers to contribute and possibly join Haiku long term, what do you think we’ll do? Ignore it and lose a chance to get more knowledgeable people in that area? Doesn’t sound like a great move to me…

I see some elitist Linux kernel developers have been very succesful at convincing everyone that operating system kernels are some kind of Dark Magic or Forbidden Knowledge. It is not. It’s just plain C++ code. Like in any project, there is some domain specific knowledge. Here it will be about interrupts, context switches, and virtual memory. In the locale kit it is about character encodings, cultural differences, alphabets and writing systems, timezones, grammar rules, …

There certainly was not much knowledge about these things in Haiku when I started the project. Much less than knowledge about kernel development. I think that is still the case today.

Well, except actually implementing variants of runtime_loader for various executable formats, making sure they can rely on a stable interface (since we don’t have stable syscalls, and it’s likely not possible to load libroot.so into your foreign executable), and probably dozens of other problems once you start really trying to do it.

We have the groundwork, sure, but there is quite a bit more to it.

8 Likes

Kindly stop continuing this nonsense. If you wanna get on topic lets do that. The purpose of this forum isn’t to generate animosity after all. I disagree with your reply but will leave it at that. I was absolutely NOT trying to discourage OP or gatekeep… frankly the continued accusations are tiring and reflect badly on you for making them over a simple mistake.

Yes, for foreign ELF files there is a lot to improve, my point way that ‘making a binfmt_misc equivalent’ as suggested in the thread is not neccesary, and thus likely wasn’t ment as a target anyway, but rather something like freebsds linux compatibility.

You keep repeating that but OP never mentioned binfmt_misc that was just a bad assumption on my and others part.

Yes… that is what I said…

1 Like

Ok LOL I will chalk it up to me being dense sometimes :smiley:

1 Like

We’re all dense sometimes! :slight_smile: