[GSOC 2023] Alternate proposal - Non-native binary execution

Community
#1

Originating from this comment, I have a proposal to implement some kind of mechanism to allow non-native binaries to be executed on Haiku.

Non-native binaries here are defined as any executable binary file that runtime_loader doesn’t accept. This includes binaries for different architectures or even different operating systems.

The most apparent use of this is allowing x86 (or more importantly, x86_gcc2) binaries on x86_64 Haiku (HoH64). In the far future the use can be extended to allowing x86_64 apps to run on different architectures through the use of a JIT compiler or running some Linux binaries to ease the process of cross-compiling some software to Haiku.

I can think of two possible implementations:

  • Kernel driver: Syscalls originating from teams running non-native binaries are dispatched to this kernel driver. It then translates into calls of the corresponding Haiku subsystems.
  • runtime_loader add-on: Certain binaries can be registered as add-ons for runtime_loader. When runtime_loader fails, it calls on these binaries by registration order, until all fail or one handler succeeds. Syscall translation should be done by the handler binary, and all syscalls dispatched by the team is native for the architecture.

The kernel driver method gives the emulator more access to kernel resources and allows better emulation in some cases where platform-specific system calls involve handling low-level information stored in the kernel, while the userspace add-on method is simpler and more stable but kernel interaction is limited to syscalls only.

The advantage of both methods is that the support does not require using an entirely different kernel version of Haiku, and the drivers or add-ons can be distributed through HaikuDepot like any other app.

7 Likes
#2

AFAIK GSOC proposals for 2023 are already submitted and set in stone so any news ideas would be for 2024.

What you are describing is basically what binfmt_misc does on Linux, it can pass off a file to an interpreter, an emulator or what have you.

Executing x86_gcc2 binaries is mostly unrelated… as those will be executed natively by the CPU… unless you were on ARM of course and wanted to run an x86 haiku APP on arm etc…

1 Like
#3

AFAIK GSOC proposals for 2023 are already submitted and set in stone so any news ideas would be for 2024.

No, GSoC proposals for 2023 will open tomorrow (my local time). The ones published on the Haiku page are suggestions, students can apply with other proposals.

What you are describing is basically what binfmt_misc does on Linux

Not really. It’s a simpler version of picoprocesses of the Windows NT kernel.

Executing x86_gcc2 binaries is mostly unrelated

x86 syscalls are different from x86_64 syscalls due to pointer sizes. Different pointer sizes also lead to different sizes of many other structures as well. These has to be taken account for by the kernel driver or binary handler.

1 Like
#4

That doesn’t sound all that useful actually… NT has always had support for libOS personalities but it has seen extremely little use. So I’m not sure why you would want to copy a little used feature of NT.

As to you second point… I fail to see how that has any relevance at all to the topic no current os uses such a strategy to implement separate syscall support, they just support both syscall types. Really all this does is make such functionality loadable… rather than part of the monolithic kernel, but again that isn’t really useful. On haiku if a feature needs to be support its just going to be built in. Most of the things that end up as modules (since it is a hybrid kernel) are drivers which may not always be loaded.

Frankly the feature doesn’t sound like a good Haiku fit… but it might be a good ReactOS fit.

#5

we already have binfmt_misc equivalent functionality through filetypes

#6

Not such a positive reply from the community :frowning: I’ll probably focus on my main proposal then.

#7

Haiku is a desktop O/S, & very few desktop users run or need to use foreign binaries. :wink:

#8

Windows is also a desktop OS, and why are there so many WSL users?

Probably the community here has a different mindset.

#9

A few comments (and none of them from any of the core developers) do not reflect any majority of the community. Please do not let this influence your plans for GSOC proposals

12 Likes
#10

Please don´t scare away possible GSOC applicants with misinformation like this.

6 Likes
#11

Haiku being a desktop OS means that it will never be a multi purposes server OS like most linux distros are. It is not only aimed at end users but at developers as well. In fact we need more of them; everything that may make them chose Haiku is a good thing. Though I don’t see many use cases, maybe because I’m not a developer myself.

#12

Though I don’t see many use cases, maybe because I’m not a developer myself.

A use case would be bootstrapping certain software that requires an older version of itself being installed on the build machine, or some other tools not available on Haiku.

Currently for this scenario I choose to use Linux and Hyclone for initial development iterations, and then test the final product on Haiku before packaging.

A method to do everything on Haiku without having to touch Linux would be nice.

4 Likes
#13

From people who don’t even really understand what the work is about.

Anyway, setting up one proposal is enough work, there’s no need for a second one unless another student is applying for the same idea and we’d like one of the two to do something else. Unlikely to be the case here since you didn’t pick one of the existing ideas.

If anything, this forum thread shows you need to start with an “elevator pitch”, picture the following situation:

You are in an elevator with Bill Gates (or whichever billionaire you want). you have 10 seconds with them before the elevator reaches its destination. You can use this opportunity to pitch your project to them and convince them to give you money. What do you say?

In this case you should probably start from the user-facing consequences of the project, something like “running Linux binaries on Haiku to allow use of closed source apps without porting”. You will quickly convince people that this sounds cool (hopefully). Then the discussion can move to more appropriate places for technical details.

4 Likes
#14

I think that Haiku-on-Haiku and running non-Haiku (Linux, Windows etc.) executables on Haiku are quite different use cases. Running non-native executables will need additional kernel modules and/or userland services (like Wine wineserver) to provide features originally implemented by a kernel of non-native executables. For example epoll, sysfs, futex.

#15

I for one would love to see binary compatibility with BeOS applications on amd64 and RISC-V.

1 Like
#16

I’m just trying out ideas to see which one seems more popular as I have a similar interest in both topics.

#17

procfs can be implemented on Haiku just like any filesystem, it simply doesn’t exist yet because Haiku doesn’t need it.

epoll should be a good addition to mainstream Haiku itself, at least according to the person who wrote the GSoC ideas site.

And futexes are just fancy mutexes that can be implemented using existing Haiku kernel infrastructure.

Linux also comes with a lot more unique syscalls that many applications can actually function without.

Also, running Linux executables is one of the long-term impacts for this project, if someone is interested enough to implement such a kernel driver/binary loader. For the proposal I would focus more on HoH and integration with other Haiku features like installing from HaikuDepot/pkgman, setarch,…

3 Likes
#18

Not really, making epoll usable would require bringing in a lot of Linux specific things that go hand in hand with it. If you want to use epoll in a real app: tour timers need to be timerfd. Your events need to be eventfd. Your signals need to be handled with signalfd. And so on.

So we would rather draw inspiration from FreeBSD kqueue instead, where various types of things can be waited onm not just file descriptors.

Well, in any case if you’re going to pursue this idea further, this should be clarified before digging further. As x512 said, the implications and implementation details would end up being quite different.

1 Like
#19

Misinformation? Cut the politically correct junk… I thought that was correct and that random ideas not submitted previously weren’t allowed.

Frankly if they are gonna be “scared” by whatever I said they are probably not good candidates anyway.

Even if technically they could late submit a project chances of finding a mentor that is also interested in something that isn’t on the list is very very low… so in practice is true anyway.

#20

Pointing out wrong information is not ‘politically correct’, it’s just correct. You should try to tone down your American culture war instincts. Not everybody is a native speaker, ‘misinformation’ might have been the wrong word here, because we can assume it wasn’t intentional. However, see below…

I’d rather not have your callousness be the arbiter of who will contribute.

If you continue to post wrong information, ‘misinformation’ may have been the right term after all.
As you now know, the period to submit projects has just started yesterday, ending April 4th. They are not late, technically or otherwise.
And, as the GSoC ideas page specifically says, projects not on the ideas list are welcome, even preferred, as they tend to be more successful.

12 Likes