The impending ABI apocalypse (if I understand it correctly)

Development OS
1

Greetings!

My name is badasahog and I’m new to Haiku (and non-Windows operating systems in general)

I wanted to fully document my understanding of the upcoming ABI crisis so that I can get confirmation that I understand it correctly, and possibly help maintainers add a section about it to the website so that new contributors can understand what’s going on.

Operating systems are (usually) written in a programming language that compiles into machine code. What the programming language actually means (ie its behavior) is defined by a standards document. This means that different compilers of a language will all create binaries that do the same thing as long as you’re not leaning into undefined behavior.

The trouble with this is that standards documents don’t often define how programs do what they do, just what they do (often referred to as the as-if rule).

This shouldn’t effect the developer, but it can in some situations. In C this isn’t such a big problem because as long as you have the same calling convention (ie the way that data is passed between function calls) you should be good to go. Binaries compiled with different compilers should be interoperable.

The trouble is that C++ is far more complex, especially when it comes to classes. What order is the vtable in? where is it located? All these things and more can create problems.

Unlike other operating systems, which have a pure-C interface (such as posix and winapi), the Haiku API uses classes. This creates a problem if a program compiled with one compiler tries to communicate with the Haiku kernel compiled with a different compiler.

In other words, we all need to agree (Haiku developers and Haiku application developers) to use one ABI.
Unfortunately this decision has already been made for us, by the developers of BeOS (the predecessor of Haiku).
In order for Haiku to run original BeOS applications it needs to be compiled with the same ABI that was used when the OS was developed.

Unfortunately, this ABI is no longer used by modern compilers so Haiku developers need to either modify the compiler they want to use after every new release to conform to the ABI, or be stuck using legacy versions which are old and don’t support the latest C++ features.

This is why Haiku is written in C++98, which is making development more difficult.

Did I get all that right?
If not please correct me! I am excited to learn more.

2

Only if you care about BeOS compatibility, which is jeopardized in some areas already.
Haiku x86 supports 2 ABI, the gcc2 one for BeOS compatibility, and the gcc11.x one.
Other Haiku versions (like AMD64/x86-64) supports only the gcc11.x one.
I see no apocalypse.

5 Likes
3

This sounds like a socio-political statement. :thinking:

I totally agree with the rest though. :grin:

4

I see apocalypse in plenty other places, like in CS, but not in the Haiku ABI. Maybe i am too optimistic.

2 Likes
5

So gcc11.x is the one that’s gonna stick around?

If so, will the kernel be migrating to C++23 once BeOS compatibility is dropped?

Alternatively, would it be possible to create an ABI translation layer to allow multiple ABIs to work?

If there’s a framework for it, it might be possible to quickly stamp out support modules for all the most common ABIs being used.

Is this too silly or am I on to something?

6

A translator could wrap the current Wine port, I suppose, and WebAssembly is a favorite of mine also. The ability of Haiku to adapt is not the problem. It’s the bloat inflicted on monolithic kernels like Windows, Linux and even Haiku itself. The bigger an operating system gets, the lesser its performance.

7

I’m not sure I understand what you mean? What do Wine and WebAssembly have to do with this?

8

They are alternative ABI formats that run on top of the existing Haiku ABI.

9

Isn’t Wine a Windows on Linux thing? And WebAsm is a way to compile code for web browsers?

I’m confident I’m not understanding you properly (which is probably my fault)

10

Wine has been ported from Linux to Mac (Darwine) and also partially to Haiku. It lets other operating systems run some Windows software.

WebAssembly can run outside the browser as well as inside the browser. For running outside the browser, see the WebAssembly Systems Interface (WASI) standard.

11

I believe the long term solution for Haiku will involve having everything move toward modern C++ compilers and ABI, with some smarts in the runtime_loader and kernel to allow running specifically BeOS gcc2 ABI 32-bit applications in a modern Haiku (regardless of whether that Haiku is itself 32-bit or 64-bit, though I expect most people will move to 64-bit on computers from the last 10 to 15 years.) At some point we will have to accept that any BeOS applications without source code are lost to time and should be replaced with something new.

I don’t think virtual machines, web assembly or something like WINE will be needed for this. Plenty of modern operating systems have a way to run 32-bit applications on 64-bit machines, and in addition there is plenty of prior work on dealing with different ABIs. APE for example is pretty insane and they make it work across many operating systems (be warned it is hard to understand and this blog post doesn’t help that much, here is another blog post from someone else about it.)

We have some existing work in Gerrit to run 32-bit apps on a 64-bit Haiku but waddlesplash did not like some aspects of how it was implemented so it has sort of stalled. In retrospect we maybe should have just merged even the imperfect version but that is sometimes how things go here. This may also solve the ABI issue too.

6 Likes
12

I’m not asking about running programs written for other platforms, I’m asking about running code that targets the Haiku API, just compiled with a different ABI. (expertly crafted diagram below)

ABI_translator
ABI_translator3464×1296 77.7 KB

13

very hard to look virtuous without a crisis upon which to pontificate and show concern

3 Likes
14

I’m not trying to play hero, just wanna learn. I’m new here

2 Likes
15

Re:the diagram

LLVM/Clang use the same ABI as GCC. The other thing with your diagram is that Windows software will require a different translation layer (Wine) than Linux sources written using its APIs (Qt etc.) and those that require a full emulator (like FS-UAE running old Amiga software).

16

Wine is a different situation, I’m talking about native applications.

This problem is unique to Haiku since other platforms’ APIs are written in C.

On Windows there’s something called COM which allows programs to share objects across DLL boundaries. Would it be possible to make something similar for Haiku so that the Haiku API could support multiple ABIs?

17

I believe the kernel already is using the gcc 11 even on 32 bit. Which is why BeOS drivers etc no longer work.

Some time ago two different 32 bit builds were offered:

  1. gcc 2 hybrid, which had the kernel and most of the OS compiled with gcc2 and was compatible with BeOS binary drivers, but also allowed applications to run using the newer ABI of gcc 4 (which was then current). So primary ABI was gcc 2 and secondary gcc 4.
  2. gcc 4 hybrid which was the other way around, and is what is offered now. It allows user space gcc2 applications to run, but the kernel and the rest of the OS are compiled with modern gcc and binary BeOS drivers cannot be used. So primary ABI is gcc current and secondary is gcc 2.

I don’t know if it’s still possible to build a gcc2 hybrid.

18

What’s the current configuration? Does Haiku currently use a hybrid system? If so, can it be expanded to include additional userspace ABI options?

19

32 bit x86 systems can. The others, including AMD64/X86_64 do not.

I suppose it theoretically could, but ideally shouldn’t. The more complexity added, the less maintainable the system becomes.

That’s why I keep bringing up wrappers like Wine: they are optional but add more compatibility.

20

I see; So if you want to develop applications for Haiku you need to use gcc11 ABI? And if the Haiku team decides to migrate to a newer version everything will need to be recompiled for the new ABI?

Also, if Haiku uses gcc11 why not migrate to a newer version of C++?