The impending ABI apocalypse (if I understand it correctly)

I think the C++ ABI has been quite stable since it was changed some time after gcc2, and Haiku, unlike Linux, does not intend to constantly break the ABI and libraries and make life hell for developers.

Though much of the hell on Linux is that it is just a kernel and there are 10,000 different distros with different libc implementations, packaging systems and everything else. Read any story about trying to write something that runs on multiple Linuxes without changes. I am amused with the massive hoops the main Zig developer Andrew Kelley went through to try to get a basic Vulkan app running on any Linux. It was a massive pain and he thought it was some sort of win, when really it just showed how much of a failure Linux is as a stable platform. I can’t find the write-up about it right now but here is the Git repo: https://github.com/andrewrk/zig-window

5 Likes

I can’t find the write-up about it right now

It turns out this was in a video, though it is 42 minutes so I can imagine people might not wish to watch, but in case you do: 2021: Year of the Linux Gaming Desktop - Andrew Kelley - YouTube

2 Likes

I thought ABI compatibility was a Haiku-specific issue because other operating systems use a pure C interface, no? Windows programs written for older versions of NT still work today (usually), and you can use any compiler you want.

Keep in mind that I’m not as smart as you guys, so I’m probably not fully understanding this very complex situation

In order to use the same linker as C uses, there is a process in C++ called “name mangling” that changes the names of the symbols in the object code to allow duplicate names to exist using suffixes determined by the namespace. The “nm” tool in GNU Binutils is probably the command you need to look up.

I know what name mangling is, but I thought that that was a C++ specific thing because C doesn’t have function overloading

That’s why functions that are imported/exported across DLL boundaries use extern “C”

I ran out of comments and your system won’t let me post for another 24 hours.

Regardless, I’m gonna leave and do some research about Haiku before I waste any more of your time (thanks for helping tho, I really appreciate it :smile: )

1 Like

Correct. The extern “C” turns off the name mangling so the C ABI can be used. Haiku doesn’t do that becuase the name mangled names are stable in the current versions. GCC2 was a buggier early implementation that is maintained only for BeOS compatibility.

Yeah I think it didn’t change since gcc 3.something

Is there any way to compile and use this 32bits compability from source? i want to experiment with it.

That or it can be run in QEmu with a 32-bit Haiku, like DOS-Box is used on Linux to run old MS-DOS software. All next-gen Amigas have 68k compatible JIT compilers but only the ones with UAE derived chipset emulation can work with most of the software. Maybe the BeBox will fall into that same category.

It may not currently compile but X512 did upload a patch in April: https://review.haiku-os.org/c/haiku/+/2874

It may be more interesting for the discussion than the code, it may not be in a state to experiment with.

1 Like

So it seems no one wanted to provide a good clear overview of the current situation and several people are already trying to offer solutions before everyone has a good understanding of the “problems” here. Let’s fix that.

First, the current situation.

Haiku currently uses the latest version of gcc (currently 11.2) in most cases. All the code can be compiled with it, and if you run anything but the 32-bit x86 version of Haiku, you won’t find any traces of gcc2 anywhere.

The current versions of gcc use a standardized ABI called the Itanium ABI (it was originally standardized for Itanium CPUs, but it is used also on other architectures). It has been broken exactly twice:

  • In gcc3, gcc switched from a previous ABI that predates C++98 to the standardized Itanium ABI
  • When implementing C++11, the std::string class was modified in GCC5

The first of these two was a major break with everything changing, and the compiler provides no way around it. Either you use gcc2, or you use later versions. It is not possible to mix both.

The second one is a minor change to one class in the standard library. The compiler can be toggled between the two versions as needed with a compile time flag. However, when running a program, all libraries and executables must agree on which style is being used. Indeed this required recompiling everything.

Now on the special case of the x86 32bit version of Haiku. This is the only version that provides compatibility with BeOS applications. Initially we aimed for full compatibility: not only applications, but also drivers, add-ons, etc. The drivers part has already been removed (because it was not needed by anyone anymore). As a result, even in this version, the kernel is compiled with a modern gcc and is not restricted to C++98. We can and will consider migrating more parts of the code to a modern gcc in the future. For example, there is no reason to compile app_server with gcc2 anymore.

Next question: where must the ABI match, and how can we mismatch things anyway?

The ABI must match between all things in a given process (or “team”, in BeOS wording). That means one executable, and all the libraries and add-ons it loads. There is no need for the kernel and userspace to use the same ABI, because the communication between the two is not traditional function calls, but system calls, which are much simpler and easier to provide compatibility for.

That also means there is no need for compatibility between the ABIs used by different apps. Here the communication will typically be done using BMessage, which does not change depending on the ABI.

So already we have reduced the surface of the problem a lot: it’s just that an application and its libraries must be compiled with the same compiler.

And even that isn’t completely true, it applies only if the interface between a library and an application uses something that has changed between the two ABIs. In the switch from gcc2 to gcc3 this was “everything C++”, but in gcc4 to gcc5 it was only std::string.

So, if your library exposes only C APIs, it is in fact possible to mix and match different compiler versions. For example, we have a version of ffmpeg that is built with a modern compiler, in a way that gcc2 applications can still use it without problems. It is not easy, but it is possible.

How do we handle this currently?

Our solution at the moment is to provide two versions of all system libraries: one compiled with gcc2, one compiled with gcc11. They are located in different directories and runtime_loader (the code tasked with starting new applications) knows how to load the correct ones for each application.

Some parts of the API are already not available for gcc2 apps. The goal is not to keep gcc2 working forever, but be able to run BeOS apps until they all have been updated or replaced. There is an ongoing effort by the HaikuArchives project to reach authors of BeOS apps and get them to update their work, or, if they don’t want to do that, publish the source under a license that allows us to update things.

Eventually, the interest for the gcc2 ABI will be sufficiently low, and we can remove it from Haiku. We have not reached that point yet.

With this setup, we have sufficient degrees of freedom to introduce modern C++ in the code for newer versions of the API and ABI, simply disabling these parts for the gcc2 build. So, only the parts of Haiku that replicate BeOS functionality really need to build with gcc2.

What will happen in the future?

We will have to break ABI every now and then for various reasons. Not only because of compiler changes, but also because of design problems in the existing API. In some cases we can make this work transparently by providing multiple versions of a function (this method is called symbol versioning).

At some point this may not be enough anymore, and we may decide to make an entirely new set of libraries. That is what we call “R2” (short for “Release 2”). There are not much written down plans for this, but I expect the R2 version will provide the R1 libraries to run old apps, and in addition also provide the new R2 libraries to run new apps, just like the current R1 version contains a build of the libraries for BeOS apps, and another set of libraries for the newer gcc11 apps.

We can easily commit to each version running the apps built on the previous one in this way. If we only do a major release every 20 to 25 years, that is more than enough. If we do releases more often, maybe we want to keep these alternate sets of libraries around for a bit longer. This is what Windows also does, you may know of the “Visual C++ redistributable” packages that some apps will install, there is one with every version of Visual Studio and it contains a copy of all the libraries to run apps compiled with that version of Visual Studio.

The “apocalypse” is when we remove support for one of these sets of libraries. It will have to happen eventually, but we will make it so people have a lot of time to adjust, by porting or replacing applications that have never been updated to a newer version. Or, if that’s not acceptable, somehow keep running these old apps by keeping their libraries around forever. Since they will be a separate set of libraries, they do not really force us to use C++98 for everything else.

I hope this answers most of your questions. Now people can discuss their ideas on how to make this work even better, or other problems that oculd happen and that I didn’t mention :smiley:

17 Likes

what would be interesting, is if there was a virtual machine, kind of how Microsoft has one for XP, but inside haiku, for beos r5, haiku gcc2 etc etc. I’m not entirely sure how that layer works in win8 and up, but it would solve all of these issues and allow a clean code break

2 Likes

Done already because BeOS kernel modules binary compatibility is not important today.

1 Like

History is more complicated. BeOS actually had 2 ABI for x86: one is MSVC ABI and PE executables (used in BeOS R4) and second is GCC ABI and ELF executables (used in BeOS R5). BeOS also run on PowerPC machines where PEF executable format is used. Early BeOS versions used AT/T Hobbit CPU and probably COFF executable format.

2 Likes

Kind of like what I suggested earlier with QEmu and the BeBox compatibility?

It is not my code, it was originally written by @korli, but he abandoned patch. I propose userland-based (Korli approach is kernel-based) compatibly layer UserlandVM that in theory can run applications for any CPU architecture, not only supported by hardware natively. UserlandVM currently supports running riscv64 applications on x86_64 with software CPU emulator.

x86 → x86_64 need more work because of syscall argument and structures translation and controlling 32 bit CPU mode.

This real-world info is very useful to would-be devs. Should all this ABI/API/GCC wrangling be in the documentation - if it isn’t already? Some block diagrams about how they all fit in with the OS and relate to the kernel would also be helpful.

4 Likes