Trying to understandHaiku build system

Hi all,

I am currently trying to understand how the jam based build system fits together and how the includes are organized. Since I am on a bug hunt regarding bfs_fuse on macOS (darwin) I started there.
Using “jam -d x ‘bfs_fuse’” in folder src/tools/fs_shell gives me some hints. First of all 2 components are build, libroot_build_function_remapper.a and libroot_build.a. Both use

-iquote …/…/…/build/user_config_headers
-iquote …/…/…/build/config_headers
-iquote …/…/build/libroot
-iquote …/…/…/generated/objects/common/build/libroot
-iquote …/…/…/generated/objects/darwin/x86_64/common/build/libroot
-iquote …/…/…/generated/objects/haiku/x86_64/common/build/libroot
-I …/…/…/headers/build
-I …/…/…/headers/build/os
-I …/…/…/headers/build/os/app
-I …/…/…/headers/build/os/drivers
-I …/…/…/headers/build/os/kernel
-I …/…/…/headers/build/os/interface
-I …/…/…/headers/build/os/storage
-I …/…/…/headers/build/os/support
-I …/…/…/headers/build/private/kernel
-I …/…/…/headers/build/private/libroot
-I …/…/…/headers/build/private/system
-I …/…/…/headers/build/host/darwin
-I …/…/…/src/build/libgnuregex
-I /opt/local/include
-I /usr/local/include
-I …/…/…/headers/build
-I …/…/…/headers/build/os
-I …/…/…/headers/build/os/add-ons/registrar
-I …/…/…/headers/build/os/app
-I …/…/…/headers/build/os/bluetooth
-I …/…/…/headers/build/os/drivers
-I …/…/…/headers/build/os/kernel
-I …/…/…/headers/build/os/interface
-I …/…/…/headers/build/os/locale
-I …/…/…/headers/build/os/storage
-I …/…/…/headers/build/os/support
-I …/…/…/headers/build/private

for includes and some defines.

  • what does this define do: ‘HAIKU_BUILD_ATTRIBUTES_DIR=“/Volumes/Haiku_Src/haiku/generated/attributes”’?

  • Why are not existing folders used (user_config_headers. build/libroot, objects/common)?

  • And why do we have some duplicate includes, like …/…/…/headers/build?

  • And what is the purpose of those duplicates of include files, like haiku/headers/build/os/kernel/OS.h, which simply consists of “#include <…/os/kernel/OS.h>”? Is it expected to point to haiku/headers/os/kernel/OS.h?

Thanks for clarification.

1 Like

This directory is where extended attributes are stored on systems which do not support them, or have a lower limit than we require on their size. (macOS’ filesystems do support them and with a large limit, so the directory is unused there.)

I am not sure what you are asking here…

This is probably a bug in the build system, but it’s a harmless bug, so nobody has bothered to investigate and/or fix it.

Yes. Note that some of the headers (Errors.h for instance) in the build hierarchy are different and not just an alias; this is required so that the Haiku-specific error codes can exist on Linux and other OSes without them, but also so that the build libroot can stay independent of the system libroot, so that if we change libroot significantly, we can continue building the newer Haiku on an older Haiku system.

A more high level view of things may be useful to stat with:

One of the problems in building Haiku is that a lot of our tools, including ones used for building haiku, make use of Haiku/BeOS specific APIs. But, of course they need to run on whatever host OS you are building on (in your case, Mac OS).

So, what we do is start with building a subset of libbe, that runs on the host operating system. It includes only some of the kits, just what we need for our build tools, and even some classes are only partially implemented.

This set of buildtools is the one using headers in headers/build. A lot of the headers are just the same as the normal ones used when running Haiku, and for these we just #include the original header. But in some cases we need to do things in a different way, and the header is replaced with a different one (removing some methods, implementing a class in a different way, …).

2 Likes

:slight_smile: Simple: why does it include non-existing folders? Also a bug in the build system, or are they needed by other components (which I did not build)?

Okay, but this could easily be solved by the order of include search paths, IMHO. I mean, if I follow those …/os/kernel/OS.h, I need to know which folder is used as a base to apply this relative path. I could not figure that out (it is not the folder of the include file), I just assumed that the “original” header is used then.

While trying to build bfs_fuse with Xcode (so I can use the complete debugging possibilities) I also found out that some includes are used with special #defines for one component (aka .cpp) and for other components the #define is not allowed (same includes). I guess there is no further documentation on all important #defines?

Also some sources need a compatibility header, which is given directly via “-include” to the compiler.
Is there any rule for that? Why not include that directly?

It is the include file. Because that’s how gcc works and there is nothing we decide about this.

These are inthe generated directory. As the name implies, files there are generated during build. Probably because they depend on which host you build from and we cannot make them a single fixed version.

It’s hard to tell without specific examples, but in general the idea would be that the file was written for use in one setup, but then the same file was reused in a different setup. In the case of filesystems, I guess you came accross KERNEL_MODE which is defined when building a file to run inside the kernel, and not defined when building to run in userspace (as is the case for fuse).

As for -include, I guess it could be used similarly, but I don’t know offhand of an example where we do this.

Documentation on the internals is indeed far from perfect. HElp improving it is welcome :slight_smile:

It’s used for the BuildCompatibility headers, to ensure that remapping of various functions and constants always occurs no matter what.

I finally was able to build the bfs_fuse component with Xcode (means without jam). This version works out of the box, while the version build with jam segfaults. Since all includes, sources and link libs are the same, I think it may have something to do with compiler/linker flags. But I can’t figure out how/where these are stored.
So please enlighten me, where are the CFLAGS for host builds stored?

They are spread out in different places. You can, however, enable some Jam command line flags which will show you the actual compile commands being used, and figure out flags from there.

Yes, I know how to get the flags from Jam. But I want to frequently replace/modify the macOS standard flags for the host Jam build to test the binaries and see if it changes something.

The flags are all set in different places. Most of them, you can grep the source tree to see where they are being set.

Use SubDirCcFlags and SubDirC++Flags to add more flags in a given subdirectory’s Jam invocations.