I do not think it will be limiting factor because each build configuration is quite big (modern userland, legacy gcc2 userland, whole kernel with add-ons, each boot loader).
Update on Ham
I’m looking more seriously at v2. I discussed it with the GSoC programmer in this GitHub issue. Among other things, he said that he started version 2 because the multithreading of Ninja build would be faster than the single-threaded file accesses of Jam and Ham v1.
Flow Control
Jamfile format supports full flow control with loops and conditionals. Ninja build was not so overbuilt. I thought I could bluff my way through using back-quoted expressions as callbacks to Ham so that I would be able to recreate the effects of flow control dynamically. The idea was that by generating conditional macro instantiation using Ham when needed, I could simulate conditionals by not generating code. Loops using tail recursion based on the macro instantiation designed for the conditionals could also be done but It’s not looking so hopeful. The break and continue statements that are allowed in loops would not be possible using tail recursion. Also, I think Ninja build AOT compiles the script inputs so that dynamic macro instantiation will be impossible.
The Ham author was going to cache the results of a Ham invocation to a Ninja build file so that at least the incremental builds would be parallelized even if the initial build was not. I was hoping that this wasn’t the case but the more I look into it, the less hopeful I am.
Jamfile Translation?
After looking at the source of Ham v2, I’m beginning to think that the recreation of the parser without using GNU Flex and GNU Bison was too ambitious for a GSoC project. I’ll have to look at ways to improve Ham v1 to cache its output directly without an additional parser rewrite. Eventually Ham v2 may come out but not before Ham 1.x gains the backside caching ability planned for v2.
It would be worth something to have the warts removed from Jam at least. At best, Ham v1 can only deliver a bugfixed experience and updated internals. Speed will likely be identical to original Jam.
1 Like
Out of curiosity what do we use loops for?
Conditionals I guess should not be done in ninja but in the step that prepares ninja files and maybe it is the same for loops.
1 Like
I’ve been comparing the Jamfile format to the Generate Ninja (GN) format. GN only has for_each as its loop construct. Jam, on the other hand, has a full contingent: while, for, if and within loops there are continue and break. Since for in Jam corresponds directly to the GN equivalent, for_each, that leaves only while and if when doing a translation.
This is not a question of “overbuilt”. These tools operate at different levels altogether.
Jam is a single-stage buildsystem, it does pretty much all steps of the build. In the case of Haiku it is simply coupled with a configure script.
Ninja, on the other hand, is designed to be coupled with a first-stage buildsystem or generator. That could be cmake, for example.
The idea is that there can be as much complexity as you need in the first stage (loops, conditionals, anything really), and then, the output of that first stage is a set of instructions for Ninja that are extremely simple: generate file A from file B using command C. Basically, what Make did initially, but Make got a lot of bloat built into it that should really have been separate generation tools.
So, the idea of Ham v2 would be that: Ham would parse all of the Jam language, determine all commands to run and all files to build, and generate a Ninja file. So, it doesn’t matter if the source Jamfile has loops. These shouldn’t be translated into loops at the Ninja level, but rather analyzed todetermin what commands end up being run during the build phase (that would correspond to invokation of “actions” in Jam side).
The approach could be added to the existing Jam codebase as well, it would be similar to what we have for generating compile_command.json for example. So I don’t expect it would be too difficult to add to Ham V2?
Ham v2 shouldn’t have also required a new parser using a single-header framework. That’s the excessive part.
Ham v2 backend part
I’m wondering “Why not create a fork of GN and put a custom Jam parser on it?” at this point. There’s already a GN port on HaikuPorts
I can think of one good reason - that adds in an extra level of complexity. You need to make the Jam syntax fit in to the GN tool capabilities. That seems like it would probably fail, as you already defined that the Jam language is richer than GN. I think GN is a dead end and will waste you a lot of time.
What you need is a Ham V2 that processes Jamfiles, generates the correct commands and spits out the correct build stage files, then those files can be in whatever build engine format you like.
People write Ninja build files by hand, so I am pretty sure you don’t need a generator pre built, you can just make Ham write the Ninja files directly.
If you really want to use GN, then I would do this: write a stub app that parses Jamfiles in to GN’s native format - this does nothing clever, justthe bare minimum to get the file to run through GN. Then implicitly run GN on them and then run Ninja. But at this point, why do you need Ham or Jamfiles at all? Then you loop back to advocating a new build system.
Not a bad idea! Now I don’t have to wonder whether to use MIT licence (Ham’s licence) or 3-clause BSD (GN’s licence).
Fuchsia and ChromeOS use GN…
Yeah - I mean, if you have got to the point where GN is able to read the output of your proxy Ham, you might as well do that once and just use GN. There starts the religious war over Jam vs other build systems.
Let’s not get mixed up here. Jam is richer than Ninja but not richer than GN. GN has filter_include and filter_exclude which make the boolean operations of if and while unnecessary on many occasions.
Amen. See ticket 18555.
I tend to agree with what @PulkoMandy is saying there. Changing a build system is not trivial and unless it is not a drop in replacement, you will hit just as many odd behaviours - as you are trying to fix here.
Edit: and by “drop in replacement” I am in no way advocating Ham. Personally, I think it is a “solution in search of a problem” as suggested in that thread.
There is still one problem: Infinite recusion needs to be detected.