Test builds of proposed changes

A warning first: these are untested, unsupported, short-lived, no-guarantees, unofficial builds from a 3rd party of not even development code, but proposed changes to it. You are on your own if you use them.

It doesn’t happen very often, but sometimes a bug has a proposed fix, the developer can’t test it due to lack of hardware and the user can’t build a package. They may find it in https://haiku.movingborders.es/testbuild/, if the machine is willing to work and has picked that change from gerrit. The interface is quite bad, though, so depending on what you want you will have to hunt and click a lot.

It may also be useful for devs that can only build for one architecture. Even though there are no tests and only the nightly-anyboot target is built, it may be enough to check if at least it continues to build in both supported architectures.

14 Likes

That’s great! Can we make it official? Can we make it post the results to Gerrit? (I can try to help with setting up that part)

I guess it depends on what making it official means. If it’s just linking to it when asking to test a patch in a bug report or to point a problem in a patch review, you can of course do that. For more than that, I’m not sure it would be a good idea.

There’s no quality backend behind that page. I didn’t bother learning how to setup, manage and integrate something like concourse or jenkins, I just wanted to write “a little script” to use gerrit’s REST API and maybe make a build. It got out of hand and I’m not a real developer, so you can guess how the code looks. No config file, hardcoded stuff as I needed it, code repetition, no separation of concerns… If you were thinking of integrating it in Haiku infrastructure, I guess it would be simpler to write whatever concourse needs. It would certainly be easier to mantain, and The Right Thing to do.

Work is done in bursts. The script will go idle for a few hours then have a conversation with gerrit and start building stuff if there are new patches. It also won’t rebuild everything when there’s a new release. So a new change may not have feedback for hours, and an existing change affected by a recently merged one may not be updated for days.

There’s also no attempt at following patches’ dependencies. When there are several commits in a branch, two builds are attempted: one rebasing all the branch and another one cherrypicking just the last commit. No check for updated intermediate changes, for example, or any other thing one should look for in a relation chain.

I don’t have alerts or even safe failure code paths. If something fails, the process dies and may leave some stuff in limbo, even with some garbage behind. It will run again a few hours later, triggering the same bug (low disk space would be an easy one for a “self denial of service”) or maybe not, in which case no one would notice until an unlucky soul tries to get the half done build.

If you think all that is just a mild inconvenience and still want to promote it as official, I’m OK with it.

If you still want it to report build status to gerrit, point me to the details and I’ll see what I can do. I guess post review with verified label +1/-1? But please let’s make it only if the score can be manually overridden or disregarded by those who can merge.

2 Likes

From my point of view as a developer, any tool for this is better than no tool at all.

I will let the sysadmins figure out the details on how to best integrate it with our infra and make it reliable.

There are normally separate labels for this, humans will set “Code-Review” to -2/-1/+1/+2, and the bot can set “Verified” to -1 (build fails) or +1 (build succeeds). It can also post a comment (there is a specific thing called “robot comment” in Gerrit APIs), for example to give a link to the build result.

We can decide to make the “Verified” label not required and not blocking, so that it is only informative for now. When we get more confident with the bot or it becomes more reliable, we can make +1 required or make -1 from the bot block merges. But let’s start without that.

I made a change request on Gerrit to enable the “Verified” label: https://review.haiku-os.org/c/All-Projects/+/3838

See also Gerrit documentation about it: Gerrit Code Review - Review Labels

I don’t know if your bot will need specific permissions for this, and if they are set like other users permissions.

3 Likes

It seems that is still not implemented (I don’t have access to that link and the discussion, but I can clone the repo and I don’t see the change merged). Would you find it useful if I started sending build status with no label?

1 Like

I just merged it. There has not been a lot of discussion from other devs so I assume they are ok with it (I see no reason why not)

403 Forbidden when adding the label, accepted when not using it. The web interface also says “You don’t have permission to edit this label.”

I guess I (lonemadmax, ID 1000223) need permissions to use that label or credentials of a ‘bot’ or somesuch user type?

1 Like

Hello, i see the errors aren’t included in the log.

https://haiku.movingborders.es/testbuild/If3a2920ba9aa0a85eaff4ba6778947f8c76ade31/1/hrev55068/x86_64/buildlog-stdout.html#n2750

Could you please check?

I have stdout and stderr in different files: invalid types ‘long int[int]’ for array subscript

Checking why it is not marked as an error, though. The only reliable info is whether the build passes or fails, capturing of specific errors is brittle, but that one looks clean.