I was curious if there are any plans to identify some release candidates for Haiku B1 to support testing prior to a formal release of Haiku B1 to ensure that Haiku B1 is at least as stable as Haiku A4 if not more so. Obviously Haiku B1 is more feature rich than Haiku A4 but ensuring that the former is better than the latter generally speaking plus making sure it’s beta quality would be a good thing.
1 Like
I feel it is a bit strange to do RC testing for a beta. What’s next, releaseing a beta of the beta? :o)
What we did in the past, is that we would switch the nightly builds to generate images from the alpha (or beta) branch during the release preparation period. This means during this time, the nightly images are getting some testing coverage. There wasn’t really a “release candidate” process.
The plan for after the beta is to have much more frequent updates to the release branch, eventually leading to R1. This means the beta is not the end, but rather the start of the RC testing process for R1. We can continue making new releases from that branch, until we reach the R1 target. What remains to be decided is how often we make new releases from that branch. Could be a fixed schedule (every 6 months, 1 year?), or could be whenever there has been enough changes since the previous one.
As for “beta quality”, that only means “all features are supposedly implemented, but there are probably bugs left”. It is a rather low requirement. But knowing the Haiku project, I would think people will set themselves much higher goals.
1 Like
PulkoMandy,
Regardless of what kind of release it is release candidates are relevant no matter if it’s an alpha, beta or final. Normally you would have internal testers that would shake down an RC prior to releasing something considered worthy to the general public. Haiku B1 should be as good if not better than Haiku A4 is you need to know that before committing to a release to avert a PR disaster. No one is expecting Haiku B1 to be perfect, but if it’s worse than Haiku A4, then people will take notice and that could/should be avoided.
-DCatt
When we will be ready for the general public, we will not put a “beta” tag on our releases anymore.
To me this seems pretty clear, in the release cycle you have alpha, then beta, then RC, and finally the “golden master” stable release. If you start putting the RCs before the beta, then something must be wrong.
But that is only a naming problem. There will be nightly images generated from the beta branch, and we will call for testing on those before we make the release. Call them “RC” if you want.
However, I think it is not a good idea to rely only on this to spot (and even more, fix) possible regressions. What we need is some continuous testing of the nightly builds to spot them shortly after they are introduced. At the moment we are largely relying on the user community to do this work, but a more formalized process may be a nice thing to have.
PulkoMandy,
In speaking on a release candidate I am talking more internally and not something you’d release by name in any official capacity. From my experience in software testing, regardless of release, that being an alpha, beta, gold master or final/stable, you draw a line in the sand to identify and commit to a ‘release candidate’ in-house and plan a more formal testing event to give sign off of readiness before ‘shipping it’. Yeah, sure, you could use RC or ‘release candidate’ at a more outward and official capacity but I’m more use to it being used internally to identify a build that seems worthy of an official release no matter what kind of release it is.
-DCatt
I have to agree with DCatt here, perceptions are important. I think an internal RC before a beta is a wise move, since the beta will probably be “the official thing that people might try” for a while, like it was with Alpha 4.
The standard process for any software is Alphas -> Betas -> Release canidates -> Release
kallisti5,
The process is not as standard as you think. Yeah, I remember Mozilla using the RC officially and publicly in the case of Firefox historically and a few other notable companies followed a similar pattern. But not all companies use ‘release candidate’ or ‘RC’ as part of their release cycle going from alpha to stable release. Take Apple for instance, they use ‘GM’ or ‘gold master’ for the release prior to the final release and I am actually use to that from previous experience (at least for software delivered on CD).
-DCatt
What more is expected than asking everyone “internally” and “externally” to give the nightlies after the Beta branch special attention? Everyone will anyway, because they know we’re on our way to Beta, and testing is done on nightlies…
It’s like PulkoMandy said:
Or do you propose not to do nightlies after the branching and do “weeklies” and call them “RC”? Not sure I see the gain…
(BTW, all the above is not meant in an aggressive way at all. I do appreciate everyone pondering how to ensure maximal quality for the Beta.)
humdinger,
You want to identify a nightly build worthy of special testing attention. I don’t assume that every nightly is stable enough to be considered a ‘release candidate’. Also you want to have some sort of testing and bugfix protocol to get at least the critical bugs sorted out that would make for a bad beta release. Some coordination and planning seems like a good thing to do.
-DCatt
When we get ready for the beta, things will go that way:
- We create a “release branch” in git.
- We switch the build bots to produce nightlies from that branch instead of trunk
- There is a strong review policy for anything that goes into the branch: only bugfixes, no new features, and systematic code review before merging
As a result, all nightlies provided during this period are supposed to be more stable than the previous ones.
The testing protocol isn’t too formalized, but there was some work on writing QA test plans during Google Code-In, which maybe we can start applying. We also ask people to test the build on as much machines as possible (from a clean install) to make sure we did not break some hardware support (not everything may be supported, but at least Haiku should boot to the desktop everywhere). We can blacklist some drivers or remove PCI IDs of devices that are known to not work (crash, black screen, etc).
If I may interject? In late 2013, the Smoothwall Express opensource project was ‘plopped’ in my lap; I became the project lead. At that time, I had almost no (zero) experience developing or releasing open source projects; I’ve improved a little in the meantime.
I chose to release aleph, beth, gemal, delat, etc, releases until I had the system good enough to release to the public. (Yeah, I chose a different alphabet.) Aleph releases were, mmm, preliminary releases that generally worked but were nowhere near complete. Beth releases had most functionality but were still not complete. Gamal releases had most everything in place, but some things still didn’t work right. At some point, most everything was working, so I switched to RC releases.
I would suggest something similar for Haiku. When Haiku-OS is mostly right but there are still some important thing that need work, switch to RC status.
But above all, be as clear as you can be about what works and what doesn’t work. No one will fault you when you produce a aleph, beth, gamal release that doesn’t fully work. Look at Debian; they don’t release until it’s ready.
When Haiku-OS is ready for a release (aleph, beth, gamal, delat, etc.) release it. When it’s good enough for RC status, release it as an RC. Otherwise, keep improving what’s there. Y’all will get there. And I’ll be waiting. Remember, no other UI allows you to click a column, then shift/ctrl/alt click a column to sort the data by the first and then the second column; only BeOS has that. 
I haven’t tried a recent build. But I have noticed that builds over the past year or two have continued to become more stable and usable. Haiku-OS needsa very stable browser and an very stable email program. Those two alone would convince plenty of people to try it.
[Aside, does Haiku-OS have a thread-safe alarm(2) function? In the old BeOS, Be included a bit of code I’d written that made the alarm(2) funtion work. It was relatively thread-safe as well.]
In summary, be pretty darn sure that the code base is ready for public consumption before you release it as such.
[BTW, I still have my original 66MHz BeBox and the replacement 133MHz BeBox. And all the CDs that were delivered. But I only have one I/O card.]