More frequent releases

Development
1

A while back we had a discussion on why so many people are using Nightlies - and one of the main reasons is because the gap between stable releases is quite long and users want newer features. Therefore, the prospect of having more frequent releases was raised in that discussion.

The Promotion Team has been going through some of the comments on the Reddit post talking about the RISC-V port, and one of the concerns raised in the comments was that the gap in between releases was too long.

I think we should have a serious discussion about more frequent releases, so to kick-start this discussion, I’ll ask the following questions:

  • What release frequency could we feasibly adopt (without impacting upon the developer workload)? Bi-annual?
  • Do we need release coordinators to ensure releases are on-time and prepared for? Who currently coordinates releases?
  • Is there anything else that needs to be considered if we’re considering more frequent releases (e.g. increased use of certain resources)
3 Likes
2

The problem is always the same: we don’t have many developers and they all work on their free time. This means the availability of developers is extremely variable.

Sometimes I am busy with my paid job, and when I get back home in the evening I don’t feel like digging into even more C++ code. Sometimes the paid job is more “spent the day in meetings” and I want to spend my weekends digging into C++ problems.

If you look at the contribution stats (http://pulkomandy.tk/stats/authors.html) you will see that usually it’s just one person doing 25 to 33% of the work in a given month, and maybe 20% in a year.

In that situation it is impossible to have any predictible planning. Basically, releases happen when someone has time and motivation to make it happen: fix all the critical bugs, and then coordinate the whole thing.

If you want more frequent releases, we need to hire someone to take care of it. Otherwise it will continue to be best effort as it is now, and personally I will not commit to any fixed schedule because it would be madness for me, I would burn out from too much work if it aligns badly with my other activities.

Now let’s look more closely at the current release and why it is behind schedule (it was initially planned late february). What are we waiting on?

  • Since february I have fixed various problems in DriveSetup. These had been added to the roadmap after the beta2 release and the observation that several people had a lot of trouble properly setting up their partitions. Indeed, several bugs were found and fixed, and several user interface improvements were identified and implemented. There are two possible improvements left in the TODO list. I think no other developer looked into these. Personally I think we are now in a good enough state for a new release
  • Another thing that got a bit in the way is updating of the haiku package mirror. This is a special repository of packages used only during Haiku build time which contains fixed versions of some packages (it does not update automatically when there are changes in haikuports). Several updates were needed in particular to allow building new versions of WebKit that will ship with the next release. After several months of asking the sysadmin team to work on this, I gave up on getting them to do it and instead they gave me access to the needed servers and updated the documentation so I could do it myself. I am still annoyed that yet another critical responsibility in the project falls, again, on me. You woul all be in deep trouble if I stopped contributing for any reason. Please do not let things be like that, it is not healthy for the project and it is not healthy for me. See https://dev.haiku-os.org/ticket/16751 for more info on this.
  • There is still one bug considered important for this release, which is a regression in BTextView leading to graphical glitches mainly when renaming files in Tracker. John Scipione has been investigating this and has got not a lot of help from anyone else. He is regularly posting updates to his patch here: https://review.haiku-os.org/c/haiku/+/3642

Note that I am linking to a few ticket here, but each of these changes also fixed several other problems in the process, required digging deep into Haiku internals in various places and/or fixing packages at Haiikuports. This work just takes time.

6 months ago, none of these issues were fixed. If we had pushed a release, people would still be angry because it would have had almost no changes from the previous one, and the same issues they identified would remain. This would be disappointing for them, and embarrassing for us. I think the slower release cycle is a more comfortable place to be.

In theory we are able to push bugfixes to the beta branch as well. This also needs people working on it, and a more clear policy on what to put there and when. You can blame me from making this not happen as much as it should, by telling people they would be wasting their time doing it because a new release is due “soon” (and then months and months pass and nothing happens…).

Well, long story short: if you want any kind of progress on Haiku, we need to hire a full-time developer to actually get some work done. A few people putting in a few hours every saturday won’t do it (even if we already achieve great things in this way, it’s just not enough).

18 Likes
3

And they do much in there free time.

2 Likes
4

I think we are all aware of the huge contribution Adrien makes to the project, and are most grateful.

In his above post I agree with almost everything. The one point where I don’t entirely agree is the implied statement that releases should only be made when the new release represents significant progress since the previous one.

My view is that we should make 6 monthly releases even if the improvements are only marginal. A release is not just better software, it’s also good PR. And the PR aspect is very important.

All over the web you see mentions that progress is very slow, and that the last release was nearly eighteen months ago. The effect of this is to make people think that the project is dying on its feet. Those who follow this forum know that this isn’t true, but many people are going to form their impression from what the Internet says and never visit the forum or download the program.

I’d like to see a situation where every six months we take a successful recent nightly and make it into a release, doing as little work as possible.

Every release should be tied in with a media blitz from the Promotion Team, with the objective of getting more people to make financial contributions to Haiku, which in turn will enable us to employ developers to lift part of the burden from Adrien and his colleagues.

2 Likes
5

First of all, thanks for giving your opinion on this :slight_smile: - we really appreciate all the work you do for Haiku!

It’s a common problem not just in development, but also volunteering in general.

Yeah, I’ve noticed things stagnate because people don’t have time or motivation to follow it up; which is perfectly reasonable - but we should draw attention to the things that stagnate for too long.

Perfectly understandable - your health and safety (and that of everyone who helps out at Haiku) is paramount.

This, I think is a common issue with the Project - I find that sysadmins and Inc members often are difficult to reach and are busy so they cannot do much. This completely is understandable, but it means that things pile up and nothing gets done, and in some cases the responsibility is passed on to a few (already busy) developers. Whilst the Inc is working on getting new members, we should consider finding more sysadmins who can help out with the workload (and anywhere else this is happening, a similar process should happen).

Ah ok. I think one thing we can do is to regularly post (and encourage posting) on the forums and on the mailing lists on things that need attention/have stagnated. If something is important enough I’m more than happy to write up a short blog post for it.

Makes sense as well - I note that Haiku’s releases have never compromised on quality.

We should definitely look into this next release cycle - I’d say that you were correct to tell people to wait until next release.

Well, this links back into the whole funding thing for Haiku. Since the Inc. does not have enough funds to sustain a full-time developer, maybe any developers who are interested in doing full-time can discuss with the community and see how we can perhaps crowdfund this.

6

Yup, this was what we found out when we went through the comments.

Already on it :slight_smile: @animortis and I are working on press releases (and we’re planning to have press releases for all future releases as well as any other significant events).

7

The last release was less than a year ago. So maybe people should just stop being wrong about it then? :sweat_smile:

This time I think we are on track to do a release less than one year from the previous one. Before getting into more ambitious goals it would be nice to reach this one already (and consistently).

Effectively, it is. If you look at the activity graphs in the stats page I linked, you will see that the activity keeps getting lower and lower over the years. Again: we need more people and we need people working full-time on things.

I’d like to reach this too, but at the same time I want (and I think it is more important) to maintain a level of quality in such releases that the current nightlies don’t really guarantee (and that’s perfectly normal with our workflow). The nightly branch is a playground for developers to test things. It gets work-in-progress changes and it gets regressions. And this is not something we should change. It is the right place for developers to do that.

This results in some difficulties when making a release because we can only do so when the nightly branch is in a stable state, with no known regressions. Ideally we would maintain a separate stable branch where things would get merged after some testing, but that requires continuous work watching the changes, deciding which are stable and which are not, etc. Eventually the occasional difficulty of getting a release out (every year or maybe a bit more often) is still a lot less than the workload needed to maintain such a branch.

Honestly I don’t reallt care for the people who are not even bothered to try Haiku because of something they overheard on forums. As a developer, I have already a ton of work to do to fulfill my own needs, and after that, the needs of the existing users (as an unpaid developer, this is the way my priorities go). And so it isn’t so much of a problem for me that the userbase remains a little small at the moment: the existing users sure can keep us all busy with bugreports and improvement ideas, and do so in a quite positive way, while being respectful of the hard work everyone is putting into this.

So, let’s start with succeeding in making yearly releases. This is a goal we have not achieved yet for the betas (beta1 and beta2 were 18 months apart indeed). I think we are on the right track for a year between beta2 and beta3, or maybe a bit more, let’s say 13 months. We are still solving our problems, getting better at planning our roadmap, and eventually we’ll manage to make the release cycle even shorter. But at this point it seems strange to say “yearly releases are not enough” when the actual complaint is “18 months between releases is too long” (and no one is going to argue against that).

7 Likes
8

Before we go further, who out there thinks they might be interested in developing full-time for Haiku?

9

Some one who get money for it, have a job using haiku.

10

For what it is worth Haiku, Inc has enough money to pay a developer a decent full time wage for at least a year, maybe more. Unfortunately this is an incredibly hot market right now for developers and in some sense Haiku is competing with all the other potential employers who can offer much more pay and benefits than we can reasonably offer. Most of the current and past Haiku developers who have experience working on the OS are older, with established careers and sometimes families, and it just is not realistic to come work for Haiku. I know that is the case for PulkoMandy and myself and I suspect people like Axel, Ingo, Stephan, etc.

We also have to be a bit picky and make sure we hire someone who can do the work (again, even normal companies have a hard time finding people who aren’t frauds trying to earn a high developer salary.) I personally take very seriously managing the donated money and don’t want to find us in a situation where we are paying someone for shoddy work.

Though our issue is more about “no applicants” than “bad applicants”. We could have the promotion team put out more of an official call but then I’m afraid if we put out that call we will get a flood of bad applicants, and then extensive volunteer time will be spent weeding those out. It feels like kind of a damned if you do, damned if you don’t situation. Haiku, Inc and the developers are already really stretched thin so adding more doesn’t help. But maybe we just have to try and push through and hope we can find someone decent.

We had two potential full time contractors who both ended up not being able or willing to take on the work. I’ve been trying to find someone, but without also essentially making this my full time job to hire another full time person, it just doesn’t seem to be happening. My current strategy is to try to wrap up my Haiku, Inc stuff as well as I can (I’m already quite late on the financial report, yet again), and make systems to make those easier to maintain, then get myself back into a regular cadence of working on Haiku as a developer. While that might help some it still won’t solve the general issue.

But maybe let this message be a bit of a “recruiting” message: please speak up if you are already in these forums and are willing to be a contractor for Haiku, Inc and earn $70,000 a year, or around $35 an hour. Unfortunately we can’t offer benefits and this would have to be a contracting position, versus full time, at least in the sense of how things work in the US, in other words, all taxes are owed by the individual. Ideally you already know how things work in Haiku and have already made some patches or something.

4 Likes
11

It would be nice if this information was on the Haiku inc website, since until now the official position was “very uncompetitve rate because we can’t do more” (IIRC it was somewhere around $12.5 an hour back in 2014, which is indeed a lot lower than market rate). Now it may be a lot more acceptable for some of the developers to accept an offer like that.

Another thing to consider is offering follow-up contract to our GSoC students if their summer projects go well. They are quite likely to be available for short term contracts during the summer, or even part time during the year, probably more than the people who already have a full time job.

3 Likes
12

Yeah, that’s a fair criticism. The idea of paying a more reasonable rate like that has only been in our minds for about 6 months, and during that time we had those prospects I mentioned so we were not really trying to “recruit” yet. But now we should put out the call.

I would also like to point out the Zig Software Foundation which has only existed for around a year yet they have already built what seems a reasonable and sustainable model. Zig is a newer programming language, which I like a lot and plan to port to Haiku. Anyhow they use GitHub Sponsors quite a bit, and I’ve started building a page there for Haiku. If we can find someone or a few people willing to work for Haiku, I think the community and the internet in general would step up to provide more funding to keep things going.

3 Likes
13

Maybe you could specify a few job descriptions? Like: work on “userland” stuff, work on kernel/drivers, etc… Also would you be willing to consider hiring half-time (something like 4 hours per day, 5 days a week, for half the pay of course :)?

1 Like
14

Hi,
I already got the stage 1 compiler for zig to work on haiku a couple of months ago, and was planning to work a bit more on that in the coming months or so, I can check if i still have that patch somewhere.

3 Likes
15

Yeah, that is a good idea. Right now it is very vague. There are plenty of things to work on, but likely the general theme for a hired developer would be to work on the things volunteers cannot find either the time or motivation to work on. Though it should still be pretty fun.

Yes, definitely. In fact that might be a wise way to start in general to make sure the arrangement is working for all parties involved.

4 Likes
16

Awesome, I wondered if someone had worked on it but did not see anything in HaikuPorts. Also the “bring your own OS” stuff they have done recently should make it easier to get the Zig standard library working on Haiku as well. Eventually I would like it if Haiku GUI apps could be written in Zig but like many things I think that will need a C wrapper around the C++ as glue code and probably a bunch of other work. Though I bet something like a Tracker add-on would not be too hard to get working quickly once Zig is ported.

2 Likes
17

As one of the people working at haikuports, I agree that regular updates could be a better thing.
Some work atm is in a “waiting” stage for the next beta because atm it can’t be merged, some of those things could bring the work at haikuports to another level? :wink: (Just my 2 cents)

18

@leavengood A more formal job description is definitely needed. Some of the potential candidates are already asking for more details. Therefore, would it be possible to announce the hiring via the website, preferably sooner than later?

3 Likes
19

[…] hiring half-time […]
Yes, definitely. In fact that might be a wise way to start in general to make sure the arrangement is working for all parties involved.

That’s good news. In that case, it would be great to have some info page, or contact info, cause i’m considering applying, as long as i can meet requirements.

3 Likes
20

Whoops, you are quite right! Mea culpa and apologies.

If we can get a regular annual release, then that would be a great start. However, I stand by my suggestion that a six monthly release, even if it contained nothing much that was new, would give a very good impression.

People, as you and your fellow developers know, lead busy lives. They don’t necessarily have time to follow up things that they hear to see if there is any truth in them. We need to establish a common view that Haiku is alive and active. That way, busy people are more likely to give Haiku a try.

…

2 Likes