So there has been discussions off-and-on about assembling a dedicated QA team for the Haiku project that could lend a hand in testing Haiku release candidates, etc… that would ensure improved quality of Haiku release to release. PulkoMandy has a desire for a QA team to be up-n-running and ready to tackle testing of release candidate(s) for Haiku B2 when its planned for release sometime next year. So this post is meant to have an active conversation about how such a QA team/process would work.
As you know, the Haiku project is very agile and doesn’t currently have a formal development process per say as the contributors work on what they feel is important and is inline with their interests and confidence level. There is a monthly report that goes out and I guess one could look at each month’s worth of development as a single sprint and maybe QA could just sample a nightly build at the end of each month and peform regression testing and report back to the Haiku development team.
Currently there are some manual test cases documented and archived in various places and would need to be collected and cataloged in a single place. I was thinking that it would be great to get an instance of TestLink hosted within Haiku’s ecosystem and migrate the (whatever existing) documented test cases and organize them in a way that makes sense for ease of maintenance, execution, etc…
So who else out there within the Haiku community has a QA background and/or is interested in such an endeavour?
Did anything come out of this? I’m currently looking into setting up Kiwi TCMS for Koder testing to avoid having embarrassing bugs in releases.
I’d be happy to provide hosting and advice for Haiku testing efforts.
There doesn’t seem to be a lot of interest in this endeavor and I have been keeping a low profile myself as of late (for personal reasons). PulkoMandy definitely would like a testing resource that is more formal and ready to go that could support the R1B2 release at the end of this year. There are obviously some QA minded folks (like myself) that exist in the Haiku community that may want to contribute to such a thing and I guess some discussions need to take place on how to proceed.
I’m probably one of those who will be on the QA team. What is kiwi and what should one know about getting into it? I see it integrates with just about every bug tracker except Trac.
My soul is developer (applications, not system), but working in software development industry I newer underestimate the importance of QA. Usually a self-respecting software company spends at least as much time and money in QA as in development.
Opensource software always suffered from little in-house testing, but this is counterbalanced by extensive user testing. If there are 2 persons willing to form QA team, and (possible) testing platform, PLEASE, go ahead! Don’t be discouraged by little feedback from developers. Believe me, the developer’s problems and priorities are very much different from tester’s ones.
senior tester (composing test plans, writing test cases, etc.)
tester (doing test runs)
user (view)
I’ll experiment later to perhaps make this a public system. Currently registration is required to see anything.
I don’t have any experience with this system, so not everything might be configured properly (permissions especially).
If you want to join, register there and send me a DM or catch me on IRC to give you permissions (depending on what you want to do).
As for writing the test cases, I think a following process will work well:
Go through User Guide for basic use-cases.
Go through tickets on Trac (active and closed) and create test cases from that.
The definition of a product is an app or preflet for now.
Why did I go with Kiwi TCMS? It looks the best out of open-source and maintained options. The only other one is TestLink I think but it doesn’t look very appealing and intuitive.
Integration with bug trackers is important if we wanted to file bugs automatically from the TCMS. I don’t think that’s desirable, manual filing should work fine. It is still possible to link bugs to test cases/runs using just it’s number (Kiwi allows adding template URLs like https://dev.haiku-os.org/ticket/%s).
I would like to explore the tool maybe over the weekend when I get time. Maybe setup a sample test project (or whatever the equivalent is) and play around to see how things flow.
Hi QA team,
It would be great if your actions and work was documented at https://www.haiku-os.org/community/getting-involved/testing/, rather than a forum thread
If you could submit an update for that page, and keep it in line with what you’re working on, how to reach you, etc, that would be awesome.
asking about PROs … related to this important development process.
I would reply for calling as a tester I could be involved especially AFTER this event
… where my development environment would be set up for good.
I would like to use the Haiku team and students developed Haiku tool that convert source code to Haiku and clang format.
I could test builds and application in it - but someone should create
—+ some scenarios what to test - basically based on existing tickets and
new features built by reviewed code.
—+ instructions how to report
—+ setup automatical checks - scripts - where applicable – to gather data
and this way useful information to analyse…
Sounds fun!