Roll call for prospective Haiku QA testers!


#1

Hello!

So there has been discussions off-and-on about assembling a dedicated QA team for the Haiku project that could lend a hand in testing Haiku release candidates, etc… that would ensure improved quality of Haiku release to release. PulkoMandy has a desire for a QA team to be up-n-running and ready to tackle testing of release candidate(s) for Haiku B2 when its planned for release sometime next year. So this post is meant to have an active conversation about how such a QA team/process would work.

As you know, the Haiku project is very agile and doesn’t currently have a formal development process per say as the contributors work on what they feel is important and is inline with their interests and confidence level. There is a monthly report that goes out and I guess one could look at each month’s worth of development as a single sprint and maybe QA could just sample a nightly build at the end of each month and peform regression testing and report back to the Haiku development team.

Currently there are some manual test cases documented and archived in various places and would need to be collected and cataloged in a single place. I was thinking that it would be great to get an instance of TestLink hosted within Haiku’s ecosystem and migrate the (whatever existing) documented test cases and organize them in a way that makes sense for ease of maintenance, execution, etc…

So who else out there within the Haiku community has a QA background and/or is interested in such an endeavour?

Just an FYI… there is a haiku-qa IRC channel (I hangout there alone now) as well as a mailing list (https://www.freelists.org/archive/haiku-qa/).


#2

Testing Haiku on real hardware? Why not? :slight_smile: Sounds fun!


#3

Count me in.


#4

I’d be down. I already have an SSD in my laptop dedicated to Beta 1.


#5

Did anything come out of this? I’m currently looking into setting up Kiwi TCMS for Koder testing to avoid having embarrassing bugs in releases.
I’d be happy to provide hosting and advice for Haiku testing efforts.


#6

If you could use a usability/consistency/user experience QA tester (all non coding) I’d be interested.


#7

There doesn’t seem to be a lot of interest in this endeavor and I have been keeping a low profile myself as of late (for personal reasons). PulkoMandy definitely would like a testing resource that is more formal and ready to go that could support the R1B2 release at the end of this year. There are obviously some QA minded folks (like myself) that exist in the Haiku community that may want to contribute to such a thing and I guess some discussions need to take place on how to proceed.


#8

I’m probably one of those who will be on the QA team. What is kiwi and what should one know about getting into it? I see it integrates with just about every bug tracker except Trac.


#9

My soul is developer (applications, not system), but working in software development industry I newer underestimate the importance of QA. Usually a self-respecting software company spends at least as much time and money in QA as in development.

Opensource software always suffered from little in-house testing, but this is counterbalanced by extensive user testing. If there are 2 persons willing to form QA team, and (possible) testing platform, PLEASE, go ahead! Don’t be discouraged by little feedback from developers. Believe me, the developer’s problems and priorities are very much different from tester’s ones.


#10

We are waiting on input from the qa team before we can give any feedback!


#11

I have set-up an instance at https://qa.kacperkasper.pl/.

Currently there are 4 roles:

  • admin (me)
  • senior tester (composing test plans, writing test cases, etc.)
  • tester (doing test runs)
  • user (view)

I’ll experiment later to perhaps make this a public system. Currently registration is required to see anything.

I don’t have any experience with this system, so not everything might be configured properly (permissions especially).

If you want to join, register there and send me a DM or catch me on IRC to give you permissions (depending on what you want to do).

As for writing the test cases, I think a following process will work well:

  1. Go through User Guide for basic use-cases.
  2. Go through tickets on Trac (active and closed) and create test cases from that.

The definition of a product is an app or preflet for now.

Why did I go with Kiwi TCMS? It looks the best out of open-source and maintained options. The only other one is TestLink I think but it doesn’t look very appealing and intuitive.

Integration with bug trackers is important if we wanted to file bugs automatically from the TCMS. I don’t think that’s desirable, manual filing should work fine. It is still possible to link bugs to test cases/runs using just it’s number (Kiwi allows adding template URLs like https://dev.haiku-os.org/ticket/%s).

I guess we’ll come up with the rest as we go.

I’ll be available on Freenode #haiku-qa too.


#12

I would like to explore the tool maybe over the weekend when I get time. Maybe setup a sample test project (or whatever the equivalent is) and play around to see how things flow.


#13

Just completed a first sweep or Haiku and found some 50 QA issues. I’ll work out the notes for publication.


#14

I’m guessing you have access to the bug tracker?


#15

Yes, but I don’t know what’s the best place for them, Haiku’s bug tracker or KapiX’ Kiwi TCMS.


#16

In the short term, Haiku Trac would be best. We still need to check out kiwi to ascertain it’s viability as a QA platform for Haiku.


#17

Hi,

I can Help in Testing RC & in QA …what to do ??
I can run on both x32-X64

PCdokter


#18

Hi QA team,
It would be great if your actions and work was documented at https://www.haiku-os.org/community/getting-involved/testing/, rather than a forum thread :slight_smile:
If you could submit an update for that page, and keep it in line with what you’re working on, how to reach you, etc, that would be awesome.