Since this change has been merged, do you think it’s a good time to start working on the BDD conversion? I believe beginning with a proof of concept in the codebase would make it more intuitive to discuss potential improvements and decide how to move forward.
Yeah, will definitely create new Test Classes for which we don’t have existing Unit Tests.
I’m a little digging from the topic, but how about writing tests in Python with Haiku Python Binding instead of writing C++ test code to try a new style of test code?
Although C++ standalone tests have a short execution time, the build time of the test is long and the maintenance cost of the test tends to increase.
If you aim to document the operation of the API by increasing the number of BDD-style test cases, it is an advantage to be able to write a lot of test code quickly.
Of course, I understand that there are aspects that cannot be tested without C++, but I think it would be faster to detect the destruction of ABI by performing repeated tests without rebuilding the same Python Binding.
Maybe I miss something in the big picture, but AFAIK, BBD make sense when someone, who don’t code or even know how to code, defines the expected behavior of features in multiple scenarii, writting down into some clear, concise format.
The unit tests are, then, here to allow to run then and build a report of the feature(s) which behave as expected in different scenarii, or not.
Without the first part, I failed to see the benefit to write unit tests according BDD style, as anyone who will want to see which feature doesn’t behave as expected will have to dive down in the unit tests to discover this expected behavior, which the unit test code by itself is already expressing, in code language. And same to specify a new feature expected behavior: if you have to write it along the test code itself, it’s actually specifiy the behavior twice : first by the decorator/macro/comment/naming convention of tests, second time by the test coding itself.
What’s the objective pursued here? Do you plan to specify in Gherkin language files the expected behavior?
Not specifically in Gherkin language, but we are considering something similar. You can look it up here.
What I mean is the expected behavior is supposed to be specified outside of the test code, which allow to specify not yet implemented feature, and detect the behavior of an implemented feature is not yet covered by any test yet.
On the test code, what is needed is to define which feature and scenario and given a test is checking, so that a report can be built, listing features which are not yet covered by test, which have tests failing to match the expected behavior, which succeed.
If you put the expectation specification in the test code, you can’t specify any expectation without actually starting the write the test code, which sounds to me the usual TDD, not BDD.
My experience with unit tests is that, without proper structure, it is often quite hard to understand what a test was meant to test.
As I understand it, BDD is one way to formalize that, it gives a structure with an initial setup (“given”), an event or action (“when”), and an expected result (“then”). This structure forces the developer writing the test to be explicit about what they are testing, in a well-defined format.
But last time I experimented with it, the given/when/then clauses where indeed implemented as code, not just part of a comment or a test method name. Otherwise there’s a risk that the test contents doesn’t match its name.
If you think the behavior can be expressed entirely with code, you don’t need unit tests: the implementation code could be read directly to understand the behavior. The difficult part is keeping track of the intended behavior and separate it from the bugs as well as possible emergent behavior (things that happen to work, but not by intentional design, and that in theory could be changed if needed). But you can have bugs in your tests as well as “emergent” testing (you often end up testing more behavior than you want to, and then unrelated changes can break a test).
I am late to the conversation as I have been away for some days. Thanks for raising this as the test style is something I have wondered about in the past.
I tend to demarcate “given when then” by commenting around the code under test like this in my Haiku tests;
BString email("wetaexample.com");
// ----------------------
bool result = ValidationUtils::IsValidEmail(email);
// ----------------------
CPPUNIT_ASSERT_EQUAL_MESSAGE("!Email invalid - no @", false, result);
…but it looks like I have also done this too 
…
// GIVEN
BLanguage languageFrench("fr_FR");
BLanguage languageEnglish("en_US");
BString name;
// WHEN
languageFrench.GetName(name, &languageEnglish);
// get the name of French in English
// THEN
CPPUNIT_ASSERT_EQUAL(BString("French (France)"), name);
However it’s demarcated, I do find the “given when then” approach very helpful even if there is the occasional situation when it does not fit. I’m happy to be more consistent with the “given when then” commenting style and to use a consistent + descriptive method naming style such as GivenInvalidEmail_WhenValidating_ThenFalseResult.
I’m also a bit concerned about introducing enforcement of “given when then” in CPPUnit tests via macros because it’s more complication and it will be harder to look at compile errors and map them back through the macros to the source code. I would be more keen on adding a style of writing tests to the coding guidelines and then pick it up in review.
I’ve used tools like Cucumber/Gerkin and Concordion in multiple projects; essentially abstracting the test logic into a natural language such as English. I’ve found these testing techniques really useful when they fit but as something alongside other types of testing such as unit and narrow-integration tests.
It is technically impossible. Making some new DSL for describing expected behavior do not make it not a part of test code.
Making some new DSL for describing expected behavior do not make it not a part of test code.
Then how do you specify an expected beharior of a feature UNTIL some code and code testing that code are written??
I’m also on that line.
Making a bit more BDD test coding policy to standardize expected comments given//when/then is a good idea, without complixify too much.
In natural human language like BeBook.
Hii Everyone,
I have written unit tests for Stopwatch. Can anyone explain to me how to test these individually?
You build the unit tests with;
jam -q unittests
Once you have it built, you can run them with;
tests/haiku/x86_64/haiku/unittests/UnitTester
Try with -h to get help on running specific tests.
You can only run them as suites - in this case SupportKit.