When working on getting the standard tests running on our solution the standard Test Tool was not making our live as easy as it could. Wanting to rerun only successful or failing tests, this was near to impossible. Manually deselecting individual tests turning of the Run field, was, euphemistically said, not the most meaningful thing to do. So, already a couple of years ago we decided to add some actions to the test tool allowing us to (1) select the Run field for (a) all tests (b) only failing tests and thus disabling all others (c) non-failing tests, thus disabling failing tests and next to that (2) deselect the Run field for all tests, thus disabling all tests. It was practically implemented in the C/SIDE objects and not very hooligan proof, I have to admit. When writing my book on test automation last year, I decided to rework this feature into an extension using this refactor project at the same time as a next exercise for myself on writing automated tests. The code has ever since then been available on GitHub and has been called Test-Tool-Extension. As it was never advertised outside of the book, so far, probably not a lot of people have been making use of it.
Test Tool Extension features
The main purpose of this project was to share with you its simple and powerful features, allowing you, as already noted above, to select and deselect specific tests. Thus enabling you to only run relevant test for your current task. For this four actions have are added to the Actions ribbon tab:
- Select on All
- Deselect on All
- Select on Failures
- Select on Non Failures
And all of these actions respect the active filter on the lines of the Test Tool. With the first action you make sure all tests (within the filter settings) are enabled. The second is its complement and does disable all tests within the filter. The third action will enable only the failing tests, where the fourth action will makes sure all others, being successful and skipped test, are only enabled. Of course when no filters are set all actions will apply to all tests in the active test suite.
The following screenshots show these actions in both Windows and Web Client:
Next to these four actions a very simple, but o, so, useful other addition has been made to the Test Tool: the Ctrl+F9 short cut to the Run Selected action (which unfortunately will not work on the Web Client). If you are working a lot on tests running them from the Test Tool, you probably will not often use the Run action, but rather Run Selected, as the latter will not throw up a dialog asking you what you want to run, but instead will immediately run the select test function or test codeunit.
Enough about the features. Let’s discuss the two other purposes of this project:
- automated test examples
- ATDD test script examples
Automated test examples
In the folder ./Solution/test/Test Codeunits/ you will find 4 test codeunits containing all together 56 (unit) tests that do test all the features. Being an advocate of the ATDD test scripting, all tests have been designed and structured in this manner. Use them as a next collection of examples (next to my book) of how I think test codeunits/functions should be organized. Readable at high level and making use of reusuable helper functions at local (specific) and library (generic) level. The 3 library codeunits can be found in ./Solution/test/Test Libraries/ and containing all kinds of reusable relevant creator, getter, setter, and verifier functions.
ATDD test script examples
With my book I introduced a 4-steps recipe helping me to effectively code new tests, given ATDD scripted tests:
- Create a test codeunit
- with name based on [FEATURE] tag
- Embed the customer wish into a test function
- with name based upon [SCENARIO] tag
- Write your test story
- based upon [GIVEN], [WHEN], and [THEN] tags
- Construct your real code
The book also discusses how to practice ATDD test scripting and do this efficiently use a Excel based setup, that I just started developing around that time. Both the 4-steps recipe and the Excel sheet having repeatable were the forerunner to what Jan Hoek and I started to conceive: ATDD.TestScriptor. For those who have never heard of it, get yourself update with this Areopa webinar recording of July 16, 2019.
All test in Test-Tool-Extension project have actually been conceived using ATDD test scripts that were converted to basic test codeunits using the ATDD.TestScriptor. Here’s on example of the basic script in PowerShell making use of the ATDD.TestScriptor syntax, followed by its AL counter part created with the ATDD-TestScriptor cmdlet ConvertTo-ALTestCodeunit:
Feature 'Enabling Actions on Failures Unfiltered' {
Scenario 1 'Unfiltered disabled failed test functions' {
Given 'One disabled test codeunit with five disabled failed test functions'
Given 'One disabled test codeunit with three disabled failed test functions'
When 'Do nothing'
Then 'Select on All is enabled'
Then 'Deselect on All is disabled'
Then 'Select on Failures is enabled'
Then 'Select on Non Failures is disabled'
}
}
codeunit 92152 "Enabl. Actns. Flrs ALTT FLX"
{
SubType = Test;
trigger OnRun()
begin
// (c) fluxxus.nl - https://github.com/fluxxus-nl/Test-Tool-Extension
// [FEATURE] Enabling Actions on Failures Unfiltered
// [FEATURE] Enabling Actions on Failures Filtered
end;
[Test]
procedure UnfilteredDisabledFailedTestFunctions()
// [FEATURE] Enabling Actions on Failures Unfiltered
begin
// [SCENARIO #0001] Unfiltered disabled failed test functions
Initialize();
// [GIVEN] One disabled test codeunit with five disabled failed test functions
CreateOneDisabledTestCodeunitWithFiveDisabledFailedTestFunctions();
// [GIVEN] One disabled test codeunit with three disabled failed test functions
CreateOneDisabledTestCodeunitWithThreeDisabledFailedTestFunctions();
// [WHEN] Do nothing
DoNothing();
// [THEN] Select on All is enabled
VerifySelectOnAllIsEnabled(NoFilterOnCodeunit());
// [THEN] Deselect on All is disabled
VerifyDeselectOnAllIsDisabled(NoFilterOnCodeunit());
// [THEN] Select on Failures is enabled
VerifySelectOnFailuresIsEnabled(NoFilterOnCodeunit());
// [THEN] Select on Non Failures is disabled
VerifySelectOnNonFailuresIsDisabled(NoFilterOnCodeunit());
end;
}
Find the ATDD test script sources in the folder ./Solution/ATDD Scenarios/.
Feel free to make use of the extension and its code. Please, share any suggestions for improvements be means of issues.