Indeed UI testing is slow

.. compared to non-UI, or so-called headless, testing.

That was one of the major reasons that the testability framework was added to the Dynamics NAV platform back in 2009 by MS. It replaced its predecessor, the NAV Test Framework (NTF). NTF was a C# based test framework that allowed you to script test in C# against the NAV UI. Having 40+ country versions to handle, and a couple of major releases, running thousands of UI tests took simply too long to check each country-version build. At that time I learned that UI tests roughly take a factor 10 longer compared to an equivalent headless test.

I knew the results and saw the differences at a high level, but I had never, I have to confess, measured it myself. Until recently, working on my book on automated testing in Business Central. And still working on it. [emoticon:4191f5ee34e248a29fa0dbe8d975f74a]

See the test results of a simple scenario populating a lookup field through code (headless) or a TestPage (UI). The average duration of the headless test yielded 0.20 seconds vs. 1.35 seconds for the UI test. Not exactly a factor 10, but quite close to it.

Indeed you will have valid reasons to build UI tests, mimicking on a testpage what a user would do, but I hope you do consider the speed of headless tests and make a balanced spread of your tests over UI and non-UI tests. And you know, testing a process doesn't need UI interactions, in general. And hell, yeah, bear in mind that your test collateral will grow over time, and continue to grow, and thus also the total execution time of all your tests.

Leave a Reply

Your email address will not be published. Required fields are marked *