Software Testing: Regression Testing
Years ago, when I was still a university student looking for entry-level jobs, I noticed many companies were looking for juniors to work on “regression tests”. As I had studied lots of mathematics, all I could think of was regression analysis. When it was companies doing products related to mathematics or signal processing, it still made sense to me, but there were so many companies in other fields, too, looking for juniors to work on regression tests. It wasn’t until years later I learned what regression testing in software means. In this blog post I give an introduction to what regression testing is and why it is important to write regression tests.
Regression testing can also be called non-regression testing.
Let’s assume one is working on a new feature in a software project. Furthermore, for the sake of an example, let’s say we don’t have any automated testing at all. We are left with the option to do manual testing or not do any testing at all. If we don’t do any testing, how do we know if we completed the feature successfully? Usually one will always do at least manual testing.
The type of manual testing that will be done depends heavily on what kind of product we are developing, what the feature is and many more things like the experience and the confidence of the developer. For example, when developing a web user interface one might click around the interface in a web browser to see how it behaves. In contrast, when developing a web backend API, testing might be concluded from the command line using curl or similar.
The developer carries on with their manual testing until they are confident the feature is completed to the quality they expect and then the feature is closed. Now, if this would be the last feature ever developed for the product, then this might actually be everything that is needed. However, in practice, we are always working on a moving target. This means that the test results from our manual testing deprecate due to the additional functionality developed over time. The feature worked when you ran the tests yesterday, but does the feature still work today?
The solution here is to introduce automated regression tests. Preferably, each completed feature should be accompanied by regression tests which show if the feature is still working. Now, as product development moves forward, the regression tests for old features can be constantly run, and if a test starts failing, developers know to react on the breaking changes. This not only saves time in manual testing as your product becomes more feature rich, but you also gain confidence in the quality of your product and developer work becomes easier as bugs are caught earlier in development cycles.
Is your feature complete before you have written regression tests for it?
What should the automated regression tests be like? Preferably, they should test the exact same things as the manual testing would. In practice, the regression test for the web backend API should run something similar to curl and the regression test for the web UI should click around in a browser. This might sound like a pipe dream, but the matter of fact is that there are very sophisticated testing frameworks which make this goal attainable. For example, postman for web backend API testing and Robot Framework for web UI testing.
Summary
In this blog post we discussed regression testing. Regression testing should aim to achieve similar testing coverage as manual testing would achieve if no automated testing would be available. Automated regression testing saves time as it replaces repeated manual testing. Furthermore, it provides confidence in the quality of the product and helps developers as bugs are found earlier in the development cycle.
This article is part of Omoroi’s blog series on Software Testing. You can reach us at blog@omoroi.fi or discuss in this LinkedIn thread.
