As you are writing your software, you need to ensure everything is working properly. You can best achieve your goal through the use of automated and manual tests. There are four main categories of tests are:
- Unit
- Integration
- Functional (AKA Systems Tests)
- Acceptance
Later in this article, we will discuss what each of these tests is, what is their role, and what it takes to automate these tests.
When do you create your tests?
There are four different times you can write automated tests.
1. As you are writing your software:
This is a very popular time to write the tests. This method ensures you have tests written throughout the software, and when the project is done, you have a good number of tests written.
2. After the software is written and you are performing validations:
This is not a great time to write your tests since you now have to go back through your entire software project and determine what each function is supposed to do. This can be time-consuming, and can easily be de-prioritized. So it may never get done.
3. When you uncover bugs in your software:
This is a good time to write an automated test. After you have fixed a bug, you can write a test to watch for that bug in the future. That way if you ever manage to re-introduce that bug into your software, you will catch it right away. And before your end users encounter the bug.
4. Before you begin writing your software:
If you have ever heard of Test Driven Development (TDD) that is what we are talking about now. You start by coming up with a basic design for your software. You then create test cases that cover the most basic functions of your software. Then you write the minimum amount of code needed to get your tests to pass. The nice thing about this model is you always have all of your automated tests written and working, and you don’t have to work backward to create the tests after you are finished with the project.
What is Unit testing?
Unit testing is testing the functionality of your functions and classes. Unit tests are the fastest tests to run and can be parallelized the easiest. For this reason, it makes sense to create the most unit tests. Unit testing also gives you the fastest feedback of any test since it is the first kind of test you can perform in your software development pipeline.
Creating unit tests is easiest when you use a testing framework. In Python, I like to use the PyTest framework. Regardless of the framework you use, unit tests consist of two parts. In the first part, you call your function and pass in some test input. In the second part, you make an assertion about what the output should be given the test input you passed into the function.
When creating your unit tests, be sure to think of the most common scenarios in which your function or class might be used, then create applicable unit tests to cover those scenarios. Unit tests are great for uncovering problems with the basic functionality of the code you write.
For information on how to write unit tests in Python, see our article on Unit testing in Python.
What is Integration Testing?
Integration testing is when you test your code and make sure it integrates with everything else. This could be an entire production-like environment. Or a small environment where you spin up the other services that need to interact with your software to make sure everything plays nicely.
While unit tests will uncover all the various scenarios you can think of. The integration testing might uncover scenarios you did not think of. For example.
Let’s assume you write a web service where people post word documents, and your web service returns a PDF file. You wrote your service, did tons of testing and it always worked when you posted a word document. You ran all your unit tests and everything passed.
Next, you put your app through the unit testing phase where you have your web front-end passing files into your new PDF conversion service. When people post word documents through the web interface, everything works fine. But next someone comes along and posts a jpg file. Your service was not set up to accept jpg files. And you maybe hadn’t thought that someone might post a jpg file. Your service crashes and doesn’t gracefully recover because of the unexpected input. This is why integration testing is important. You can catch these unexpected problems with your integration tests.
Now that you have identified a problem, you can go back to your code and fix it to either handle jpg inputs and convert them to PDF. Or you can configure your code to fail more gracefully when it encounters unexpected input. After you have your code fixed you can run it through the integration tests again and make sure there is nothing else you have missed.
What is Functional Testing?
Functional testing is when you check to make sure the software you wrote does what you think it is supposed to do. If it is a web application you might script this out using selenium. You are checking things like is there a button in the right place. And when I click this button does it do what I was expecting.
In this phase, you are checking to see if you can check all the boxes for your technical requirements. And that all the expected functionality works.
What is Acceptance Testing?
Acceptance testing is the last stage of testing before you roll out your code. In acceptance testing or sometimes called User Acceptance testing (UAT) your users/customers or product owners are testing your code to make sure it works the way they were expecting
Functional testing is really about the developers making sure it worked as they expected. And acceptance testing is making sure it works as the customer expects it to. In this stage, you might find it functions to the original specifications. But there is a design flaw or oversight that makes it not work at all.
Or you might find there was a technical requirement that was missed and it made its way all the way through to this stage. Finally, the users/customers might uncover a bug that you don’t want to push to production during this stage. UAT is not easily automatable because it is your end users testing. Generally, you can’t expect that your end users are savvy enough to automate this testing phase. And they may not even want to.
If you can automate this step, then you are really just combining it with the functional testing.
Summary
Today we discussed the four major kinds of testing that you might perform during software development. All four kinds of tests are important and will help to ensure you don’t push buggy code that misses your goals all the way out to your customers. The first 3 categories can and should be automated. Whereas the 4th kind of testing really involves a human to sign off on it.
