Are you a total beginner to testing mobile apps? Same! I’m Matt, the newest member of the test team at The Distance, and I’ll be going over a few basic concepts.
What is the purpose of testing an app?
Broadly speaking, the primary purpose of testing is to provide assurance in two regards. Firstly, it is important to ensure that the application under test performs as expected, under normal operating conditions. An email client that sent emails to the wrong address would be no use at all, and this is exactly what we are looking to catch here. Typically in a commercial environment, a set of requirements is produced, and all expected behaviour documented – the tester will work closely with this document, in order to discover any discrepancies in behaviour.
We then need to consider what happens when an application is taken out of its comfort zone. Essentially, the tester is given free reign to try and spoil the fun for everyone else by breaking the app. What happens to our email client if the user disables their internet connection part way through sending an email, or if we paste 100000 emojis into the log-in form? Quite often, this testing skirts around the edge of what is actually specified in the designs, meaning the testing will potentially inform additions or modifications to the current requirements, by identifying edge cases or deficiencies in the current design.
Generally, we will keep finding issues with an application until it has passed through the testing process a few times – this is fine! Software systems are increasingly complicated, and nigh impossible to get 100% correct the first time. In a nutshell, this is the reason that test departments exist at all – we want to catch these issues, or bugs, before we release an app, so that our users won’t see them.
What is a bug?
In Sep 1945, Harvard University operators removed a moth from the inside of a primitive computer called the Mark II Aiken Relay Calculator. They taped it to that day’s log book with the entry: “First actual case of bug being found.” And thus the terms “Bug” and “Debugging” were coined in software development.
Fortunately, our devices are somewhat more moth-proof these days. The term bug refers to problems we find in an app, which would normally be one of the following:
- Some functionality contained in the requirements is missing, or not correctly implemented in the app.
- The app has additional screens/functionality that are not required by the designs.
- The app is not meeting our general expectations for how an app should perform. This could include things like crashes or UI elements not displaying correctly.
How much testing is enough?
Testing is fairly unique in the software development process, in that there is no distinct point where the testing of an app is “complete”. Modern software is incredibly complex, and the number of possible input variations mean it is not possible for us to test exhaustively – that is, test every single possible combination of inputs to the system.
If we compare this to say, design or development – both processes have clear deliverables, that when complete provide an obvious end to the task. This is exactly what we need to decide when testing – we call it our “Exit Criteria”.
Typically, this would be decided on a context appropriate basis – the exit criteria of a banking app would be much more stringent than a noughts and crosses game, for example. We might use a measure such as one of the following:
- A full system test has occurred, and no major bugs have been found.
- All identified test cases have been executed.
- Testing budget has expired.
How does a tester fit into the process as a whole?
So far, we have talked about the primary responsibility of a test analyst – identifying and documenting bugs, to be given back to the developers. However, the tester also has another important role to play, as “product expert”.
The tester typically will work very closely with the requirements, and also will be the primary individual viewing the system as an actual user. Comparatively, a developer will tend to be focused on small sections of implementation of the code at a time, and thus it is more difficult to see the “big picture” of the full system. Therefore, the tester is ideally placed to cast a critical eye over the user experience of the application – this can then drive changes in development, and can really add value to a finished product. The test perspective can also be very useful to project managers, to give top level information on the quality and completeness of an app.
TL;DR
- We test apps in order to identify and fix problems with apps, before they get to the user.
- These problems are called bugs.
- We need to define a sensible point to know when to stop testing.
- A good tester can also use their perspective in order to add value to a product.
Thanks for those of you that stuck around this far. I have only briefly touched on a few concepts today, so I hope we can go into some more topics another time. Class dismissed!
Article by Matt Chambers, Test Analyst at The Distance