Overview
If you perused our previous testing blog, you would have read about the difference between manual and automated testing, and the pros and cons of the two.
In this blog, I’m going to discuss what the test department at The Distance actually does, and how we ensure the delivery of the highest standard of mobile applications that we can.
To Start
When an app goes through its pre-development process, a feature document is created by our Business Analyst (BA). Once a first draft has been written, it goes through a review process both internally and externally. After the feature document has been through its review process, it gets signed off by the client. This, along with a full set of app designs, ultimately becomes the blueprint that we follow to develop and test the application.
When the feature document has been completed and signed off, it is delivered through to test. This document is then utilised as the foundation from which we create test scripts.
At The Distance, we use a programme called PractiTest as our test management tool; it grants us the ability to create and maintain our test scripts. Once written, the scripts are reviewed usually by another member of the test department and our BA. This is to ensure that they cover everything specified in the requirements.
Testing Process
Once we have created test scripts, we have our basis to test the application.
During development, the app goes through multiple development stages. These stages are what we use for our sprint plans. Within our sprint plans, we book in test cycles. These cycles usually follow the below plan but can change depending on the project:
Initial Release
- Operations will schedule in testing when the iteration is ready for review.
- The test department is provided with all requirements developed in the release.
- A Test Analyst (TA) will run any test scripts relevant to the iteration release.
- Any test scripts that fail will be raised as bugs and given an appropriate priority.
- The Creative Director will also review build UI and pass any bugs to the test department to raise.
- Before the application is released to the client, all bugs that have been raised are fixed and retested to ensure the application’s contents are working.
Subsequent Releases and Full End-to-End
- As well as the points stated in the Initial Release above, a TA will re-run any test scripts from previous iterations and append any and all new tests based on new or altered requirements.
Further Iterations
- If applicable, further cycles can be completed depending on the complexity of app. In many cases, multiple iterations are required before the full application is complete.
User Acceptance Testing (UAT) Release
- The application is released to the client as a whole. The client is then given time to review the app in full, and raise any bugs related to the release.
- Any UAT bugs raised on Freshdesk (customer support software) by the client will be triaged by the PM. The status of each ticket will, at this point, be changed from open to pending and added to the test department’s queue to raise as normal.
- Any outstanding bugs raised on the previous iterations to be tested.
Final Smoke Test
- Once all UAT (user acceptance testing) bugs have been developed and passed testing, a final smoke test is completed.
- Once full test is completed bugs will be raised and fixed, these fixes will then be re-tested.
- Client will be provided with a Full test report for them to review all tests completed and any outstanding Issues (Dependant on Acceptance Criteria)
Test Cycles
Our test cycles are run depending on the requirements that have been developed in the current sprint. When a sprint has been developed, the test department is informed of what has been developed. Based upon this, we set up test sprints that include all the test scripts created before development.
During these cycles, any and all bugs that are found are placed back into development to be fixed and are retested before any releases are sent to the client.
All bugs raised are given a priority. These are:
- P1 – Critical
- P2 – Major
- P3 – Standard
- P4 – Minor
This is to ensure that the there is no severe issues within the application.
Throughout the development process, when a sprint has been completed, a test cycle will run that includes any and all previous requirements and any and all new requirements. This ensures that there are no regression issues that have been come back from previous builds.
Exploratory Testing
Once all test cycles have been completed, an exploratory cycle commences. This is only done when we have ensured that the application meets the requirements. This step allows us to make use of our testers’ collective knowledge of mobile apps and the experience that we have gained of the workings of the application. We make sure that the app looks and feels how we would expect. The extra step ensures that users have a better experience when utilising the app. This cycle does not normally result in many bugs, but it can identify some improvements that may possibly be implemented. A design review is completed at the end of every cycle to ensure that each section is built as per the designs.
After Testing
When an application has gone through the test process, it gets a final release to the client. If applicable, the application is released to any and all relevant stores and the client is provided with a document that contains a full breakdown of the testing that has been completed.
Article by Ben Cossins, Senior Test Analyst at The Distance