According to British Standard 7925 -1:
"Testing is supposed to demonstrate that the software does what it is supposed to do as well as demonstrating that software does not do what it is not supposed to do."
It is worth noting that testing can only show the presence of faults, it cannot prove that the software is 100% correct and it cannot prove that there are no faults. Risk will reduce if subsequent re-work and re-testing takes place.
During any System Development Life Cycle (SDLC) faults will occur. This could be in the initial feature or application request, requirement documentation, testing scripts or code. Perfection is not possible - this is true of any product, system or process.
Assessing and Reducing Risk
The level/risk of the fault(s) will differ depending upon the level of risk if the system fails. In addition the exit criteria of a system will differ as well. A life threatening system, for example a new on-board airplane computer will have a very strict and lengthy exit criteria, whereas a simple calculator program will not have a particularly strict set of exit criteria. The risk for a calculator is a lot lower than that of the on board computer.
In QA we have to assess the risk to the system, and of the system itself. During any SDLC decisions have to be made as to what faults are corrected and how this is done, as well as which faults may be left until a later system release.
When testing a fault we do our best to cover as many angles as possible to reduce any risk to the user, be it testing a fix or a new feature. We have to consider the potential knock-on effects of a fix. For example, a fix may have been done in the Purchase Ordering of Stock so we will also test this in the Sales Ordering of Stock. This is because code uses procedures that are shared throughout an application, so a fix to a shared procedure could cause issues in all the places that procedure is used.
This understanding comes with experience, if we are not sure what knock on effects a fix may have, then we speak to the developer, or a more experienced tester in the team.
Environmental Considerations
Environment is also a key part of testing. This could be the operating system in use, the computer hardware, the monitor etc. For cars, this could also include a wind tunnel. In QA not only do knock on effects of fixes have to be considered, but so does the environment. This applies throughout the SDLC.
When considering what new feature to implement or which faults to fix, an additional consideration has to be the existing users of that system, will they be able to continue to use the system as they do now, what benefit would it bring. And what knock on effects could this feature/fault fix have to the rest of the system, what is the QA impact. This is then weighed against the due release date.
Many things are considered when a system is under test - these are just a few of those things.
In a further blog I will be talking about how a new feature is chosen in more depth, and how that feature is implemented.
Key:
Error – human action that produces an incorrect result
Fault/Defect/Bug – manifestation of an error in the software. This could be something wrong in the specification, design or code.
Failure – deviation of the software from its expected delivery or service
Reliability – the probability that the software will not cause the failure of a system for a specified time under specified conditions
Exit Criteria - a set of criteria that have to be passed in order to release the system
Kirsty Lowe, QA department
Comments