Tackling complexity
Posted 12 years ago by Gabor Puhalla
Software complexity is one of the most significant challenges that
a software tester may face. Testing software that is complex usually requires a breadth of knowledge and experience. The goal that the tester needs to accomplish with regards to complexity is: to test the software in a reasonable time and at a reasonable cost.
As an example, think about ways to test upgrades of a database system with the following support matrix that gives altogether billions of combinations to test. Can this test scope be reasonably covered? Maybe. Let’s see several techniques that can help addressing complexity.
| Operating Systems | the database system should run on various versions of Windows, Solaris, RedHat Linux, Ubuntu Linux and Suse Linux |
| Deployment types | the database system can be deployed as stand-alone, client-server or client server with multiple redundant instances of the server |
| Upgrade from | 2 prior versions of the database system |
| Integrated products | the database system should work with multiple versions of associated software products from different vendors (application server, sso, identity management, soa, portal, plug-in for integration with sso) |
| Interfaces | thousands of GUI, CLI and API that exercise features of the database system |
I don’t intend to suggest a single practice for addressing testing complexity here. The specific test approach always depends on the software in hand and other factors, such as available resources. One usually needs to employ a good mix of multiple test techniques to test both effectively and efficiently. Instead, let me discuss techniques and approaches that we’ve used in practice, hoping that you could leverage them for creating your own mix and extend them based on your own experience.
- Understand your customer’s needs – to understand your test objectives
- Understand your product – to understand your test scope
- Get involved in product development early – to reflect past testing experience with product complexity in subsequent product releases
- Identify testing priorities – to know, where to focus
- Rotation of test configurations – use the advantage of iterative processes to increase coverage
- Domain partitioning – sample your test scope to avoid executing tests that add low value
- Pair-wise testing – a brute force approach that dramatically decreases the number of test combinations to test
- Exploratory testing – to use the experience of your test experts to complement the pre-scripted way of testing
- Test automation – to speed up and lower the cost of execution
Each of them offers evident business benefits, if used appropriately. Each of them needs certain knowledge and involves some risk as well. I’m going to talk about these in the following series of articles.
| What is software complexity? This article discusses complexity from the point of view of a possibly huge number of test combinations. Not from the point of view of complexity of the code structure nor the business-processes that the software implements. |
And, BTW, most of these approaches are described in books and taught on trainings about testing. I’m going to refer to those books and articles at least and reflect to their descriptions critically based on practical experience.
