Maximizing Test Coverage

We’ve written about test tools and processes before, but we were recently reminded of the importance of maximizing test coverage. Ideally, applications are tested from inception, but we all know that is rarely the case. Testing happens when an application matures, in most cases, when there’s traction and the problems that ensue from having real users. We regularly experiment with testing tools and techniques, looking for better ways to deliver high quality modern applications. We share our insights to help you too…

“Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.”
― Rick Cook, The Wizardry Compiled

Test Coverage 101

Test coverage is “what are we testing and how well are we testing?” It is the monitoring of test quality to locate missing or unvalidated areas of an application. At a basic level is unit testing, or code coverage. Developers should write tests that target all areas of the code at least once. Test coverage should facilitate testing every requirement at least once. Some teams consider a requirement covered if there is at least one test case against it. Others may consider it covered if at least one team member is assigned to it or if all test cases against it are executed. Three areas should be reviewed: product, risk, and requirements. Product coverage determines what parts of the application have been tested. Risk coverage evaluates risks that the application could face when users get their hands on it. And requirements coverage ensures that the business requirements were fully implemented.

The Big Picture

Software testing is critical in software development and maintenance. An application should do what it is supposed to do, right? We’ve worked in organizations where testing was a “luxury”, and it made release days into nightmares (we appeased unhappy customers with frequent rollbacks and patches). Modern application development makes it easier to create comprehensive testing programs and measure testing completeness and effectiveness.

Test Plans

Test coverage is a job for the quality assurance team (not development). Software test plans guide testing activities using an organized process. Testing should validate as much functionality and accuracy as feasible. The goal of test coverage is to determine how well the application is tested in relation to its requirements. A test plan describes the test strategy, objectives, schedule, deliverables, and resources needed to maximize test coverage.

Unit Testing

The first step in a good test plan is unit testing. During the development (coding) phase, deveiopers test individual components to validate that the code performs as expected. A unit may be an individual function, method, procedure, module, or object. It is first level of testing done before integration testing.

Integration Testing

After unit testing comes integration testing where modules are integrated logically and tested as a group. Since systems typically have modules written by different programmers, integrating and testing can expose defects in the interaction between integrated software modules. Integration testing checks data communication between modules.

Manual Testing

Most applications require manual testing before it is possible to automate any testing. A tester manually performs test cases to identify any issues or defects in the application. It requires more effort but helps determine the feasibility of automation.

Cross Browser Testing

Cross Browser Testing verifies that an application works across different browsers. Each browser — Safari, Edge, Chrome, Firefox — interprets display information differently. Some browsers lack the features that an application relies on so it appears broken on that browser.

User Acceptance Testing

The final testing phase before go-live is User Acceptance Testing (UAT). UAT helps determine if the application or module performs as designed according to the organization’s business requirements.

Automation

Manual testing requires considerable time and effort to ensure the software performs as expected. Each tester is responsible for verifying the functionality of the software and documenting issues. Testing with an automation tool helps increase test coverage and scalability. Automated testing is best for large projects, projects that require testing the same areas over and over, and projects that have been through manual testing. Automated testing strategy is most beneficial in modern software development.