Agile Testing

Agile testing is a software testing practice that follows the principles of the agile manifesto, emphasizing testing from the perspective of customers who will utilize the system. Agile testing does not emphasize rigidly defined testing procedures, but rather focuses on testing iteratively against newly developed code until quality is achieved from an end customer's perspective. In other words, the emphasis is shifted from "testers as quality police" to something more like "entire project team working toward demonstrable quality."

Agile testing involves testing from the customer perspective as early as possible, testing early and often as code becomes available and stable enough from module/unit level testing.
Since working increments of the software are released often in agile software development, there is also a need to test often. This is commonly done by using automated acceptance testing to minimize the amount of manual labor involved. Doing only manual testing in agile development may result in either buggy software or slipping schedules because it may not be possible to test the entire build manually before each release.

Testing Agenda in Agile Projects:
Agile methodologies are designed to break the software down into manageable parts that can be delivered earlier to the customer. The aim of any Agile project is to deliver a basic working product as quickly as possible and then to go through a process of continual improvement. An Agile project is characterized by having a large number of short delivery cycles (sprints) and priority is given to the feedback-loops from one cycle to the next. The feedback- loops drive continuous improvement and allow the issues that inevitably occur (including the way requirements have been defined or the quality of the solution provided) to be dealt with much earlier in the development life cycle. To achieve this, companies need to re-think their approach to delivery and have their previously independent contributors (Business Analysts, Developers, Testers, End Users etc.) work together in teams.
The testing agenda during the development phase is very different to the traditional approaches that support the Waterfall or V-Model development methodologies because Agile uses a Test Driven Development (TDD) model. However, many of the requirements of the extended testing model will still apply.

Project Initiation Phase:
For an Agile project to succeed, it requires effective planning and management. As part of the project team the Test Manager is responsible for establishing the quality processes, identifying test resources and delivering the test strategy. The test strategy will include details of the Agile development process being used. It will also include details of test phases not directly related to the Agile development for example, Operational Acceptance or Performance Testing.

Development Phase:
True Agile development uses a “Test Driven Development” (TDD) approach. The testers, developers, business analysts and project stakeholders all contribute to kick-off meetings where the “user stories” are selected for the next sprint. Sprints are usually of fixed duration and selecting the right number of stories and estimating the time to deliver them is the key to success. The estimation process can often be inaccurate at the beginning of an Agile project but improves as the team’s experience grows within the specific implementation they are working on. Once the user stories are selected for the sprint they are used as the basis for a set of tests. The testers create test scenarios which are presented to the business analysts and project stakeholders for their approval and signoff. These test scenarios are then broken down to test cases that offer adequate test coverage for the given functionality.
The developers then write code that will pass the tests. In this approach the development and testing take place continuously throughout the sprint – there is no separate testing phase. In the TDD approach the features are tested throughout the sprint and the results presented to the stakeholders for immediate feedback. The test scenarios defined are not limited to functional testing but can include other types of testing including performance and integration testing when the product is mature enough.
While the development is underway the user stories for the next sprint are written. These include the stories specified in the delivery plan but will also include additional stories required to cover any issues that have been identified as part of the feedback process from previous sprints. Sprints in an Agile project can extend to multiple levels in a complex system. A sprint might not lead to a product release if it does not add enough functionality to the product being developed. The stakeholders take a decision on when the application should be moved to the release phase depending on the market need or the level of key functionality being added to the system. While multiple iterations may be required to release a product, there may also be cases where releases are more regular owing to the additional value delivered at each iteration level. Whichever release approach is adopted, the testing team’s goal is to have a release available with minimal defects and low implementation risk at the end of the Sprint.
As functionality grows with each iteration, regression testing must be performed to ensure that existing functionality has not been impacted by the introduction of new functionality in each iteration cycle. Defect fixes should also be followed by extensive regression testing. The scale of the regression testing grows with each sprint and to ensure that this remains a manageable task the test team should use test automation for the regression suite and focus their manual testing effort towards locating new defects during the build phase.

Release Phase:
The aim of any Agile project is to deliver a basic working product as quickly as possible and then to go through a process of continual improvement. This means that there is no single release phase for a product. The product will move into production when enough of the functionality is working and the quality of the product is acceptable to the stakeholders. Prior to release, a final acceptance test is performed before transitioning the application into production. The testing activities listed above are not exhaustive but broadly cover the areas which the testing team contributes to the Agile approach.

Getting Equipped for Agile Testing:
Agile projects present their own challenges to the testing team. Unclear project scope, multiple iterations, minimal documentation, early and frequent testing needs and active stakeholder involvement all demand special and diverse skills from the testing team. Some of the essential skills are illustrated here:

Resource Management
The Agile approach required a mixture of test skills – usually held across one team. Test resource will be required to define the scenarios and test cases, conduct manual testing alongside the developers, write automated regression tests and execute the automated regression packs. As the project progresses, specialist skills will be required to cover further test areas that might include integration and performance testing. The availability of a pool of professional test resources offers the scope for effective resource management across the agile project life cycle. There should be an appropriate mix of domain specialists who plan and gather requirements in addition to test engineers who will have multiple skill sets and own the test execution.

Communication
The benefits of independent testing will not be realized unless good communication exists between the testers, developers, business analysts and project stakeholders. The iterative model of development and the frequency of releases mandate that all teams have a common understanding of the user requirements. The testing teams should be skilled in the use of change management tools. The communication model used by the testing team itself must enable both regular and ad-hoc project updates from various parties engaged in the project. The testing team should adopt the most efficient and effective methods of conveying information to the developers, project stakeholders, domain specialists using a combination of face-to-face conversation, meetings and workshops, phone calls, email and WebEx meetings.

Processes
Another key success factor for Agile Development is the implementation of quality governance processes such as configuration management. Typically, an organization that has no processes in place will be in chaos. As formal processes are implemented the organization will be able to follow a repeatable and consistent delivery method. Organizations considering the use of Agile should ensure that configuration management, change management, project management and release management are in place. Testing teams which bring with them best practices and are accredited with globally recognized certifications (e.g. TMMi, CMMi, ISO etc.) will be able to help organizations accelerate testing and enable lower cost of quality to be achieved.

Conclusion:
Companies which adopt Agile projects should note the importance of engaging test teams at project initiation. This will ensure an accelerated delivery of working software. If Agile projects are to achieve customer satisfaction and ROI then time, cost and quality must be controlled and balanced. To ensure accelerated delivery of working software which conforms to the desired quality, the testing team should be involved from the beginning of every iterative development cycle and not just after the first couple of sprints. The testing team must develop the necessary mindset for an Agile project. Their own agility and flexibility is essentially the key to their success in the project.