We have all heard and seen over the last few
years how products are moving into an Agile mode of development.
Although they may not be explicitly termed Agile, the core
characteristics of the model including faster releases, an iterative
development model with demonstrable software at the end of each
milestone, close collaboration amongst teams De-emphasizing on product
documentation and formal processes, are all becoming more visible and
inevitable.
When time to market is one of the critical factors in determining the product's acceptance in the market place, effort estimation for all disciplines in very important to ensure the product can be released within the committed timelines yet not compromising on its feature set or performance. In this article, I would like to share with you all the most important points along with some best practices to keep in mind in estimating an Agile test effort.
Firstly, as you may be aware the product release typically consists of several sprints. Each sprint may be anywhere between a week to a month in duration, with specific features that need to be targeted in that interval. Daily scrum meetings are conducted to discuss the team's progress, plan of action for the next day and any blocking issues that need to be addressed.
In the traditional waterfall model, one would typically base estimates off of the requirements defined, the complexity of the requirements, potential test cases for each of the requirements, complexity of test cases and estimate test efforts depending on the complexity factor and the number of expected test cases.
In Agile, estimation is based off of user stories. The user story's priority, complexity, associated use cases, types of testing needed are the primary determinants of the testing effort estimate. You may see that there is not much difference here compared to the traditional estimation model. However what is different is the agility of your estimates. Given how short the milestones are, effort estimation tends to be an ongoing activity, where one needs to keep tabs on the numbers and raise any alarms as soon as they are sensed. On the same note, if one is able to finish tasks ahead of the original estimates, such extra bandwidth should be leveraged to take on tasks from the subsequent sprint, to give you a head start. Agile world also offers you the advantage of precedence setting where one is able to learn from the mistakes of the previous effort estimates which are often comparable, because user stories from previous milestones are relatively of the same magnitude and were implemented not too long ago. So, there is often a good baseline to refer to, to help produce more robust estimates.
Also, some of the more time consuming test activities such as test maintenance are typically separated out of the core agile test cycle. In some projects I've seen the team set aside a milestone for quality every quarter or so, where about a week is reserved for various disciplines to focus on maintenance efforts. The test team can use such times for activities such as test case updates, add tests for ad hoc bugs, develop test automation frameworks etc. not including these as part of the core test effort estimation.
Similarly identify tasks that are Sprint contained and ones that span across Sprints. Typically, functional, UI, compatibility tests are specific to Sprints and associated with features that are developed in each Sprint. However, tasks such as Performance, Security testing may run across Sprints. Specific to your product determine what makes the most sense, depending on whether a V1 product is developed or the effort is sustenance in nature.
A lot of unit testing is typically undertaken in Agile model, where the tester tends to focus on the number of units to be tested, the kind of asserts to verify for each feature. This is an area, where collaborative estimation along with the development team becomes important. If the team is taking on Test Driven Development, where automated tests are designed upfront, which become the basis for development coding, test estimation actually becomes more straightforward as the tester has most visibility into the product scenarios compared to the rest of the product team.
Agile by nature is a self-correcting model which promotes a lot of collaboration and frequent communication. So, it is inherently designed to ease the estimation process and promote accuracy of estimates by keeping everyone in sync on updates, changes. The Sprint planning meeting that happens at the start of every Sprint, forces every team member to come prepared with their effort estimates. This allows for everyone to know what the overall product team's estimates are and what ratios they would like to maintain as a team. In most cases, I have seen a 1:1.25 development: test effort estimate. Keeping such high level ratios in mind, such team meetings help review everyone's estimates specific to the Sprint at hand. This is an excellent opportunity for everyone to collaborate, brainstorm, question and review each other's estimates to ensure they are not unreasonable unlike in the traditional waterfall model, where efforts are estimated in isolation, without a complete view into the product dynamics. So in some sense, estimation becomes a group activity bringing in all the checks and balances needed to make it more accurate.
That said, every team has to play its part in making the process a success. If one is not responsive to change, ready to learn from past experience, collaborate with the team, they are setting not just themselves for failure, but are pulling the entire team down. So, it is in the best interest of the entire team, to align oneself with the product dynamics, and help provide robust and near-to-accurate estimates in building and shipping products of exceptional quality.
Along similar lines of test effort estimation in agile, is an interesting topic on test metrics to gauge product and test effort's quality in the agile world. I will touch upon this in my subsequent post, exclusively for the Silicon India audience.
When time to market is one of the critical factors in determining the product's acceptance in the market place, effort estimation for all disciplines in very important to ensure the product can be released within the committed timelines yet not compromising on its feature set or performance. In this article, I would like to share with you all the most important points along with some best practices to keep in mind in estimating an Agile test effort.
Firstly, as you may be aware the product release typically consists of several sprints. Each sprint may be anywhere between a week to a month in duration, with specific features that need to be targeted in that interval. Daily scrum meetings are conducted to discuss the team's progress, plan of action for the next day and any blocking issues that need to be addressed.
In the traditional waterfall model, one would typically base estimates off of the requirements defined, the complexity of the requirements, potential test cases for each of the requirements, complexity of test cases and estimate test efforts depending on the complexity factor and the number of expected test cases.
In Agile, estimation is based off of user stories. The user story's priority, complexity, associated use cases, types of testing needed are the primary determinants of the testing effort estimate. You may see that there is not much difference here compared to the traditional estimation model. However what is different is the agility of your estimates. Given how short the milestones are, effort estimation tends to be an ongoing activity, where one needs to keep tabs on the numbers and raise any alarms as soon as they are sensed. On the same note, if one is able to finish tasks ahead of the original estimates, such extra bandwidth should be leveraged to take on tasks from the subsequent sprint, to give you a head start. Agile world also offers you the advantage of precedence setting where one is able to learn from the mistakes of the previous effort estimates which are often comparable, because user stories from previous milestones are relatively of the same magnitude and were implemented not too long ago. So, there is often a good baseline to refer to, to help produce more robust estimates.
Also, some of the more time consuming test activities such as test maintenance are typically separated out of the core agile test cycle. In some projects I've seen the team set aside a milestone for quality every quarter or so, where about a week is reserved for various disciplines to focus on maintenance efforts. The test team can use such times for activities such as test case updates, add tests for ad hoc bugs, develop test automation frameworks etc. not including these as part of the core test effort estimation.
Similarly identify tasks that are Sprint contained and ones that span across Sprints. Typically, functional, UI, compatibility tests are specific to Sprints and associated with features that are developed in each Sprint. However, tasks such as Performance, Security testing may run across Sprints. Specific to your product determine what makes the most sense, depending on whether a V1 product is developed or the effort is sustenance in nature.
A lot of unit testing is typically undertaken in Agile model, where the tester tends to focus on the number of units to be tested, the kind of asserts to verify for each feature. This is an area, where collaborative estimation along with the development team becomes important. If the team is taking on Test Driven Development, where automated tests are designed upfront, which become the basis for development coding, test estimation actually becomes more straightforward as the tester has most visibility into the product scenarios compared to the rest of the product team.
Agile by nature is a self-correcting model which promotes a lot of collaboration and frequent communication. So, it is inherently designed to ease the estimation process and promote accuracy of estimates by keeping everyone in sync on updates, changes. The Sprint planning meeting that happens at the start of every Sprint, forces every team member to come prepared with their effort estimates. This allows for everyone to know what the overall product team's estimates are and what ratios they would like to maintain as a team. In most cases, I have seen a 1:1.25 development: test effort estimate. Keeping such high level ratios in mind, such team meetings help review everyone's estimates specific to the Sprint at hand. This is an excellent opportunity for everyone to collaborate, brainstorm, question and review each other's estimates to ensure they are not unreasonable unlike in the traditional waterfall model, where efforts are estimated in isolation, without a complete view into the product dynamics. So in some sense, estimation becomes a group activity bringing in all the checks and balances needed to make it more accurate.
That said, every team has to play its part in making the process a success. If one is not responsive to change, ready to learn from past experience, collaborate with the team, they are setting not just themselves for failure, but are pulling the entire team down. So, it is in the best interest of the entire team, to align oneself with the product dynamics, and help provide robust and near-to-accurate estimates in building and shipping products of exceptional quality.
Along similar lines of test effort estimation in agile, is an interesting topic on test metrics to gauge product and test effort's quality in the agile world. I will touch upon this in my subsequent post, exclusively for the Silicon India audience.
