We have all heard and seen over the last few
years how products are moving into an Agile mode of development.
Although they may not be explicitly termed Agile, the core
characteristics of the model including faster releases, an iterative
development model with demonstrable software at the end of each
milestone, close collaboration amongst teams De-emphasizing on product
documentation and formal processes, are all becoming more visible and
inevitable.
When time to market is one of the critical factors in
determining the product's acceptance in the market place, effort
estimation for all disciplines in very important to ensure the product
can be released within the committed timelines yet not compromising on
its feature set or performance. In this article, I would like to share
with you all the most important points along with some best practices to
keep in mind in estimating an Agile test effort.
Firstly, as you
may be aware the product release typically consists of several sprints.
Each sprint may be anywhere between a week to a month in duration, with
specific features that need to be targeted in that interval. Daily
scrum meetings are conducted to discuss the team's progress, plan of
action for the next day and any blocking issues that need to be
addressed.
In the traditional waterfall model, one would
typically base estimates off of the requirements defined, the complexity
of the requirements, potential test cases for each of the requirements,
complexity of test cases and estimate test efforts depending on the
complexity factor and the number of expected test cases.
In
Agile, estimation is based off of user stories. The user story's
priority, complexity, associated use cases, types of testing needed are
the primary determinants of the testing effort estimate. You may see
that there is not much difference here compared to the traditional
estimation model. However what is different is the agility of your
estimates. Given how short the milestones are, effort estimation tends
to be an ongoing activity, where one needs to keep tabs on the numbers
and raise any alarms as soon as they are sensed. On the same note, if
one is able to finish tasks ahead of the original estimates, such extra
bandwidth should be leveraged to take on tasks from the subsequent
sprint, to give you a head start. Agile world also offers you the
advantage of precedence setting where one is able to learn from the
mistakes of the previous effort estimates which are often comparable,
because user stories from previous milestones are relatively of the same
magnitude and were implemented not too long ago. So, there is often a
good baseline to refer to, to help produce more robust estimates.
Also,
some of the more time consuming test activities such as test
maintenance are typically separated out of the core agile test cycle. In
some projects I've seen the team set aside a milestone for quality
every quarter or so, where about a week is reserved for various
disciplines to focus on maintenance efforts. The test team can use such
times for activities such as test case updates, add tests for ad hoc
bugs, develop test automation frameworks etc. not including these as
part of the core test effort estimation.
Similarly identify tasks
that are Sprint contained and ones that span across Sprints. Typically,
functional, UI, compatibility tests are specific to Sprints and
associated with features that are developed in each Sprint. However,
tasks such as Performance, Security testing may run across Sprints.
Specific to your product determine what makes the most sense, depending
on whether a V1 product is developed or the effort is sustenance in
nature.
A lot of unit testing is typically undertaken in Agile
model, where the tester tends to focus on the number of units to be
tested, the kind of asserts to verify for each feature. This is an area,
where collaborative estimation along with the development team becomes
important. If the team is taking on Test Driven Development, where
automated tests are designed upfront, which become the basis for
development coding, test estimation actually becomes more
straightforward as the tester has most visibility into the product
scenarios compared to the rest of the product team.
Agile by
nature is a self-correcting model which promotes a lot of collaboration
and frequent communication. So, it is inherently designed to ease the
estimation process and promote accuracy of estimates by keeping everyone
in sync on updates, changes. The Sprint planning meeting that happens
at the start of every Sprint, forces every team member to come prepared
with their effort estimates. This allows for everyone to know what the
overall product team's estimates are and what ratios they would like to
maintain as a team. In most cases, I have seen a 1:1.25 development:
test effort estimate. Keeping such high level ratios in mind, such team
meetings help review everyone's estimates specific to the Sprint at
hand. This is an excellent opportunity for everyone to collaborate,
brainstorm, question and review each other's estimates to ensure they
are not unreasonable unlike in the traditional waterfall model, where
efforts are estimated in isolation, without a complete view into the
product dynamics. So in some sense, estimation becomes a group activity
bringing in all the checks and balances needed to make it more accurate.
That
said, every team has to play its part in making the process a success.
If one is not responsive to change, ready to learn from past experience,
collaborate with the team, they are setting not just themselves for
failure, but are pulling the entire team down. So, it is in the best
interest of the entire team, to align oneself with the product dynamics,
and help provide robust and near-to-accurate estimates in building and
shipping products of exceptional quality.
Along similar lines of test
effort estimation in agile, is an interesting topic on test metrics to
gauge product and test effort's quality in the agile world. I will touch
upon this in my subsequent post, exclusively for the Silicon India
audience.
