Test Efforts in the Agile World

We have all heard and seen over the last few years how products are moving into an Agile mode of development. Although they may not be explicitly termed Agile, the core characteristics of the model including faster releases, an iterative development model with demonstrable software at the end of each milestone, close collaboration amongst teams De-emphasizing on product documentation and formal processes, are all becoming more visible and inevitable.

When time to market is one of the critical factors in determining the product's acceptance in the market place, effort estimation for all disciplines in very important to ensure the product can be released within the committed timelines yet not compromising on its feature set or performance. In this article, I would like to share with you all the most important points along with some best practices to keep in mind in estimating an Agile test effort.

Firstly, as you may be aware the product release typically consists of several sprints. Each sprint may be anywhere between a week to a month in duration, with specific features that need to be targeted in that interval. Daily scrum meetings are conducted to discuss the team's progress, plan of action for the next day and any blocking issues that need to be addressed.

In the traditional waterfall model, one would typically base estimates off of the requirements defined, the complexity of the requirements, potential test cases for each of the requirements, complexity of test cases and estimate test efforts depending on the complexity factor and the number of expected test cases.

In Agile, estimation is based off of user stories. The user story's priority, complexity, associated use cases, types of testing needed are the primary determinants of the testing effort estimate. You may see that there is not much difference here compared to the traditional estimation model. However what is different is the agility of your estimates. Given how short the milestones are, effort estimation tends to be an ongoing activity, where one needs to keep tabs on the numbers and raise any alarms as soon as they are sensed. On the same note, if one is able to finish tasks ahead of the original estimates, such extra bandwidth should be leveraged to take on tasks from the subsequent sprint, to give you a head start. Agile world also offers you the advantage of precedence setting where one is able to learn from the mistakes of the previous effort estimates which are often comparable, because user stories from previous milestones are relatively of the same magnitude and were implemented not too long ago. So, there is often a good baseline to refer to, to help produce more robust estimates.

Also, some of the more time consuming test activities such as test maintenance are typically separated out of the core agile test cycle. In some projects I've seen the team set aside a milestone for quality every quarter or so, where about a week is reserved for various disciplines to focus on maintenance efforts. The test team can use such times for activities such as test case updates, add tests for ad hoc bugs, develop test automation frameworks etc. not including these as part of the core test effort estimation.

Similarly identify tasks that are Sprint contained and ones that span across Sprints. Typically, functional, UI, compatibility tests are specific to Sprints and associated with features that are developed in each Sprint. However, tasks such as Performance, Security testing may run across Sprints. Specific to your product determine what makes the most sense, depending on whether a V1 product is developed or the effort is sustenance in nature.

A lot of unit testing is typically undertaken in Agile model, where the tester tends to focus on the number of units to be tested, the kind of asserts to verify for each feature. This is an area, where collaborative estimation along with the development team becomes important. If the team is taking on Test Driven Development, where automated tests are designed upfront, which become the basis for development coding, test estimation actually becomes more straightforward as the tester has most visibility into the product scenarios compared to the rest of the product team.

Agile by nature is a self-correcting model which promotes a lot of collaboration and frequent communication. So, it is inherently designed to ease the estimation process and promote accuracy of estimates by keeping everyone in sync on updates, changes. The Sprint planning meeting that happens at the start of every Sprint, forces every team member to come prepared with their effort estimates. This allows for everyone to know what the overall product team's estimates are and what ratios they would like to maintain as a team. In most cases, I have seen a 1:1.25 development: test effort estimate. Keeping such high level ratios in mind, such team meetings help review everyone's estimates specific to the Sprint at hand. This is an excellent opportunity for everyone to collaborate, brainstorm, question and review each other's estimates to ensure they are not unreasonable unlike in the traditional waterfall model, where efforts are estimated in isolation, without a complete view into the product dynamics. So in some sense, estimation becomes a group activity bringing in all the checks and balances needed to make it more accurate.

That said, every team has to play its part in making the process a success. If one is not responsive to change, ready to learn from past experience, collaborate with the team, they are setting not just themselves for failure, but are pulling the entire team down. So, it is in the best interest of the entire team, to align oneself with the product dynamics, and help provide robust and near-to-accurate estimates in building and shipping products of exceptional quality.
Along similar lines of test effort estimation in agile, is an interesting topic on test metrics to gauge product and test effort's quality in the agile world. I will touch upon this in my subsequent post, exclusively for the Silicon India audience.

Cloud Computing

A Field Guide to the Wonderful World of Clients

Test Estimation Techniques

Estimation is done often because it is expected to help in predicting how much will the project cost and when will the project get completed. Proper analysis and effort estimation is necessary for successfully planning for a testing project. Any flaw in critical estimation phase, results in missing the project deadlines, reduces ROI and loses of customer’s faith. Remember - Bad estimation can lead to poor distribution of work. 

There are different standard and non standard methods for test estimation. In many product based companies, people utilize non standardized but conventional estimation methods to make things work. These methods might have developed over a continuous period accommodating hidden factors like nature of application under test, environment, and risk factors for that specific product/market. But these methods can’t be adopted as a generalized organization standard for a mature operation model.

Fact: Managers/Leads are not comfortable with software estimation work. But it is a required activity, so based on their past experience on one particular Product, Test Leads/Test Managers estimate the entire testing project (but for that Product only). Because they spent 1-2 or 2-3 or even more years on that particular product. But if they are asked to change company or shift to new product of entire different domain, then it is very hard for them to do estimation.
In this topic, we will discuss following methods for test estimation:
1. FIA (finger in the air) or best guess
2. Ad-hoc method
3. Experience Based - Analogies and experts
4. WBS
5. Delphi technique
6. Three-point estimation (successive calculation)
7. Function points / Test point Analysis
8. Percentage of development effort method
9. Percentage distribution
10. Use case point estimation method
 
Lets discuss one by one:
1. Best Guess – This technique is purely guesswork and based on the some sort of experience.
The method is very common, but since it is based on your gut feeling, its uncertainty contingency is probably around 200% or even higher.

2. Ad-hoc method - The test efforts are based on tentative timeframe. The timeline set by managerial or marketing personnel or by client without any guess/experience. Alternatively, it is done until the budgeted finances run out.
This is very common practise in extremely immature organizations and has error margins of over 100% at times.

3. Experience Based - Analogies and experts:

• Metrics collected from previous tests.
• You already tested similar application in previous project.
• Inputs are taken from Subject Matter experts who know the application (as well as testing) very well.

4. Work Breakdown Structure - It is created by breaking down the test project into small pieces. Modules are divided into sub-modules. Sub modules are further divided into functionalities and functionalities are divided in sub-functionalities.
Review all the requirements from Requirement Document to make sure they are added in WBS. Now you figure out the number of tasks your team needs to complete. Estimate the duration of each task.

5. Delphi technique – Same as above WBS. Here functionalities and each task is allocated to each team member. Then team member gives estimate that they will take this much hours to complete the task.
Averagely, this technique gives good confidence in the estimation. This technique can be combined with other techniques.

6. Three-point estimation – This technique is based on statistical methods In this technique, task is broken down into subtasks (similar to WBS) and then three types on estimation are done on each chunk –

• Optimistic Estimate (Best case scenario in which nothing goes wrong and all conditions are optimal.) = a
• Most Likely Estimate (most likely duration and there may be some problem but most of the things will go right.) = m
• Pessimistic Estimate (worst case scenario which everything goes wrong.) = b

Formula to find Value for Estimate (E) = a + (4*m) + b / 6
Standard Deviation (SD) = = (b - a)/6

7. Function Point/Testing Point Analysis: The FP technique is a direct indicator of the functionality of software application from the user's perspective. This is the most accepted technique used to estimate the size of a software project.
This technique is a part of TMap. Base of this technique is function point technique. Here we convert function points into test points. In Test Point analysis, we usually carry out the following:

• Dynamic Test Points
• Static Test Points
• Environmental Factor
• Productivity Factor
• Primary Test Hours
• Control Factor
• Total Test Hours

In Testing, This estimation is based on requirement specification document, or a previously created prototype of the application. To calculate FP for a project, some major components are required.
The major components are:
Unadjusted Data Function Points: i. Internal Files, ii. External Interfaces
Unadjusted Transaction Function Points: User Inputs, ii. User Outputs & iii. User Inquiries
Capers Jones basic formula:
Number of Test cases = [Number of Function Points] x 1.2
Total Actual Effort, TAE = (Number of Test cases) * (Percentage of development effort /100)
This method is done in a case when a detailed low level design document or requirement document is available (i.e measure of function point is available) & Previous data for development and testing is available. But now days, when we are using agile and iterative methodologies to deliver projects, so most of the times all this documentation is not available.

8. Percentage of development effort method
Here the assumption is that a more complex business application may require more testing effort. The test effort required is a direct proportionate or percentage of the development effort.


Note: The development effort can be estimated using line of code (LOC) or function point (FP) which is not in the our scope.
Example:
If a previous project with 500 FPs required 50 man hours for testing, the percentage of testing effort is calculated as:
P = (50 / 500) * 100 =10%
For the current project with a development effort, say 1500 FPs, the testing effort is:
Total Actual Effort, TAE = 1500 * (P/100) = 1500 * (10/100) = 150 man hours.

9. Percentage distribution – Here all the phases of SDLC are divided in parts and assigned effort in %. Like –
Project management 7%
Requirements 9%
Design 16%
Coding 26%
Test (all test phases) 27%
Documentation 9%
Installation and training 6%
Now testing % is further distributed into all testing phases:

OR

All phases	%
Component testing	16
Independent testing	84
Total	100
Independent testing	%
Integration testing	24
System testing	52
Acceptance testing	24
Total	100
System testing	%
Functional system testing	65
Non-functional system testing	35
Total	100
Test Planning and Design Architecture	50%
Review phase	50%

10. Use case point estimation method: Use case point (UCP) method is gaining popularity because now-a-days application development is modelled around use case specification. The test case development is normally kicked off after baseline use case. So the various factors in use case give a direct proportion to the testing effort. 
Use case is a document which well specifies different users, systems or other stakeholders interacting with the concerned application. They are named as ‘Actors’. The interactions accomplish some defined goals protecting the interest of all stakeholders through different behaviour or flow termed as scenarios.

UCP Estimation Method in brief
1. Obtain unadjusted actor weight (UAW)
2. Determine unadjusted use case weight (UUCW)  
3. Calculate unadjusted use case points (UUCP) UUCP = UAW + UUCW
4. Determine the technical/environmental factor (TEF)
5. Compute the adjusted use case point (AUCP) AUCP = UUCP * [ 0.65 + (0.01 * TEF)]  
6. Arrive at final effort using a conversion factor Total Actual Effort = AUCP * CF

Conclusion
There can’t be a sole hard and fast rule for estimating the testing effort for a project. There may be different other methods also which can be effectively used for the purpose. It is recommended to add on to the possible knowledge base of test estimation methods and estimation templates constantly revised based upon new findings.

5 Reasons Why You Need a TCoE n Your IT Company

Some of the benefits of having a TCoE in an organization are-

1. Produce better products faster: One of the primary reasons for having a TCoE implemented in the organization is that it takes less time to release a product in the market. In this competitive market of software, everyone is busy producing better products faster and bringing it out to market. So having a test center within the compound will help maintain a competition.

2. Cost cutting: Integrating the testing tools and resources can help in reducing the redundancy and lead to cost saving in resource utilization, software setup, maintenance costs and software procurement.

3. Quality Improvement: This is one of the important benefits of having a test center because it enhances the reputation of an organization. TCoE helps in delivering improvements in quality of application software, visibility of QA processes, reduces the risk of failure and enables a good customer experience.

4. Maximizing talent:  Assigning people to work outside their work station can cause talent to diminish. So having a TCoE can help retain the talent and use the available in-house capabilities and knowledge. This also increases the job opportunities and provides new career opportunities.

5. Industrialization: In response to market pressures many organizations are turning to TCoEs to strengthen the quality assurance. That’s because testing centers use standardized testing methods, automation, testing tools and they get the best practices to establish the project based training into a standard process.

SQL Injection

Knowing about Agile Methodology

1) What is Agile Development model and explain about different methodologies in Agile Software development?

Agile Development model is an Incremental or iterative model,
It has several methodologies,

Types of Agile Methodologies

o Scrum

o Extreme Programming (XP)

o Agile Unified Process (AUP)

o Dynamic Systems Development Method (DSDM)

o Essential Unified Process (EssUP)

o Exia Process (ExP)

o Feature Driven Development (FDD)

o Open Unified Process (OpenUP)

o Crystal Clear

o Velocity tracking

2) What is the difference between Agile and Waterfall Model?

Difference between Agile and Waterfall Model:
1. The main advantage is the backward scalability in Agile. Under waterfall approach we cannot change the decisions and implementations that we had made under the previous stages. If we want to make changes under waterfall we will have to build the entire project from the scratch once again.
2. The flexibility to error check under any part of the development stage makes Agile more bug free and less erroneous as compared to Waterfall which can only test bugs at the end of the development module.
3. Since Agile provides flexibility to make changes as per customer requirements it is more inclined towards better client satisfaction. This is a real set back for the Waterfall model which doesn’t allow any modifications once the module has been completed.
4. Under Agile development modular partitioning of the software can be effectively carried out as compared to its counterpart. Though both of them allows option for segregation the later lacks the modifications in the implementation stage. The rules are set down before the commencement of the project hence it hinders further break down of the logical module. Whereas Agile can be of great help under such situations and can allow simultaneous development of different modules at the same time as per time bound requirement. If we want the project to be more segregated Agile comes as a pain relief for developers.

3) How Agile is different?
Agile is different because
a) Greater Collaboration.
b) Shorter work cycle and constant feedback.
c)  Need to embrace Change.
d) Greater flexibility.
e) Greater discipline.
f) Greater stakeholder accountability.
g) Greater range of skills.

4) What are the implications for testing in Agile?

- Agile testing must be iterative.
- Testers cannot rely on having complete specification.
- Agile testers must be flexible.

5) What are the Agile quality strategies?

Agile quality strategies are:
a) Re-factoring.
b) Non-solo development.
c) Static and dynamic code analysis
d) Reviews and inspection
- Iteration/sprint demos
- All-hands demo
- Light-weight milestone reviews
e) Short feedback cycles.
f) Standards and guidelines.

6) Explain about the Agile process?

The agile process follows the software development life cycle which includes requirements gathering, analysis, design, coding, testing and delivers partially implemented software and waits for the customer feedback. In the whole process, customer satisfaction is at highest priority with faster development time.

7) What are the new features of agile process?
In agile process new features can be added easily by using multiple iterations.

1. Iterative: The main objective of agile software processes is satisfaction of customers, so it focuses on single requirement with multiple iterations.
2. Modularity
Agile process decomposes the complete system into manageable pieces called modules. Modularity plays a major role in software development processes.
3. Time Boxing
As agile process is iterative in nature, it requires the time limits on each module with respective cycle.
4. Parsimony
In agile processes parsimony is required to mitigate risks and achieve the goals by minimal number of modules.
5. Incremental
As the agile process is iterative in nature, it requires the system to be developed in increments, each increment is independent of others, and at last all increments are integrated into complete system.
6. Adaptive
Due to the iterative nature of agile process new risks may occurs. The adaptive characteristic of agile process allows adapting the processes to attack the new risks and allows changes in the real time requirements.
7. Convergent
All the risks associated with each increment are convergent in agile process by using iterative and incremental approach.
9. Collaborative
As agile process is modular in nature, it needs a good communication among software development team.
Different modules need to be integrated at the end of the software development process.
10. People Oriented
In the agile processes customer satisfaction is the first priority over the technology and process. A good software development team increases the performance and productivity of the software.

8) Where the agile methods focus on?

The agile methods are focused on different aspects of the software development life cycle. Some focus on the practices (extreme programming, pair programming), while others focus on managing the software projects (the scrum approach).

9) What are the benefits of Agile Modeling?
 
Agile methodology has an adaptive team which is able to respond to the changing requirements.
The team does not have to invest time and effort and finally find that by the time they delivered the product, the requirement of the customer has changed.
Face to face communication and continuous inputs from customer representative leaves no space for guesswork.
The documentation is crisp and to the point to save time.
The end result is the high quality software in least possible time duration and satisfied customer

Benefits of Crowdsourced Usability Testing

Crowdsourced Usability testing is an alternative method widely used by software testing professionals to test mobile, web and desktop applications. It involves a large group of people from different backgrounds. Although it may look chaotic, however it has its own benefits.

As compared to in-house usability testing, the participants in crowdsourced usability testing are not influenced by any politics in the company. Moreover, as they belong to a particular community, they are more motivated and loyal towards the test instructor. The participants provide high quality feedbacks that make the data highly valid.

Let us take a look at some of the benefits of Crowdsourced Usability Testing as mentioned in a whitepaper by uTest titled “Crowdsourcing Eight Ways to Launch Higher Quality Applications and Get to Market Faster with the Crowd.”

1.    Cost Containment
Crowdsourcing brings down the costs involved as the company doesn’t have any long term commitments with the participants. Furthermore, as compared to traditional outsourcing, this method is more accountable and cost effecting.

2.    Decreases the Time to Market
Crowdsourcing decreases the time for the product to reach the market by removing any delays or problems that usually crop up before launching a product or the software. It had also proved its usefulness to projects with tight budgets.

3.    Meeting Users Expectations
Users expect their products to be of high quality and functional without any glitches from the very first day. Crowdsourcing ensures that the product or the application is of high quality.

4.    Keeping your Customers
If a user is satisfied with the application or the product, they will look forward for products / applications from the company. Thus, they become loyal customers of the company.

5.    Empower the User
Crowdsourcing bridges the gap between a software user and a software builder which includes the designers, developers and the testers of the application. Further, it ushers in a strong emotional commitment towards the company or the application as the users feel that they have a hand in the success of the product.