Software Testing Practices, Test Methodologies and Test Competency Services


Welcome to the world of Software Testing & Best Test Practices.

All you wanted to know about Software Testing, Test Practices, Test Methodologies and building the Test Competency Services !!


Tuesday, October 30, 2012

10 Tips to be an Agile Tester

1. Focus on Risk/Change Based Testing – In an Agile environment, the goal is always to get the highest quality product to market in the shortest amount of time. Therefore an Agile tester must be able to assess which areas of the application are changing, how they all fit together and prioritize tests based on areas that pose the greatest risk of failure.

2. Understanding Product Architecture – Put simply, this means understanding exactly how data flows through an application. This allows us to test the impact of sub-system component failures, as well as address potential security vulnerabilities. As a result, when defects are found an Agile tester will be able to help developers fix the issues quickly and thoroughly early on.

3. Understanding Business Objects and Context – Since testing should always align with customer context, an Agile tester must understand how the end user will interact with the product. If you focus on the applications and scenarios that provide value to the end user you will be able to dimensionalize your testing strategy. Meaning, if you can divide a product up based on the product architecture and customer context you will be better equipped to test for potential issues that would negatively impact end users.

The added benefit of dimensionalizing the product from the customer view point is that the tester will be able to tag defects with the affected business objects. That way when you begin to review your test sets, you will be able to recognize areas with higher instances of defects. This will allow you to address the issues in a more focused and effective way.

4. Application Logs - While reporting ‘when feature X breaks we do Y’ provides value, it is not enough in an Agile environment. Agile testers should be leveraging the great amount of information contained in application logs for a number of reasons. First, it provides insight into the underlying architecture that is occurring on the system level, which gives you a better understanding of what the defect is. Second, application logs also allow you to address ‘silent errors’ – i.e. errors that occur without the end user ever even knowing. Lastly, it allows testers to gain credibility and work more closely with the development team, which is an integral component of being Agile.

5. Based Tools – In addition to application logs, browser based tools are incredibly important for Agile testers to be able to quickly troubleshoot defects. Two examples of easy-to-use browser tools are Developer Tools in Google Chrome and Firebug in Firefox. Tools such as these provide value in the form of metrics, errors, analysis, java script consoles and debuggers.

6. Test Document Requirements – Many Agile organizations utilize some variant of SCRUM or Kanban, in which there is an element of work tackled by a team to ensure that all aspects function as intended. In order to do this, the team will need to create test scenarios. The expectance criterion of these test scenarios often becomes a requirement repository over the long term. While some people view the written test scenario as a waste, Brian Rock contends that these test scenarios become very valuable to the Agile tester for future automation, regression testing and product analysis.

7. Automation – Automation is an excellent tool but it goes without saying that it is not the ‘silver bullet’ to fix the testing problem. Frameworks can be expensive to create and even more expensive to maintain. However, Agile testers need to be able to leverage automation for quick, easily repeatable tasks. If the delta of change is low, chances are you should automate it.

8. Exploratory Testing – Exploratory testing (ET) is common sense testing. ET focuses on instantaneous testing and learning and thus should be a primary tool of all Agile testing. Agile testers can leverage knowledge gained through ET on every future product iteration. Additionally, it allows the Agile tester to make variations in testing to quickly ferret out bugs.

9. Testing From the Customer Perspective – In many ways, this goes back to understanding the business objects and how testing should focus the end user – despite the fact that the end user is going to use the product in ways that you never thought feasible, wise or even sane. Testing from the customer perspective goes a step further to state that the fitness of the product for use by the end user is the primary standard of quality. After all, without customers there is no product. Agile testers are in a unique position to face QA challenges from this context.
    
      10. Know That Change is Constant –Perhaps this is a bit cliché, but a static product is a dead product. As an Agile tester you should be able to give a quality assessment of the product at any stage in its life cycle. If you are able to do this, you will be better equipped to handle the ever changing landscape that is Agile.

Wednesday, October 24, 2012

Top 10 ways to conduct Security Testing on web applications

1. Finding the /admin/ interface by "Fuzzing" or "Guessing" technique

In this, as testers; we need to find the admin path for the URL. The path names should not be easily guessable like,

http://example.com/admin/
http://example.com/admin.aspx
http://example.com/login
http://example.com/administrator
http://example.com/admin.php

The guessing keeps on going. Out of 10 e-commerce websites, I have seen people using very easily guessable path names for admin interface. Other sites belonged to "Content Management Systems", "Health Care", "e-Ticketing" and others.

Now, what's the fix here?

First of all, admin interface should not be publicly allowed to be accessed. There need to be white-list of IP addresses or hosts which can access the path. Also, have the admin path name in such a fashion that, it is not easily guessable and have it in totally different path which could be like http://example.com/jKMe23292mMjs/luKseqA/LogMeInAsAdmin -- You might want to change "LogMeInAsAdmin" to something else which is not easy to guess.

You got to instruct the spiders from indexing this URL. So, do not make a mistake of considering this pathname in the sitemap.XML file. It will be a disaster!

2. Forgot Password being used as spam machine

Often I see, Forgot Password code is written in a sloppy way or security layers are not built for the feature. Here is a vulnerability that you could use on most of the available websites -
You got to go to Forgot Password webpage
You will see an input text field where you can enter an e-mail address
Now, repeat the above 2 steps and you can keep sending e-mails to the same address again and again

This could result in spamming where attacker enters the target e-mail address which might belong to anyone and keeps them sending bulk e-mails which makes the customer look at it as, spamming which is pathetic.

Now, the developer might ask you a question; "Come on, man! Who has time to keep sending 100 or 1000 e-mails by submitting the form again and again".

Your answer could be this, "Attacker would automate HTTP requests and keep sending the e-mails" repeatedly. You can demonstrate this automation by using iMacros (Go to http://add-ons.mozilla.org/ and download "iMacros"). Once downloaded, record the flow of "Forgot Password" web-page | Enter e-mail address | And submit the form - Then you got to stop the recording session. Then loop it how many ever time you want; if you say 1000 loop time(s) then 1000 e-mails would be sent. So, while you sip the coffee, the target e-mail address is victimized with lots of e-mails in inbox.

This could also be an attack on SMTP server.

Now, what's the fix?
CAPTCHA could be shown after 3 or 5 attempts to send e-mails using same e-mail address. There are chances of bounced e-mails; in this case we got to give some attempts without CAPTCHA. CAPTCHA would help in stopping these automated attacks.

Or else, put an upper limit on sending e-mails within 24 hours. If there are 10 e-mails sent for same e-mail address, then you can say; "Please try after 24 hours from now as you have exceeded your requests for the day already."

3. Application Caching Bug - BlackMagic BackMagic

Do not be scared, I am not talking about some BlackMagic Mantra here. It's just the humor stuff by using strikethrough *Grins*.

You can remember "Application Cache" test as "BackMagic" (It need not be new terminology and interviewers, please do not treat this as technique - It is just a humor thing and for my reference). What is the test exactly?

You login to your web application with your credentials and browse different web-pages which are shown to only for the logged in end-users. Now, finally you logout and let us say you are landed on homepage or any other web-page used in the re-direction based on the decision taken by business folks. Now, you just need to click on "Back" button of your web browser and see if you are taken to the pages which you browsed previously when you were logged in. That's the bug.

The fix here is to disable application caching or clearing the cache once the logout action is invoked.

The risk here is, when people logout and just do not close the web browser. Next user just comes in and clicks on "Back" and looks at all the information which violated the privacy policy set by your organization.

Now, again there are 2 variants of it,
You can click "Back" and just view web-pages which were browsed recently.
You can click "Back" and view the web-pages and also perform actions like "Change e-mail address", "Edit Personal Details" and so on.

Second one could be disaster. Be careful.

4. Directory Listing - Access to Treasure

When Directory Listing is not disabled, you could go through the all the directory structure which includes sub-directories and files. Example could be: http://example.com/images/ -- Try accessing your web application with directories and see if Directory Listing is enabled. It should be set to off if there is no business reason behind it.

Example:


One tip to testers is: Do not conclude on saying, "Directory Listing is Off" by just navigating to one directory. You must understand that, there is way to set On / Off to particulate set of directories. So, you got to have an automation solution if there are lots of directories and then run it and give a report of which directory has listing on and which is off.

5. Form server side tests - Unleashing the power of Firebug

I know that you always stopped when the form text fields,

a. Did not allow you to type more than 20 characters or any "X" number of characters and you thought it is the ultimate boundary.

b. Did not allow you to enter specific characters in the text field (Example: Asterisk symbol, @ symbol, less than, greater than symbols etc.)

c. And many other reasons because of client-side validation rules set by your Javascript or any Javascript framework like jQuery etc.

Now, that's one of the test that you did but, you did not go deeper and see if the client-side validation could be by-passed and submit the values to the database.

Here, I will show you, how you can bypass the text field boundary and enter more characters and submit the form. Well, if your application gives you an error for entering more characters after you by-pass client side validation, then it's good news; which means server side validation is being done. If the form is submitted with those many characters which are not intended by the business then, it's a bad news (No server-side validation implemented).

Here is how you can do it,

Install Firebug add-on in your Mozilla Firefox web browser (URL: http://addons.mozilla.org/)
Open your website and navigate to the form. Example text fields would be: FirstName, LastName, MiddleName etc.
Now, go to any text field and right-click on it. Choose "Inspect Element" option just like below,



You see that, I performed right-click on "E-mail address" text field and chose "Inspect Element with Firebug". Once I do that, Firebug opens and you will see something like this as below,



There is a maxlength attribute which is set to 100 which means; the client-side boundary is 100 characters and once you reach that, even if you type anything (Say 101th character), it doesn't get displayed.

Now, you can edit this by just double-clicking on the 100 value and just editing it to 300 or any other value or you can just make it NULL. Now, you will be able to enter more than 100 characters. After entering more than 100 characters, submit the form by entering valid data in all other fields of the form so that the test is successful. Wohoo! You learned how to do server side validation tests now. I hope that makes you feel proud. This could be done for any form in your web application. Now, your test coverage can give a different confidence to your customer and you as you include "Server Side Tests" in your test ideas.

6. Account Lockout Policy and CAPTCHA in Login

Can I use brute force mechanism on your web application's login and crack the password? Well, most of the websites I have seen do not have Account Lockout Policy or CAPTCHA shown after "X" number of invalid attempts. This will be a cool stuff for hackers who can exploit by deploying their brute force suite on the login to crack the password. To avoid this, you need to challenge the user with CAPTCHA after "3" or "5" invalid attempts to login using the same username. However; there is one more attack called as "Shot-gun style" (Let us not talk about this for a while).

You can try using wrong login attempts at Gmail and later you are asked to enter CAPTCHA so that you do not employ brute force tools to crack the password. That's how you can implement even for your web application login. Or else there is another way where you lock the account for 24 hours or so (This depends on the decision the business has taken). Locking of account after "X" number of invalid attempts is "Account Lockout Policy" (Policy will define the way of working when account is locked out). You can try HDFC bank internet banking where your account gets locked out if you enter 3 or 5 invalid IPIN.

Finally, I want to finish this article saying; "You got be explorer, if you are just ending up doing specific set of steps and stop there; then it is hard or impossible for you to gain that mind-set of hacking & skill-set of hacking".

7. HTTPS re-direction - Developer forgot to put a re-direction

First question would be where do we need HTTPS to be used? The answer is; wherever you think confidential data is being sent over the wire, you got to use HTTPS to encrypt the data and send over the wire. Now, let us say: Checkout web-page has HTTPS when you click on "Checkout" button. However; forcibly remove HTTPS and just use HTTP for the Checkout web-page URL. If the web-page loads checkout in HTTP web-page then it's vulnerability. Voila!

The fix for this considering security is, re-direct HTTP to HTTPS even when HTTP is used forcibly.

8. Token Expiry Test - Expire the token or you get expired *Grins*

Let us say, you are testing a web application where you have Forgot Password or Activation e-mail sent with a unique token URL which need to be used in order to proceed further. We will pick forgot password URL at moment, what you got to test is - Use your own e-mail address in Forgot Password. Once you receive the e-mail in your inbox / spam / junk / bulk folder, use the link and then you will be taken to your web application where you set new password. Now, after setting new password; re-use the same link and try setting the new password again. If you are able to set two new passwords using the same link then the token has not expired and it is security vulnerability. Hackers and attackers can just use brute force mechanism to generate the URL and use it.

If you don't kill the token on one use, then it will kill you. Disaster!

9. SSL Tests from SSLShopper.com

You can visit http://sslshopper.com/ and perform the SSL tests. Well, this works only for the web applications which are on the production and are accessible by public. You might want to explore and find out similar tools to support for your intranet based applications. Also, learn about SSL / HTTPS about their working and what kind of tests you can derive based on the learning.

10. Mantra Browser - http://getmantra.com/

Well, here exists awesome add-on(s) that you can use to perform tests and run automated checks for your web application. This really rocks and is one of my favorite ones.

There are so many web browser add-on(s) which will rock your day for sure in the area of "Security Testing" / "Penetration Testing".

6 Things You Should Never Say to a Software Tester



1. “Can you hurry up and finish testing soon? We’re under a very tight deadline.”
Good testing cannot be rushed. Ideally, testing should begin with the design phase of a project and continue all the way through. Software testing is ultimately about validating the design requirements, but to be effective at that, the testing process must begin with the design stage itself. When testing is started later, it can be less effective. Designers must sync with testers, developers must sync with testers, and valuable time is wasted getting teams and technology expectations to mesh. This is a recipe for delays, confusion, and budget overruns.

Starting the testing process with design lets testers be involved  from the beginning. They can contribute early, get to know the product early, and be involved early. That makes for better testing and that makes for a better product.

2. “Promise me there’s no bugs in this, okay?”
It’s been argued (correctly, in my opinion) that the role of testing is not “quality assurance” but rather “quality assistance.” I believe that phrase originated with Jon Bach, but don’t quote me on that. Basically, quality cannot be tested into a product, thus a tester’s job is not to verify that a product is error-free, but that it has passed (or failed) to meet a certain set of standards, which vary from company to company. Leave the outlandish promises to the marketing and sales departments. Keep testing in the realm of reality.
3. “You must be bored out of your mind.”
This is a huge myth of software testing, but one that’s easy to dispel. Go to a testing conference, read a few blogs and articles and you’ll quickly see how passionate testers can be with their craft. 

Here’s a nice quote from Michael Bolton that pretty much sums it up:


“Testing is something that we do with the motivation of finding new information.  Testing is a process of exploration, discovery, investigation, and learning.  When we configure, operate, and observe a product with the intention of evaluating it, or with the intention of recognizing a problem that we hadn’t anticipated, we’re testing.  We’re testing when we’re trying to find out about the extents and limitations of the product and its design, and when we’re largely driven by questions that haven’t been answered or even asked before.”
4. “I didn’t have time to read your bug report. Give me the 5-second version.”
Testers (the good ones, anyway) spent a great deal of time crafting their bug reports with precision, accuracy and steps to re-produce. When a developer (or other department) doesn’t care to understand the full severity of a bug, it not only wastes the tester’s time, but it can lead to huge problems down the road. The reasons for this should be rather obvious. Instead of the five second version, how about the five word version? Read the freaking bug report.
5. “You’re actually a very nice person. I don’t care what the developers say.”
I had to throw a little humor into the post. On a more serious note, the relationship between testers and developers is not always – how shall I say this – pleasant. As testing guru James Bach once wrote: “Anyone who creates a piece of work and submits it for judgment is going to feel judged. That’s not a pleasant feeling. And the problem is compounded by testers who glibly declare that this or that little nit or nat is a “defect,” as if anything they personally don’t like is a quality problem for everybody.” In other words, don’t stir the pot.

6. “Couldn’t a machine do this much easier?”
Depends on what they’re doing, but even if the answer to that question were “yes” who would manage the automation? Who would analyze the results? Who would choose the tool to use and decide where and when it’s going to be implemented? This doesn’t even touch on the need for manual testing. Machines can do a lot of things, but replacing testers is not – and never will be – one of them.

Qualities of a Good Tester



A good tester is one who knows all the techniques that is involved in testing an IT system.
But do you think being a knowledgeable tester will ensure success in this domain?  

A ‘good tester’ possesses certain qualities that distinguish him/her from a ‘bad tester’.


  • ·         A ‘good tester’ tests an application thoroughly

  • ·         A ‘good tester’ has the ability to explain all the requirements and errors in a clear and concise manner

  • ·         A ‘good tester’ is always curious and enthusiastic to learn more about the system

  • ·         A ‘good tester’ will have the eagerness to learn and keep themselves up-to-date

  • ·         Moreover, one should have a sharp and quick analyzing power along with good domain knowledge and most importantly the patience to do research and development on the domain


Take a look at the infographic to get an idea of the qualities that makes a ‘Good Tester’.


Monday, October 22, 2012

10 Benefits of Unit Testing



Some Benefits of Unit Testing:
1. Unit Testing reduces the level of bugs in production code.
2. Unit Testing saves you development time.
3. Automated tests can be run as frequently as required.
4. Unit Testing makes it easier to change and refactor code.
5. Unit Testing can improve the design of code especially with Test-Driven Development.
6. Unit Tests are a form of documentation.
7. Unit Testing forces you to confront the problem head on.
8. Unit Testing inspires confidence!
9. Unit Tests are a measure of completion.
10. Unit Testing is FUN!

Wednesday, October 17, 2012

Automation Basics

1. What is AUTOMATION

Automation is to execute your existing regression test cases without any human intervention. And your script should keep on updating stakeholders about the current status.

For good automation, manager and resources have to plan lot of things in advance, so that your automation suite is robust, maintainable and error free.

Following are basic points you should think of:

a. Automation framework installation in new machine
b. Proper documentation for each step needed by a QA professional to start suite
c. Easy to learn and easy to execute
d. Easy to add New Test Cases
e. Easy to Add new product
f. Easy to maintain Input data for different products
g. Good Logging for each test case, so that it can be used later if some error is found by automation.
h. Reporting to Stakeholder at each stage
i. Scaling of your automation suite


  • Increase in number of cases
  • Increase in number of system
j. Script to check performance stats and Accuracy Stats

2. Automation Effort

There is an easy way to get automation effort needed to implement system in use. All the different steps can be divided into 4 high level verticals (Plan, Design, Test, and Maintain).
Again it will depend how many automation engineers you have and how many machines you have to achieve your goal.
Think of PDTM when you want to automate any application.

PDTM stands for:


P: Plan
D: Design
T: Test
M: Maintenance


Below is the chart which describes 4 different levels of Automation:


2.1 Task Layout Plan:

1. Estimate Time
2. Define Scope
3. Create Test Plan
4. Form team, with different level of scripting knowledge on different tools needed in automation
5. Train team on different tools and different scripts integration
6. Finalize all features and test cases.
7. Create Final Schedule
8. Finalize on different environment your test case will run
9. Identify Test Data set.

Design:


1. Read Documents
2. Run Manual cases
3. Design Framework
4. Different feature and test cases can be selected as users will.
5. Review Framework
6. Design Reporting
7. Design Error Handling
8. Verify that different feature and test cases should be picked properly

Test:


1. Verify each test case
2. Verify test cases in batch
3. Run test cases with different dataset
4. Introduce defects to verify that your script catches those defects.

Maintenance:


1. Make Scripts version control
2. Make changes as and when feature changes
3. Maintain Script
4. Store script in Secure location
5. Script should be updated with latest release

Automation Effort Graph


Overall effort calculation may have the following components:-

1. Test Requirement gathering & Analysis
2. Framework design and development
3. Test Case development (incase the available manual test cases not compatible)
4. Script Development
5. Integration Testing and Baseline.
6. Test Management.

2.2 Roles and Responsibilities

Manager


1. Estimate Time of automation
2. Understand Scope
3. Create a Plan
4. Form Team
5. Train Team with the product
6. Finalize features to automate
7. Create a Schedule
8. Check Environment Availability
9. Identify Test Data Set to be used for automation

QA Professional


Understand Phase:

1. Go through documents of application
2. Install and execute few cases manually

Design Phase:


1. Understand / Design Framework

Development Phase:

1. Write Code for Framework
2. Write code for Reporting
3. Write code for error handling
4. Write code for important functions in library

Coding Phase:


1. Code case and test case
2. Do thorough testing on each script
3. Run you script for several Test Data set
4. Introduce defect and check whether your script catches

Maintenance Phase:


1. Make your script as version control
2. Make changes as and when feature changes
3. Maintain for new environment and DATA set
4. Store scripts in secure location
5. Script should match with Application Release