How to Choose an Automation Testing Framework

Tweet

Automation testing is a process in which manual testing steps are mimicked by a script that can be run automatically on a system to test its correctness. An automation testing framework defines a set of automation testing guidelines which when followed can increase the overall productivity and produce better results apart from saving effort and time from the testing perspective. 

 

 

There are many automation testing frameworks available to choose from.

 

 

 

 

Let us look at these frameworks in a little bit more detail.

 

Record & Playback Automation Framework

 

This is the simplest of frameworks where the tester manually records each and every test step such as navigation, user inputs and checkpoints that are to be performed in order to execute a test case. There is no coding involved in this approach, thus simplifying the overall process. Once the recording is over, the tester can playback the script that will automatically perform the testing.

 

Advantages

 

• No coding is involved thus generation of script is fast and easy.
• No technical expertise required.
• Great way to start learning the process of automation testing.

 

Disadvantages

 

• Test data is hardcoded into the testing scripts.
• Test scripts cannot be reused.
• Synchronization problems may occur in UI testing because of which a test script may not re-run.
• Manageability is difficult as the test area/scope widens.

 

Data-Driven Automation Framework

 

This approach has larger test coverage as compared to the record and playback method. In this approach, each test case is viewed as a function call to which data is fed from an external source. In the data-driven automation framework, test data is stored in a separate external file thus eliminating the hard coding of test data into test scripts. Thus, it is possible to run the same test case with different sets of test data.

 

Advantages

 

• Test data is separately maintained thus making it easier to make changes to the test script.
• Better test coverage possible by using different test data for the same test case.

 

Disadvantages

 

• It is not possible to test all the real-time business functionalities of the system under test.
• There is no easy way to specify which data file must be associated with which test script.
• It needs the tester to have some basic programming skills in the tool that has been chosen to automate the testing.

 

Keyword-Driven Automation Framework

 

This approach is also known as table driven testing framework, in which data tables and keywords are to be developed (independent of testing tool) for the test case. Each test script is viewed as a keyword prompted action and the test data directs the testing here. Thus when the automated test case is run, the test data is read which points to the corresponding keyword and then finally the relevant script is called/executed.

 

Advantages

 

• The testers can decide on which test cases are to be executed and in which order.
• This approach provides code reusability.
• The keywords are also reusable.
• The test scripts can be developed independent of test tool/application.

 

Disadvantages

 

• This needs the tester to have a good technical expertise in the field of automation testing.
• When the UI of the application changes, it is quite a challenge to update the test scripts accordingly.

 

UI Page Maps Automation Framework

 

 

This approach is by far the best of approaches since it solves most of the problems encountered while using any of the above-mentioned frameworks. The tester writes test scripts for classes of the individual UI page under specific window/page. Instructions are provided through test data, which recognizes the object on which action has to be taken which subsequently calls a script for the specific object.

 

Advantages

 

• The maintainability and reliability are handled quite well in this framework.
• The test script for a UI object can be reused anywhere in the application.
• It makes the development of test cases a tad easier.

 

Disadvantages

 

• When new UI objects are introduced, new test scripts may need to be written.
• Any changes to the UI object class will need revision of the associated test scripts.

 

The Hybrid Automation Framework

 

As in life, one size does not fit all. Therefore, the test manager needs to decide which testing framework is most suitable for the application to be tested. A hybrid approach may be considered.

 

As the name suggests, a sensible combination of two or more testing frameworks can be adapted to obtain the best results. Depending upon the application to be tested, the test data pool available and consistency of the application, the test team may chose to use more than one of the above frameworks to automate their testing processes.

 

Cygnet Infotech offers a wide range of software testing services to its clients. We have good experience in a number of automation frameworks. Read more about our automated testing services.

 

 

 

Software Testing Life Cycle

Tweet

Software testing is not a single activity but a series of planned tasks that need to be executed along with the software development activities to ensure that a product is delivered without any errors. This post aims to explain the overview of steps in a Software Testing Life Cycle. Take this as a simple guide to plan your testing activities.

 

 

 

 

  

Requirement Analysis

 

In the Requirements Analysis phase,  

• You start by analyzing the business functionalities and business rules by studying the requirements document and application architecture.
• You then identify the modules, transactions and user roles associated with the system to be tested.
• You decide the tests to be performed and assign priorities to them. These details are put together in the Requirements Traceability Matrix (RTM)
• You may also perform automation testing feasibility (recommended). 

 

Test Planning 

 

Once the Requirements Traceability Matrix is finalized, you can proceed with planning activities to start the testing. 

• You will first evaluate various testing methodologies available in order to decide which approaches suit the system best.
• Test Strategy/Test Plan document need to be prepared. This includes the test approach, resources, test management tool, defect management tool and schedule details. 

  

Test Case Development 

 

With the test strategy document and requirements specification document in place, you can begin to write the test cases.

 

• You create test cases and automation test scripts. An appropriate test management tool can be used if possible. It is important to plan for automation right at the beginning since it will help you save a lot of time in testing as the product develops.
• You then review the test cases
• Finally, creation of test data for all the test cases is done.

 

Test Execution

 

Once the test cases and test data are ready, the next thing to do is execute the test cases and log the results. It is essential to set up the test environment and then begin testing activities. 

• Understand the required architecture to carry out the testing.
• Set up a test environment by getting the system ready with all the requisite hardware and software in place.
• The networking and connectivity needs are taken care of with the help of network management staff.
• Finalize the test data that will be used for testing. Once all is set, a run of smoke testing is carried to ensure that the basic functionalities of the system works as expected. Depending upon the smoke test results, you then accept or reject the application. 
• If the smoke test results are good enough for the product to be tested, you can commence the planned testing activities.
• The test execution results are logged in to the test management tool and the defects are logged and passed on to the development team. A separate defect management tool can be used or you can use the built-in defect logging functionality of your test management tool. 
• When a defect is fixed, you need to retest the function and either closes the defect or reopens it. You are responsible to track every defect to its closure.
• With every new release of the software, you must do a round of regression testing to ensure that no new defects are introduced into the system.

 

Test Cycle Closure

 

With this, the end of software testing life cycle has been reached. You have successfully completed the testing, results of which are recorded and logged.

• You now prepare test metrics report with the details including software quality, test coverage attained, critical business objectives and cost-time factor. 
• Defect report must be prepared which gives a chart representation of all the defects by its severity and/or type.
• You are responsible for creating a final report about the system that was tested to help the client take a go live/no live decision. 
• Test closure report must be signed off by the client to officially call an end to the testing cycle. 

Remember that it is essential for you to start the software testing activities alongside the development activities, rather than waiting for the development cycle to end. It minimizes the overall cost and effort that is spent on both testing and development activities.

Cygnet Infotech has several years’ experience in performing software testing for its clients. We adopt a comprehensive testing methodology which is designed in conformance to standards set by ISO 9001 and CMMi Level 3.

Understanding White box Testing and Black box Testing Approaches

Tweet

Software testing is one of the best means to affirm the quality of software and deliver an error-free application. Over the years, software testing has matured into a separate discipline giving way to several different testing techniques that have been introduced, analyzed and studied in this area. Black box testing and white box testing are two such testing approaches that are quite commonly used by software testers.

What is Black Box Testing

Crudely put, when the tester has no idea of the internal working of the system which he is testing, that approach is called black box testing. In this case, the system under test is viewed as a “black box”. 

Requirements Document or Functional Specification Document forms the basis of this testing, which requires the user to understand the processes within the software.

 

How to write Test Cases for Black Box Testing

• The tester examines requirements and specifications of the system. 
• The tester explores the system’s UI and functionality to understand how the processes on the system are expected to work. 
• Tester designs test cases with valid inputs and the corresponding expected outputs. 
• Tester also includes some negative test cases with invalid inputs and expected outputs (error messages/program termination) as applicable.

Techniques of Black Box Testing

In case of black box testing, inputs to the test cases are the driving factor. Any one of the three techniques discussed below can be used to choose the inputs during the black box testing process.

• Boundary Value Analysis: This approach is focused on testing the boundary values associated with the system. This approach aims at testing the boundaries of the input domain that have the highest probability of giving erroneous outputs.
• Equivalence Class Partitioning: In this approach, a limited set of functions is identified along with its corresponding valid and invalid inputs and expected outputs. This approach aims at identifying classes of errors and therefore reducing the number of test cases required.
• Error Guessing: An experienced tester most often uses this approach to first identify the defects and then develop corresponding test cases.

What is White Box Testing

In white box testing methodology, the tester has the knowledge of the internals of a system and knows how the system is implemented. The tester uses this knowledge to develop test cases that will examine the control flow, information flow, data flow, exception and error handling as well as coding practices of the system.

 

How to write Test Cases for White Box Testing

• The tester analyzes and understands the structure of the system by examining its code.
• The tester understands the weak spots within the code that is most prone to defects.
• The tester develops test cases to cover individual data/information/ control flows and branches within the code.
• The tester also develops test cases to test proper working of all the functionalities and error handing of the system.

Techniques of White Box Testing

When it comes to white box testing, the knowledge that the tester possesses about the system is the driving factor, which helps the tester to devise test cases aimed at discovering defects with the internal working of the system.

• Statement Tests: All the statements within the code must have a test case associated with it such that each statement must be executed at least once during the testing cycle.
• Decision Tests: All the decision directions must be executed at least once during the testing life cycle.
• Branch Condition Tests: All the conditions in a specific decision must be tested for proper working at least once.
• Decision/Condition Tests: All the combination of the possible conditions within a specific decision for all the decisions is to be tested.
• Data Flow Tests: This will ensure that all the variables and data that are used within the system are tested by passing the specific variables through each possible calculation.
• Multiple Condition Tests: This will ensure that each point of entry with in the code is tested at least once during the testing life cycle.

What is Grey Box Testing

This is an approach that lies between white box and black box testing. In this, the test cases are written with the knowledge of internal source code of the application. However, the testing is done at the interface level only.

Cygnet's Services

Read more about Cygnet Infotech's Software Testing Services. If you have a testing requirement, please leave a message for us.  

About Testing Whiz

Tweet

Testing Whiz is a provider of software testing services. This is an initiative of Cygnet Infotech, Ahmedabad, India.

Visit Testing Whiz