FAQ on Mobile Application Testing

Tweet

Here are some frequently asked questions on mobile application testing and their answers

1) How is mobile application testing different from mobile testing?

When you are asked to do mobile testing, you must test the mobile phone itself. It may include protocol testing, battery testing, network connectivity testing, software compatibility testing and more.

On the other hand, mobile application testing is to test applications and software built for the mobile phones. In this case, the application needs to be tested to verify whether it meets the specified functional and non-functional requirements.

2) What are the different types of mobile application testing?

a) Functional Testing

b) User Interface Testing

c) Usability Testing

d) Performance Testing

e) Stress Testing

f) Compatibility Testing

g) Interruption Testing


3) What is the most important type of mobile application testing?

While you cannot neglect any of the above testing types; some of them definitely deserve more attention than the others. For example user interface testing must be done extensively to make sure that the application’s color schemes, consistency, menu styles, navigation etc. are perfect.

Apart from the listed ones, it is also important to carry out monkey testing of the application to examine the stability of application when continuous invalid user inputs are provided.

Another important aspect to mobile application testing is to examine how the application reacts when there is no or little network coverage.

4) What are the best ways to carry out mobile application testing?

Mobile application testing is not just about writing test cases and executing them. Below are some pointers that can help a tester test mobile applications in the most effective manner.
Explore and learn about mobile phones and their attributes. Domain knowledge will really help you see beyond the obvious.

Understand when, how and where the application will be used and then create test cases.
Study the mobile phones on which the applications will run and write appropriate test cases.
Use simulators as often as possible to execute the test cases.
Use remote device services (RDA) as well.

5) Can mobile application testing be automated?

Yes, it can. The fact that mobile application testing is gradually grasping the attention of IT industry may be the reason why several automation tools like TestComplete, M-Eux, Robotium etc. for mobile app testing are being released into the market. It is best to keep your options open and explore all forms of mobile application testing and choose the ones that best suit your needs.

Do you have any more pointers on mobile application testing? Please add on!

Heisenbug - An Elusive Bug

Tweet

An eluder by nature, Heisenbug, is a software bug that disappears when you attempt to study it. The name Heisenbug refers to the physicist Werner Heisenberg and his studies of quantum mechanics, the conclusion of which is that studying a system may alter its state. 

 

So, What Causes Heisenbugs?

 

Most commonly, heisenbugs are caused due to external factors and disturbances, over which a programmer has little or no control. Heisenbugs may come and go depending upon:

 

• The compiler with which the program is being compiled. An optimizing compiler may throw the bug, but a non-optimizing compiler might not.   
• Using non-initialized variables in the program. 
• Presence of an invalid pointer in the code. 
• Use of external interfaces while debugging a program might also lead to a Heisenbug. 
• Generation of random numbers leading to table overflow. 
• Un-flushed cache contents get underwritten. 
• Analog user inputs. 

 

However, Heisenbugs cannot be disregarded or neglected. They very much exist and need to be taken care of. It is best to eliminate all reasons that could possibly cause the random behavior of the bug and then study it.  

 

And, How Does One Handle Heisenbugs? 

 

Source:simonb.com

 

 

It is logically impossible to fix a defect if it vanishes every time you attempt to study it. So, the first step is to make it stay. To do so, below are some pointers: 

 

• Study the frequency of the Heisenbug. Make a note of the bug as and when it occurs.  
• Write down all the circumstances under which the bug shows itself.  
• Try to reproduce the bug (with the help of a tester).  
• Build a profile of the bug and keep adding more information as you proceed.  
• Try to learn the location and cause of the bug – some bugs occur only when compiled, some occur during testing while some occur in live environments only.  

 

Until you are sure that code alteration will fix the bug, don't do it. Because code alternation may just temporarily make the Heisenbug go away. Rather, eliminate chunks of codes by removing them and studying the effect on the bug. When a piece of code is missing, and the bug is still there, it clearly means that the piece of code is totally unrelated to the bug.  

 

Finally, it is important not to get totally absorbed in to studying of Heisenbugs. It is rather necessary for the developers to get acquainted with debugging methods of Heisenbugs, such as CHESS, Jinx etc. 

 

Have you ever encountered a Heisenbug in your IT career? Please share your experiences, thoughts and views for our benefit.  

 

Capability Maturity Model Integration CMMi In the Context of Software Testing

Tweet

In this post, lets learn about CMMi and how it is relevant to software testing

 

1) What is CMMI model?

 

CMMI (Capability Maturity Model Integration) is all about processes. It is a process improvement approach that examines your current processes in place and identifies their weaknesses and strengths. Then appropriate process changes, improvements and modifications are made to change these weaknesses into strengths.

 

CMMI model is a collection of set of very effective and reliable best practices that can help an organization improve quality, standards and efficiency. It consists of several process areas such as configuration management, project planning, etc. 

 

2) What is the difference between capability level and maturity level?

 

Both these metrics are closely related, and often rely upon each other. Capability level can be used to measure the attained improvement in individual process areas; maturity level can determine the achieved improvement in multiple process areas. 

 

3) Why is CMMI Model important? 

 

CMMI model is being widely adapted by organizations to prove their efficiency and reputation. Some of the advantages of adapting CMMI model are:

 

• It provides good RoI (Return on Investment).
• When it is adapted, business sees heighted success.
• It is compatible with other quality related methodologies such as 6-Sigma, ISO Standards and ITIL.
• CMMI model is always improving and evolving. 

 

4) How does it contribute to Software Testing?

 

 

When CMMI model is implemented: 

 

• It improves quality of software delivered.
• Customer satisfaction is increased.
• It helps in achieving targeted cost savings.
• The model also ensures stability and consistent high performance.

 

5) How can it be effectively implemented?

 

CMMI model does not only apply to processes. It applies to work groups, people, rules, teams and projects. To effectively implement CMMI in your organization, 

• Choose staff members who will be trained.
• Work on creating Engineering Process Groups.
• Consult with CMMI consultants.
• Implement quality processes and practices.
• Select appropriate tools.
• Make your staff learn quality assurance, configuration management and project planning.
• Introduce the CMMI model to a few projects.
• Get feedback and reviews from customers
• Learn and improve the practices implemented.
• Add on more projects under the CMMI model. 

6) What are the CMMI levels?

 

• Level 0 - Incomplete
• Level 1 - Performed
• Level 2 - Managed
• Level 3 - Defined
• Level 4 - Quantitatively Managed
• Level 5 - Optimizing

 

Cygnet Infotech, the parent company of Testing Whiz is CMMi Level 3 certified.

Is Software Quality Assurance same as Software Testing?

Tweet

Have you ever wondered whether there is actually a difference between Software Testing and Software Quality Assurance? Are these just two names to the same concept? Well, this blog attempts to answer your question and help you understand whether these two are synonyms, or actually indicate different areas in the area of quality.

What are they, really?

Software Quality Assurance is a systematic and planned approach to identifying, defining and modifying set of processes that can help one attain an error-free end product. Now that the processes are in place, implementing these processes and making them a reality to deliver an error-free end product is what testing is about. 

While Software Quality Assurance talks about creating and implementing methods and procedures to improve the overall development cycle; software testing concentrates on verifying and validating the product to discover bugs and locate defects.

The Fine Line

There is this distinct difference between quality assurance and testing which one needs to understand. Executing test cases and raising defects does not mean quality assurance is in place. It just means that testing is being done and attempts are being made to discover and fix as many bugs as possible in the product.

For Quality Assurance to become a part of the project, standards that the project will follow to deliver a high quality product need to be defined. Proper metrics must be defined to measure the quality of processes that will be followed by the project members to develop the product. 

It’s a ‘name’ thing

Sometimes, in some projects, Software Quality Assurance and Software Testing may be used interchangeably. There is nothing wrong in this. As long as the processes, activities and steps to ensure quality are clearly defined and specified, nomenclature doesn't really matter. 

There can be an overlap between quality assurance and testing in projects, especially small scale ones. Testing might provide important quality related information about the product, but quality assurance is what uses this information to improve the quality related processes. In some projects, an external Quality Assurance might be engaged to conduct an audit of the quality related processes. 

Conclusion

In short, Testing concentrates on the end product while Quality Assurance concentrate on the process.

There might be instances when testing is given a higher priority than Quality Assurance. This may be acceptable as sometimes when time and resources are a constraint, getting things done is more important than waiting for quality related procedures and standards to be in place. However, in the longer run, Quality Assurance must be incorporated since continuous improvement and overall success depend on it.

Thus, the ultimate aim of Quality Assurance and Testing is to deliver an error-free product that meets its functional and non-functional requirements. Consequently, efforts must also be put to improvise quality by learning from experience and with the help of a proper feedback loop.

Hypothesis Based Testing (HBT)

Tweet

Hypothesis Based Testing is a unique, scientific and personal methodology that helps the testers deliver clean software since it is defect centric and adapts a quality growth model. 

 

How is HBT Different from Traditional Testing?

As opposed to traditional test methodologies that focus on defining and implementing activities and tools that can help discover potential defects in the software, Hypothesis Based Testing focuses on identifying and defining the most probable defects first. 

Once the potential defects are identified, testing activities to ensure that these defects are uncovered and fixed are carried out. In case of traditional test methodologies, the ultimate discovery of defects still depends upon the experience of the tester. But with HBT, that is not the case. 

So, while traditional testing - 

• focuses on testing activities, 
• is driven by processes, 
• is powered by testing tools/utilities and 
• depends a lot on the tester’s experience to achieve the ultimate goal of finding defects; 

When it comes to HBT,

• the focus is on the goal of finding defects first and
• then, on the activities that need to be carried out to achieve that goal.

 

 

How does HBT work?

Take any software, the below are the guidelines as to how HBT actually is implemented.

• First identify the defects that may occur in the software. Define the cleanliness criteria for the software.
• Then define the places, loops or functionalities where the defects can be found.
• Also, write the other conditions that may cause the defect to occur.
• Based on the above, create test plan, test strategy and test cases.
• Choose a tool that can assist you with the defect detection. If possible, opt for sensible automation.
• Perform the test cases and record the results.
• Based on the results, certify the cleanliness of the software.
• Repeat the HBT and code fixing until the software is defect-free.

 

 

Advantages of HBT

• The probability of discovery of defects is high as testing activities are formulated keeping the defects in mind.
• Goal focused evaluation of the software is adapted.
• Higher and wider test coverage is possible.
• Effort can be prioritized with respect to product features by concentrating on the most critical areas of the product first.
• It is possible to define an objective quality growth model.
• The overall test assessment and product analysis is accelerated.
• The overall validation strategy is effective.
• The amount of defect leakage is reduced since most of the defects are found at the first/initial stage itself.
• Minimal/No defects are found in production/post-production cycle.
• The overall productivity is boosted since HBT helps in selecting an appropriate tool.
• HBT can be customized as per individual project requirements.

 

 

Summary

In short, HBT consists of the below six stages –

Stage I - Understand the expectations with respect to the software/product by grasping its features, attributes, business criticality etc. Align the expectations and objectives with business goals.

Stage II - Understand the context of testing in terms of project requirements and the cleanliness criteria.

Stage III - Formulate an appropriate hypothesis and potential defects catalog.

Stage IV - Devise proof with test strategy, test design and requirements traceability matrix. Also define measurements and a clear-cut plan of action.

Stage V- Identify and Select appropriate tools. Perform tooling benefit analysis.

Stage VI - Assess the software product and perform analysis based on testing results. Carry out test management and defect management activities.

Thus, HBT is defect centric testing which can help the test team prioritize the requirements and utilize the available resources in the most appropriate and beneficial manner.

The Four Quadrants of Agile Testing

Tweet

With Agile testing gaining importance to deliver high quality products that keep the customers content, it is imperative to understand the Agile testing quadrants and how it can help you perform Agile testing better.

These quadrants can help the whole team communicate better and deliver a high quality software product by adapting to changing requirements and developing in iterations. Another advantage of the quadrants is that they can explain the whole testing in common language that is easy to understand.

Agile Quadrant I consists of test cases that are technology driven and are performed to support the team 
This quadrant focuses mainly on internal code quality. The unit testing done in this quadrant acts as an initial safety net for the product and helps developer get the story right. By performing the tests in Q1, instant feedback about the product can be obtained. When Test Driven Development is adapted, design becomes better and stronger. The testability is build into the code in this quadrant. Very few bugs will be found after coding is done. The kinds of tests in this quadrant are –                                       
1. Unit Tests including testing a piece of code and ensuring that it fulfills the requirements.                      
2. Component Tests including testing architect intent and ensuring that components work together. 

Agile Quadrant II consists of test cases that are business driven and are performed to support the team
This quadrant focuses on eliciting the requirements. In this quadrant, developers can code as long as the test cases give the expected results. The test cases are business facing and so development is driven keeping the business objectives in mind. The objective of this quadrant is to obtain enough requirements so that coding can commence without any hiccups. An increased customer-tester-developer collaboration is possible in this quadrant. The kind of tests in this quadrant are – 

1. Testing of examples of possible scenarios and workflows.
2. Testing of User Experience such as prototypes.
3. Pair testing.

Agile Quadrant III consists of business facing test cases that critique the product
The main focus in this quadrant is to provide feedback to quadrants one and two. This quadrant enables one to learn as test progresses. The test cases can be used as the basis to perform automation testing. The product can be evaluated since realistic use of the product is kept in mind in this quadrant. The process can be changed as needed. The many number of iteration reviews in this quadrant builds confidence in the product and ensures the presence of a quick feedback loop. The demo can be given even on unfinished code to gain insight into the working of the product. Exploratory testing can be paired with the customer. The kind of tests in this quadrant are - 

1. Exploratory Testing
2. Usability Testing
3. Pair testing with customers
4. User Acceptance Testing
5. Collaborative Testing

Agile Quadrant IV consists of technology driven test cases that critique the product
This quadrant focuses on the non-functional requirements such as performance, security, stability etc. This quadrant is responsible to deliver the ultimately finished product. The application is made to deliver the expected value and non-functional qualities with help of this quadrant. Kind of tests in this quadrant - 

1. Non-functional tests such as stress and performance testing
2. Security testing with respect to hacking and authentication.
3. Infrastructure testing.
4. Data migration testing.
5. Scalability testing
6. Load testing

Summary
Depending upon the project requirements, scope, risks and priorities, the quadrants need to be chosen and implemented. Appropriate tools also need to be selected to assist and carry out the testing. Implementing the quadrants help in collaborating the efforts of programmers, testers and customers. Knowing when to use the quadrants and how to implement them together is to be decided by the test team. 

Testing Whiz assists French IT Company in Application Testing

Tweet

Application testing is a critical phase in any software product development activity and insufficient knowledge of proper testing methodology and approaches can seriously hamper the quality of the product. This is what a French software application development company realized that in order to ensure quality of product, it was essential to take assistance from a specialized software testing services provider.

Profile of Application to be tested
The application consisted of software development tools on technologies like PowerBuilder, PL/SQL, Transact-SQL, .NET and VB with the central functionality being that of role based access control for application users. 

The application adds security features to the other applications integrated with it. It provides user level security with easy access to the applications. It maintains user permissions and its roles to access different systems. It can integrate with various web as well as desktop applications. The application supports various databases such as Oracle and SQL Server 2000.

Challenges Addressed During Testing
Testing Whiz faced several challenges during the initial stages including testing scope being unclear, lack of functional documentation, and significant language barrier between English and French. However, Testing Whiz testers started with face to face meetings with the client’s team to get a better understanding of the application. 

Testing Approach and Tools
The test team comprised of one test lead, 2 team leads, 1 senior developer and a tester. A test plan was prepared which listed the test modules and test approaches to be used in the project. Proper test logs were defined to keep track of bugs and test results. 

The test team set up the tools for performing functional testing, regression testing, integration and system testing. Fogbugz was used as a bug tracking tool and Selenium for functional test automation. 

Benefits to Client
Client has been extremely delighted with Testing Whiz’s services and has been a client since more than 3 years. 

“Like many software companies in Europe, we face international competition, lowering our production costs, without degrading the quality of our products. That's why we started our partnership with Testing-WHIZ. We began by relocating tasks devices (but expensive) such as testing and documentation. This also enabled us to transfer our knowledge of functional software Testing-WHIZ. We then have them on developments. Testing-WHIZ's consultants are competent and well-organized. The few identified corrective actions were immediately implemented. The build-up is facilitated by the fact that we bill Testing-WHIZ work on time and assume the costs of inter-contract. We are thus passed from one person part-time, several people full time. Each euro is therefore a concrete result. In the end, Testing-WHIZ has always responded quickly and effectively to all our requests.”

Read the whole story