Mobile App Testing

The successful companies will look at current mobile market as something different and rethink their product strategy, taking a mobile first perspective. This begins by looking at the target market, the intention of the app and the risk profile. Plus, if anything, we are seeing mobile applications which have more features or more functionality than the web equivalent. We need to test at least the same as that web equivalent and usually more.

For example, an airline application requires check-in abilities on the web. But on the mobile application, you also need to have ticket presentation ability, baggage tracking and many more functions. So the mobile app testing for such an app would be different for an app from eBay.

How is Web Testing different than Mobile App Testing

The complexity of test has increased making the test matrix exponentially larger than it was before, specifically in mobile. For example, right now on the market, there are some 12,000 unique Android devices alone. The fragmented mobile market has created huge challenges for app developers.
They need to understand these trade offs in relation to the time-to-market goals and the quality tolerances of the consumer-base.

You cannot test it all because the number of different vectors that you need to cover with mobile are much greater than in traditional testing if you want to keep up with the market.

High quality no longer means merely the absence of bugs. Mobile apps are now measured against a user-centric, more 360 degree view of app quality, which from our perspective includes factoring in usability, performance, content and functional quality,as well as security. If an app fails to meet the expectations of users, it can often be detrimental to the entire brand.

Are there any industry specific examples that demonstrate a different focus on testing?

Many companies are trying to get new features out in the market. One of their marketing techniques is to provide free applications to consumers. They may sacrifice inconsequential bugs to be able to launch a new feature. If a bug is not going to impact significant number of people, they may or may not fix it.

However, certain industries, like banking, really cannot have any functional bugs that make consumers think they can’t be trusted. If there are security or even minor functionality flaws, it could lead to poor perceptions and damage that trust. Remember though, that the drive in the banking industry is for mobile applications.

This is supported by an Alexis Partners study that suggested nearly half of smart phone users that switched banks said that mobile banking was an important factor in their decision. That’s up from only 7% in 2010. Financial institutions feel they have to add new features and capabilities to differentiate themselves from other providers, but at the same time, they have to maintain an extremely high level of quality and security in their applications.


Keeping some of these factors into consideration, here are some of the things that you need to should consider when creating a test strategy for a mobile app:

Functional Testing

  • Validation of Functionality
  • Smoke/Regression Testing
  • Orientation changes
  • Network Access testing
  • Negative/Boundary condition Testing
Non-Functional Testing

  • Different Network Strength / Outage / Recovery testing
  • Different network types e.g. Wi-Fi, 3G, 4G etc.
  • Different hardware, peripheral testing.
  • Stopping activities and not killing the app.
Interoperability or Multi tasking

  • Voice/SMS interrupts
  • Notifications
  • Battery/Cable removal
  • Sending app in background/ Multi-tasking
Memory Profiling

  • Memory usage under different tests
  • Memory leaks after long usages or other situations like multi tasking.
Performance testing

  • Device CPU usage testing.
  • Network data usage
  • Page render times or Activity render times
Installation testing

  • New App install
  • Uninstall and reinstall
  • Upgrade
Usability testing

  • User experience
  • Expert reviews
  • Competitive analysis
Security testing

  • OWASP vulnerabilities
  • Dynamic testing
  • Static code analysis
  • Data encryption
  • Penetration testing
International-device support testing

  • Different locales
  • Different currencies, time zones
  • Images and text

Some of the areas may be more relevant than others based on the type of application under test. Obviously, being able to test everything on all devices can be a daunting task. So that’s where you would want to automate a lot of different areas.

Automation:

It’s a key to any testing strategy. But the questions are:

  • How do you create efficient testing automation which can run on multiple devices?
  • How do you create efficient automation which uses a minimal amount of changes between your versions?
  • How do you create automation which can be truly unattended for multiple devices?

The answers to these questions are very important when it comes to mobile. Practically speaking, they determine the level of efficiency that can be derived from your testing efforts and the coverage reach you hope to achieve.

In my future post, I would talk about different options available to automate!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>