Performance Testing of Mobile Apps – How is this Different?

Below is a quick start guide on performance testing a mobile application.

To begin, learn how your end-users access applications on their devices and then establish an expectation from performance testing. Once users’ needs are identified, create a strategy document to simulate realistic load conditions.

Almost all performance testing tools available in the industry support performance testing of Mobile apps – native, hybrid or app accessed via browser from the device. Most tools come with out-of-the-box features to conduct tests irrespective of the communication protocol (HTTP or HTTPs).

As modern applications get complex and feature-rich, it is pivotal to build a clear strategy to overcome challenges based on the tool you choose.

Below are few prerequisites to help you develop a detailed test strategy –

• Type of app – native or hybrid; communication protocol
• Security features – almost all communication via devices are secured using security encryption logics
• Network conditions
• Virtual users, real user; distributed load
• Device-level (client-side) performance testing
• Server-side performance testing
• Types of testing & Load conditions (workload modelling)
• Business-critical scenarios & test data
• Availability of devices – physical & cloud-hosted
• Application monitoring
• Features that may need to be bypassed or disabled such as OTP/Captcha etc.,
• Test data & Load test environment
• Back-up/restore of assets

Once a good test strategy is defined, the phases that would follow will be planning, development, execution & Analysis.

Following are some of our recommend approaches for the above phases –

Scripting 

Choose a proxy recording feature provided by load testing tools. The steps are simple & straight forward. Set a proxy on the device & ensure both the tool and the device are on the same subnet. Once done, start capturing the traffic. If your application security features restrict you to capture the traffic, consider using packet capture tools from the play store or app store. Export the network capture to other tools. You may use JMeter, LoadRunner, Fiddler, Blazemeter, Wireshark or any packet capture tool to record the HTTP calls from the client to the server. You may also record each for Android and IOS.

If the application uses SOA, REST or Jason calls, seek help from the application vendor or developers to assist you in understanding the various HTTP request formats (HTTP GET, POST, PUT) and parameters. These would be useful when dealing with applications that do not allow you to capture traffic using various tools, for security reasons.

Script Enhancements: In this phase, test using parameters instead of hardcoded values. Check the need for changing the header file (If you prefer to simulate load from both Android & IOS). Ensure all dynamic parameters are correlated & properly handled to ensure the script works for multiple users for various test data.

One of the key parameters is to simulate realistic network conditions such as 2G, 3G, 4G LTE etc. Several tools have built-in features to simulate these conditions. Some tools allow you to make changes to the configuration file and so it’s critical to consider the right tool depending on your requirements.

Execution

In this phase, set-up a test environment using controller & agents to support the desired type of test & volume of users. If you are keen to simulate load from various geo-locations, set-up your load test agents to simulate similar behaviour.

We recommend running a few sanity dry runs for a few users to ensure that there are no issues with the implementation.

Ensure your end-users or BA’s access the application during peak load conditions to help you monitor application behavior from an end user’s perspective. Request your team members to perform a few businesses transactions and record their experiences. As most tools do not include rendering time or client-side processing in the response time, it’s a good practice to ask someone familiar with the application to perform a few transactions manually or run tests that work at the GUI layer.

We recommend involving all key stakeholders (BA’s, dev & infra) during the load test execution. Most issues or bottlenecks identified during execution are much easier to fix when all key members are involved during the formal load test runs.

If these processes are remotely coordinated, get into a conference call to ensure each layer of the application is proactively monitored.

You may integrate the APM – Application Performance Monitoring tool with load testing tool to get a detailed breakdown of transaction response times. Though this would be a time-consuming process, it is an effective way to discover application issues, which could be fixed immediately.

Analysis & Reporting

Spend a considerable amount of time in this phase to determine how systems behaved before load test, during load tests and post-load tests. We recommend you to seek assistance from DBA’s or the concerned team to help you gauge the number of transactions that were performed during the load test. The data pattern would indicate any performance degradation.

We recommend you to create separate reports for your target audience and stakeholders. The reason being, while your business team may be interested in end-user experience or response time during a peak load, the IT infrastructure team would be more interested to know about the server utilization.

Create detailed reports for people who can help you assess the system and can work with you in fixing bottlenecks.

So finally, how is this different?

If you are experienced in load testing of web-based applications, then load testing of a mobile application is no different. The major difference is only during script preparation. As you cannot install load testing tools on the device itself the proxy feature helps to record the HTTP traffic from devices and export to load testing tool.

The other notable differences are: –

• Network conditionssuch as 2G/3G/4G. Do not run your tests in LAN if you are keen on understanding the end-user experience.
• Device for recording– Instead of the browser use the device itself, as the header file (user-agent) in each device are unique and servers now understand requests originated from different devices by reading these header files.
• Device-level performance – Record the device level performance manually during peak loads. The functional testing tools that can help record can be integrated with load testing tools. At the least measure response time, CPU, memory & battery utilization per each step during the business transaction.