Selecting Mobile Devices for Testing

The selection of mobile devices is a fundamental part of our strategic test planning or test definition that tracks success and assures a high adaption rate amongst our end users.

Given the bond that the majority of people develop with their mobile devices and the applications they use to manage all tasks related to their job, home, health and environment just to mention a few, demand high quality standards in their apps that go a lot further than just having software free of functional defects.  If for example, the app isn’t appealing, or the usability isn’t adequate, or it has performance problems, users will forget about the app and won’t hesitate to uninstall it after one use and look for an app that meets their expectations.

Under this premise it’s important to understand that the mobile device selection for testing is a fundamental part of the process given that the performance and functionality can vary considerably depending on the mobile device tested.  This is because in the market we can find a wide range of devices with distinct processing capacities, screen resolutions, or even hardware pieces that should be integrated into our app.  For example, notifications via LED lights with custom colors.

Many times this task can get complicated, especially if we take into consideration the major defragmentation of devices, in particular, ones that concern the Android platform.

So, how do we assure ourselves that we are selecting adequate mobile devices for our tests?

Due to time and cost limitations, we could say that it is impossible to perform tests on every single device on the market, and for that reason we’ll present a series of tips in the next section that will help us make accurate decisions when selecting mobile devices for the testing process.

1. Review the technical specifications

The first step in selecting a mobile device is to review the technical specifications of the app.  If there are existing established restrictions like a minimal operating system version or a low screen resolution, then that would be the first criteria to reduce the list of eligible devices.

2. Know your user’s goals

Knowing the goals of the users using the app is of utmost importance.  For example if the app is targeted at millennials then for sure we will see that the majority of users will have high performance mobile devices that are new on the market.

3. Review the statistics of the most popular devices

Reviewing the statistics of the most popular devices, or better said, most used in the global and local market, will also help us trim the list of eligible devices for testing.  Websites like StatCounter constantly update statistics related to operating system versions, screen resolutions and amongst others, the most used mobile device models.  

4. Create a device matrix

With the information gathered previously, we can create a device matrix that contains information similar to the following:

5. Look for similar devices

Once the matrix is complete, the next step is to find mobile devices that have similar characteristics and begin to narrow the possibilities discarding devices that we consider unnecessary because their characteristics are already covered by another device in the matrix.

6. Generate the final list of devices

From the resulting matrix, we should prioritize the mobile devices to be used in testing.  These devices can even be prioritized by category such as:

• High performance devices
• Average performance devices
• Low performance devices

Even after having completed this exercise to narrow the list of possible test mobile devices we can find ourselves unable to complete full coverage for all the devices due to available time for testing.  In these cases, it is recommended to do 100% priority device test coverage and complete compatibility tests with the rest of the devices in the matrix.

In conclusion, the selection of mobile devices is a task that depends on the particularities and necessities of each project where we can’t define a fixed list of devices that indistinctly applies to all the apps we are going to test.  However, with the help of some tips like the ones described before, we can develop a consistent process that correctly selects mobile devices for testing.