This article will be useful to everyone who is involved in the development process, both from the agency and from the client: managers, designers, developers, testers. It covers the main features of testing mobile applications. In my comments, I will reveal in more detail the nuances of each type of testing, as well as give several examples of use.
One of the following types will suit you, depending on what purpose you are pursuing:
- functional testing;
- performance testing;
- security testing;
- usability testing (usability testing);
- compatibility testing;
- recovery testing.
Functional testing of mobile applications usually covers user experience testing as well as transaction testing.
Factors important for this type of testing:
- The type of application defined by its business functionality (banking, gaming, social media, education).
- Target audience (user, company, educational environment).
- The channel through which the app is distributed (for example, App Store, Google Play, or direct distribution).
In simple terms, we check if the application performs the expected functionality, which is usually described in a specification or dictated by business processes.
Therefore functional testing can be done based on requirements. In this case, test cases are formed, for their creation, a technical task based on business processes is used. After that, so-called use cases are created. They describe scenarios for daily or continuous use of the application.
Basic functional test scripts:
- Check the correctness of the required fields.
- Make sure required fields are displayed differently than optional fields.
- Make sure that the application runs at startup / exit time meets the basic requirements.
- Make sure the app goes into the background in case of an incoming call. To do this, you will need another phone.
- Check if the phone can store, receive and send SMS messages while the application is running. To do this, you need another phone from which you can send a message to the device under test with the application already running.
- Make sure the device is multitasking when needed.
- Check how the necessary options for working with social networks function – Share, Publish, Navigate.
- Make sure that the application supports payment transactions through payment systems Visa, Mastercard, Paypal, etc.
- Check the adequacy of the page scrolling scripts.
- Check if there is proper navigation between important application modules.
- Ensure that the number of round-off errors is minimal.
- Check for error messages such as “Network error. Please try again later ”in case of network malfunction.
- Make sure that the installed application does not interfere with the normal operation of other applications and does not eat up their memory.
- Check if the application is capable of returning to the state it was in before being suspended (for example, a hard restart or a system crash).
- The installation of the application should proceed without significant errors, provided that the device meets the system requirements.
- Make sure that the automatic launch of the application works correctly.
- Check how the app works on all devices of 2G, 3G and 4G generations.
- Perform regression testing to identify new software errors in existing and already modified areas of the system. Additional conduct of all previous tests to verify the behavior of the program after the changes.
- Make sure there is a user manual available.
The system often has a large number of functions, and it is not always possible to test all of them. Therefore, before starting functional testing, they usually prioritize certain test cases and use cases, allocate time in accordance with the priorities and then pay attention to the most important ones. It is quite difficult to select some standard scenarios for functional tests due to the variety of applications, but you can select common modules, compose test cases for them and use them in the future, modifying them for specific requirements.
For each function, both positive and negative scenarios need to be tested. The scenario is considered positive if the user eventually reaches his goal (creates an item, sends a message, etc.). Negative, respectively, on the contrary – at some of the steps an error occurs, and the goal cannot be achieved.
For example, consider login / logout and creating a contact (section, user, or any other item). Standard login / logout may include options:
- registration: with a login and password, without a password, via social networks, etc .;
- authorization: with login and password, via social networks, etc .;
- password recovery;
- logout: independent, after session expiration, etc.
- Registration in the application is available in all ways described in the TOR.
- You can register by filling in only required fields.
- You can register by filling in all the fields completely.
- After registration, you can log in to the application. In this case, the entered data is correctly saved in the profile (e-mail, password, personal information, etc.).
- After registering on one device, you can log in to another – the data is correctly saved on the server and available.
- Logging out is working correctly.
- Password recovery works correctly.
Negative scenarios (most obvious):
- Repeated registration to the same e-mail, with the same login is not available.
- Registration without filling in the required fields is not available.
- Registration, if all fields are left blank, is not available.
- Registration is not available if the format of the entered data does not meet the requirements.
- Authorization with empty fields is not available.
- Authorization with wrong / deleted / blocked login is not available.
- Wrong password authorization is not available.
It is logical to assume that if a user creates a contact, then it should be possible to view, edit and delete it. This is the basic set of functions that item can have.
Create, edit, view and delete contacts are available.
Creating a contact with a minimal set of data is available.
Creating a contact with the maximum data set is available.
When creating, all data types described in the TK are processed correctly.
Once created, the contact is available for viewing.
The change takes into account required fields / data / elements. It is not possible to save a contact without them.
Once deleted, the contact is no longer available.
Creating two identical contacts is not available (this could be a positive scenario as well).
Contact creation with missing required elements / data is not available.
Here, as functional testing, I will include checking the user interface:
Checking screens for matching layouts.
Testing the work of “native” gestures: swipe, multitouch, etc. – the application must react to them in a certain way.
Checking element states: buttons change color if pressed; lists collapse and expand, etc.
Localization check, if such is stated in the application. It is important to pay attention to the layout – many names in other languages are much longer.
Also known as load testing. This is automated testing that simulates the work of a certain number of users of a shared resource.
- Determine the number of users who can simultaneously work with the application.
- Check how the application behaves when the intensity of some operations increases.
- Check the performance of the application with many hours of use at an average load.
- Check application behavior under stress conditions.
- Check the work in the conditions of the “grown” database – how quickly the queries are executed.
The main purpose of this type of testing is to make sure that the application works acceptable under certain performance requirements: access to a large number of users, elimination of an important infrastructure element, such as a database server, etc.
The main scenarios for testing the performance of mobile applications:
- Determine if the application performs the same under different network conditions.
- Determine if the current network coverage is capable of supporting the application at various levels of user load.
- Determine if the existing client / server configuration provides optimal performance.
- Find various application and infrastructure bottlenecks that are slowing down application performance.
- Check if the application response time meets the requirements.
- Assess the ability of the product and / or hardware to handle the planned load.
- Estimate how long the battery can keep the application running under the planned load.
- Check the operation of the application in cases of switching from a Wi-Fi network to a mobile 2G / 3G network and vice versa.
- Verify that each of the processor memory levels is performing optimally.
- Make sure that battery consumption and memory leaks are within the normal range, and that various resources and services, such as GPS navigation or the camera, are working properly.
- Check the durability of the application under severe user load conditions.
- Check network efficiency while the device is in motion.
- Check the performance of the application if it works with an unstable internet connection.
It is a testing strategy used to verify the security of the system, as well as to analyze the risks associated with providing a holistic approach to protecting the application, attacks by hackers, viruses, and unauthorized access to confidential data.
The main purpose of this type of testing is to ensure the security of the network and application data.
Following are the key steps to check the security of a mobile app.
- Make sure that the data of application users – logins, passwords, bank card numbers – are protected from network attacks of automated systems and cannot be found by brute force.
- Ensure that the application does not provide access to sensitive content or functionality without proper authentication.
- Ensure that the application’s security system requires a strong password and does not allow an attacker to get hold of the passwords of other users.
- Make sure the session timeout is adequate for the application.
- Find dynamic dependencies and take steps to protect these vulnerabilities from attackers.
- Protect the application from SQL injection attacks.
- Find cases of unmanaged code and fix its consequences.
- Verify that certificates have not expired, regardless of whether the Certificate Pinnig application is using or not.
- Protect your application and network from DoS attacks.
- Analyze data storage and validation requirements.
- Provide session management to protect information from unauthorized users.
- Check all cryptographic codes and correct errors if necessary.
- Ensure that the business logic of the application is protected and not vulnerable to external attacks.
- Analyze the interaction of system files, identify and correct vulnerabilities.
- Check protocol handlers (for example, whether they are trying to reconfigure the default landing page using malicious floating frames).
- Protect the application from malicious attacks on clients.
- Protect the system from malicious injections while the program is running.
- Prevent possible malicious consequences of file caching.
- Prevent unreliable data storage in the device keyboard cache.
- Prevent possible malicious actions of cookies.
- Ensure regular data security controls.
- Examine user files and prevent their possible malicious influence.
- Protect the system from buffer overflows or memory integrity violations.
- Analyze various data streams and protect the system from their possible harmful effects.