In today’s world, achieving high levels of customer satisfaction is no simple task. Any small hiccup in application functionality, performance, or usability can affect the software’s reputation and result in lost customers and lost prospects.
In many cases, software companies testing their product invest a huge amount of effort and resources in testing the functionality of the product and its features. Performance and usability testing, however, get less focus, even though they have a similar or even greater effect on the end user.
Ironically, it’s the performance and usability that ultimately makes the difference between a successful product and a failed one.
Contrary to popular belief, performance and UI/UX testing does not require a large effort or even a different mindset.
If the testing environment is continuously being distributed with production data, all testing activities being carried out are already contributing to the performance effort. And every effect that the data load will have in production can be easily found and prevented in testing or even DEV environments. The key is to connect production and development for an optimized testing process.
Use Case: Forex Trading Room
A good example of linking production and development is a foreign exchange trading room that also supplies an electronic trading platform. Just like other Forex platforms, it has thousands of users: internal traders and analysts, marketing and sales personnel, and clients. The platform manages millions of dollars in different currencies and thousands of transactions every month, and even a small glitch can cost millions of dollars to clients or the trading room.
A Forex trading platform must be tested thoroughly before every new release, and the only way to account for “real-life” conditions is to load real production data to the testing environment in order to take it a step closer to production. The main data types that affect the behavior of the trading platform and that Dev/QA need to posses in their environments are the transactions, customers information, billing (depositing and withdrawing money) activities and audit information of all activities in the system. These data types give R&D a deeper understanding of the user experience, but in order to do so, the testing database must contain a replica of the relevant data.
Having said all of that and understanding the importance of the connection between production and development for proper production data testing, it’s equally important to understand that managing production data outside of the production environment requires addressing security loopholes and managing risks.
Even though distributing production data to other environments is not a hard task to do and can be achieved using a simple duplication process, when put to practice, this link between production and development requires planning, the right resources, and understanding local security requirements.
In the case of a foreign exchange trading room, the trading platform is designed with a data distribution process that takes into consideration the relevant data and possible hazards as and also improves focus on critical data that is relevant for the product’s quality.
What Data Is Required in Testing Environments?
Connecting production and development comes with additional caveats. It’s clear to everyone that copying ALL production data without singling out the data affecting tests is not the best practice. Unnecessary data will burden the environment, the testers, the copy process and, worse yet, will get exposed to unauthorized eyes.
Data masking, which will be discussed later, is a method of creating a structurally similar but inauthentic version of the organization’s data so as to protect sensitive information.
Understanding different data types and deciding on what to copy and how it affects the testing environment is crucial as it impacts the success of the process. For the Forex trading platform, the load of the transactions and their information, customer data, and audit of prices in the right mix allows the platform to behave as in the real-life environment. Other data types do not create added value to better simulate the production environment.
How Often Should You Replicate Data Pre-Production?
When addressing this question, there are several approaches to consider. The always up-to-date approach pushes for daily updates, which will keep QA closest to production and help recreate a real customer experience. On the other hand, there’s also the approach of bringing data from production to QA just one time for every release — before the stabilization period begins.
This second approach lowers the cost of the process while still allowing QA to test features and regression on production data. Another common approach taken in connecting production and development is a mix between the two, where data is copied ad-hoc based on the input of QA (whenever they determine that the environment is not up to date and its behavior differs significantly from production behavior).
No one approach is necessarily superior to the other. The right approach should be determined considering the resources allocated to QA, DB size and capacity, process cost, and other variables that affect the data managed in testing and the operational fluidity between production and development.
What About Resources?
Database capacity is never a question with production, but a QA environment does not share the same luxury. When QA tests to certify the data volume, it requires a higher amount of data capacity than is currently in production. Accordingly, it also requires added resources to accommodate that capacity.
It is therefore critical to test environment resources when planning the production data distribution process.
Interchangeable Production and Development: What Can Go Wrong?
Storing production data in less secure environments such as QA/staging environments produces many vulnerabilities. In the previous section we mentioned local security requirements. Today, almost every business is a universal business and has users spread around the globe. This means that when securing user information, every country’s personal information rules and regulations must be studied and addressed.
There are many ways to ensure personal information will not leak. Customer information like email addresses, phone numbers, names, passport numbers, bank details, CC details, IP addresses, signatures, and official documents must not be accessible or even present in the test environment. Achieving this requires an understanding of the information types and what has the potential of creating data exposure.
After understanding what information is important to disguise, there are several best practices that every organization needs to pursue. These include:
- Replace sensitive data with fake data. The main reason for getting data from production to testing environments is to get a similar load of data. Usually, once the data load is in place, it’s authenticity is less important. A DB procedure that runs through all sensitive information fields and substitutes the data with fake data using a scrambling algorithm or masking can help disguise vulnerable information.
- Maintain a one-way connection from production to the QA environment. There’s a well-known story about a programmer in his first week on the job who was working on a testing environment with credentials identical to production. He was asked to execute a short script on the QA DB and ended up executing a delete table script on production. All security information must be managed and modified so it will not be exposed to non-production individuals.
- A lot of data is stored in the environment audit tables. It’s a good practice to replicate audit tables to testing environments to simulate real DB load, but audit tables also contain sensitive data which must be screened and go through a censorship process before becoming available.
- Know who’s looking at what data, even in a QA environment. Even though sensitive data was replaced and there should be no valuable information in the DB, it’s still important to manage data access permissions so that even if vulnerable data is exposed, the exposure can be controlled.
Another dangerous pitfall of having production data in testing environments is related to the product communication systems. If users get notifications, such as emails, text messages, or any other form of communication, the notification system must be turned off so production users will not receive notifications on actions and activities done in the QA environment.
The Hawaii ballistic missile alert incident represents a perfect case in point on how bad it can get. The same restriction should be applied on integrations with external systems. If the external systems’ connection information is replicated to QA environments, operations can generate a call to a production environment of the external system and contaminate it with false data.
Summary
Testing is crucial for the development process. And testing with production data to get a realistic sense of customers’ usage elevates QA validation and verification to a whole new level. Rather than creating a full duplication of the whole production DB, it’s important to pinpoint the data that is relevant for the testing activity or the feature being tested and fetch the specific data type required for that test.
Testing a new feature with real production data allows for and is sometimes required of QA to authenticate the functionality and experience it the same way the customers will. Testing the performance of software and defining a threshold for concurrent users or data volume requires data generation; and duplicating production data into testing environments is the most common way of generating such data.
When testing the UI and UX of a product, it’s crucial to have the volumes and the same extent of data as customers have so that the effect the software has on users can be reproduced.
The link between production and development must be carefully set up and handled. Duplicating and protecting this data must be considered a standalone project and should be managed and tracked in order to maintain the security and protection of the data that is so vital to the organization.
