It’s Never Too Late Until It Is: Making Most of Cloud Computing Performance Testing

Cloud computing delivers significant benefits to its users. It enables the scalability, speed, and agility crucial for delivering exceptional digital experiences. However, does this mean that you can gain all these advantages by simply moving to the cloud? No. Your cloud provider simplifies many details, of course. However, to leverage cloud possibilities to their fullest and not to be caught off guard by unanticipated challenges, you shouldn’t overlook a critical component: performance testing of cloud-based applications.

What happens if you don’t test your cloud performance?

What’s the worst that can happen if your cloud performance is not tested in an appropriate way? It depends, but the options are not promising at all. No more than six hours was enough for Facebook to lose about $65 million and taint its reputation — the company’s family of apps, including WhatsApp, Messenger, and Instagram, were unavailable all over the world. The problems that Facebook faced, no pun intended, or maybe, just a little, refer not only to social media giants. Even if your company is not that big and the downtime costs are not that tremendous, losing money and business opportunities is still frustrating and disappointing. 

So, by escaping the testing stage, you jeopardize your business in three ways: operational, financial, and reputational. 

1. Resources need careful monitoring; otherwise, you miss the chance to scale them appropriately to reduce costs

Without regular testing, you won’t be able to prove that your system is reliable and scalable. If you don’t find the bottlenecks in time, you won’t be able to make the right decision to evaluate and change your disaster recovery plan. And this is fundamental for minimizing downtime and the impact on the end-users of the app. 

Using cloud providers such as AWS, Google Cloud, Microsoft Azure, OVHcloud, etc. without understanding the limits of your application might be too costly due to the possibly unnecessary power. Knowing your systems’ “turning points” helps to understand the real capacity needed and, thus, choose an appropriate server. Without performance testing, you won’t find the infrastructure’s limitations, and, in the case of issues, you will most likely have to request a more powerful server from the hosting provider to boost the capacity. But it’s about paying extra money. 

Conversely, adopting performance testing as part of cloud computing allows a business to shorten the time to release new features and improve the system’s flexibility due to cloud providers’ development of new solutions. 

2. Downtimes happen more frequently, and you lose money

According to Dunn & Bradstreet, 59% of Fortune 500 companies experience a minimum of 1.6 hours of downtime per week. You can look for another statistic in the hope of consolation, but according to Gartner, the average IT downtime costs $5,600 per minute and this money could be used in a much more effective way. Earlier, in the article about the reasons for DevOps failures, we’ve mentioned the case of Knight Capital that went bankrupt in 45 minutes due to failed deployment. It’s a vivid example of when skipping the performance testing phase for the cloud-based application became critical for a business beyond the cloud migration process.

Downtimes might cripple your business, by allowing your competitors to outperform you. E.g., during the six-hour outage of Facebook and WhatsApp, 70 million new users registered in messaging app Telegram, wilfully abandoning the network in favor of quick communication.

Speaking of Facebook, testing is crucial for apps fueled by ads. 98% of its revenue comes from advertisers’ contributions. The outage of the system affects the company by losing advertisers. Also, the organization loses its employee confidence, as they may begin to think of finding a stable company to work in. 

However, it’s not only tech giants who suffer serious repercussions because of system failures. Imagine you roll out a large-scale advertising campaign right before the Christmas season, getting ready for an overwhelming flow of customers and unprecedented revenue increase, but your website or app crashes. Don’t let your opportunities, time, and money go down the drain and ensure a high level of stability for your system.

3. Poor usability pushes customers away, and, guess what, you lose money

The early 2010s, when users could close their eyes to app-related bloopers are long gone. Don’t cross-testing off your priority list if you don’t want to bleed money because of poor performance or suffer public embarrassment after a system crash. 
Thanks to performance testing of cloud-based applications, testers can set aside time for in-depth inspections in the security and accessibility of the app for end-users. Don’t miss the chance to make your app more user-friendly and, thus, improve customer experience, which ranks highly as the fourth priority for IT technology initiatives in companies worldwide from 2020 to 2021 after digital transformation, cybersecurity, and cloud migration. 

How to test cloud applications? It’s natural to choose cloud testing instead of in-house performance testing. In this case, you don’t need to pay for the maintenance of a whole infrastructure and pay only for the time of using the infrastructure resources, saving you money overall.

The cloud performance testing process in six steps

When you’re ready to switch from “Why do we need performance testing?” to “How do we implement it in our workflow?”, it’s time to follow this algorithm. 

1. Set testing objectives and choose the relevant types of tests 

Speed, scalability, and stability are three key parameters that you should check during testing. Finding bugs is essential, but you should also look for bottlenecks. You can’t see how quickly the app responds, understand the maximum user load it can handle, and check its stability under changeable workload without performance testing. 

Choose the types of tests to run first according to your goals: 

• Load testing in cloud computing is used to verify the system’s quality criteria. Is your system’s performance on the right level to be used by multiple users simultaneously? Find out with a load test in which testers induce a normal or expected workload to the system and observe its general behavior, throughput, and latency. 
• Stress testing is about a steadily increasing higher-than-expected workload on the system. To describe the application capacity, QA engineers need to know when, where, and how your system breaks.
• Spike testing is similar to the previous types, but it identifies how the app can handle rapid traffic overload. Such an approach helps take a fresh look at the usage of the elasticity of the cloud. 
• Failover testing in performance testing is needed to prove or disprove if the app can provide extra resources and replace a failed component under the heavy traffic load. At the same time, the end user’s experience shouldn’t be affected, and that’s the main focus here. 
• Scalability testing is a series of stress tests to measure if the system can translate additional resources into additional capacity. So, you can define and remove bottlenecks or just use more resources to temporarily fix scalability issues.
• Availability & Resilience testing is needed to check specific processes under the load. E.g., testers can audit if database migration, deployments, or automatic scaling environments are doable under variable workload levels.

Take advantage of cloud consulting with *instinctools experts to identify the most relevant types of tests for your business to run.

2. Design user scenarios

With the help of product owners and stakeholders, you should select relevant user roles for cloud computing performance testing. Based on this information, test engineers design user scenarios for the key user roles. E.g., for an e-commerce solution, you can define the roles of a guest or logged-in user, new buyer, or regular customer. According to this, you can use such basic scenarios as registration or signing into an account, searching for products and viewing detailed information on their pages, adding items to the cart, or reviewing previous purchases to focus the tests on.

3. Identify key performance metrics and design the tests

There are several essential metrics to track:

• Requests per minute

When should you boost the capacity of the cloud resources to use them most beneficially? To adapt your system to a changeable workload, you first need to define the cyclic increase of requests rate in different periods and predict the following increases. 

• Time to acknowledge

Knowing the time it takes for your system to respond to a request is necessary to uncover problems with load balancers. Also, a slow time to acknowledge may mark that the system is underprovisioned. Keep an eye on each cloud region since latency issues can differ from one cloud region to another. This approach will save you time and help pinpoint any issues. If the system is running well, you can go the extra mile and think about minimizing latency. For this, compare the time to acknowledge when a given request is handled and not directed by a content delivery network (CDN). 

• Response duration

 Being defined by load testing in cloud computing, this metric represents the time it takes the system to respond to a request and helps indicate if the application can handle the incoming workload. Apart from that, response duration can uncover issues with internal communication, such as the inability to communicate between different microservices inside the system.

• Error rates and types of errors

 You can use them to keep track of the application’s overall health and the cloud hosting environment.   

• Servers and nodes available

You should monitor the number of available servers within the total amount that you have. It’s better not to rely only on cloud orchestration and automation tools. Although they might be good at redistributing workloads from servers that crash to well-functioning ones, you need to ensure that the number of available servers doesn’t fall below 90% of the total deployed. Because if it does, there’s a problem with your cloud server instances that you can’t ignore. 

Note that you don’t necessarily need to constantly keep a check on all these metrics. But instead, focus on the ones most crucial to your business. 

4. Configure the test environment

Performance testing of your cloud-based application won’t be effective if the test environment isn’t as close to the real conditions as possible. At this stage, you should also arrange tools to run near-realistic tests.

Real users’ activity may change the test results and it would be problematic to identify reasons for bottlenecks, and, thus, eliminate them. So, don’t forget to make sure the test environment is isolated.

5. Run the tests

Run the tests, collect the data, and ensure that testing is a mandatory stage of every release of your app. It will save you money and your reputation in the long run. 

6. Analyze. Improve. Retest

Analyze the data to get insights into the robustness, availability, and scalability of the system. To identify bottlenecks, check the results in every iteration, and share them with your team to think of ways to remove barriers, improve performance and the system as a whole. Then, tune the test plan, change the application’s infrastructure if needed, and test again. Still have something to improve? Repeat the cycle.

The reliability of test results: how to ensure them?

There are three tips to be done:

• Make sure that test and production environments are as close to each other as possible. E.g., the test environment should include the same number of database records as the production environment. Think of a cloud-based test environment because, in that case, setting up a test environment is cheaper and faster — you can reduce costs and save time.  
• Take care of sufficient network bandwidth. Its low level can undermine the results of testing because of time-out errors due to user requests.
• Consider removing the proxy server from the network path. While it’s running, the users will get data from the cache and stop requesting the server. This situation can lead to a slower response time. It means you won’t be able to run the near-realistic tests. Cloud computing performance testing is a heavy hitter only if it gives you a valuable result. You can resolve this issue in two ways: transfer the web server to a secluded environment or strike directly to the web server (include server IP address to the HOSTS file).    

Cloud computing performance testing tools: *instinctools suggestions

The tools can differ according to their accessibility. Some are suitable both for small companies and enterprises, and others are targeted mostly at tech giants. So, be aware of your organization’s size and the most frequent types of performance tests. Here are the recommendations of our engineers to help you choose the tools that match your needs:

• Apache JMeter. With it, you can test Java-scripted and web applications for both static and dynamic resources. Consider this option if you need to simulate a heavy load on a server or a group of servers, objects, or networks to analyze its performance under various load conditions or test its sustainability and scalability. Due to the open-source format, this tool is accessible to software organizations of all sizes, from start-ups to enterprises. 
• BlazeMeter. It’s an enterprise-ready tool. Except for creating new load tests, it allows you to reuse existing scripts. To improve your software performance, use BlazeMeter’s detailed historical reports. 
• Gatling. This open-source tool for performance testing of cloud-based applications is able to simulate thousands of requests just with a few load generators. Gatling’s dynamic automated reports can be useful even for employees without data-oriented hard skills because the information is organized in an easy-to-understand way.
• LoadStorm. Test and manage the performance of your entire cloud infrastructure under excessive load. It’s useful for stress and failover testing. 

Take care of in-depth performance testing of cloud-based applications to secure your business

Here’s the thing with the digital era: performance is EVERYTHING. Whenever your application or a website fails, so does your company. It’s not enough to choose the proper tools for the successful implementation of cloud computing performance testing. Define the goals to select the relevant types of tests to run and metrics to monitor.