Last year, Glastonbury Festival made the news after its 118,200 standard tickets sold out within 30 minutes of going on sale. With the festival trending on Twitter as the tickets were released, the ticketing site, See Tickets, crashed due to the volume of traffic received. I was one among the many hundreds of thousands of online visitors attempting to get on the site, only to be met with slow loading pages which had often timed out by the time I got to them.
This situation is of course an extreme example, and we only expect some of the most popular websites to experience this level of traffic on a regular basis. However, new features and products, marketing campaigns, and annual events such as the festival could all cause spikes in traffic to any site. And so, to ensure your promotion isn’t missed because people can’t access your site, you need to be confident that it can withstand high numbers of users and still provide the optimal experience to every visitor. This is where load testing comes in.
Load testing means that we put demands on a software system/computing device by simulating high levels of traffic, in order to measure response times. But what information exactly are we gathering?
A good load test is well-planned with clear goals. We want to write user journeys which accurately recreate expected user behaviours, then run these scenarios on the site with varying numbers of concurrent virtual users. Of course, we always strive to create a situation relevant to typical site conditions – so if your site gets a few thousand visits at peak time, then there’s no need to test with a million virtual users.
For example, we might compare site performance when facing 200 users in 10 minutes against 200 users in 30 minutes. By ensuring the user journeys are accurate and realistic, and by pre-planning the scenarios against which we want to test, we can make sure that the data gathered will provide valuable answers to our questions – not just a lot of fruitless information.
Following successful load testing, we analyse the data gathered and put together a report which clearly shows the site’s performance capabilities and, from which, improvements can be made as required. Rather than stopping here though, we’ll run the same load test again once the improvements have been applied, to confirm whether there has been a positive impact, i.e. that the performance of the site has been enhanced. By keeping testing lean and iterative in this way we aim to answer and solve the most important questions – only conducting tests that add value, to ensure we get back the data that we require.
Throughout the process, communication between project team and client is vital – to ensure that we are testing on the site at the most convenient time; usually when it receives the lowest recorded site traffic. Ideally, however, load tests will be completed on an isolated site which has no other users, in order to most successfully control the environment and data received.
Constant quality checks along the way are crucial, after all starting a test which will error not only wastes your time but also your money. A load test that doesn’t have a well-planned set-up, schedule and documentation process will more than likely fail, and not give you the information you need to measure your software’s performance.
We use a handy tool called Load Impact which is both flexible and simple to use (on the site you can watch a free load test to see how it looks, and what data is generated), taking the data outputted and manipulating it to generate a series of clear graphs and tables.
Below is one example of this output, where for each minute of the test you can see the average page load time versus the maximum page load time, alongside the number of active users and erroring pages. You’ll see that as the amount of active users increases the max page load time becomes more sporadic. This indicates that the software is struggling to cope with the increased capacity and that the issue should be investigated.
An example of the graphs we generate at Box UK using load test data
If you’re concerned that your software is running slowly, or you’re about to release something new which you know will draw more users to your site, we can work to resolve these issues and safeguard your success. Load testing is a key part of our standard quality and testing processes but we can also look at such issues in isolation as we did for Thinkbox. Either way we can pinpoint the specific areas where your software may be having performance problems so that the issue can be investigated and solved as quickly as possible. Simply brilliant.