EСommerce Performance Testing: How to Test an eCommerce Application With DogQ

Nowadays you can hardly find a person who doesn’t shop online. ECommerce industry offers a huge variety of applications, platforms, and e-stores of all kinds in order to satisfy growing customers' needs, which include a user-friendly interface, intuitive site navigation, quick product image loading, multiple payment options, deep personalization, and customer support, as well as timely shipment, and a system of cashback and bonuses.

All these features must work properly, as a good eCommerce platform is key to overall retail business success. Since these applications include so many vital features, it is critical that it undergoes thorough testing.

Today we’ve compiled all our professional advice and expertise on eCommerce testing into one detailed guide to help you test any eCommerce website successfully. It consists of two parts: the first one will tell you about the concept of eCommerce testing, its types, techniques, eCom apps core features to be tested, as well as our tips to do it right.

In the second part, we’ll proceed with our own detailed example of using DogQ for Otomate eCommerce system testing and then provide you with a testing checklist, which is a short summary of the article backed with handy advice. Here we go!

ECommerce Automation Testing: What Is It?

First of all, let’s answer the question: what is eCommerce testing? It’s a type of testing that involves reviewing the eCom website or app for any issues before it goes live and is open to the public, as well as after adding new features to the site based on customer feedback. It requires the use of multiple testing techniques to cover all eCommerce app features like catching design rich for images and links, intuitive navigation, security, functionality, etc.

So, what advantages can eCommerce testing bring to your shopping application? There are definitely more than a few, so we enlisted all of them below:

• Makes your e-store more tested, user-friendly, intuitive, and efficient;
• Improves the overall user experience and helps to eliminate any related bugs;
• Reduces the risk of the site crashing, data loss, security breaches, and other unforeseen situations;
• Helps to attract more customers as the interface is becoming smarter and simpler;
• Makes it possible to evaluate the incoming traffic, shape the sales strategy, and increase KPIs.

Thus, the eCommerce testing process is beneficial not only for the technical side of the application but for business as well, if only you implement the right testing strategy and check the functionality of the eCommerce website on possible bugs.

ECommerce Website Testing Types

Of course, eCommerce testing is not a single task, but rather a complex of testing strategies of various types, which altogether help to improve an eCommerce website's performance. Let’s consider in detail how to test an eCommerce website using various testing types.

For this purpose, we've cooperated with our client, Otomate eCommerce company, who are successfully using DogQ to improve system performance and UI of their PIM management tool. We also used illustrations provided by them to create this handy and detailed guide as an example of eCommerce testing.

1. Functional Testing

We start, as usual, by checking the app’s basic usability, which is its core functions and accessibility. The following test cases are a few of many test cases that can be considered for checking the functional testing:

• Registration on the website: this includes entering login, username, and password, with a personal account creation as a result. Here, you should check whether a user’s able to enter the data, and does a new account creation follow this operation.
• Searching for a product: check the proper work of a search string, so that after a user inputs some product item, all the corresponding results are shown on the page;
• Selecting the product and placing it into a user’s shopping cart: you should make sure that a user can successfully log in, find the needed item, and place it to a shopping cart.

Apart from the testing automation, such testing techniques as BVA, decision table testing, and equivalence class testing may be also used during functional testing of eCommerce apps.

2. User Interface Testing

No eCom website testing can go without UI testing. Here various user interaction aspects are tested to ensure that the program responds appropriately to mouse and keyboard inputs, displaying hyperlinks, tables, menus, error message boxes, dialogue boxes, frames, buttons, and toolbars correctly. There are mainly 2 types of user interface testing:

- Navigation Testing: here all internal and external links are tested to make sure that they are reliable and well-organized. The following elements here can be checked:

• Check the proper work of every link on the site, so that an appropriate page opens in every single case;
• Choose the proper work of the back button;
• Check whether the info on products is displayed correctly, as well as horizontal and vertical scroll bars.

- Form-Based Testing: there are plenty of various forms on retail websites, for example, while the user is registering, customizing a product, or when the user is providing payment and shipping details. Form-based test cases for online shopping websites verify that the data submitted is in the correct format, that all the fields are filled in, and check the correct use of the tab key to go from one form field to another. The following examples of test cases can be used here:

• Check the maximum and minimum value of the length of string that can be put in the field;
• Check all mandatory fields in log in are entered in the form;
• Check if an error sign is displayed if not all mandatory fields have been filled in or they are too lengthy;
• Check that the correct sequence is followed when using navigation.

3. Browser Testing

Testing the app functionality with multiple browsers is a must nowadays, as each of them behaves differently. Though we still recommend choosing 3 or 4 the most important of them, as checking the app compatibility with all existing browser types can be too resource- and time-consuming. The best strategy is to test the app compatibility and functionality with the most popular browsers like Chrome, Safari, Firefox, and Edge, as well as with the browsers your customers prefer according to user acquisition statistics (if you have such).

4. Security Testing

How to test an eCommerce website on security? ECommerce apps in most cases function as web portals with huge customer databases, so it’s vital to secure the personal information of its users (such as financial details, date of birth, product preferences, etc.) from being stolen or misused. Test cases for an ecommerce websites here can be the following:

• Make sure that the data is secure against unauthorized users;
• Check whether the data sent by authorized users has the proper assignment;
• Test whether the chosen information reaches the needed recipient;
• Evaluate how often a failure occurs and how long it takes for the network to recover.

5. Performance Testing

ECommerce testing strategy won’t be full without performance testing, which allows defining such important metrics as request per second, transaction per minute, execution per click, the response time of page load, duration of the task, length of time between click and page display, as well as DNS lookup.

It has a bunch of sub-types, but we recommend paying special attention to load and stress testing types. Load and stress testing is vital to make sure an eCommerce app can handle the demand during peak hours and serve customers without delay. It puts the web application through its paces by overloading it. The following factors should be included while carrying out performance testing: maximum and average number of users, their type, response time, network bandwidth, memory available at peak times, and the average amount of time spent by a user.

If taken in detail, load testing should include the following factors to guarantee good performance of an eCommerce app during peak hours:

• Defining the load test environment;
• Calculating the number of users to define their limit number;
• Identifying suitable load metrics: webpage view per hour/day/ week, the number of concurrent users, wait time, elapsed time, etc.;
• Choosing the proper tool ( like Microsoft’s Application Center Test (ACT) or Empirix’s e-Test tools).

Stress testing is, vice versa, running a web application under more than maximum, assessing the system’s response when it is subjected to a load that exceeds its defined limits. Thus, the system’s behavior is observed to identify when the system under stress test fails and how it recovers from the failure. The following metrics here can be monitored: CPU and memory utilization, response time, backend database, user types, and concurrent users.

6. Database Testing

Due to the fact that eCommerce apps are database-driven, it is critical that they function effectively and protect the user’s sensitive data, such as personal and credit card info. The following aspects should be considered while database testing:

• The proper work of database operations;
• Make sure that the database returns to a consistent state after a crash recovery;
• Check the backup and recovery processes to assure the user’s uninterrupted service;
• Make that the database has enough memory to store records, and manage concurrent user access, as well as various administrative and user operations.

ECommerce App Features to Be Tested

Now let’s explore one by one the core features of any eCommerce app that should be tested. We also backed up them with professional advice from the DogQ testing team to draw your attention to tiny peculiarities you may face whilst testing. Let’s go!

1. Home Page

Here the main menu and the links to all other important pages are located, so your task is to check that everything works correctly, and then make it intuitive, and functional, with images and text blocks displayed correctly.

The following questions should be answered during the testing process:

• Is the auto scroll working correctly? What happens if the user hovers over it?
• Are all the links clickable and whether they redirect a user to the right page?
• How long does the whole page is loading?
• Can all the content be viewed clearly?
• Does the page render correctly in different browsers and various screen resolutions?

2. Product Catalog

This is the core stage in choosing a product by a customer, so you need to make sure every element is working fine and quickly, such as lists of products with all the explanatory images, product descriptions, and buttons. Thus, you need to test here the following elements:

• Make sure that images of the product are of good quality and resolution;
• Check the correctness of product prices and their specifications
• Test the proper work of in-stock/out-of-stock, as well as customization functions like multiple colors, sizes, or other elements;
• Ensure that customers can leave reviews on the product they bought;
• Make sure every element in breadcrumb navigation is functional.

3. Search Function

In order to get the top experience from using an eCom app, users should be able to search for the desired product right on the platform, and after they get the search results, they should be directed to the needed product page just in a few clicks. Common tests here can be the following:

• Test the proper work of the search filter. It should be based on the products’ names, brands, and categories. It’s good if it can search by product characteristics like color, size, etc.;
• Make sure that search results are relevant, displayed correctly on the page, and clickable;
• Ensure that a search result redirects the user to the right page or product.
  

NB! It’s also good to place the search in various places of an eCom app, which is connected with different purposes, like general info search, product characteristics search, FAQ navigation, etc. For example, when users search for info in the Otomate eCommerce tool, they might see a search filtered by a product name, type, or characteristics:

4. Order Processing

When a user places an order, they should be able to select the preferred shipping and billing options and see their order details displayed correctly. You need to check the following things at this step:

• The correct functioning of different payment options;
• For a guest user the app should finish the purchase and offer to register at the end;
• For returning customers the system should offer login to check out;
• Check the proper work of login and registration options;
• For a user which is signed up for a long time, make sure the session is timed out;
• Test adding products to the cart while continuing shopping;
• If the user adds the same item to the cart while continuing to shop, the item count in the shopping cart should be doubled;
• Whether a user can successfully delete one or several items from the shopping cart;
• Check whether a personal email or text confirmation is generated properly when the order is finished.

5. Payment Function

At this step, security testing is the most important, as you need to check customer payment and shipping details, the proper generation of order confirmation, the receipt with details, etc. We recommend using the following checklist at this step:

• If an eCom app stores customer debit or credit card details, make sure it is secure. Here PCI DSS (payment card compliance data security standard) compliance is a must;
• Check whether all products and their totals are displayed correctly in the cart;
• If a user adds or removes items from the cart, check that the total should change correspondingly;
• Test how calculating shipping costs and taxes are applied with different locations;
• Check whether coupons and discounts if any are working correctly;
• Make sure that if a user closes the site and comes back later, it still retains the items in the cart for some period (usually 2 weeks/1 month).

6. After-Order Tests

After a user places an order, some other manipulations can still be needed, for example, change or cancel the order, track the shipment, or check the refund, so the corresponding test cases may be needed.

7. Other Tests

There also can be other important features, which do not fall into a certain category, but still you need to test them, such as login, FAQs, contact us, and customer service pages, so make sure your test cases cover the full functionality of the retail application with no gaps.

ECommerce Application Testing Example With DogQ

Let’s consider a short example of using DogQ for eCommerce web application testing via our recent case study for Otomate eCommerce company.

Otomate is a robust eCommerce tool that allows managing various product catalogs and editing items in them from 1 place. When the client just started using DogQ, there were as many as 15 bugs that needed to be fixed. So they started by writing the documentation for the Otomate product catalog system: possible test cases and checklists (to go through the regression testing faster and step by step) and then they created test cases for the DogQ testing tool.

As soon as new functionality appeared on the Otomate staging, the client’s team started covering them with functional and regression tests. Using DogQ, they quickly created corresponding test cases, and checklists (which were super handy as they allowed the client to go through the regression much faster and with no errors).

Then they used the DogQ scheduling function and simply got notifiers from DogQ in Slack: if the test was successful, they just moved on to the next scenario, and if the tests failed, they figured out why this happened (this could be due to a bug in the product, or just because a request from the API didn’t come as fast as DogQ run all tests), fixed the bug and reset the testing.

As for now, the client’s team is refactoring product types and items, so not all the Otomate functionality has been covered with tests, only positive scenarios. But nevertheless, in just 3 months of using DogQ, they have already checked the main functionality of Otomate, checked all the elements, forms, text blocks, and the correct redirect to pages. Using DogQ for UI testing, the client’s team managed to significantly improve the work of the media gallery, optimize the designs, and improve the UI of the app.

They also extended the functionality in 2 of the Otomate main entities: item and product type. Now the system is able to add not only pictures of different formats to items (item), but also video, audio formats, documents, and 3D models, and this is happening not by simply loading from a browser, but with the drag&drop feature or through an URL. In addition, Otomate can now add files to an item that has already been uploaded to their system.

Otomate’s team also really likes the DogQ feature of entering random text into the input, as well as the sequential launch of scripts through the scheduling option. This is very convenient and reduces the time of manual testing of the entire application. Using of the DogQ testing tool allowed for a significant improvement and stabilization of the Otomate PIM management system.

“The testing process has been improved greatly. If earlier I constantly registered around 10 bugs (we somehow had even 10+ bugs on the board) in each release, now we always have around 2-4, and the development goes much faster. After each deployment of new features to the testing environment, tests are automatically launched via DogQ, we get the results via the DogQ notifier in Slack (for example, 26 tests passed – 24 passed successfully, 2 failed) and we quickly understand if the developer’s code has broken something that worked well for us before. This is, of course, very convenient and reduces the time needed for the testing of the entire CMP (content management platform)” – Ekaterina Chernigina, the Leading QA Specialist at Otomate

ECommerce Website Testing Checklist

Let’s now accumulate all the above info said above into a detailed test plan for any eCommerce website. Here we go!

Step 1. Remember that your testing strategy should provide an eCommerce app with the following benefits:

• Make your e-store more tested, user-friendly, intuitive, and efficient;
• Improve the overall user experience and helps to eliminate any related bugs;
• Reduce the risk of the site crashing, data loss, security breaches, and other unforeseen situations;
• Help to attract more customers as the interface is becoming smarter and simpler;
• Make it possible to evaluate the incoming traffic, shape the sales strategy, and increase KPIs.
• Improve your security, scalability, and accessibility standards used on the website.

So build your strategy in order to get as many as possible of them and make your eCom web app a handy solution for customers.

Step 2. Check the full eCommerce application functionality, which includes:

• Home page and all the links placed on it;
• Images and videos of the product;
• Price, dimensions, text description, and specifications of products;
• Search filter;
• In-stock, order later, favorite, and customization buttons;
• Customer reviews;
• Shopping cart functionality;
• Payment, shipment, and delivery options;
• Discounts, special offers, and promo codes;
• Return and refund options;
• Contact us form.

At this step, you need to use a bunch of different eCommerce test scenarios, techniques, and strategies to cover the full application functionality. We recommend using the functional, user interface, browser, performance, security, and database testing as obligatory techniques.

Also, below you can find the top 7 advice and recommendations prepared by our professional QA specialists in order to make your eCommerce testing strategy perfect:

1. ECommerce apps have numerous changes and updates on a daily basis, so make sure to run regression tests at least every day to keep track of the effects of change.

2. Homepage of a retail app is the most important in catching customers’ attention, and the most complex from the technical side as well, as it contains tons of info and links associated with each product. Moreover, their number is changing every day as new offers or product is added to a page. So before proceeding to regression testing it's best to verify all sitemap links by HTTP status codes.

3. Always create eCommerce test cases that cover all the steps from selecting a link on the home page to the checkout and payment gateway pages. Thus, you will cover maximum user experience and achieve the best testing results through an automation regression cycle.

4. Make sure to verify each product's details, no matter if it’s 10 or 1000 products, as the correctness of the info displayed related to products builds trust with customers and 1 slight mistake may ruin customer loyalty forever.

5. Web element of the eCommerce website keeps changing, so make sure to create a manual XPath. Some web elements' attributes will be the same, and there will be no unique way of distinguishing them. In such a scenario, use contains() method of Xpaths, or scroll into view.

6. Testing keyboard actions without using mouse action will definitely help you to detect new problems and fix them. This plays a significant role in user interface testing, especially in eCommerce mobile app testing.

7. Don’t forget about eCommerce a/b testing. Test up to five different versions of a page for several target audience groups to determine which version generates the most conversions. It helps in usability and performance testing.

Step 3. Use the right tools for testing an eCommerce site

In order to get your eCommerce testing performed quickly, efficiently, and successfully, you need to embrace the test automation advantages of various eCommerce website testing tools available on the market. Below you can find a top list with our recommendations.

DogQ

Speaking about functional, regression, and UI testing, DogQ is a perfect automation tool. It follows best practices of continuous testing and integration, helping you to get the perfect UI for an eCommerce website.

When you’ve finished testing your eCommerce app, it’s good to check the website’s overall performance. For this purpose, you can follow our relative testing checklist or use it to compile your own.

SonarQube

When it comes to security testing, it’s hard to find a tool better than SonarQube. It offers meticulous testing of the source code, finds vulnerabilities and tricky bugs to prevent undefined behavior, and protects your app on multiple fronts.

Apache JMeter

Apache JMeter is an excellent tool to test performance both on static and dynamic resources. It can simulate a heavy load on 1 or several servers, groups of servers, or networks, analyzing overall performance under different load types.

sqlmap

It analyzes the source code of SQL queries, and finds potential vulnerabilities and SQL injections in it, ensuring stable database operation and safe query execution.

ECommerce testing is more than automation, as also includes manual testing and checking user experience. Below you can find handy state-of-the-art tools that can help you perform usability testing in eCommerce apps.

HotJar

It shows the most clicked and unclicked zones of sites by visitors. This tool reveals the online behavior and voice of the users.

Feng-GUI

It simulates human vision during the first five seconds and predicts what a real human would most likely look at.

Wrapping Up

ECommerce testing is a complex process that involves several testing strategies to cover the full functionality of modern retail applications. In this article, we considered various types of testing used during eCommerce testing scenarios, provided several examples of test cases, as well as showed how we successfully performed eCommerce testing services using DogQ codeless test automation tool.

Our QA specialists are always ready to build your own eCommerce testing strategy using DogQ, as well as answer any related questions for free. So don’t hesitate to contact us to get a consultation and subscribe to our blog to stay updated with the latest news from the world of QA!