ML-Powered Demand Forecasting For a Canadian Ecommerce Company

How replacing a traditional forecasting model in Excel with ML-based demand forecasting and Power BI visualization empowered a Canadian clothing retailer to step up their long-term forecasting accuracy from 23% to 92%, lower stockouts to 10%, and keep overstock within the 5% range.

Industry:
Ecommerce

Download

Software Product Development

AI Development

Business challenge

Understanding exactly what, when, and in what quantities your customers want is a prerequisite to building a stable ecommerce business. Therefore, all ecommerce companies use some form of demand forecasting, often relying on rule-based linear regression analysis in Excel. However, as the ecommerce landscape continues to evolve, reaching the point when good enough isn’t enough anymore.

While traditional methods might still work for short-term planning, they keep failing to highlight longer-term insights. Quickly processing data from multiple external and internal sources or providing a comprehensive view of future trends is simply beyond the rigid systems’ capacity. The complexities of modern consumer behavior, influenced by myriads of fluctuating factors, require more advanced solutions — specifically, machine learning (ML) algorithms. For instance, present-day ML-based demand forecasting models are known to provide up to 98% accurate predictions.

Our client — a Canadian clothing manufacturer and online retailer — wanted to go beyond Excel-based analytics and myopic demand forecasting, solely grounded on historical dynamics and static consumer buying patterns. With garment factories in Vietnam and a customer base in Canada, they expected an intelligent forecasting model to optimize inventory planning and supply chain management.

The major shortcomings of their current spreadsheets-based prediction model included:

  • Manual data analysis
  • High dependence on the analysts for data interpretation
  • Low accuracy of long-term predictions
  • ‘Bullwhip effect’ of supply chain inefficiencies
  • Inability to adjust forecasts to new data and market changes right away
  • Permanent stockouts and overstock

As the limitations of their current approach became more apparent, the client decided to explore ML-driven solutions and started scanning the IT market for a tech partner to craft a highly accurate demand forecasting model. They also wanted to combine ML forecasting implementation with the adoption of data visualization software to make demand forecasts easy to grasp for a wide range of users.

After evaluating several potential ISO-certified vendors, they delegated the project’s heavy lifting to *instinctools’ dedicated team of top-tier data scientists, ML engineers, and BI specialists.

The outcomes they expected to achieve within our cooperation included:

Long-term predictions

with over 90% accuracy

Automation

of the forecasting processes

Stockouts

< 10%

Overstock

< 5%

The capability

to instantly factor in new data and market changes

Multidimensional modeling

and comparisons of what-if scenarios for
promotions, market fluctuations, etc.

Wide accessibility

of ML-powered insights thanks to integration
with data visualization tool

See how we approached the task, what challenges we faced and overcame.

Solution

The client was already using Microsoft corporate products, such as Dynamics365, Azure SQL database, etc., so they wanted to keep their tech stack consistent and opt for Azure ML. Moreover, they were also leveraging Power BI for visual insights in other departments.

However, they lacked in-house capacity and expertise to manage this software implementation initiative. Therefore, the *instinctools team was tasked with developing and integrating demand forecasting and data visualization tools into the client’s software ecosystem.

Here’s a system overview.

  1. Training a recurrent neural network (RNN) on the client’s data

Azure ML allows you to train your own ML model, save it in the registry, and then deploy in production. Based on our hands-on experience in similar projects, we chose a recurrent neural network (RNN) as a go-to option for demand forecasting tasks. After getting the client’s approval, our data scientists and ML engineers proceeded to work on the model’s exact architecture to incorporate the client’s dataset.

As the client has been considering implementing an ML-powered demand forecasting model for a while, they’ve put data cleanliness, accuracy, and completeness on the front burner long before taking on the ML adoption. Nevertheless, our data scientists run data cleaning procedures to ensure the dataset with diverse information from the client’s website, social media, third-party marketplaces, etc. is neatly organized. It greatly contributed to fast-tracking the project and getting a precise forecasting model that accurately reflects customer behavior and preferences while taking into account macro- and microeconomic events.

We split the data into training and validation sets, following the standard 80/20 ratio — 80% of data for training, and the rest 20% for validation. Splitting was performed randomly to eliminate bias in both sets and, thus, boost the model’s prediction reliability.

As the client didn’t have the necessary high-performing GPUs in place and didn’t plan to update their hardware park in the near future, it was more budget-wise for them to leverage Azure-based computing resources for the RNN training rather than invest in their own GPUs.

Through multiple re-training cycles of the ML-powered prediction model, we hit the 92% accuracy mark, with potential to reach 98% over time.

  1. Addressing two major challenges of ML-driven demand forecasting
Ensuring a seamless ML lifecycle

The adoption of AI and ML isn’t all sunshine and rainbows. To preempt issues that might affect the model’s performance, we established a structured and transparent ML workflow, relying on best MLOps practices:

  • Automated data pipelines for data collection and preprocessing
  • Version control of the model training
  • Standardized and traceable deployments
  • Auto re-training mechanism in case of data changes
  • Continuous model performance monitoring with auto alerting of any issues
Tackling software maintenance complexity

You still need human oversight to make the most out of artificial intelligence. Since the client doesn’t have in-house data scientists and ML engineers, they decided to retain our experts for adaptive software maintenance to modify the model in line with the updates in the company’s policies, regulations such as GDPR, and other possible post-deployment changes. Our specialists also stay in charge of regular re-evaluation and updates of the training and validation sets.

  1. Making ML-based demand insights accessible across the company

Accurate demand forecasting means nothing if not followed with timely actions. However, outputs generated by ML models are primarily designed for data scientists, limiting their accessibility to other employees. To democratize ML tools across various departments, the client decided to use Power BI for data visualization.

Both Azure ML and Power BI being part of the Microsoft stack allowed for seamless integration, enabling automatic updates to visualizations whenever the prediction model is re-trained.

Here’re the examples of analytical dashboards the client was provided with.

Before

  • Excel-based demand forecasting limited to linear regression analysis
  • Failure to take into account all valuable external and internal data
  • High reliance on manual processes
  • Poor accuracy in long-term forecasts
  • Permanent stockouts and overstock

After

  • Advanced demand forecasting with a deep learning engine at its core
  • Holistic ML model incorporating multiple internal and external data sources
  • Automated data processing and forecasting
  • High-fidelity long-term demand forecasts
  • Mitigated shortages and overstock

Business value

  • 92% demand forecasting accuracy
  • Stockouts reduced to 10%
  • Overstocks below 5%

Multiplier effect

Is there an industry where you can hit it big without meticulous planning backed by accurate demand forecasts? Be it ecommerce, healthcare, manufacturing, entertainment and media, or any other sector, as a business owner, you have to constantly balance your offering against customer needs and market changes. Switching from spreadsheets to frontier forecasting can improve operational efficiency by a mile in predicting hospital visits and the need for medications, anticipating power consumption, or planning workforce — the list can go on and on.

Related capabilities

Do you have a similar project idea?

Anna Vasilevskaya
Anna Vasilevskaya Account Executive

Get in touch

Drop us a line about your project at contact@instinctools.com or via the contact form below, and we will contact you soon.