1. Introduction: What Is Demand Forecasting, and How Is It Done?
1.1 What is Demand Forecasting?
Demand forecasting is, in essence, developing the best possible understanding of future demand. In practice, this means analyzing the impact of a range of variables that affect demand—from historical demand patterns to internal business decisions and even external factors—to increase the accuracy of these predictions. Accurate demand forecasts can be leveraged throughout retail operations to improve decision-making and outcomes in areas such as store and distribution center replenishment, capacity planning, and resource planning.
Demand forecasts can be developed on different levels of granularity—monthly, weekly, daily, or even hourly—to support different planning processes and business decisions, but highly granular forecasts are always extremely valuable. The benefits of a granular forecast are obvious when thinking of fresh food products whose short shelf-lives sometimes call for intra-day forecasts at the product-location level to prevent spoilage.
Why, then, would slow-moving items that sell only a couple units per location per day, if even that, require the same level of forecast granularity? Even if the day-product-location level forecast for a slow-moving item is itself somewhat inaccurate, forecasting at this level of granularity ultimately makes it easier to aggregate demand—whether for different periods of time, across products (for example, total demand per product per distribution center), or by total order lines per DC per day, etc.
To effectively execute store replenishment, capacity planning, and other business decisions, retailers need multiple forecasts with different levels of granularity that look at different time spans. This is why flexible aggregation across products or over different planning horizons is critical to a retailer’s ability to leverage the same demand forecast in all their retail planning.
Planograms, for example, are not adjusted on a daily basis, but are generally revised every few months as part of a larger assortment review process, with smaller adjustments often made between review periods. While many retailers still base planograms on past sales examined in weekly or monthly time buckets, demand-driven planograms offer a much higher degree of accuracy because they take advantage of this highly granular, day-product-location level forecast data. The same applies to markdown optimization, workforce shift optimization, and any other planning process with a longer planning period than daily replenishment.
Despite the knock-on benefits of accurate demand forecasts throughout retail operations, many retailers’ forecasting capabilities are unfortunately still quite limited. In a recent study of North American grocery retailers, less than half of the retailers surveyed were able to produce forecasts on the day-product-location level with their existing software.
1.2 What Are the Benefits of Accurate Demand Forecasting?
Reliable demand forecasts are integral to the health of retail operations because, simply put, they reduce uncertainty. With an accurate calculation of how many items they will sell at any given time, retailers can order, allocate, and replenish those items accordingly. Beyond supply chain and inventory management, though, an accurate forecast in a unified retail planning environment can be used to optimize capacity management, workforce scheduling, planograms, and more. In short, the demand forecast is the foundation from which retailers can drive a wide range of benefits across retail functions.
Benefits of Accurate Demand Forecasting in Retail:
- Increased sales from better product availability
- Reduced spoilage and fresher, more appealing products through more accurate stock allocation
- Increased inventory turnover through reduced need for safety stock
- Better margins through proactive, optimized markdowns
- Improved capacity utilization and more reliable fulfillment through better visibility into capacity requirements and proactive bottleneck resolution
- Reduced personnel costs through forecast-based shift optimization in stores and distribution centers
1.3 How Do Retailers Actually Forecast Demand?
From omnichannel giants to the smallest brick and mortar boutiques, all retailers rely on demand forecasts to order items based on their best estimate of how much they will sell. Modern demand forecasting is a sophisticated statistical analysis that takes into account numerous variables to optimize that prediction.
While some retailers still rely on spreadsheets and manual calculations, such high-powered statistical analysis is best executed by specialized software designed to process enormous, retail-scale data sets. The best of these software transparently show users what data is being used to build forecasts and how the forecasts are being calculated. Modern demand forecasting software automates difficult and time-consuming decisions, using machine learning to optimize predictions.
2. Demand Forecasting Methodology: How to Create Your Forecast
Traditionally, retailers have created their baseline forecasts using time-series modeling, looking at historical data to make predictions about future demand. These forecasts were often adjusted based on causal modeling and manual input. Modern retailers, though, have replaced these older approaches to demand forecasting with machine learning.
2.1 Machine Learning Improves Demand Forecast Accuracy
Machine learning not only increases the accuracy of demand forecasts, it also automates large amounts of planner work and can process enormous data sets—far more than any human planner would be capable of.
To generate an accurate demand forecast, a system must be able to process an enormous amount of data on the wide range of variables that can potentially impact demand. With advancements in large-scale data processing and in-memory computing, modern demand planning systems can make millions of forecast calculations within a minute’s time, taking into consideration more variables than ever before possible.
Consider the three broad areas of variability that continuously impact demand: recurring variations in baseline demand patterns, your own internal business decisions, and external factors such as weather or local events.
A demand forecasting software must be transparent about the models it’s using to calculate forecasts using all of this data. Transparency, after all, is critical to retail planners’ ability to fully understand and trust their automated forecasts.
For a more in-depth discussion of retail forecasting approaches, take a look at The Complete Guide to Machine Learning in Retail Demand Forecasting.
2.2 Internal Business Decisions: Price Changes, Promotions, Product Introductions, and More
The commercial choices you make as a retailer—promotions, price changes, new product introductions, or changes in how you display products—have an enormous impact on sales volumes. Because these decisions can introduce so much hard-to-predict variation, they absolutely must be accounted for in forecast calculations.
To predict the impact of business decisions, you must leverage machine learning algorithms that can process large amounts of retail data and integrate them into the baseline demand forecast to be accounted for.
The variables most commonly included in these calculations include:
- Promotion type, such as price reduction or multi-buy
- Marketing activities like newspaper ads or in-store signage
- In-store display, such as presenting a promoted product on an endcap or table
- Price elasticity, or how a change in price impacts a product’s demand
- Impact of a price change on other products within that category
Accurate price elasticity modeling is especially important for markdown optimization, as it provides planners a clear picture of how to price markdown stock to sell quickly while maintaining the highest possible margin.
To learn more about how to accurately take the impact of commercial choices into account, read our white paper, More Accurate Promotion Forecasting with Machine Learning.
2.2.1 Accounting for Sales Cannibalization
When you lower one product’s price, you usually also reduce demand for other products in that category as demand shifts to the cheaper product—a phenomenon known as “cannibalization.” To avoid over-ordering cannibalized products, retailers should adjust forecasts for these non-promoted products down, then incorporate the adjusted forecasts into their replenishment planning. This level of accuracy, of course, is especially relevant when replenishing short shelf-life products. Make sure your demand forecasting software can accurately account for cannibalization.
If you are interested in learning more, check out our white paper, Considering Cannibalization and Halo Effects to Improve Demand Forecasts.
2.2.2 New Product Introductions
New product introductions present a different kind of challenge to sales forecasts because you don’t have the luxury of historical sales data to serve as a foundation. Reference products with historical data serve as a blueprint until you’ve gathered enough data to create an actual forecast for the new product. Typically in product introduction scenarios, planners must choose reference products themselves—a time-consuming process that is also often inaccurate when you consider wide assortments and high renewal rates.
A far more efficient and accurate way to solve this problem is to utilize a planning system capable of automatically selecting the appropriate reference product based on the most relevant attributes in the product category (brand, size, use, color, flavor, etc.). Of course, this system should also be able to quickly update its SKU-store/channel forecasts as actual sales patterns for the new product emerge. To learn more about introducing new products, have a look at our article, Top Challenges in Demand Forecasting.
2.3 External Factors that Impact Demand
When we say “external factors,” we’re referring broadly to anything not in your power to decide as a retailer. Examples of external factors might include weather forecasts, local events, or even your competitors’ business decisions, all of which can cause significant changes to your demand.
We know, for example, that a heatwave almost always boosts ice cream sales and, conversely, that the first snowfall of the year sends customers flocking to buy winter coats. That seems straightforward enough, but identifying and accurately predicting all demand shifts for all external events across the entire product range at multiple store locations is enormously difficult.
To demonstrate how impossible a task this would be for human calculation, let’s consider a small retailer with 100 stores and a range of 5,000 products. If their planners were to attempt to manually account for the effects of weather alone on a reasonably granular level, they would have to examine some 280 million potential relationships between variables (100 stores x 5,000 products x 20 weather variables x 7 weekdays x 4 seasons). The data quickly becomes impossible for any team of humans to compute.
Fortunately, machine learning automates a large portion of this work and can integrate these external factors into your forecast. These algorithms can secure your ice cream supply just before a heatwave rolls through, or they can reduce your supply before a torrential rain settles in for the week. In our experience, incorporating weather can reduce forecast errors by 5–15% on the product level and by up to 40% on the product group and store level.
When external data is taken into consideration, a demand planning software can give planners a clear understanding of how different factors impact the forecast—for example, the effect of local weather on sales. Demand forecast accuracy depends on a system’s ability to incorporate a wide range of potential external data sources, including but by no means limited to:
- Local weather data and weather forecasts
- Passenger number data and passenger forecasts for transportation hubs (read the case study)
- Data about past and upcoming events, such as fairs or concerts
- Competitor pricing data
Note that while machine learning in demand forecasting helps automate the bulk of the work, it does have its limitations. After all, consumer trends are always changing, and the unexpected always happens when you least expect it. There will always be a risk that forecasts reflect how things happened in the past instead of how things will actually be. That’s why machine learning can never replace human expertise and experience. We will always need demand planners who can observe and understand real-world changes and correct the automated forecasts accordingly.
We’ve now built a highly automated demand forecast that leverages machine learning to build an accurate baseline forecast that identifies recurring demand patterns, incorporates promotions and your other internal business decisions, then factors in external data such as local events and competitor pricing. Next, we’ll look at one of the most disruptive developments in retail as a whole, and certainly in demand forecasting: omnichannel operations.
3. Demand Forecasting in Omnichannel Retail
Retailers who execute an omnichannel strategy must deliver a good customer experience in every channel, whether in-store, online, or through hybrid channels like click-and-collect. From product availability to fulfillment speed, customer experience relies heavily on a retailer’s supply chain efficiency and visibility.
To successfully forecast demand across multiple channels, retailers must link online sales to the correct fulfillment channel. For example, if your online orders get picked from your local stores, this online demand must be incorporated into the store-level demand forecast to ensure accurate replenishment that meets both online and in-store demand. But that’s just the beginning.
Online orders tend to follow a different sales pattern than brick-and-mortar sales. There are many reasons behind this difference—for example, the fact that price comparison is far faster and easier online than when shopping a physical store. Holiday seasons are another example of variation; retailers usually see online orders placed well ahead of time, followed by a wave of procrastinators rushing to brick-and-mortar stores for last-minute purchases. Of course, this pattern may change as consumers expect faster online fulfillment, even during rushed holiday seasons.
Because demand patterns can vary so significantly by channel, demand planning systems must be able to separate the forecasts for online and in-store sales, adding even more granularity. These forecasts can be used to enable virtual ringfencing at warehouses and distribution centers, ensuring availability across channels. Omnichannel retailers must be able to forecast by store, by sales channel, and by fulfillment channel to ensure the right stock is available in the right places and maintain customer satisfaction across the board.
4. Demand Forecast Accuracy
No matter how advanced technology grows, it will always be impossible to predict demand with 100% accuracy. That’s why retail planning processes must always be able to accommodate a certain degree of uncertainty. But how accurate do you need your forecasts to be?
Ultimately, accuracy is always important, but should be analyzed on different levels. Depending on the use case, forecast accuracy should be evaluated for different periods of time and different levels of aggregation. When replenishing quick-selling, short shelf-life perishable products, for example, highly accurate day-product-location level forecasts are imperative. For slow-moving products, on the other hand, it’s more important to get the total volume fulfilled by a specific DC correct and to avoid bias. No matter what your business goals or assortment characteristics, though, there’s always a point of diminishing returns on forecast accuracy. All retailers reach a plateau where it becomes more cost-effective to accept and prepare for a certain degree of inaccuracy than it would be to invest more time and money on further improvements. If a product has a long shelf-life, for example, the cost of a small increase in safety stock may well be less than the cost of asking demand planners to further refine the forecast.
Keep in mind, too, that demand forecasting is just one part of the larger retail planning operation. Even near-perfect forecasts will fail to produce the desired results if the other parts of your planning process fall short. You’ll still end up with overly large batch sizes or with too much presentation stock. For a full review of retail planning processes, consult our Forecast Accuracy Guide for more strategies to increase accuracy.
So, when is your forecast accurate enough? In short, when further improvements in accuracy will only marginally improve your actual business results.
5. Picking a Demand Forecasting Software
We’re not too humble to say honestly that we believe our own demand forecasting solution is a powerful asset for most retailers with a large, continuous assortment. Whatever solution you pick, though, make sure it leverages a full AI toolbox to capture the complexity of demand-influencing factors from all available internal and external data sources. The only way to calculate a good forecast is by feeding your demand planning system with enough relevant data. The data required to make accurate predictions is far too heavy and complex for humans to calculate, so retailers must look for an AI-driven solution whose machine learning algorithms can automate the majority of that work.
It’s essential, though, that this complexity doesn’t result in a “black box” of a system that forces planners to simply trust automated recommendations. The best demand forecasting systems are transparent. They provide planners with a clear understanding of how they calculate forecasts to support a wide range of short and long-term planning activities—from replenishment planning to workforce optimization and capacity management.
Furthermore, the demand forecasting system must be powerful enough to process data at retail-scale. Retailers generate enormous amounts of sales data every single day. Add to that the data from internal business decisions and external factors and the level of granularity required for accurate planning, and the importance of data processing power becomes apparent. Choose a solution with a fast, modern database, or you’ll end up waiting hours each and every time you need to run a calculation.
If a solution doesn’t satisfy these basic requirements, move on to the next one. To read in more detail about evaluating and choosing the demand forecasting software that will best serve your business needs, read Supply Chain Transformation: The Complete Guide.
6. Further Resources on Demand Forecasting
This guide is only a starting point. There’s more granularity to demand forecasting than what we’ve covered here—that’s why we’ve gathered these helpful resources. Dig deeper into the world of demand forecasting to find out how you can improve your retail operations and bottom line at the same time.
Challenges in Demand Forecasting
Managing Promotions and Weather in Demand Forecasts
Demand Forecast Accuracy
Demand Forecasting Software