1. Introduction: What Is Demand Forecasting, and How Is It Done?
1.1 What is Demand Forecasting?
Demand forecasting is, in essence, developing the best possible understanding of future demand. In practice, this means analyzing the impact of a range of variables that affect demand—from historical demand patterns to internal business decisions and even external factors—to increase the accuracy of these predictions. Accurate demand forecasts can be leveraged throughout your supply chain to improve decision-making and outcomes in areas such as store and distribution center replenishment, capacity planning, and resource planning.
Demand forecasts can be developed on different levels of granularity—monthly, weekly, daily, or even hourly—to support different planning processes and business decisions, but highly granular forecasts are always extremely valuable. The benefits of a granular forecast are obvious when thinking of fresh food products whose short shelf-lives sometimes call for intra-day forecasts at the product-location level to prevent spoilage.
For CPG companies, detailed forecasts allow planners to model the effects on demand based on sale or promotional information. Another key benefit of granular forecasting lies in its ability to allow a manufacturer to aggregate data on products that share a certain raw material and create a forecast that will reveal the true amount of materials needed to produce each product, helping them to reduce costs by optimizing bulk purchasing and reducing the number of shipments needed. Additionally, modeling can help optimize production capacity.
Why, then, would slow-moving items that sell only a couple of units per location per day, if even that, require the same level of forecast granularity? Even if the day-product-location level forecast for a slow-moving item is itself somewhat inaccurate, forecasting at this level of granularity ultimately makes it easier to aggregate demand—whether for different periods of time, across products (for example, total demand per product per distribution center), or by total demand per product in a month or customer group, etc.
To effectively execute replenishment, capacity planning, and other business decisions, retailers and CPG companies need multiple forecasts with different levels of granularity that look at different time spans. This is why flexible aggregation across products or over different planning horizons is critical to a company’s ability to leverage the same demand forecast in all their supply chain planning.
Consider planograms, for example, which are not adjusted daily but are generally revised every few months as part of a larger assortment review process, with smaller adjustments often made between review periods. While many retailers still base planograms on past sales examined in weekly or monthly time buckets, demand-driven planograms offer a much higher degree of accuracy because they take advantage of this highly granular, day-product-location level forecast data. The same applies to markdown optimization, workforce shift optimization, and any other planning process with a longer planning period than daily replenishment.
For CPG companies, granular forecasts allow them to leverage retail sales and assortment data automatically for more accurate short-term forecasts. A forecasting system that uses machine learning can calculate complex and variable demand drivers automatically allowing manufacturers to quickly adapt their forecasts to changes in demand patterns at the right level of granularity. With automated and centralized assortment reviews, CPG companies can see marked benefits including more than 90% weekly forecast accuracy, a 9-pp improvement in peak season forecast accuracy, and a 10% increase in forecast accuracy when using retailer data.
Despite the knock-on benefits of accurate demand forecasts throughout retail and CPG operations, many companies’ forecasting capabilities are unfortunately still quite limited.
1.2 What Are the Benefits of Accurate Demand Forecasting?
Reliable demand forecasts are integral to the health of your supply chain because, simply put, they reduce uncertainty. With an accurate calculation of how many items they will sell at any given time, retail and CPG companies can order, allocate, and replenish those items accordingly. Beyond supply chain and inventory management, though, an accurate forecast in a unified supply chain and retail planning platform aligns sales and operations, supply planning, and raw materials replenishment. In short, the demand forecast is the foundation from which retailers and CPG companies can drive a wide range of benefits across their business functions.
Benefits of Accurate Demand Forecasting in Supply Chain:
- Increased sales from better product availability.
- Reduced spoilage and fresher, more appealing products through more accurate stock allocation.
- Increased inventory turnover through reduced need for safety stock.
- Better margins through proactive, optimized markdowns.
- Improved capacity utilization and more reliable fulfillment through better visibility into capacity requirements and proactive bottleneck resolution.
- Reduced personnel costs through forecast-based shift optimization in stores and distribution centers.
1.3 How Do Retailers and CPG Companies Actually Forecast Demand?
From omnichannel giants to the smallest local manufacturers, all retailers and CPG companies rely on demand forecasts to order items based on their best estimate of how much they will sell. Modern demand forecasting is a sophisticated statistical analysis that considers numerous variables to optimize that prediction.
While some retailers and CPG companies still rely on spreadsheets and manual calculations, such high-powered statistical analysis is best executed by specialized software designed to process enormous supply chain data sets. The best of these software transparently show users what data is being used to build forecasts and how the forecasts are being calculated. Modern demand forecasting software automates difficult and time-consuming decisions, using machine learning to optimize predictions.
2. Demand Forecasting Methodology: How to Create Your Forecast
Traditionally, retailers and CPG companies have created their baseline forecasts using time-series modeling, looking at historical data to make predictions about future demand. These forecasts were often adjusted based on causal modeling and manual input. Modern companies, though, have replaced these older approaches to demand forecasting with machine learning.
2.1 Machine Learning Improves Demand Forecast Accuracy
Machine learning not only increases the accuracy of demand forecasts, but it also automates large amounts of planner work and can process enormous data sets—far more than any human planner would be capable of.
To generate an accurate demand forecast, a system must be able to process an enormous amount of data on the wide range of variables that can potentially impact demand. With advancements in large-scale data processing and in-memory computing, modern demand planning systems can make millions of forecast calculations within a minute’s time, taking into consideration more variables than ever before possible.
Consider the three broad areas of variability that continuously impact demand: recurring variations in baseline demand patterns, your own internal business decisions, and external factors such as weather or local events.
A demand forecasting software must be transparent about the models it’s using to calculate forecasts using all of this data. Transparency, after all, is critical to retail and CPG planners’ ability to fully understand and trust their automated forecasts.
For a more in-depth discussion of different approaches to forecasting, take a look at The Complete Guide to Machine Learning in Retail Demand Forecasting.
2.2 Predicting the Effects of Business Decisions: Price Changes, Promotions, Product Introductions, and More
Commercial choices such as a promotion or price change, or new product introductions, have an enormous impact on sales volumes. Because these decisions can introduce so much hard-to-predict variation, they absolutely must be accounted for in forecast calculations.
To predict the impact of business decisions, you must leverage machine learning algorithms that can process large amounts of data and integrate them into the baseline demand forecast to be accounted for.
The variables most commonly included in these calculations include:
- Promotion type, such as price reduction or multi-buy
- Marketing activities like newspaper ads or in-store signage
- In-store display, such as when retailers present a promoted product on an endcap or table, or when CPG companies provide a product-specific display to retailers
- Price elasticity, or how a change in priceimpacts a product’s demand
- Visibility, such as when a CPG company wins or loses placement in a grocery store
- Impact of a price change on other products within that category
Accurate price elasticity modeling is especially important for markdown optimization, as it provides planners with a clear picture of how to price markdown stock to sell quickly while maintaining the highest possible margin.
To learn more about how to accurately take the impact of commercial choices into account, read our white paper, More Accurate Promotion Forecasting with Machine Learning.
2.2.1 Accounting for Price Changes
Within your organization, when you lower one product’s price, you usually also reduce demand for other products in that category as demand shifts to the cheaper product—a phenomenon known as “cannibalization.” To avoid over-ordering cannibalized products, retailers should adjust forecasts for these non-promoted products down, then incorporate the adjusted forecasts into their replenishment planning. This level of accuracy, of course, is especially relevant when replenishing products with short shelf-life.
However, for factors outside of your organization’s control, effective supply chain collaboration can make sure your demand forecasting software will accurately account for price changes impacting demand. When retailers and CPG companies establish a regular data-sharing routine, it allows both companies visibility into their end-to-end supply chain, which can help mitigate a broad range of issues that arise from unexpected changes to demand signals.
If you are interested in learning more, check out our white paper, Considering Cannibalization and Halo Effects to Improve Demand Forecasts.
2.2.2 New Product Introductions
New product introductions present a different kind of challenge to sales forecasts because you don’t have the luxury of historical sales data to serve as a foundation. Reference products with historical data serve as a blueprint until you’ve gathered enough data to create an actual forecast for the new product. Typically, in product introduction scenarios, planners must choose reference products themselves—a time-consuming process that is also often inaccurate when you consider wide assortments and high renewal rates.
A far more efficient and accurate way to solve this problem is to utilize a planning system capable of automatically selecting the appropriate reference product based on the most relevant attributes in the product category (brand, size, use, color, flavor, etc.). Of course, this system should also be able to quickly update its SKU-store/channel forecasts as actual sales patterns for the new product emerge. To learn more about introducing new products, have a look at our article, Top Challenges in Demand Forecasting.
2.3 External Factors that Impact Demand
When we say, “external factors,” we’re referring broadly to anything not in your power to decide as a retailer or CPG company. Examples of external factors might include weather forecasts, local events, or even your competitors’ business decisions, all of which can cause significant changes to your demand.
We know, for example, that a heatwave almost always boosts ice cream sales and, conversely, that the first snowfall of the year sends customers flocking to buy winter coats. That seems straightforward enough but identifying and accurately predicting all demand shifts for all external events across the entire product range at multiple locations is enormously difficult.
To demonstrate how impossible a task this would be for human calculation, let’s consider a small retailer with 100 stores and a range of 5,000 products. If their planners were to attempt to manually account for the effects of weather alone on a reasonably granular level, they would have to examine some 280 million potential relationships between variables (100 stores x 5,000 products x 20 weather variables x 7 weekdays x 4 seasons). The same logic applied to CPG companies, who tend to have fewer locations and smaller product ranges but can serve many different retail channels. The data quickly becomes impossible for any team of humans to compute.
Fortunately, machine learning automates a large portion of this work and can integrate these external factors into your forecast. These algorithms can secure your ice cream supply just before a heatwave rolls through, or they can reduce your supply before a torrential rain settles in for the week. In our experience, incorporating weather can reduce forecast errors by 5–15% on the product level and by up to 40% on the product group and location level.
Retailer’s commercial decisions on store assortments, including product displays, promotions, and in-store events, impact demand at the store level, which in turn impacts demand from the CPG company. With efficient supply chain collaboration, CPG companies can leverage retailer data in their own demand planning to attain high forecast accuracy and meet demand in every situation.
When external data is taken into consideration, a demand planning software can give planners a clear understanding of how different factors impact the forecast—for example, the effect of local weather on sales. Demand forecast accuracy depends on a system’s ability to incorporate a wide range of potential external data sources, including but by no means limited to:
- Local weather data and weather forecasts.
- Passenger number data and passenger forecasts for transportation hubs (read the case study).
- Data about past and upcoming events, such as fairs or concerts.
- Competitor pricing data.
- For CPGs, retailer data, like POS, retailer promotion and assortment, including store count.
Note that while machine learning in demand forecasting helps automate the bulk of the work, it does have its limitations. After all, consumer trends are always changing, and the unexpected always happens when you least expect it. There will always be a risk that forecasts reflect how things happened in the past instead of how things will actually be. That’s why machine learning can never replace human expertise and experience. We will always need demand planners who can observe and understand real-world changes and correct the automated forecasts accordingly.
We’ve now built a highly automated demand forecast that leverages machine learning to build an accurate baseline forecast that identifies recurring demand patterns, incorporates promotions and your other internal business decisions, then factors in external data such as local events and competitor pricing.
Next, we’ll look at one of the most disruptive developments in the supply chain, and certainly in demand forecasting: omnichannel operations.
3. Demand Forecasting in Omnichannel
Retailers and CPG companies who execute an omnichannel strategy must deliver a good customer experience in every channel, whether in-store, online, or through hybrid channels like click-and-collect. From product availability to fulfillment speed, customer experience relies heavily on a retailer’s supply chain efficiency and visibility.
To successfully forecast demand across multiple channels, companies must link online sales to the correct fulfillment channel. For example, if your online orders get picked from your local stores, this online demand must be incorporated into the store-level demand forecast to ensure accurate replenishment that meets both online and in-store demand. But that’s just the beginning.
Online orders tend to follow a different sales pattern than brick-and-mortar sales. There are many reasons behind this difference — for example, the fact that price comparison is far faster and easier online than when shopping at a physical store. Holiday seasons are another example of variation; retailers and CPG companies usually see online orders placed well ahead of time, followed by a wave of procrastinators rushing to brick-and-mortar stores for last-minute purchases. Of course, this pattern may change as consumers expect faster online fulfillment, even during rushed holiday seasons.
Because demand patterns can vary significantly by channel, demand planning systems must be able to separate the forecasts for online and in-store sales, adding even more granularity. These forecasts can be used to enable virtual ringfencing at warehouses and distribution centers, ensuring availability across channels.
Omnichannel retailers and CPG companies must be able to forecast by store, by sales channel, and by fulfillment channel to ensure the right stock is available in the right places and maintain customer satisfaction across the board.
4. Demand Forecast Accuracy
No matter how advanced technology grows, it will always be impossible to predict demand with 100% accuracy. That’s why supply chain planning processes must always be able to accommodate a certain degree of uncertainty. But how accurate do you need your forecasts to be?
Ultimately, accuracy is always important, but should be analyzed on different levels. Depending on the use case, forecast accuracy should be evaluated for different periods of time and different levels of aggregation. When replenishing quick-selling, short shelf-life perishable products, for example, highly accurate day-product-location level forecasts are imperative.
For slow-moving products, on the other hand, it’s more important to get the total volume fulfilled by a specific DC correct and to avoid bias. No matter what your business goals or assortment characteristics, though, there’s always a point of diminishing returns on forecast accuracy.
All retailers and CPG companies reach a plateau where it becomes more cost-effective to accept and prepare for a certain degree of inaccuracy than it would be to invest more time and money on further improvements. If a product has a long shelf-life, for example, the cost of a small increase in safety stock may well be less than the cost of asking demand planners to further refine the forecast.
Keep in mind, too, that demand forecasting is just one part of the larger supply chain planning operation. Even near-perfect forecasts will fail to produce the desired results if the other parts of your planning process fall short. You’ll still end up with overly large batch sizes or with too much presentation stock. For a full review of demand planning processes, consult our Forecast Accuracy Guide for more strategies to increase accuracy.
So, when is your forecast accurate enough? In short, further improvements in accuracy will only marginally improve your actual business results.
5. Picking a Demand Forecasting Software
We’re not too humble to say honestly that we believe our own demand forecasting solution is a powerful asset for most companies with a large continuous assortment, fresh products, and for those with a wide product range offering direct-to-store deliveries. Our solution can improve end-to-end supply chain collaboration for both retailers and CPG companies.
Whatever solution you pick, make sure it leverages a full AI toolbox to capture the complexity of demand-influencing factors from all available internal and external data sources. The only way to calculate a good forecast is by feeding your demand planning system with enough relevant data. The data required to make accurate predictions is far too heavy and complex for humans to calculate, so retailers and CPG companies must look for an AI-driven solution whose machine learning algorithms can automate the majority of that work.
It’s essential, though, that this complexity doesn’t result in a “black box” of a system that forces planners to simply trust automated recommendations. The best demand forecasting systems are transparent. They provide planners with a clear understanding of how they calculate forecasts to support a wide range of short and long-term planning activities—from replenishment planning to workforce optimization and capacity management.
Furthermore, the demand forecasting system must be powerful enough to process data at scale. Both retailers and CPG companies generate enormous amounts of sales data every single day. Add to that the data from internal business decisions and external factors and the level of granularity required for accurate planning, and the importance of data processing power becomes apparent. Choose a solution with a fast, modern database, or you’ll end up waiting hours each and every time you need to run a calculation.
If a solution doesn’t satisfy these basic requirements, move on to the next one. To read in more detail about evaluating and choosing the demand forecasting software that will best serve your business needs, read Supply Chain Transformation: The Complete Guide.
6. Further Resources on Demand Forecasting
This guide is only a starting point. There’s more granularity to demand forecasting than what we’ve covered here—that’s why we’ve gathered these helpful resources. Dig deeper into the world of demand forecasting to find out how you can improve your supply chain and bottom line at the same time.
Challenges in Demand Forecasting
Managing Promotions and Weather in Demand Forecasts
Demand Forecast Accuracy
Demand Forecasting Software
Machine Learning in CPG Demand Forecasting