Supply Chain Optimization using Big Data

As we witness a pivotal change in the way big data is revolutionizing and redefining all aspects of our lives, it becomes increasingly necessary for professionals from all domains to think radically on its application in their industries. The inventions around the Hadoop ecosystem has enabled ground-breaking technologies from driverless cars to intelligent assistants like Siri. It is not surprising, that the crucially important field of supply chain optimization, is ripe for a major breakthrough in how it has been approached until today.


Traditionally, procurement has been planned around either predefined reorder points triggering a procurement request, or around fixed forecasting period using safety stock and average sales forecast. The problem with this approach was that there was no feedback loop to react in real time as business scenarios changed. This lead to either a “lost opportunity” in terms of not having the right inventory or the right price, or “dead stock” due to wrong stocking or purchasing decision. 




This problem of not being agile and responsive to the events occurring in the marketplace can be addressed by using big data technologies. The process starts by dividing the various steps involved in supply chain automation into multiple operational windows. This facilitates the prioritizations of various decisions based on how frequently they need to be evaluated. The results of each phase in the process feeds into the decisions of the next process thus creating a positive feedback loop which makes the entire process more responsive to external events.

The process starts with Strategic planning which involves the high-level analytics process in Hadoop to baseline the data. In this we automatically calculate the various parameters which impacts the supply chain decision process. This process will generally be an iterative process, run on a quarterly or monthly schedule, based on the type of business. The metrics from previous period will feed into this process and the performance of various parameters is evaluated and tweaked accordingly.

The next phase involves tactical decisions making, where various decisions regarding procurement and transfers are made based on the parameters and demand forecast. In this phase decisions related to what to buy, when and from which vendor are made. The decisions on how to stock a multi-echelon distribution network is also made in this step.

After this step, the next phase involves continuous evaluation of the performance of the supply chain and making tweaks to the inventory placement, the price at which to sell etc. These techniques of near real-time decisions are also referred to as “Demand Sensing”.


The Details

Strategic Planning:

  • Inventory categorization: In this part, various methodologies of categorization of Inventory is used. This includes FNS classification, Order frequency analysis, Price sensitivity analysis.
  • Multivariate clustering: The various parameters which influences the demand are then automatically evaluated by creating clusters using techniques like Principal Component Analysis and other clustering models.
  • Determining best-fit algorithm: Each Item in the inventory has a different demand pattern, it could have a trend, seasonality etc. The model which will be the best to forecast the demand would vary for Items in different clusters. The best model is identified and stored for forecasting.
  • Multi-echelon network calculation: If the company has multiple warehouses which form a part of the distribution network, we need to determine the best strategy of roll-up and aggregation for each Item in the network.
  • Supply chain parameters: The various parameters which influence the procurement and transfers are calculated based on the demand pattern and historical receiving performance.

Tactical Planning:

  • Demand forecast: The demand forecast for the various Items in the inventory for the selected period. The best-fit algorithm and clusters determined in the Strategic planning process is used to calculate the forecast.
  • Procurement plan: The projected demand and the forecasted inventory position in the period is used to calculate the procurement plan. The historical performance of the vendor is used to determine the date of order and the quantities. The EoQ, Safety Stock and other inventory parameters are used to create the procurement plan for the period.
  • Inventory transfers: For a distribution network, the stock placements at various locations are calculated and the transfers are created.

Demand Sensing:

  • The most crucial aspect of the big data architecture is the ability to respond to changes in the actual sales and adapt the strategy to it.
  • The “Lost sales” can be tracked and compared against the forecasted sales to evaluate a under or over-demand scenario. If the demand is more than the forecasted sales, the Purchase orders can be expedited to meet the unexpected demand. This can also lead to decisions to internally transfer inventory across various locations (Inventory levelling).
  • The price sensitivity determined during the strategic planning phase can be used to increase lagging sales. It can be decided to run promotions to boost the sales to the expected values.
  • Some of the variations in supply chain, like delay in shipments by vendors can be handled by either inventory levelling or expediting other POs on order.
  • The advanced feature of Text analytics can be used to forewarn of potentials disruptions to the supply chain and precautionary steps can be taken to avoid any impact to the Inventory.



The new age of data science and big data technology opens new vistas for automating the hitherto manual process of supply chain optimization. Technologies like Hadoop enable working with SKUs running into millions of counts and historical data running into several years with billions of transactions. The integration of machine learning libraries in tools like Spark has brought predictive analytics into the mainstream.  Latest Lambda and Kappa architectures enable streaming processing of near real-time data and creation of predictive models which can respond to changes in business patterns. The above process can greatly improve the performance of the supply chain and thus the overall business.

Video of the webinar :-

Contact us at for a free evaluation of how big data can be leveraged to provide you a competitive advantage.