The Data Needed to Transform Supply Chains

Last fall after attending the CSCMP conference in Atlanta, I shared my observation that digital transformation is now a central theme in the industry. Many of you have had the same observation. This isn’t breaking news, but it is new for the industry. Again and again, digitization and data were at the heart of panel and networking conversations. Even headline speakers were professing “data got sexy” and data is now a core strategy for companies looking to succeed.

Yet amongst this buzz, I was challenged to find success stories. There was optimism and excitement, but the stories of transformative success were few and far between. Why? Why had transformation initiatives fallen short? What were the barriers and stumbling blocks? And how could a company succeed in digitally transforming?

Transform Into What?

The Hackett Group provides a useful visual for describing the maturity stages that supply chain is moving through as the industry strives to leverage data and advanced analytics.

Supply chain leaders are wanting their organizations to be data-driven. This means that companies must move from descriptive, rear-facing reports (stage 1) to prescriptive and forward-looking analytics to drive meaningful decision making (stage 4).

At ClearMetal we’ve observed that most companies are in Stage 1 and Stage 2, and small pockets in the industry are beginning to build towards Stage 3. Hackett appears to agree with this observation. But as we dug deeper, each executive we questioned, each transformation and analytics project we heard of falling short, comes back to the same root failure; the data was undependable and full of holes. Again and again, the quality of supply chain and logistics data is too low for companies to jump to the next maturity stage.

The industry has been complaining about data quality for years and the research says the same. A Deloitte study in 2017  found 49% of procurement officers believe the quality of data is a major barrier, and the lack of data integration was the number two barrier (42%). This data quality issue isn’t news to anyone, but it has continually hamstrung every company.

We all know how the industry handles the data issue. Thousands of hours are spent each year cleaning and reconciling the data by hand. This laborious process has been scaled in-house, then out-sourced, and partnered with rigid, error-prone ETL automation. These methods have reached their limits, even when used in conjunction with one another. No more scale or speed is achievable, and companies that have maximized these methods have only reached Stage 2.

So what does this mean for companies attempting to mature into an analytics-driven business? How do you go about continually maturing without stalling at Stage 2? What are those few companies doing that have a chance of jumping to Stage 3?

Data Challenges And Solutions By Supply Chain Maturity Stage

Each stage of supply chain maturity confronts unique challenges in collecting, managing, and leveraging data to accomplish the business objectives.

STAGE 1:  Descriptive Analytics Requires Consistent Data Collection

The first maturity stage is all about determining what has happened in the supply chain.

The challenge companies face in standing up Descriptive Analytics is simply consistent collection and/or access. Companies are uncertain what data is collected and where it is stored. Data is spread across internal silos, both known and unknown, as well as third-party providers who for years have controlled access to maintain control.

Establishing automated or consistent manual activity to audit and manage the collection and storage of data is the requirement to enter Stage 1. Knowing what data you have, what you’re missing, and where it’s stored enables teams to manually run their own analysis, report KPIs, and create a record of past activity.

STAGE 2:  Diagnostic Analytics Requires Data Canonicalization

The second stage, Diagnostic Analytics, allows organizations to execute root cause analysis for issues and inefficiencies across functions, departments, and operations. Put simply, why did something happen?

Supply Chains are a network of interdependencies and being able to analyze the data across functions requires a common language in the data, a referenceable canon. An event in one function or system needs to have an equivalent, translation, or handling rule for another function or system to understand that event. An example, when the transport team says a shipment has arrived at port, the Finance team needs to know this means a shipment has been unloaded from a vessel and ownership has changed hands and not that the vessel is at berth but the shipment won’t be unloaded for 24hrs. Canonicalization, sometimes called standardization or normalization, represents and protects the business logic that data supports.

This protection can be accomplished manually or via automation, likely a hybrid. This is much of what offshoring data cleansing and ETL mappings accomplish for companies. In Stage 2 businesses begin to leverage business intelligence (BI) tools for data visualizations and receive analysis that used to be done via pivot tables. Reporting moves to persistent dashboards and away from spreadsheets in emails.

STAGE 3: Predictive Analytics Requires Data Validation

In Stage 3, organizations begin to benefit from predictive intelligence and probabilistic decision making in designing and operating supply chains. Simply, what WILL happen?

The promise of predictive analytics offers the greatest value to operations teams running scenarios looking a few weeks into the future. It assists decision makers in being more precisely aware and proactive. Predictive Analytics take the best wisdom and “gut know-how” of top-performing team members and turns those skills into algorithmic detection, insights, and alerting.

The level of data quality required to unlock Predictive Analytics is significantly higher than the previous two stages. In the previous stages human comprehension can fill in the holes and gaps using business experience and analysis, but when jumping to predictive analytics this know-how must be captured in the algorithm that is fed by data. Previously a team member can say to disregard a questionable data point because they suspect bad data was utilized, but the algorithm can’t do that.

Some companies have been able to start implementing very basic predictive analytics that are marginally better than regression trending. They do this by using the same manual processes of offshoring the data cleaning process to accumulate a higher quality historical dataset to then power algorithms or begin experimenting with machine learning. This manual cleaning is focused on human detective work to validate what data is true, is questionable, or is outright wrong. This validation process is key to accessing the value of predictive analytics which require the most truthful data that represents operational reality.

STAGE 4: Prescriptive Analytics Requires Real-time Data Validation

The promise of Prescriptive Analytics continues to build the analytics value story. First know what happened, then anticipate what will happen, and then be told what to do.

Stage 4 offers the first assisted decision-making in current market conditions. Through AI simulations and probabilistic predictions teams will manage supply chain design, procurement, and transportation with greater precision and will balance the trade-offs using confidence of near-term future. Your operations and decision-making will be able to respond to ever changing market conditions in real-time and utilize agility to gain a competitive edge.

The jump from Stage 3 to Stage 4 will be very difficult for companies that are not already prioritizing digitization and IT resourcing. Data science approaches to cleaning and instantly analyzing granular events at a global scale unlock access to the value of Prescriptive Analytics. Even businesses that will utilize 3rd party solutions, vendors, and partners to access Stage 4 will still need substantial internal resources for managing, integrating, and leveraging the power of prescriptive. The validation activity that can be done by hand in Stage 3 must reach multiples of scale and speed in Stage 4. Machine Learning and AI must be used to validate and correct incoming data in real-time, enrich with additional contextual data, throw out invalid data, probabilistically fill the holes, and finally manage the canonicalization across a network of partners.

Reaching Stage 4 requires long term planning and strategic commitment.

Where To Begin Your Supply Chain Transformation?

If you have already begun a digital transformation process then you are on the right path. The key question is how much are you prioritizing the resourcing and automation to create the data foundation you need to support your journey along the Supply Chain Maturity Model.

Cleaning up data starts by asking “What problem are we trying to solve?” and “What are we using the data for?” Once that ultimate use case is identified, you can then determine the right data to collect and the format in which to process it. Focus on the data you use daily for KPIs and decision making. Focus on that first to ensure you get it right. That’s the data you use most often for business decisions.

Organizations must create standards and procedures for entering information into systems. Companies will need to create strategies for data governance, and then create long-term views of how to leverage it.

This is a long term strategy and commitment. It is easy to seek low-hanging fruit and look for the wins that payout today, but deciding to take the long road and put data first will pay much larger dividends over the next decade.

The supply chain industry is changing. Supply chains are digitizing. Leadership has mandated a data-driven future. Shouldn’t you be asking how you are managing your data today? Tomorrow? And how will you use data to grow?

If you always consider the data first, then you have the best chance of success as you begin the long journey into the digital world.

Adam Compain is the CEO of ClearMetal, a predictive logistics company that uses data science and machine learning to unlock new efficiencies for global trade. Compain co-founded ClearMetal after working in Hong Kong at one of the world’s largest container-shipping companies. For five years prior, Adam deployed the newest geo-commerce technologies at Google, and for 16 years he has been the Executive Director of the nonprofit he founded to export charitable goods. Adam holds five technology patents, two degrees from the University of Michigan, and an MBA from Stanford University.