As companies have moved to make more and more supply chain and purchasing activity digital, the data formats utilized to share that digital information have multiplied in kind. The advent of multiple format options brings flexibility and the power of choice, but can also lead to additional cost, even in today’s world of transform engines and configurable system connectors.
In this blog post, we’ll consider the effort required to maintain integration of these various data feed formats, the feasibility of standardizing formats across various partners, and how that standardization can be accomplished.
Data Format Proliferation
The explosion of data formats surely didn’t happen in a big bang, but was an evolution over time. As with other business tools, existing formats were improved and new formats were developed to meet changing requirements. This has created one of the widest ranges of data formats in existence today. It is no surprise then, that as companies have added new suppliers or partners, the variety formats required for integration have kept pace.
A manufacturer with multiple integrated supplier and distribution partners might be working with data feeds in a number of formats such as XML, CSV, EDI, JSON, etc. This is quite beneficial from a business standpoint; however the side effect is that each format feed requires maintenance and support whenever software or hardware updates are done, requiring in turn dedicated effort from the internal IT team. Clearly it would be more cost and labor efficient to simplify and limit the number of data formats to be managed.
Controlling the Data Format Explosion
Your company may have accepted the non-ideal situation above because it seemed too costly or impossible to alleviate; however, that is not necessarily the case. There are now opportunities to achieve a higher degree of standardization of data feed formats.
One such opportunity is when you are planning to implement a new collaboration system/portal with your trading partners. Instead of simply moving all existing processes and data streams to the new platform, this is an ideal time to review your data format policies and consider what standardizations you can put in place. You could find that more of your trading partners are in a position to help standardize feed formats than you presumed. We have outlined several steps to take:
1. Identify collaboration systems/portals which have capabilities to integrate with your partners in a variety of formats (EDI/XML/CSV/etc.), but deliver information through a single console. While many systems may claim format flexibility, this claim can sometimes be deceiving. Often there are a few formats still relegated to direct back office system integration, leaving you with disparate feed formats and disparate data channels to manage. The better systems will be architected with powerful messaging layers and transform engines that can either pass data through to other systems or enable transaction monitoring and control through a centralized management console. Learn more with our recent blog post on successful supply chain collaboration implementation.
2. Plan for a single format data feed between your internal system and the 3rd party system. This would help reduced the support required from your internal IT team going forward. For example, in the current environment the internal IT team must support, maintain, and troubleshoot different types of data formats (in terms of processing the data received as well as processing the internal data to send out.) If a company has a few partners on EDI and also some requiring data in XML, a company needs to be able to process all these feeds which require different technologies. Consolidating all these feeds into a single designed format suitable and easy to support based on internal IT team experience will simplify operations significantly.
3. Finally, and this is crucial, avoid the “all at once” approach – the better idea is to stagger onboarding so that the risk is minimized and focus is maintained on successful implementation. Below is one suggested approach:
- EDI formats – Plan for transitioning partners one at a time. Even though there are EDI standards for each transaction, EDI format integration is likely to be more complicated and require additional discussions and testing of the feed for each partner.
- XML formats – Plan for transitioning 1 to 2 partners at a time. Because XML is less complicated than EDI, but more than CSV, this becomes the “just right” format for integration.
- CSV/JSON – Plan for transitioning 3 to 5 partners at a time. Compared to XML, CSV files are easier to read and hence slightly easier to integrate.
We currently have a customer that divides their suppliers into Low, Middle and High Tiers. Low-Tier suppliers have low transaction volume and only input data manually into the online Portal without any automated data feeds. Mid-Tier suppliers have some level of data automation, while High-Tier suppliers have EDI and high volume. The customer has on-boarded the Low- and Mid-Tier suppliers working out any issues along the way. This has eased the path for transitioning the High-Tier suppliers that make up the bulk of the transactions and $ volume.
Standardizing Supply Chain Data Streams for Simplified Support and Management
It may seem that your data streams and the corresponding formats have multiplied like gremlins eating after midnight, but it’s not too late or too costly to move to standardization even now. Implementing a new automated supplier collaboration system is an ideal time to discuss and negotiate this transition with your trading partners. There are advantages and disadvantages to each format that your partners will recognize as well. Leveraging the advantages and the benefit of standardization will only make relations with your trading partners (and your IT team) stronger.