EDI is the transfer of data between two computers without human intervention, with the data structured to meet a common standard. The sentence you just read is probably as painful as a Lego brick under a barefoot, yet, as long as you overcome a few barriers to entry, the EDI universe is an ideal case study to materialize the issues associated with interoperability, the multiplicity of languages (human or computer) and the ambivalence of "real" data.

‍

‍

To clarify some acronyms:

• EDI refers to Electronic Data Interchange, a technology that allows two companies to exchange data.
• EDIFACT stands for Electronic Data Interchange for Administration, Commerce, and Transport, which is a term within the EDI ecosystem that refers to a standard defining the rules to be followed in terms of content and syntax for an EDI message, and in specific contexts.
• SMDG stands for Ship Message Design Group, which is a term from the EDI ecosystem of container shipping that refers to a non-profit organization responsible for writing and validating the EDIFACT standard specifications with the mandate of the United Nations.
• UN/CEFACT stands for United Nations Centre for Trade Facilitation and Electronic Business, which is a term from the United Nations ecosystem that refers to a body responsible for establishing or validating recommendations, standards, or specifications for international trade and electronic commerce.

‍

Why talk about EDI, EDI standards, and standards organizations in this blog? Because interoperability, which eFTI advocates, is already at the core of the EDI ecosystem. We can learn from our forefathers and peers who have both succeeded and failed before us.

‍

Talent in the industry is like mushroom clearings in Bearn or the truth in negotiation. No matter how well we know the right corner, we can never be sure of finding it, and no matter how much we know the truth, we are not immune to a sophism that will publicly prove us wrong. The edifying history of EDI standards will always be there to remind us that the right technical solution will not be **the** solution if all other parameters do not lead to it. The other parameters are abundantly listed in the literature: implementation cost, human factor, learning curve, maturity of organizations, power of habits, "side effects" and other "interfaces", social or political power relations, lack of knowledge of operational practices, learning curve, a critical mass of users, competition from other technologies, lack of visibility, pedagogy, ambassadors. As a technology, EDI is an old subject in the world of enterprise computing, but it is still predominant in practices, we probably have to learn from it in the practices of the present and the future. Eddy's word - Long-term philosopher

‍

‍

Before delving into EDIFACT and the SMDG, let's take a digression into the Far West, which has always nurtured avant-garde ideas, especially when it comes to helping its European cousins during the Cold War.

Like many of its technological counterparts, EDI has its roots in the military. Its emergence dates back to the post-World War II era during the airlift to West Berlin. While it's unclear whether Christopher Nolan's next film will be an ode to American Sergeant Ed Guilbert, his 1948 anecdote is worth telling.

The urgency of supplying the city led to questions of who would fly the planes, what goods were contained in which package, and how to keep track of them. Ed Guilbert and other officers created a standard manifest system (similar to today's ASN) that could be transmitted by telex, radio teletype, or telephone. The trick was to erase the diversity of document formats and reduce the language barrier. With this standard, thousands of tons of goods were delivered every day for over a year until the roads were reopened in 1949. This could anachronistically be called the first POC that allowed Guilbert to re-exploit the concept in the early 1960s at Du Pont Co. to exchange information with his carrier Chemical Leahman Tank Lines. In 1965, the Holland-America Steamship Line relied on a procedure that would make the RPA champions swoon:

• Transatlantic telex transmission
• Printing on perforated strips
• Import into a computer

‍

‍

Many companies have embraced EDI.
The horizon looks bright for large companies in the position of prescribers and customers (of physical and data flows), but smaller suppliers are struggling as they are not yet equipped with the expensive software and network infrastructure required, known as VAN (Value Added Network). Sometimes a single supplier has to comply with several standards, sometimes one standard per customer. This dialogue illustrates that the technical relevance of a solution is one thing, but that the will of the market and its "big players" is probably even more essential.To fully realize the benefits of EDI across the market, more efforts are needed to standardize implementation and reduce costs for smaller suppliers.

‍

Justice only enters into the reasoning of men if the forces are equal on both sides; otherwise, the strong exercise their power, and the weak must yield to them. Thucydides

‍

To take another step towards fluidity, the Transportation Data Coordinating Committee (TDCC) was created in 1968 and Buddy Bass was involved. Their work led to the first EDI specification in 1975, giving rise to 45 "transaction sets". Large companies in all industries followed the lead of the pioneers: food processing in 1977, automobiles in the early 1980s, and mass retailing, all eager to save time and money by getting rid of paperwork. Today, the ASC X12, governed by ANSI, has succeeded the TDCC and there are now more than 300 transaction sets in use in the United States.

A study by the Royal Society of Medicine has demonstrated the benefits of reading the long list of ASC X12 transaction sets on insomnia. It should be noted, however, that there is no effect on EDI experts, who see in these glyphs what the average person cannot see.

‍

‍

EDIFACT, SMDG

Outside of the American perimeter, EDIFACT and the SMDG are the keystones of the maritime EDI universe, a universe shrouded in the aura of the 90's, which some nostalgic people remember with a tear in their eye, eyes moistened by the evocation of a free internet where the content had not succumbed to the form and of which here are some witnesses:

• The official website of the SMDG before its recent redesign
• Technical specifications of the messages on the official site of the UN
• More or less shady tools that make you wonder what they do with your data
• Antediluvian tools that nevertheless provide many services
• Legendary tools for insiders. Every self-respecting vessel planner has once blessed Capt. Dragan Milatovic for his free BAPLIE reading program, which is used extensively in some ports more than 10 years after its last update.

‍

EDIFACT 101

In 1987, the UN and ISO recognized EDIFACT as an international standard. The principle is simple: a sender wishes to send a series of information to a recipient. To do this, a communication is started that will be called an Interchange (often called an intercourse for obvious reasons). It can be imagined as a communication pipe between the two interlocutors, which allows for sending emails. Within this Interchange, the sender sends as many envelopes as they want, which are called Message. Each Message can be seen as an envelope duly filled in which one slips the mail, the letter, or the postcard that one wants to transmit: the Body of the message. Concretely, we translate these steps and this information into a "simple" text file (that anyone can have fun writing in the notepad of his favorite operating system) structured in such a way that it is optimal for a computer system to decipher it (the "parser" in good franglais) but also for a human being to decipher it (the "debugger" if he wants). Each line is a segment that starts with 3 characters (BGM, LOC, DTM...) that define the "meaning" of this line. Within a line, we separate the different information by using the "+" sign when they are not linked and ":" when they are linked. Each segment ends with a ' (among other things to ensure that computer systems detect the end of each line. While line break are practical for human readability, they are less practical for computers). It is the SMDG that thinks about which message to use, the structure of the messages, and the coding of the data.

Below, we attempt to explain a CODECO message exchanged between Jimmy Page and Ringo Star recently, which pertains to the notification of a container's entry or exit from a terminal. We have also provided a more concise summary of the message without any additional comments.

‍

Jimmy Page initiates a transaction with Ringo Star on 04/19/2022, it is a CODECO Version D transaction, Release number 00B, the one validated by the UN and managed by the SMDG. It is a brand new message to notify you that the cargo for booking 800255846 passed through the gatehouse on 18/04/2022 at 12:12, that it was transported by Steve Vai in PONU4863849, that it weighs 17t, certified by Jean Ederme PESEE, which is a nice weight for a 40 feet baby (ISO 45G1 container). The seal is in good condition, or the fraudsters are getting good at it. If you need more info we call you, as usual.

‍

With a little practice, it may be healthy to prefer this more concise format (without the comments present for educational purposes).

‍

UNB+UNOA:2+JimmyPage+RingoStar+20220419:1009+030410' -- Interchange start
UNOA:2 is a technical convention

JimmyPage sends a message to RingoStar on 04/19/2022 at 10:09 am

030410 is the interchange identifier repeated at the end of the interchange

UNH+241443010001+CODECO:D:00B:UN:SMDG21' -- Message start
241443010001 is the identifier of Message

We send a CODECO that respects Version D, sub-version 00B of the standard validated by the United Nations (UN) and created by the SMDG in 2021. This is what allows a computer tool to apply the right reading grid to the following information.

Here we have three elements separated by +, these are three pieces of information that live independently of each other. The last element contains sub-elements separated by :, these pieces of information are linked to each other (We are talking about a CODECO Version D Release 00D approved by the UN and drafted by the SMDG in 2021, all this is in one piece)

BGM+109+800255846+9' -- Beginning of message
Jimmy sends Ringo an initial (Code 9) gatehouse entry notification (Code 109) with the identifier 800255846. If Jimmy wants to send an update or correction on this entry, he can send back a message with code 5 (Replacement message) instead of code 9 (Original message).

DTM+137:202204181212:203' -- Date & Time
Date of sending (Code 137) of the message in YYYYMMDDHHMM format (Code 203)

RFF+BN:800255846' -- Reference
Reference number of a booking

RFF+VOR:V00012345' -- Reference
Reference number at time of weighing

NAD+CF+Steve Vai:172:20' - Name & Address
Identification of the carrier (Code 172) according to the BIC (Bureau Internation des Conteneurs) code (Code 20)

NAD+WPA++JEAN-EDERME PESEE:Rue du Molinel:Lefrinckoukhe:France' -- Name & Address
Identification of the entity that weighed the goods

EQD+CN+PONU4863849+45G1:102:5+1+2+5' -- Equipment details
Identification of a container (CN) via its number PONU4863849, but also of the code 4532 which we are told corresponds to a type and size identification code (Code 102) which respects the ISO standard (Code 5). It is a container provided by the shipper (Code 1), direction Export (Code 2) and it is full (Code 5).

MEA+AAE+VGM+KGM:12373' -- Measurement
The container weighs 12373 kg and the weight is VGM certified

SEL+V0465672+SH+1' -- Seal number
The shipper lead (SH code) is in good condition (Code 1) and the identification number is V0465672*.

FTX+ABS++SM1:ZZZ:SMD' -- Free text
"Free text" area that everyone ignores because nothing is standardized as a rule. A bit of an unfortunate catch-all for putting additional information.

CNT+16:1' -- Control total
We summarize the quantity of information in the message we have just read: here we have 1 container (Code 16) in the message. We could have made a weight total or something else.

UNT+17+241443010001' -- Message end
Message tail that includes the UNH segment identifier

UNZ+030410' -- Interchange fin
Interchange tail which takes the identifier of the UNB segment

Here it is in its simplest form:

UNB+UNOA:2+JimmyPage+RingoStar+20220419:1009+030410' -- Interchange beginning
UNH+241443010001+CODECO:D:00B:UN:SMDG21' -- Beginning message
BGM+109+800255846+9' -- Beginning of message
DTM+137:202204181212:203' -- Date & Time
RFF+BN:800255846' -- Reference
RFF+VOR:V00012345' -- Reference
NAD+CF+Steve Vai:172:20' - Name & Address
NAD+WPA++JEAN-EDERME PESEE:Rue du Molinel:Lefrinckoukhe:France' -- Name & Address
EQD+CN+PONU4863849+45G1:102:5+1+2+5' -- Equipment details
MEA+AAE+VGM+KGM:12373' -- Measurement
SEL+V0465672+SH+1' -- Seal number
FTX+ABS++SM1:ZZZ:SMD' -- Free text
CNT+16:1' -- Control total
UNT+17+241443010001' -- Message fin
UNZ+030410' -- Interchange fin

‍

An EDIFACT message is rarely much more complex, but that does not foreshadow anything that comes from the information transmitted. Such a CODECO will be for some actors and cargo managers the only source of information to know the VGM weight of a cargo and its date of entry in a port for example. With other messages such as COPRAR (Confirmation of Loading List on board a vessel), the entire loading plan of the vessel is derived from it, and the entire organization of a container terminal for the upcoming call (with all that this implies in terms of searching for containers, reconciling what is actually in the possession of the terminal and what is not, numbering errors, etc.).

‍

There are geniuses whose glory resounds unanimously in the plains of knowledge: Socrates, Louis Pasteur, Albert Einstein. There are other, slightly less singular minds whose posterity is assured but sometimes controversial: Diogenes, Thomas Edison, Steve Jobs. And then there are the obscure inventors, the creators of the shade: Ed Wood, Jonathan Sifflé-Ceutrin or, the one which interests us, the SMDG. These are people who doubt but who act humbly, with passion and dedication, without imagining that sometimes the fruit of their work has an extraordinary repercussion, namely the creation of the only EDI standard deployed internationally. As a standard, EDIFACT blows a wind of pragmatism that is effective, although not without flaws. As an organization, the SMDG embodies a frail, determined and hard-working skiff (it seems that it pays off). Jean-Jacques Codeman - Divulgacheur of the EDI of Nantes

‍

The impossibility of the MIG

EDIFACT and ASC X12 are necessary, and their coexistence is probably inevitable to manage the differences at the international level, especially in customs and transport. Harmonization has its limits, otherwise, we would fall into the trap of leveling everything out. After all, why should identity claims spare the technological community? And is it reasonable to shamelessly flout local specificities under the pretext of excessive optimization in the service of galloping globalization? More seriously, the position of the standardization cursor is a crucial and complex subject because every standard has its interpretation, and everyone has their truth.

As early as the 1970s, the potential was fully identified: transmission of information with unbeatable speed, unprecedented reliability of data entry, and reduction of human intervention (and the security virtues that this can generate): Eden. However, this is without taking into account that, paradoxically, the incommunicability of computer systems sometimes mirrors that of people, and the path toward the fluidity of exchanges is full of pitfalls. We try to remove the obstacles with a lot of reference material, standardization, or harmonization, all terms that always mean the same thing: making sure that we understand each other. One of the big words we often hear is MIG for Message Implementation Guideline. Each message has its own MIG, or rather its MIGs.

On paper, everything seems clear when it's written down. But in reality, while a standard may be unique, interpretations can vary, and each actor has their own implementation. While the variations may be small for a given message, they require adjustments in IT systems. Therefore, the 'one standard per customer' syndrome remains unbreakable, particularly for huge consortia who don't negotiate on their specifications.

‍

• One message per container or multiple containers in one message?
• Is the loading port transmitted behind code 9 or 11?
• When there are several terminals for the same UNLOC code, what syntax should be applied?
• When two terminals in two different countries have the same terminal code and my recipient does not ingest the associated UNLOC code to distinguish them, how do we resolve the ambiguity? (TICT)
• I may know that 172 is the carrier identification code according to BIC, but it doesn't tell me how I translate ER-72BJ-ZZ in MY software database.
• As a terminal operator, I may feel that it is the vessel operator's responsibility to send a consolidated COPRAR, but if the local practice is for each slotter to send their own load list, it is up to me to do the consolidation work.

‍

SMDG but not only

Among the organizations whose activities are closely related to the work of the SMDG are:

• BIC
• DCSA
• EIPP SAB
• EXIS
• ITIGG
• PROTECT
• UN/CEFACT

‍

‍

Here is a paragraph to look at the bigger picture through the lens of EDIFACT and SMDG, which have many cousins.

‍

Some standards bodies (The cousins of the SMDG)

• GS1 EDI set of standards developed the GS1 predominant in global supply chain
• Odette
• GALIA
• OpenPEPPOL
• ANSI (American National Standard Institute)
• AIAG (Automotive Industry Action Group)

‍

Some EDI standards/message standards (EDIFACT cousins)

The essentials of logistics

• ASC X12 (Accredited Standard Committee) in North America.
• UN/EDIFACT in almost all the world

Health

• HL7 data standard that aims at interoperability for health services
• SCRIPT is a National Council for Prescription Drug Programs (NCPDP) standard for the electronic transmission of medical prescriptions.

High-tech

• RosettaNet is widely used in the electronic component manufacturing community. It is developed by GS1.

Government exchanges

• Peppol (Pan-European Public Procurement Online) on cross-border procurement by implementing electronic invoicing. It is based on AS2 or AS4, the PEPPOL access points accept EDIFACT, Factur-X, XML and PDF...

The automobile

• VDA (Verband der Automobilindustrie) is used in the European automotive industry, mainly in Germany.

The snake that bites its tail

• SAP IDoc (Intermediate documents). Standardization to exchange information between an SAP system and a non-SAP system. It is a re-implementation of EDIFACT and X12 in the SAP ecosystem. We fight as best we can against the monsters we create.

The meta thing

• SEF (Standard Exchange Format). A text file schema that contains the guideline implementation in a hybrid format that can be read by humans and decoded by a computer system. It is a form of perfectionism that encodes the standard and would allow the software to be configured automatically via a text file (modulo the prior configuration of the translation of this file in each software). If someone has implemented this standard, let him renounce himself and enlighten the community.

‍

Some EDI subsets/message sub-standards (EDIFACT's little brothers and nephews)

• EANCOM (European Article Number Communication). EDIFACT subset on a large scope of commercial activities between a buyer, a carrier, a supplier and a bank (quotation, purchase order)
• IATA Cargo-IMP (International Air Transport Association Cargo Interchange Message Procedures). For airlines and their third parties.
• ODETTE (Organization for Data Exchange by Tele Transmission in Europe). EDIFACT subset on the European automobile.
• Edig@s (EDIGAS). EDIFACT subset also XML compatible on gas trade, transport and storage (via pipeline or container). Derived from EDIFACT but also compatible with an XML syntax.
• TRADACOMS. Standard developed by the ANA (Article Number Association, now GS1 UK) for the UK retail industry. In reality it is a subset of an earlier EDIFACT standard: UN/GTDI. TRADACOMS has been superseded by EANCOM but is still widely used in the UK.
• HIPAA (Health Insurance Portability and Accountability Act). X12 subset in the area of health insurance.

‍

Some communication protocols/standards (VAN cousins)

• AS1
• AS2 (created in 2002 by Walmart)
• AS4
• Email
• FTP, SFTP and FTPS
• HTTP/HTTPS
• OFTP (and OFTP2)

‍

‍

These orders of magnitude only commit their sources (and those who believe in them)

• With Factur-X, the reduction of the gross cost of an invoice, which today averages €8.75/invoice, would be achieved. This cost would be divided by 3, going from an invoice of almost 9€ to 3.2€(source GALIA / GS1).
• According to GS1, with the Factur-X, the overall cost of an invoice throughout the processing chain (estimated at 22e in paper format) is reduced by half (Arthur D. Little study for Deskom) and processing time is reduced by 30%.
• According to the 2008 Aberdeen report "A Comparison of Supplier Enablement around the World", only 34% of purchase orders are transmitted electronically in North America. In EMEA, 36% of orders are transmitted electronically and in APAC, 41% of orders are transmitted electronically. They also report that the average paper requisition to order costs a company $37.45 in North America, $42.90 in EMEA and $23.90 in APAC. With an EDI requisition to order, costs are reduced to $23.83 in North America, $34.05 in EMEA and $14.78 in APAC.

‍

‍

The evolution of the market seems to prove wrong the clichés convinced that EDI technology is doomed to disappear because it is prehistoric, probably because it has invested such a large perimeter that it can only be replaced within the framework of heavy, risky IT projects with not always obvious benefits. We are dealing with a kind of "too big to fail" syndrome, even though its shortcomings are widely recognized and more recent technologies are filling these gaps.

Among the developments we have seen:

• IoT sensors embedded in the packaging of shipped packages and linked to regular EDI messages to improve visibility of package status in near real-time.
• Blockchain technology as the underlying technology for EDI information flows for shipments to share facts and help quickly resolve or even eliminate chargeback disputes.
• An AI agent to monitor all relevant events and information related to a shipment and able to identify a non-compliant event. AI agents can also determine if a reshipment is necessary, analyze the most efficient alternative source, initiate a new shipment and accept an authorized return.

‍

‍

EDI may not need to be included in the curriculum of French elementary schools to combat radicalization, but demystifying what is ultimately just a silly human-readable text file could simplify some debates.