tekBlueprint

A short article about a little-known, or maybe not, XML representation format. Once upon a time, when mobile devices were small and weak, people were looking for a way to save on traffic and computing resources when transferring data between client and server using XML. One option is to use a compact representation in the form of Efficient XML Interchange (EXI).

Introduction

Now there are quite a lot of data formats for compact storage and transmission of data. The most common of them is JSON, but it is not always suitable for tasks where strict typing or data validation is required. In this case, you can use XML, which is also a human-readable data format. But despite the flexibility and availability of various standards (XSD, XPath, XSLT), it is not very efficient in size, resource consumption (memory consumption if SAX is not used) and parsing speed. This is where EXI can be used.

What’s wrong with XML?

XML is a text format that can be easily read by humans. However, when working with large amounts of data, it may be ineffective for the following reasons:

• Suboptimal size.
• Difficulty of parsing.
• Typing problem.

At the same time, it also has advantages:

• Easy to read and understand the data structure, albeit cumbersome in some cases.
• Support for XML Schema Definition (XSD) to describe data structure and validation.
• XPath and XSLT support for searching and transforming data.

Not optimal size

Text data takes up more space than binary data. And XML itself can be quite verbose compared to JSON (it happens the other way around).

For example,

<sensor>
    <id>device-001</id>
    <temperature unit="C">24.5</temperature>
    <timestamp>2026-03-26T10:00:00Z</timestamp>
</sensor>

In text form, this event takes up about 136 bytes. Of these, the payload (ID, value, time) is approximately 30% (39 bytes). The rest is tag names, attributes, brackets and service symbols. If you transfer this kind of data every 5 seconds from a million devices, the extra 97 bytes turns into gigabytes of unnecessary traffic. In conditions of limited communication channels with poor connectivity, such as when data is transmitted over a cellular network or satellite channels, this can be critical. The situation is even more aggravated when transferring large amounts of data.

Parsing difficulty

XML parsing is more resource intensive than working with binary data. The parser must read a stream of characters, recognize lexemes, build a document model tree (DOM) or process events (SAX), and check encodings (UTF-8, UTF-16). This is a CPU intensive process. On weak CPUs, a full XML parser can take up a significant portion of available resources.

Let me give you an example - it is very old; now this is more likely to be true for microcontrollers. The mobile application was written for PDA (Windows CE) in JavaME and an XML-based protocol was used for communication with the server. Messages did not always arrive due to unstable connections, and sometimes the application itself crashed due to memory overflow. I had to look for possible memory leaks and edit in the VM (already abandoned at that time), and rebuild my version, which was a dubious pleasure.

Typing problem

XML is textual in nature. The value of <temperature>24.5</temperature> for the parser is a string. Conversion to floating point occurs at the application level. In binary protocols, the data type is explicitly encoded, which eliminates conversion errors and speeds up processing. This can be partially compensated for by using XSD with input validation.

EXI (Efficient XML Interchange)

EXI (Efficient XML Interchange) is a W3C standard (W3C EXI 1.0 recommendation). Its basic idea is to encode XML into a compact binary representation that significantly reduces data size and speeds up processing. EXI uses lookup tables (dictionary) and fixed length encoding. A complete description can be found in the [W3C EXI 1.0] standard (https://www.w3.org/TR/exi/#principles).

Essentially, EXI operates on a byte stream (similar to SAX), rather than an XML document, with the following events:

• Start Document
• End Document
• Start Element
• End Element
• Attribute
• Characters
• Namespace Declaration
• Comment
• Processing Instruction
• DOCTYPE
• Entity Reference
• Self Contained

Can work in two modes:

• schema-less – the encoder and decoder do not have access to the XSD schema. They only work with the stream of events. Element and attribute names are passed in the stream, but are compressed through the built-in dictionary. Frequently occurring strings are replaced with short indexes.
• schema-informed – both sides (client and server) have an identical XSD schema. A grammar (EXI Grammar) is built on the basis of the scheme. Grammar determines the structure of a document. Since it is known, the element names are not transmitted at all. Instead of the <temperature> tag, an integer index from the grammar is passed (for example, 5).

Compression Mechanisms

Using grammars and bit packing. Details can be found in the official specification, where everything is described in some detail. The approach itself is similar to dictionary compression algorithms (for example, LZ77), but using EXI grammars and special rules for packing bits. Almost always used in conjunction with EXI Schema-aware, as it allows the use of schemas to define data types and simplify coding.

For example, in text XML the unit="C" attribute occupies 9 bytes. In EXI schema-aware (I could be wrong in the numbers, but I think the meaning is clear):

1. The fact of the presence of the unit attribute is encoded by 1 bit (flag).
2. The value of "C" can be encoded as an enum index (if the schema defines unit as an enum), occupying 2 bits.

As a result, 3 bits versus 72 bits.

Additionally, classic compression algorithms such as Deflate (used in ZIP/GZIP) can be applied to the entire EXI stream. This gives maximum compression but requires CPU resources.

Comparison with other formats

Available libraries to use

Implementations of the standard are available for some popular programming languages:

• Java is the most mature implementation. The Exificient library is the reference library. It integrates with JAXB (Java Architecture for XML Binding).
• C/C++, .NET – library OpenEXI.
• Python – exi or pyexi library.
• JavaScript – there are implementations in Exificient.

Example of work in Java

Below is a small example in Java with XML and POJO classes generated via JAXB. The source XML was taken from the one at the beginning of the article (<sensor/>).

The task is to convert POJO to EXI format for storage or transmission and get POJO back from it.

Dependencies (Maven)

<dependencies>
    <dependency>
        <groupId>com.siemens.ct.exi</groupId>
        <artifactId>exificient</artifactId>
        <version>1.0.7</version>
    </dependency>

    <dependency>
        <groupId>jakarta.xml.bind</groupId>
        <artifactId>jakarta.xml.bind-api</artifactId>
        <version>4.0.2</version>
    </dependency>
    <dependency>
        <groupId>org.glassfish.jaxb</groupId>
        <artifactId>jaxb-runtime</artifactId>
        <version>4.0.5</version>
    </dependency>
</dependencies>

Converting POJO to EXI via JAXB

private static void marshal() throws Exception {
    final var sensor = new Sensor();
    sensor.setId("device-001");
    sensor.setTemperature(24.5);
    sensor.setTimestamp("2026-03-26T10:00:00Z");

    try (var osEXI = new FileOutputStream(EXI_FILE_JAXB)) {
        final var exiResult = new EXIResult(DefaultEXIFactory.newInstance());
        exiResult.setOutputStream(osEXI);

        final var factory = JAXBContext.newInstance(sensor.getClass());
        final var marshaller = factory.createMarshaller();
        marshaller.marshal(sensor, exiResult);
    }

    final var exiSize = Files.size(Paths.get(EXI_FILE_JAXB));

    System.out.printf("JAXB (serialize): EXI size: %d bytes%n", exiSize);
}

Convert EXI to POJO via JAXB

private static void unmarshal() throws Exception {
    final var exiSource = new EXISource(DefaultEXIFactory.newInstance());
    exiSource.setInputSource(new InputSource(EXI_FILE_JAXB));

    final var factory = JAXBContext.newInstance(Sensor.class);
    final var unmarshaller = factory.createUnmarshaller();
    final var sensor = (Sensor) unmarshaller.unmarshal(exiSource);

    System.out.printf("JAXB (deserialize):  %s%n", sensor.toString());
}

Direct XML to EXI conversion

private static void encode() throws IOException, EXIException, SAXException {
    try (var osEXI = new FileOutputStream(EXI_FILE)) {
        final var exiResult = new EXIResult(DefaultEXIFactory.newInstance());
        exiResult.setOutputStream(osEXI);

        // java.xml

        final var xmlReader = XMLReaderFactory.createXMLReader();
        xmlReader.setContentHandler(exiResult.getHandler());
        xmlReader.parse(XML_FILE); // parse XML input
    }

    final var xmlSize = Files.size(Paths.get(XML_FILE));
    final var exiSize = Files.size(Paths.get(EXI_FILE));

    System.out.printf("XML size: %d bytes%n", xmlSize);
    System.out.printf("EXI size: %d bytes%n", exiSize);
    System.out.printf("Ratio: %.2f%%%n", (double) exiSize / xmlSize * 100);
    
    /*
    Encode:
     XML size: 136 bytes
     EXI size: 81 bytes
     Ratio: 59.56%
    */
}

The final EXI size is more than half the size of the original XML.

Direct EXI to XML conversion

private static void decode() throws Exception {
    final var exiSource = new EXISource(DefaultEXIFactory.newInstance());
    exiSource.setInputSource(new InputSource(EXI_FILE));

    // java.xml

    final var transformer = TransformerFactory.newInstance().newTransformer();
    transformer.transform(exiSource, new StreamResult(XML_FILE_DECODE));

    final var xmlSize = Files.size(Paths.get(XML_FILE_DECODE));
    final var exiSize = Files.size(Paths.get(EXI_FILE));

    System.out.println("Decode:");
    System.out.printf("\t XML size: %d bytes%n", xmlSize);
    System.out.printf("\t EXI size: %d bytes%n", exiSize);
    System.out.printf("\t Ratio: %.2f%%%n", (double) xmlSize / exiSize * 100);
    
    /*
    Decode:
     XML size: 211 bytes
     EXI size: 81 bytes
     Ratio: 260.49%
    */
}

ℹ️ Important

When using JAXB, the business logic does not change. The work is still done through JAXBContext and with regular Java objects.

Risks of using EXI

Probably the biggest problem is the “unreadability” of the format. You can no longer simply open messages or documents in a text editor. As a solution, you can make a debugging tool with EXI decoding similar to exificient-gui. For example, this may be necessary for logging in test environments. On industrial applications this may be disabled for security and performance reasons. If this is not done, then the time to diagnose incidents (MTTR) increases.

There may be problems with versioning. You can simply add a new field to JSON and legacy clients will ignore it if deserialization is configured to behave that way. In EXI with schema, changing the schema may break compatibility. How to get around:

1. Use xsd:any for expandable parts of the document.
2. Never remove fields from the schema, only make them optional (as in protobuf).
3. Use namespaces (as with regular XML) for versioning (for example, v1, v2).

Conclusion

It doesn’t always make sense to use EXI if you have sufficient computing resources and good network bandwidth. If you need to reduce traffic and the required data storage space, then even conventional text compression algorithms from the PPM* family can be more efficient.

tekBlueprint applications store a lot of data in XML format, and the documents can be quite voluminous. Components between applications and tekBlueprint.kbase are also transferred as XML. Therefore, EXI was considered as an intermediate storage format.

ℹ️ Note

Sample project is available in GitHub.