SOAP Protocol and Binary Payload Challenges

Web Services rely on SOAP, a lightweight XML-based protocol, to exchange structured data across distributed systems. Platform-independent and language-agnostic, SOAP structures messages using an Envelope, which contains an optional Header for extendability (like security or transaction contexts) and a mandatory Body for the primary payload. When errors occur, the Body encapsulates a Fault element detailing the failure.

Because SOAP is fundamentally a text-based XML format, incorporating binary files—such as media or documents—poses a significant hurdle. The most straightforward approach involves Base64 encoding, which transforms binary streams into ASCII characters using a 64-character alphabet. According to XML Schema definitions, this data is typed as xs:base64Binary.

<?xml version='1.0' ?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <soap:Body>
    <Product id="item01">
      <Thumbnail xsi:type="xsd:base64Binary">aW1hZ2VfZGF0YV9lbmNvZGVkX2hlcmU=</Thumbnail>
    </Product>
  </soap:Body>
</soap:Envelope>

However, Base64 introduces a 33% payload bloat by converting every three octets into four 6-bit encoded characters. The associated CPU overhead for serializing and deserializing these payloads heavily impacts application performance and network latency.

SOAP Messages with Attachments (SwA)

To bypass Base64 inefficiencies, the W3C introduced SwA in 2000. SwA leverages Multipurpose Internet Mail Extensions (MIME) to package the primary SOAP envelope and binary attachments separately within a single HTTP request. The payload uses Multipart/Related content type, where the root part is the XML document, and subsequent parts hold the binary data. Attachments are linked from the SOAP body using href attributes referencing the MIME part's Content-ID.

POST /inventory HTTP/1.0
Content-Type: Multipart/Related; boundary=MIME_separator; type=text/xml
Content-Length: 2048
SOAPAction: http://example.org/checkStock

--MIME_separator
Content-Type: text/xml; charset=UTF-8
Content-Transfer-Encoding: 8bit

<?xml version='1.0'?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <Product id="item01">
      <Image href="cid:logo@shop.example.org"/>
    </Product>
  </soap:Body>
</soap:Envelope>

--MIME_separator
Content-Type: image/jpeg
Content-Transfer-Encoding: binary
Content-ID: <logo@shop.example.org>

...binary JPEG data...

--MIME_separator--

While SwA eliminates encoding bloat, parsing MIME boundaries by scanning for string delimiters is computationally expensive, especially for large payloads. More critically, SwA breaks the XML Infoset; the attachments exist outside the XML document, rendering standard XML technologies like XPath, XSLT, and XML Schema unable to process or validate the linked binary content, leading to severe interoperability issues.

DIME and WS-Attachments

In 2002, Microsoft and IBM proposed Direct Internet Message Encapsulation (DIME) and WS-Attachments to address SwA's parsing inefficiencies. DIME is a binary record-based format where each record contains a fixed-length header specifying the payload length, type, and ID. Flags like MB (Message Begin) and ME (Message End) mark record boundaries, allowing parsers to jump directly to specific records without scanning the entire stream.

WS-Attachments dictates how to wrap SOAP messages inside DIME records. The primary SOAP envelope must be the first record, while attachments follow as secondary records referenced by their IDs.

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <order:ItemDetail xmlns:order="http://example.net/order">
      <order:attachment href="uuid:A1B2C3D4-E5F6-7890-ABCD-EF1234567890"/>
    </order:ItemDetail>
  </soap:Body>
</soap:Envelope>

Despite its parsing advantages, DIME required developers to abandon the widely adopted MIME ecosystem entirely. This radical shift prevented DIME and WS-Attachments from gaining broad industry traction.

XOP and MTOM: Unifying Infoset and Performance

To resolve the Infoset fragmentation caused by SwA, vendors proposed PASwA, which later evolved into W3C's XML-binary Optimized Packaging (XOP) and Message Transmission Optimization Mechanism (MTOM). XOP provides a standardized way to extract Base64-encoded text from an XML Infoset and package it as separate MIME parts, while still keeping the data logically inside the XML document.

Instead of the raw Base64 string, the XML body uses a <xop:Include> element pointing to the MIME part containing the binary data. An original XML Infoset containing binary data looks like this:

<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"
 xmlns:xmlmime="http://www.w3.org/2004/06/xmlmime">
  <soap:Body>
    <prod:Item id="item01" xmlns:prod="http://example.org/product">
      <prod:Graphic xmlmime:content-type='image/png'>U3ltYm9sRGF0YQ==</prod:Graphic>
    </prod:Item>
  </soap:Body>
</soap:Envelope>

When serialized using XOP, the Base64 string is replaced by an inclusion reference, and the raw binary data is moved to a MIME attachment:

MIME-Version: 1.0
Content-Type: Multipart/Related; boundary=MIME_separator;
type=application/xop+xml;
start="<root@shop.example.org>";
start-info="application/xop+xml"

--MIME_separator
Content-Type: application/xop+xml; charset=UTF-8
Content-Transfer-Encoding: 8bit
Content-ID: <root@shop.example.org>

<?xml version='1.0'?>
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"
 xmlns:xmlmime="http://www.w3.org/2004/06/xmlmime">
  <soap:Body>
    <prod:Item id="item01" xmlns:prod="http://example.org/product">
      <prod:Graphic xmlmime:content-type='image/png'>
        <xop:Include xmlns:xop='http://www.w3.org/2004/08/xop/include' 
        href="cid:graphic@shop.example.org"/>
      </prod:Graphic>
    </prod:Item>
  </soap:Body>
</soap:Envelope>

--MIME_separator
Content-Type: image/png
Content-Transfer-Encoding: binary
Content-ID: <graphic@shop.example.org>

...raw PNG binary data...

--MIME_separator--

MTOM is the mechanism defining how a SOAP message is transmitted using XOP serialization. Because the <xop:Include> element is a well-defined part of the XML structure, the logical Infoset remains intact. This allows XPath, XML Schema, and other standard XML processing tools to function seamlessly, effectively resolving the performance limitations of Base64 and the interoperability flaws of SwA.