SIAA FISD MDDL Streaming
In a recent article 'Why has XML swept the World?' I said that bandwidth is now so cheap and therefore the verbosity of XML is not an issue. Although this is generally true there are cases where the XML overhead compared to the real data content is staggering in some cases in the region of a hundred to one. If this message is sent a few times a day nobody would care but if it may be sent hundreds of times a minute the money men begin to twitch (and when an accountant twitches an IT wallah gets flattened).
An example of this problem is streaming of financial market data feeds. At the ultimate every trade done on a stock exchange turns into a market data feed message. There is now a draft standard for Market Data Definition Language, from mddl.org, which is supported by many of the main financial institutions worldwide including Morgan Stanley, London Stock Exchange, Reuters and Deutsche Bank. It is vitally important to these organisations because they need a standard automated method of transferring data about the instruments, corporate data and the trades. It is a great step forward but it is verbose for example a trade message looks something like this:
<mddl version="2.2-beta"> <header> <dateTime>2003-10-15T17:00:00.000+05:00<dateTime> <source>XTC Demonstration</source> </header> <snap><equityDomain><commonClass> <instrumentIdentifier> <code scheme="http://www.mddl.org/ext/scheme/symbol?SRC=XTKS"> 6501 </code> <name>A Company in Your Neighborhood</name> </instrumentIdentifier> <sequence>0306</sequence> <session>1</session> <trade> <last>12375</last> <dateTime>2003-10-15T16:58:32.234+05:00</dateTime> <marketCenter> <code scheme="http://www.mddl.org/xtc/Examples/scheme/iso10383.xml"> XTKS </code> </marketCenter> <size>200</size> <currency>JPY</currency> <status scheme="http://wws.mddl.org/xtc/Examples/scheme/tradeStatus.xml"> normal </status> </trade> </commonClass></equityDomain></snap> </mddl>
(This example is borrowed from a presentation by James E. Hartley to WFIC 2003)
Of that only the highlighted bits are variable data, the rest is a standard template. Much of the variable data can be encoded or compressed in one way or another, and in fact it can be reduced from the original 890 bytes down to less than ten, nearly 100:1.
The problem is that to do that requires going back to a proprietary protocol that only the sender and receiver will understand. That was until last week when the Financial Information Services Division of the Software Information Industry Association (SIIA/FISD) announced a beta version of the fisdMessage specification which defines a standards-based way of delivering XML content like MDDL between a data provider and consumer, and in particular how to define the templates and compression techniques. An excellent Holiday Season present for all.
This technique means that MDDL can be created in XML with all the benefits of that standard but can be compressed when required to reduce bandwidth requirements.
Although this standard has been developed specifically by the financial institutions it is obviously relevant to any other industry that sends high volumes of standard messages. I would expect other industries to pick it up over the next year or two.
Copyright 2004. Originally published by IT-Director.com, reprinted with permission. IT-Director.com provides IT decision makers with free daily e-mails containing news analysis, member-only discussion forums, free research, technology spotlights and free on-line consultancy. To register for a free e-mail subscription, click here.
For more information:
- Looking for free research? Browse our comprehensive White Papers section by topic, author or keyword.
- Are you tired of technospeak? The Web Services Advisor column uses plain talk and avoids the hype.
- For insightful opinion and commentary from today's industry leaders, read our Guest Commentary columns.
- Hey Codeheads! Start benefiting from these time-saving XML Developer Tips and .NET Developer Tips.
- Visit our huge Best Web Links for Web Services collection for the freshest editor-selected resources.
- Visit Ask the Experts for answers to your Web services, SOAP, WSDL, XML, .NET, Java and EAI questions.
- Couldn't attend one of our Webcasts? Don't miss out. Visit our archive to watch at your own convenience.
- Choking on the alphabet soup of industry acronyms? Visit our helpful Glossary for the latest lingo.
- Discuss this article, voice your opinion or talk with your peers in the SearchWebServices Discussion Forums.