Industry Trends: Breaking the Surface - How Data Management Innovation Will Save an Industry Drowning in Data
MiFID, Basel II, RegNMS, and other regulatory directives all seem to be coming at the same time. From front through middle to back office and across the market’s infrastructures, data and its usage is exploding. At the centre of this transformation lies a data challenge. The challenge rests not only with the relatively well-known disciplines of data volumes, handling, maintenance, and storage, but also with the emergence of new interactions between producers and consumers of data—and the information derived from it.
![]() |
Nigel Woodward Financial Services Director UK Intel |
Business behaviors will adjust, and current systems, although yet to be proved inadequate, will have to perform different tasks and handle massive incremental volumes. Current predictions are that significant failures in infrastructure are on the cards. In the short term, systems are likely to require major adaptation—in the medium term, wholesale overhaul. The market must remain positive about these changes. We must embrace new paradigms rather than try to force-fit old models. The time is now for an overhaul of financial services technologies, systems, and services.
The Data-Processing Frontier
Exactly how the future data-management landscape will shape up across the entire transaction life-cycle remains undefined. So far, attention has focused mostly on what changes to expect—high storage levels necessary for record keeping, possible on-demand regulatory scrutiny, and the audit trails to support professional practices across the transaction life-cycle. While MiFID is undoubtedly spearheading these changes in the short term, it fundamentally sets the scene for a new generation of data-management practices and technologies from front to back office.
Front-office data processes are mostly about receiving market data and distributing it to trading positions efficiently. This is followed somewhat later with contributions to the data vendors from the pricing engines. However, current market data systems (MDSs), which consume the market data, are not sophisticated enough to cope with the explosion of data publication that becomes necessary for systematic internalizers (SIs) in a post-MiFID world.
Data latency is another primary concern in the automated trade environment, especially in the areas of statistical arbitrage and intra-day markets, where it is a means of competitive advantage. As news spreads of performance benefits achieved by competitors, front-office participants will start to exert performance demands on aging systems and processes.
Direct Market Access (DMA) and the emergence of new execution venues are also transforming the trade data environment. A number of new forces exist here, including speed, cross-border, and inter-firm trading and democratic access to market prices. When combined, they start to place massive demands on software, hardware, the network, and overall system reliability.
The middle office is where data comes to life—or at least where it should. Capture data up front, settle it out back, and analyze and leverage it in the middle. With the inappropriate, poorly integrated designs of today’s systems, however, processes compensate by translating, consolidating, massaging, and transporting data between systems. In future, we will see increasing levels of direct access to information, bypassing much of today’s mainstream technology middleware installed purely to bridge the gaps in information generation.
If DMA is one of the key drivers in the front office, the back office is the data engine room. This is where the strain is taken for record keeping, storage, retention, and audit trails. Extrapolating the numbers gives monumental statistics in terms of the bytes of storage required. If we look at the landscape of inputs, signals, and advice that drives trades, the depth of implied data is almost infinite. A “Google Earth” approach has to be taken to this engine room so that it is actively managing massive amounts of data as a norm, as opposed to storing transaction details purely for resolving reconciliation errors during settlement or later dispute.
We have seen horizontal and vertical consolidation of exchange businesses with the big players. Equally, several new commercial propositions have emerged, enabled by the availability of low-cost technology and a proportional reduction in entry barriers. Latency, intelligent networks, and standards all factor here.
The concept of large, dominant domestic exchanges is fading, as liquidity becomes electronically data-driven. Value-added services and efficiency of execution are now the data differentiators, while manageability is the critical enabler.
The Future
The front office will always be an area of outright competitive edge, but, as middle- and back-office systems look to tackle the information explosion and shake off legacy stovepipes of inefficiency, service provision will see yet another renaissance.
Data grids, semantic Web, and other information-centric technologies are promising to open up new levels of processing. These technologies will provide the compensation for the stovepipe data management in the back office and offer seamless information access and manipulation across the middle office.
New value propositions are also coming to market, such as market data, routing, and systematic internalizer-hosting. This trend will continue, fuelled by lower barriers to entry as a result of technology-enablement and a thirst to experiment and break paradigms. After all, there is no status quo in this as-yet-uncharted world. Perhaps a start-up can get to a new information-management solution quicker than unraveling the organic data tangle that exists.
Old habits, cultures, and necessities die hard, but change is inevitable. The future belongs to those who can get to it first. Innovate or die.
Read the full version of this article in the new book, The Future of Investing in Europe’s Markets after MiFID, edited by Chris Skinner and published by Wiley Finance Series http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0470510382.htm
Filed under: Issue 2 - Summer 07


