ETF systems design & development: meeting the challenges

A brief history of index systems - the spreadsheet analogy

In all areas of investment there is a need for a benchmark of performance and an "index" has long been the entity used to identify winners, losers and non-movers. A single value is able to guide investors as to relative performance of an instrument or basket of instruments against a theoretical portfolio, chosen by a fund manager or an institutional index provider. For the greater part, these indexes used to be calculated on an intra-day or end of day basis using spreadsheets and data input either manually or from a file. Increasing sophistication in terminal products from vendors such as Reuters combined with spreadsheet products such as Excel has enabled indexes to be calculated in real time as the price of each component instrument in the index moves. Spreadsheets are not however a good solution for the provision of high calibre real-time indexes.

The theoretical portfolio

The constituents of an index are described as a theoretical because they are not generally speaking available to a single investor, and acquisition of the entire portfolio is impracticable. The theoretical portfolio is chosen to represent a market or sector of a market and is designed carefully to ensure that movements on individual instruments can not severely impact the index.

ETF - an opportunity to own the theoretical

An Exchange Traded Fund provides an investor with the opportunity to own the theoretical. Instead of owning individual equities the investor is in essence purchasing a share in the index. A single tradable instrument thus effectively allows investment in the performance of all of the constituent stocks. This is similar in concept to investing in an index-tracking fund, however an ETF can be traded throughout the day and has no tracking errors. Moreover, with no actual equity ownership there may be no capital gain realised when the ETF rises above the purchase price and so this presents the investor with a tax efficient unit. The ETF market started in the USA and has now crossed to Europe. In index calculation terms there is little or no difference between ETF and non-traded indexes and thus demand has increased for index calculation systems, albeit for use in another field. This increased demand has fuelled investment in developing new technology with smarter and faster systems being required.

Designing for today and designing for the future

If one looks at the growth in indexes over the last 15 years it is easy to understand why systems designed at the time are no longer able to cope with current demands. As the volume of data to be processed and the activity of those instruments ramps upwards with little sign of abating, combined with demands for new and simultaneous calculation methodologies, index system designers are faced with design criteria reaching well ahead of current requirements. Given the time taken to test and integrate new systems into operational use, designers are having to take a view at least three to five years ahead with a design infrastructure capable of at least ten years growth.

The challenges of data throughput: instruments and currencies

As trading has become more automated so the price activity of instruments has increased. Decimalization of the US markets has, for example, adjusted the spread on trades and thus shifted the volume of trading as margins are found in ever smaller offsets. With indexes more frequently representing geographic sectors, such as Europe or Asia Pacific, the need for normalisation from the currency of each of the host nations requires currency cross calculations to take place. The exchange rate markets in themselves represent a fast moving information stream and when combined with increased instrument movements the demands on systems are magnified. Traditionally, real-time indexes were calculated at one-minute intervals; however, this has shifted to 15 seconds or less, creating even more processing demand.

Going global - mixed data with overlapping hours

In an ideal index calculation world all data would start and stop at the same time within agreed tolerances. In the real world each exchange has its own rules for the start of day and end of day, with some having multiple trading or other complicating factors. For example, one exchange may issue an official closing price for each instrument, which may occur at any time after the exchange close at 17:30 local time. A neighbouring exchange may define a closing price to be that price which is available at 17:00, even if there are later price changes in the same day. Each exchange has devised its own technology solution and creating a credible global index requires an understanding of the complexities of each market.

Add to the equation governmental enforcement of trading rules such as overseas investibility and the demands increase. Local holidays add to the complexity with rule engines defining the participation in a global or regional index of a country that is not trading on a given day. Even though a stock may not be traded on a country's national holiday the local currency will still be active in global markets, which will cause stock prices to vary in global terms. How many countries must be on holiday before the index is suspended? Going global creates many problems.

Going 24 hour - time zone travelling and corporate actions

In a 24-hour day, where is the start and where is the end? In global terms there is neither, but it is vital to take account of weekends, public holidays and corporate actions in the management of an index. When should corporate actions be applied to stocks that may appear in many different local and global indexes, when should an index be re-balanced and when should divisor changes be made so that they can be disseminated to the market in time for the next day? How will the system be maintained if it is running 24 hours a day, seven days a week? How can data be backed up?

The demands of a 24-hour index are intensive and from an operational perspective add to the nightmare. Operating a system on a 24-hour basis defines it as being global by virtue of the movement of the main sources of data content, "following the sun" around the world. Who will provide the alert management - the user input required to make quality judgements - in each time zone? A good system must allow for handover between different offices around the world, and simultaneous monitoring on a global basis in real-time.

Keeping control of the numbers: data quality checking

There is a time-honoured expression in the IT industry: Garbage In, Garbage Out. An index requires absolute data accuracy; however, the data itself is often derived from manually input data. If a broker enters a trade of 699 as 996 in error then a blip in the index could theoretically occur which in turn could drive automated trading. The ensuing chain reaction could be difficult to contain and often impossible to unwind.

Quality checking on both input and output data is vital, with degrees of threshold tolerance for both warnings and errors. A warning will alert the operator to a possible bad price value, while an error might be triggered by a larger price movement, resulting in a constituent automatically being held at its last known good value while the operator investigates.

Threshold checking also needs to take place on exchange rates and the calculated index values. The thresholds for error and warning movements are different at the market open from those used during the day. Equally important is checking for an illiquid state, that is an instrument not moving, which may indicate a data vendor failure or a mistake in corporate action maintenance.

Slicing and dicing: categorising indices by sector and economic grouping

As the market for indices has increased, so has the diversity of custom designed indices available to the professional. No longer is it satisfactory to have a pan European index without also wanting the Retail Index derivative in each country, in local currencies or indeed the pan European index excluding the Retail sector. A sophisticated system must be able to dice and slice, and drill down through an unlimited number of hierarchical groups. Additionally, each index vendor will wish to use their own classification scheme for the description of sectors.

Contingency in data sources

Tying a system to an individual data vendor is often seen as an unacceptable risk. What happens if the vendor has an incomplete market data set or suffers from technical issues due to exchange connectivity? But in selecting multiple vendors for the supply of data, how does one overcome temporal disordering, that is the lack of simultaneous arrival of the same data through disparate sources? How does one identify the data within the indexes if two or more vendors describe the data in different ways, for example using proprietary naming conventions as seen with Reuters and Bloomberg. How can the system tell whose data is timelier and whose is more accurate?

In providing for contingency one must define the rules used to select data either by explicit event or by using fuzzy logic to determine the most likely correct data. Nyquist theory deals with the need for over-sampling to reconstruct data points and the over-sampling in this case is a multitude of information vendors. The cost of high quality multiple data sources is often the defining factor in choosing a lead vendor who has proven their investment in systems to deliver the highest quality data.

Dissemination issues: handling handlers and standardisation with exchanges.

In an ideal world, everybody would speak the same language and use the same accepted protocols for behaviour and manners. As global exchanges have evolved, purchasing and developing systems with local technical design, so a multitude of disparate exchange systems interfaces are required for dissemination of index values and associated data. Requirements vary from exchange to exchange with some exchanges requiring only an index name and value, whereas others require more fundamental data. Moves to standardise data contribution have been thwarted by the proprietary nature of exchanges and vendors and by legacy issues. XML-style contribution handlers were heralded as the new age; however, they are by nature inefficient in their use of communications bandwidth, often using 50 characters to define a data value where 5 would have sufficed.

Processing the different methodologies

Index providers compete not only on the range of headline and sub sector indices that they produce but on the trust and proof of their methodologies. An index calculator must be able to calculate using different algorithms or methodologies at the same time. In addition, clients may require widely differing methodologies to handle the various exceptional conditions that occur with stocks such as suspensions and corporate action events. In order to maintain common code for the greater part of the system, thus keeping bugs and cost to a minimum, the design of a generic index calculation system demands careful attention in design to this area and the insertion of coding hooks at almost every part of a calculation.

Software design versus system horsepower - why the boundaries are distinct

Imagine an index of 100 constituents all in the same currency. A move in any constituent price simply triggers a recalculation at the appropriate time. If the number of instruments increases so we can apply a simple linear progression to determine the time taken to calculate the index. If the index now contains instruments in multiple currencies we need to normalise the currencies to the base currency of the index and then perform calculations for each currency that the index is calculated in.

As calculation times increase and the number of indexes increases so we run the risk of not being able to recalculate an index before it is due for dissemination. If the index calculator attempts to disseminate more information than receiving systems can accommodate, no amount of additional processing power can overcome the bottlenecks. Forecasting the growth in instrument activity based around new market events such as decimalization is a black art and system design cannot rely on simply adding bigger and stronger processors. Software designers have to allow for and define the limits for systems, rather than rely on new and better technology being available in the months or years to come.

Playing safe with dualled systems

Index systems are required to operate on a movement by movement basis with zero down time. How can systems be serviced and maintained and what happens if a system fails?

Dualling is a technique of binding multiple independent systems together in order that they operate in a seamless manner and that, should any part of the system fail, another system will continue to operate in its place. There are solutions available that offer hardware redundancy in the event of a system failure; however they cannot offer application level software recovery. Kinetic systems are designed from the platform level up with dualled software components throughout, which constantly monitor each other looking both for correct operation and at performance measurements to ensure stable system operation. Should a system software component fail, the operation of that particular component will be switched to another machine rather than switching all components, which is inefficient and time consuming.

Facility managing - not for the squeamish

Being armed with the highest quality data and software is simply not enough to deliver indexes to a demanding market place. The ability to guarantee delivery is as important as the index value itself and the hosting of systems is a minefield, the traversing of which is not for the squeamish. Ensuring the continued operation of third party systems demands a level of understanding and commitment that is often not cost efficient unless provided as part of either a bigger commercial arrangement or for branding purposes. As an index is a sum of many parts, so index systems require facility managers to be able to understand all the parts of the sum from data sourcing through operation, remote client control and global dissemination.

Chinese walls in development

When licensing systems to clients in the same market place, the most common question asked is how we maintain confidentiality between customer systems. Our index calculation system clients are the industry leaders in the field and compete with each other for acceptance of their indexes and at the same time endeavour to keep their calculation methodologies as secret as possible. Our calculation engines are universal, with plug in methodologies, and clients are able to use our screen development tools to customise the interface to suit their workflow requirements. Every client system is separately designed, tested and hosted down to separate uninterruptible power systems to ensure as individual a system as possible.

Summary

Index calculation systems consume an enormous amount of thought, design, development, processing horsepower and memory and are in a process of continual evolution. The changing demands of clients to slice and dice indexes by sector and geographic region as well as shifting through broader time zones ensures a continuing growth in demand for hardware and hosting capability. Increased business drives increased dissemination and the growth of ETFs, with their revenue return, ensures that the value and calibre of the system will be a key buying requirement. The days of spreadsheets for index calculation are long gone. As the role of the index as a tradable instrument grows so will the demand for state of the art systems provided by craftsmen in this field.

The diagram below demonstrates how the Kinetic Integrated Platform is used to provide a solution for index calculation.

Kinetic is a London-based technology provider whose KIP platform is used as the basis for its Index Calculation solution, which is used by FTSE, Dow Jones, Standard & Poor's and many other global clients to calculate and disseminate real-time indexes across Europe, North America, Asia, Oceania and Latin America.
www.kinetic.co.uk