Modern market surveillance applications process billions of transactions in real-time every business day. The current trend is that new regulatory and compliance requirements result in market data sets getting larger and an increasing need to process data from longer time periods.
In recent years a number of Big Data tools have arisen to handle these massive quantities of data. The tools parallelize large data sets across shared clusters built on low-cost commodity hardware and provide easy scaling while dramatically reducing hardware costs.
Fredrik Lydén, CTO at Scila and the co-author of the paper is very pleased with the result. "We are continuously evaluating various techniques as complement to traditional methods for detecting financial market anomalies. Big Data technologies combined with cloud technologies offer new interesting possibilities for data mining."
The white paper can be downloaded at www.scila.se