.jpg)
Kx Systems, the leader in high-performance database and timeseries analysis, has announced today the release of version 2.6 of kdb+. The most significant feature of this release is the addition of nanosecond timestamps as built-in data types. Kx is the first to provide the capability to work with such a high degree of accuracy and to easily identify unique events, both of which are becoming important in low-latency environments.
The constant increase in data volumes and the resulting need for greater accuracy and granularity were the drivers behind Kx’s decision to make the nanosecond timestamp available to its client base. Until now the industry standard has been millisecond accuracy, i.e. up to three decimal places (HH:MM:SS.123). There is a new trend among exchanges to provide greater precision, but while some data providers support microsecond accuracy (i.e. 6 decimal places), it does not always mean that this can be utilised by end users. In kdb+ nanosecond timestamps (9 decimal places) are built-in data types making the unique identification of the most recent events, such as trades, very straightforward and enabling much faster processing and simpler and more elegant code. This allows Kx users to perform more complex operations, creating a competitive advantage.
Kx chief strategist, Simon Garland, comments: "The feedback we’ve had from our users is that a millisecond timestamp was not accurate enough for state-of-the-art applications. Adding support for microseconds would have been a bit of a stop-gap measure as we would have had to revisit it in a couple of years’ time thus inconveniencing our users." Garland adds: “Although a nanosecond timestamp goes beyond the needs of most people at the moment, we have decided to do this now, to give our clients a significant amount of flexibility. This is similar to our leading the market to 64-bits years ago, at a time when this was not seen as essential – now it has become the market standard.”
There are a number of other enhancements in 2.6, including:- Network Compression – additional compression between kdb+ instances now speeds data transfer, especially in distributed and cloud environments.
- Windowed database join – this feature is now more general than the join previously available in kdb+. For example, a user could retrieve all the quotes in a user-defined window around each individual trade in a group of thousands or even millions of trades. This allows for very sophisticated calculations and facilitates targeted back-testing on historical data. This feature takes advantage of the enormous speed of kdb+, a pre-requisite to being able to filter vast quantities of data.
- Network bottleneck tracking and management – this feature gives the ability to monitor the activity of server clients. Servers running 2.6 could, for example, automatically check whether all clients are consuming data sufficiently quickly. If any are found to be falling behind they could be disconnected from the server to prevent other clients being affected, or they could be throttled until they catch up, by for example being sent just trades rather than both trades and quotes.
- Memory use – based on user feedback this was dramatically enhanced in version 2.5 and has now been fine-tuned further, making even more efficient use of memory.
- Named services – kdb+ now fully supports named services making kdb+ easier to deploy in complex, enterprise-wide environments; it also simplifies administration.
Garland says: “As usual, this version is faster than the previous version, and it takes advantage of the latest advances in Intel's chip technology. We never force our clients to migrate to new versions of software, but the upgrade process is so simple and the benefits offered are so immediate that we would expect the vast majority of our clients to move to the new version as quickly as they usually do.”
