Kx Systems Kdb+ and Kx Technology

InBrief Philip Howard – Research Director, Information Management www.kx.com 3 Canal Quay, Newry, BT35 6BP, N. Ireland Tel: +44 (0)28 3025 2242 Email: [email protected] Kx Systems kdb+

and Kx technology CREATIVITY SCALE

The company Kx Systems was founded in Palo Alto, California in 1993. The company is a subsidiary of First Derivatives Inc. plc, which acquired a majority shareholding in Kx in October 2014. FD employs over 2,000 people worldwide and has operations in cities around the world including London, “ Through its New York, Stockholm, Singapore, ability to rapidly process Hong Kong, Tokyo, Sydney, Toronto, vast amounts of time-series Philadelphia, Dublin, Belfast, Zurich, EXECUTION TECHNOLOGY data, provide analytics in real- Seoul and San Francisco. The image in this Mutable Quadrant is derived from 13 high level time, and integrate with In capital markets, where the metrics, the more the image covers a section the better. our machine learning pipelines, company originally focused, Kx Execution metrics relate to the company, Technology to the Kx is ideally placed to power used both direct marketing and product, Creativity to both techincal and business innovation and our neuroscience Scale covers the potential business and market impact. platform. channels. It has partners around BrainWaveBank” the world that provide product sales and consulting. In addition to these memory and historical data, and combinations companies, the company also has OEM thereof, with very low latency. partners. However, for Internet of Things (IoT) Kx technology is built on the kdb+ database, applications Kx primarily goes to market directly. which the company has recently taken to calling Kx started with the smart energy and utility as a time-series database. While this is valid as sectors, and since has added customers in several a description it does not reflect the architecture industries including precision manufacturing and of Kdb+, which is columnar. On top of Kdb+ the IoT more generally. company provides a built-in array processing language called q. The array-based nature of this processing means that it can work across both columns and rows simultaneously, making it much more efficient that traditional approaches. It is also worth noting the terseness of q, which is actually SQL-like. The combination of Kdb+ and with array processing facilitates concurrent execution and parallelism. The environment supports deployment across multiple machines with a distributed capability for scale-out in clustered environments. Resilience and automatic recovery are provided, together with load balancing and replication. Figure 1 – In practice, the combination of stream processing, together with in-memory processing What is it? and persistent data storage, means that Kx Kx technology supports the rapid analysis provides a so-called Kappa architecture (a of time-series data – something that has simplified version of a Lambda architectures), historically been a rare capability – and, more as shown in Figure 1, even though this is hidden generally, the analysis of streaming, in- under the covers. Where it differs from many

other potential providers of Kappa, or indeed Of these tools, Analyst for Kx provides data Lambda, architectures – particularly where preparation capabilities, native statistical libraries these are to be built on a variety of open source and integration with machine learning environments offerings – is that Kx offers a consistent, unified such as TensorFlow, Theano and others. You can approach to supporting the combination of real- also call python libraries from within q code. Kx for time and batch analytics that Lambda and Monitoring, on other hand, is specifically targeted Kappa architectures are aimed at, at network, IT and similar environments where you with a single code base. need to monitor the state of your infrastructure. “ There are This is illustrated in Figure 2. It supports the many valuable applications What does it do? real-time analysis of the network to display critical of satellite imagery across information and raise alerts, and it can combine this As can be seen in Figure 1, a range of industries, many of with historic information to support such things as Kx is not limited to providing which are time sensitive and require trend analysis. an engine but also offers powerful analytic processing. By combining our data and Control for Kx, Monitor Why should you care? Kx technology we expect to be able for Kx, Analyst for Kx, to provide new and existing Dashboard for Kx and various Kx has proved itself in what is unarguably the most customers with unique and industry-specific solutions that demanding big data market: financial trading and valuable insights. risk management. The company has been targeting Airbus Defence ” reflect the company’s target and Space markets. Some relevant points IoT and related environments over the last few years include the fact that Kx has a small and has had some notable success across a range deployment footprint (600KB) so is suitable of applications, where the collection and analysis of for embedding in edge devices; is available in the time-series based data is important. cloud as well as on premises; and includes an We would have to say that Kx has been ODBC driver for connection to popular third-party fortunate. We do not believe that it anticipated dashboard platforms such as Tableau, as an the Internet of Things. What it did was to focus alternative to the tools Kx offers. on offering the best possible performance for its chosen (financial) market. It just happens that the (big data) issues faced by that market are exactly analogous to many Internet of Things environments. Not only is Kdb+ suitable for implementation for many of these use cases but, more importantly, the product has significant technical advantages in these areas, not least that it provides both real-time and batch-based capabilities from a single platform.

The Bottom Line Kx offers a small deployment footprint, a single platform for streaming analytics and a fast (time- Figure 2 – series) database, and the optimisation of in-memory and persisted data processing. It therefore fulfills the requirements for embedding in edge and gateway devices as well as, more generally, as a streaming analytics platform for live sensor data.

FOR FURTHER INFORMATION AND RESEARCH CLICK HERE