DSE 6: Delivering on Serious Improvements

Introduction

Without doubt, the highlight of the Advanced Performance feature in DSE 6 is the new Thread Per Core architecture, an architecture that offers a coordination-free design to maximize throughput.

This and other performance gains have already been discussed in the above linked blog at depth and serve as an appropriate segue into the next topic. This blog post continues exploring the innovation of DSE 6 and discusses additional benefits outside of performance optimizations, analytic speed improvements, and our new bulk loader that makes this an exciting and very impactful (for our customers) release for DataStax.

Performance Improvements Recap

While performance gains around throughput have been discussed and even externally benchmarked, other architectural improvements have allowed for improved latencies, which come from storage engine optimizations. This is important because it’s often simple to achieve more throughput, typically by adding more resources for the tasks at hand. However, improving latency, which would translate to improving how fast a resource can finish a task, isn’t as trivial. These storage engine improvements also optimize compaction performance which can often become a bottleneck.

Anti-entropy mechanisms such as the classic repair are meant for data resiliency as it pertains to data consistency. DSE 6 also introduced NodeSync for data resiliency and consistency across servers. As the new standard for maintaining data consistency between replicas, it is vital to ensure that mechanism itself is capable resilient operations.

Resiliency Improvements

Relatively speaking, throughput is a strength for Cassandra when compared to most databases due to the architecture and Cassandra’s ability to linearly scale as more servers are added. What can be difficult to manage in production scenarios are the spikes in loads or bursts of user activity that can overwhelm a Cassandra cluster before clusters can be properly scaled to handle the load.

DSE 6 introduces Traffic Control, a back-pressure mechanism to avoid overloading a server with any requests, whether external or internal. This mechanism safely queues requests as thresholds are met, which would otherwise make a server unresponsive.

In a test designed to overload a node by exhausting all its resources, we can see the impact of Traffic Control. The first image shows exactly the scenario users would want to avoid. We can clearly see that the node goes from increasingly processing requests to a drastic descent into the downed state.

The next image shows the same test but with DSE and Traffic Control. The difference is drastic. Not only does this node stay “up”, but it is also able to maintain a stable and high throughput.

While Cassandra is architected to handle failures like a node or two going down, there is no doubt that having this resiliency built into DSE better enables users to never be down. This mechanism is enabled by default in DSE 6 and comes “pre-tuned” based on our internal, multi-workload testing.

But Wait! There’s More!

These huge gains don’t stop with just the database. There are also search and analytics enhancements that greatly improve user and operator experience. For DSE Search, a profound improvement on the indexing path through the Advance Performance framework brings another level of resiliency and stability to the platform. Writes with DSE Search become more stable, synchronous, and simple. The indexing pipeline has been flattened to work more harmoniously with the Advanced Performance framework, which also removes many tuning parameters and provides a more synchronous search index write.

The image above illustrates the indexing behavior of inserting 500 million rows into DSE 6 and DSE 5.1. DSE 6 not only completes in less time but does so with more efficiency and stability.

DSE Analytics brings resiliency to SQL through AlwaysOn SQL by not only improving the availability mechanisms in the Analytics server but also providing failover and caching between instances. SQL in DSE Analytics does not have a single point of failure.

DSE 6 has been out for some time now and we invite you to give the latest version a try. Download it here.