How to Manage the Data Challenges of a Cloud-Native Platform

The daily operations of a cloud-native platform can be challenging and time-consuming. Especially when it involves persistent data. One minute you’re scaling up nodes, because your application grows. The next you’re restoring backups. The good news is, you can automate many operations with K8ssandra – the operational data platform for Kubernetes.

A typical cloud-native platform has one or more of the following features:

• Infrastructure-as-a-service (IaaS) offering computation power and storage
• A Kubernetes engine to help manage and scale containerized nodes and applications
• A database-as-a-service (DBaaS) to help with scaling and operations (like DataStax Astra DB)
• Microservices that help orchestrate the complete solution

You can spend a lot of your time on day-to-day challenges, if you’re manually managing a cloud-native platform. Thankfully there are automated solutions to help you create backups, access data, and serve simple monitoring.

Using K8ssandra to manage containers

If you want to automate processes like deploying, managing, and scaling containerized applications – and still get the benefits of a distributed NoSQL database – then K8ssandra is your solution.

K8ssandra is a cloud-native distribution of Apache Cassandra® that runs on Kubernetes. You get Cassandra’s power of scale and high availability combined with Kubernetes’ capability to manage containers. Whether you’re installing locally or in the cloud, K8ssandra is easy to install with a Helm command and ability to provide custom values in k8ssandra-values.yaml. 

Day-to-day challenges in cloud-native platforms

K8ssandra comes drop-loaded with different components, each with the goal to help automate your daily operations.

Monitoring Kubernetes with K8ssandra

Monitoring is an essential aspect of all systems and applications. The sooner you know something is wrong, the faster you can start to fix it. K8ssandra deploys metrics collectors for Apache Cassandra (MCAC) to the Kubernetes environment in order to provide useful metrics.

Figure 1. Visuals of the dashboard that can be created by MCAC.

MCAC is built on the open-source solutions Prometheus and Grafana for monitoring and dashboards. MCAC tracks these metrics via a Java agent and can give you over 100,000 unique metric series per node, such as operations per second or heap usage.

Prometheus gathers all the data and stores it as a multi-dimensional data model. PromQL, the flexible query language for Prometheus, is used to access this data model. You’ll find that PromQL is much more powerful and rich for queries than its competitors.

Having lots of metrics doesn’t do you any good if you can’t read them easily. And that’s where Grafana can help. Grafana turns collected data into highly customizable dashboards.

Management API for Cassandra

Managing Cassandra operations is mainly command-line driven. It’s usually outsourced to a team of experts who understand management of the operational tools. This approach produces an uneven set of best practices and risks corners being cut in the process.

With K8ssandra, you get the power of a management API and the ability to:

• Start nodes
• Stop nodes
• Customize configuration
• Perform Kubernetes liveness/readiness checks

Scaling up and down Cassandra

If your web application is flourishing, and more people are using it, your database will likely need to grow quickly. You can add nodes and remove them in Cassandra simply by altering the k8ssandra-values.yaml because of the distributed database structure.

Figure 2. A graph showing the linear progression in the number of operations per second

These are the properties within the k8ssandra-values.yaml:

• Cluster name for your Cassandra installation
• Name of data center used with your Stargate installation
• The size of the data center

A simple YAML used in the Helm install could look something like this:

cassandra:

clusterName: my-k8ssandra

datacenters:

- name: dc1

size: 3

Altering the size of the installation is as simple as changing the size property and applying the changes with helm upgrade:

helm upgrade my-k8ssandra k8ssandra/k8ssandra -f k8ssandra-values.yaml

API access with K8ssandra

A database isn’t much use if it’s just sitting there storing data. And how useful will your application be if a user can’t interact with it? You need to be able to complete the four basic operations:

• Create
• Read
• Update
• Delete

To help you with this K8ssandra deploys Stargate. Stargate is a unified API gateway that sits between your app and your database. It gives you a range of different APIs to do the above basic operations. Some of the APIs are:

• REST
• GraphQL
• Document API
• gRPC

Figure 3. A diagram showing the various API extensions available through Stargate.io.

You can either use an API you’re familiar with or pick the one that’s just right for the job from the mix of different APIs.

Backup and restore with K8ssandra

No matter how careful you may be, sometimes the worst happens. Fortunately, K8ssandra allows you to backup and restore your Cassandra database with the help of Medusa.

You can backup and restore single nodes with Medusa or do a full database backup. It supports Amazon S3, Google Cloud Storage, MinIO and Azure. You can even create a snapshot of your database. So if disaster strikes, you can roll back to your previous backup and continue as if nothing happened.

Repairs with K8ssandra

Data can go out of sync within Cassandra because of entropy. Entropy is technically a natural event caused by downed nodes and data deletions among other things. If left alone, data across nodes may become inconsistent and eventually unstable.

That’s why running repairs is an important task. Running repairs will help keep your nodes healthy and battle entropy. You can run incremental repairs daily and full repairs weekly or monthly.

K8ssandra deploys Reaper for anti-entropy repair operations on Cassandra. Reaper helps automate repairs and schedule repairs. It also has a simple UI where you can quickly see the health of your clusters.

Technical issues no longer have to be a major headache for you, even though they are a fact of life when running cloud-native apps. Certain tools and technology can greatly simplify many of the operational tasks you come across. Learn more about K8ssandra to see what challenges it can solve for you.

Check out our YouTube channel for tutorials and follow DataStax Developers on Twitter for the latest news about our developer community.

Resources

1. DataStax Astra DB 
2. Metrics Collectors for Apache Cassandra
3. Stargate
4. Medusa
5. Reaper
6. DataStax Community Platform
7. DataStax Academy