How Modern Big Data Architecture Thrives with NoSQL

The database world is changing fast, with no signs of slowing down. Open source solutions and frameworks have become the norm. More services and data now reside in the cloud. Today’s data managers must respond to business challenges with scalable and responsive systems that can handle both structured and unstructured data, with the ability to respond to any and all queries, all at a moment’s notice. 

What’s needed is a modern data architecture built on flexible, modular technology, either from open source frameworks and software or through cloud services. Here are eight key ways to prepare for and manage a modern data architecture.

Focus on a modern customer experience

All data management initiatives should ultimately focus on one thing: serving the customer or end user. In today’s digital economy, competitive advantage comes from delivering a superior customer experience, and that’s only possible with enough of the right data at the right time. Data architects need to work with business owners to determine customer requirements, and work backward from there.

Ensure data portability

The goal of a modern data architecture is to abstract all underlying infrastructure—servers, storage, and networks—from the applications and data being used. Thus, data should be capable of being moved expediently between platforms with little-to-no disruption to the end user. This may be from an on-premise system to the cloud, from cloud to cloud, or from the cloud back to an on-premise environment. 

While cloud platforms offer many compelling advantages to data managers and developers, they also pose the risk of vendor lock-in by potentially making it too costly and messy to attempt to switch providers. It may not even be possible to extract databases from cloud providers. That’s why data portability needs to be an element of a modern data architecture—data and applications will inevitably be moved to the platform that makes the most sense for the business. 

Build a data service layer into the architecture

Agility and flexibility are fundamental to data architecture success. Data and insights should be available to end users at a moment’s notice. A modern data architecture should support a highly networked business, marked by a continuous flow of information across all boundaries and organizational walls. 

Establishing a standardized interface that can access all applications and data brought into the infrastructure is the essence of a modern data architecture. Data will reside in many places in the years ahead, including on-premise legacy systems, private clouds, and public clouds. The experience in using this data needs to be the same, with the underlying platform virtually invisible to the end user.

Support edge computing

“Fog computing” is a concept gaining traction. It enables organizations to access, store, and process data anywhere between a centralized server and an edge device. This is critical, as much of the information that will drive businesses—operational issues, product usage, and customer trends—will come from devices and sensors situated at the edges of enterprises. 

Sensors in production systems may be streaming time series data on the health of the system, for example. A modern data architecture needs to support edge computing, as well as streaming data.

Disaster-proof your architecture

Big data architecture needs to ensure that data and applications are available to end users at all times, even if there are disruptions or outages to the system. This is accomplished through a well-defined resiliency strategy that, in the past, typically required a secondary data center, but today may involve cloud resources. 

Disaster recovery, failover, load balancing, backup, and data restoration processes need to be built into architectural planning. This requires greater attention than was necessary in the simpler days of data mirroring or replication between primary and hot standby systems.

Automate for big data

There are many database tasks and operations, such as backups, patching, security, and anomaly detection, that take up inordinate amounts of staff time. Big data architecture incorporates automation of such tasks. The time saved can be spent on higher-level activities. AI and machine learning can play important roles in increasing the ability of databases to manage themselves.

Promote collaboration in design and implementation

Importantly, a modern big data architecture is not solely the domain of the IT department. It is a set of services and capabilities built and used by just about every department across the organization. A modern big data architecture needs to be closely tied to the needs of the business, and needs to have the flexibility to adapt to and meet business changes. While data managers and IT departments will continue to oversee infrastructure and security matters, their colleagues will help set the agenda for the types of analytics delivered, the speed of delivery, and the capabilities offered.

Ensure a highly secure and compliant infrastructure

Cybersecurity needs to be baked into big data architecture right from the beginning. The goal of a modern data architecture should be to reduce the potential attack surface as much as possible. In addition, as data is subject to many regulations and mandates, the data architecture needs to build in checks and balances, as well as transparency, to ensure data is well-maintained and secured. A modern data architecture is a living, breathing entity that grows and adapts as the enterprise itself evolves. Today’s enterprises run on data, and their success depends on how well they leverage their data assets.

Working smarter with big data architecture

Adopt a modern big data architecture and treat your data like the invaluable, mission-critical asset that it is. It’s the proven path to giving your data the level of protection and disaster-proofing it deserves, while making it easier and faster to work with and providing your customers with superior experiences.

Thanks for reading this excerpt from the DataStax whitepaper, “Moving to a Modern Architecture.”