At Skypoint, we have a strict SLA of five seconds to generate responses for our frontline healthcare providers," Mathew said. "Hitting this SLA is especially difficult in the scenario that there are multiple LLM and vector search queries. Being able to shave off time from generating embeddings is of vast importance to improving the user experience."
NVIDIA and DataStax
Enterprise RAG with NVIDIA and DataStax
Astra DB integrates the NVIDIA NeMo GenAI framework to bring developers faster embeddings for Enterprise RAG (Retrieval-Augmented Generation) in an easy-to-use API.
Why Astra DB and NVIDIA?
GenAI end-users expect fast responses without perceptible latency. To deliver this, developers need high-performance vector embeddings generation and indexing for Enterprise data.
Astra DB directly integrates NVIDIA's NeMo framework so that developers can create high-performance embeddings directly with an easy to use Data API for more responsive RAG (Retrieval-Augmented Generation) applications.
This means developers and companies can provide better GenAI experiences to end-users with higher performance and lower TCO (total cost of ownership).
20x Faster Embedding and Indexing
9x Throughput
74x Faster Response Time*
80% Lower TCO*
*Based on findings from Dec 2023 GigaOm Report: Vector Databases Compared v1.0
Trusted by
Get Started with Astra DB, RAGStack and NVIDIA
NVIDIA NeMo
Developers can use NVIDIA NeMo to create high performance vector embeddings directly through the Astra DB Data API.
Get Started with NeMo
RAGStack for NVIDIA NeMo
DataStax RAGSTack, a supported, one-stop Generative AI stack for developers integrates NVIDIA NeMo.
Use NVIDIA with RAGStackFAQ
What is Astra DB?
DataStax Astra DB is a cloud-native, scalable Database-as-a-Service built on Apache Cassandra. New vector search capabilities enable complex, context-sensitive searches across diverse data formats for use in Generative AI applications.
Can I use Astra DB with NVIDIA?
Yes, there is an integration between Astra DB and NVIDIA NeMo that enables developers to create embeddings with NeMo through the Astra DB Data API to deliver high-performance, retrieval-augmented data solutions.
How do I run Astra DB on NVIDIA?
Sign up for DataStax Astra DB and NVIDIA's NeMo microservices. Integrate these services using the RAGStack in your development environment, which is powered by LangChain and LlamaIndex for out-of-the-box integration. This setup allows you to leverage the fast vector search capabilities provided by the partnership for your GenAI projects.
Is Astra DB free on NVIDIA?
Astra DB has a free tier that’s ideal for trying out the service as you get started. Beyond the free tier, you’re only billed for what you use. You can get started with NVIDIA NeMo here.
What benefits does the partnership between DataStax and NVIDIA offer for GenAI applications?
The partnership between DataStax and NVIDIA offers several benefits for GenAI applications, including drastically reduced latency in generating embeddings and indexing documents, higher throughput, and significantly lower operational costs.