TechnologyMarch 7, 2024

Airbyte and DataStax Simplify GenAI and RAG App Development

Announcing a new partnership that offers a DataStax Astra DB data destination in the Airbyte Cloud data pipeline.
Aaron Ploetz
Aaron PloetzDeveloper Relations, DataStax
Airbyte and DataStax Simplify GenAI and RAG App Development
pip install ragstack-ai python-dotenv
ASTRA_DB_APPLICATION_TOKEN=AstraCS:blahblah:blahblahYourTokenGoesHere
ASTRA_DB_API_ENDPOINT=https://your-endpoint.apps.astra.datastax.com
ASTRA_DB_KEYSPACE_NAME=default_keyspace
ASTRA_DB_COLLECTION_NAME=airbyte
OPENAI_API_KEY=sk-blahblahYourOpenAIKeyFromAbove 
import os
from dotenv import load_dotenv
from astrapy.db import AstraDB
from langchain_openai import OpenAIEmbeddings
load_dotenv()
model = OpenAIEmbeddings()
db = AstraDB(
    token=os.environ["ASTRA_DB_APPLICATION_TOKEN"],
    api_endpoint=os.environ["ASTRA_DB_API_ENDPOINT"],
    namespace=os.environ["ASTRA_DB_KEYSPACE_NAME"],
)
collection = db.collection(os.environ["ASTRA_DB_COLLECTION_NAME"])
if (len(sys.argv) > 1):
    query = sys.argv[1]
else:
    query = "Kepler"

vector = model.embed_query(query)
res = collection.vector_find_one(vector,fields=['title','img','alt'])
print(res)
> python airbyte_query.py

Query="Kepler"
{'_id': 'f3201311-f41c-40ac-be3f-ca1679be8242', 'title': 'Kepler', 'img': 'https://imgs.xkcd.com/comics/kepler.jpg', 'alt': 'Science joke.  You should probably just move along.', '$similarity': 0.96341634}
> python airbyte_query.py "orbital mechanics"

Query="orbital mechanics"
{'_id': '05f8c7ac-1aa4-429f-a9b3-d97f398514f4', 'title': 'Orbital Mechanics', 'img': 'https://imgs.xkcd.com/comics/orbital_mechanics.png', 'alt': "To be fair, my job at NASA was working on robots and didn't actually involve any orbital mechanics. The small positive slope over that period is because it turns out that if you hang around at NASA, you get in a lot of conversations about space.", '$similarity': 0.96553665}
Discover more
DataStax Astra DB
Share

One-stop Data API for Production GenAI

Astra DB gives JavaScript developers a complete data API and out-of-the-box integrations that make it easier to build production RAG apps with high relevancy and low latency.