How Artera Built its Art Discovery Platform with Astra DB, Vector Search, and Astra Streaming

London-based Artera is dedicated to transforming how art is experienced on mobile platforms worldwide. Cutting-edge generative AI technology is the key to accomplishing this.

The Artera Art Discovery Platform enables users to load their artwork, curate personal collections, and share them within the platform. By integrating vector search, streaming, and other technologies, the platform allows effortless discovery of similar artworks or exploration of recommended pieces, offering a seamless and engaging way to experience art.

Discovering similar artworks with Astra DB vector search and real-time social interactions with Astra Streaming 

Leveraging DataStax Astra DB has revolutionized Artera's art discovery platform. By integrating Astra’s vector search capabilities and Astra Streaming, Artera has unlocked real-time data interactions and precise art-matching capabilities, enriching our users' experience. With Astra, Artera has empowered art enthusiasts worldwide to effortlessly explore, discover, and engage with art in a truly personalized and inclusive manner.

The Artera app combines a React Native interface with Django, managing the backend and Astra DB, powering real-time data interactions for the mobile application and vector similarity features. It uses Astra DB as the database for the application data and the vector database to manage art embeddings. The Artera application database has a catalog of various activities, including user profiles, artwork details, collections, feedback, browsing history, likes, comments, and follower relationships.

The "artwork recommendation" feature helps users discover art that is similar to what they're viewing. Text and image searches are possible through Astra DB, enabling users to snap a photo to find similar artworks with Astra's vector search capabilities. Additionally, Astra Streaming seamlessly integrates various events—including liking posts, creating collections, and commenting—into the non-vector Astra database.

Artera evaluated other vector search technologies, including Zilliz and Pinecone. Its decision to use Astra DB was driven by its competitive pricing, scalability, and the seamless integration of its suite of services with its existing infrastructure. Astra DB and Astra Streaming provide real-time data interactions and vector similarity features, essential for a dynamic art discovery platform. Astra can handle diverse data from user interactions efficiently, while its vector search capabilities enable accurate matching of similar artworks. Astra Streaming integrates various user activities seamlessly, improving the interactive experience on the platform.  

The Artera Art Discovery Platform also enables users to enrich their art experience with audio comments, a next-generation feature that creates engagement through multimedia commentary. This enables visitors to record and share their insights in audio directly alongside artwork, merging the visual with the vocal. Artera uses the OpenAI Whisper model to transcribe these audio comments into text, ensuring the platform's content is accessible to everyone. This fusion of text, audio, and visual elements not only broadens accessibility but also deepens the connection between art and its audience, making art discovery an inclusive experience for users of all abilities.

Artera’s Architecture

Vector use case

Astra DB vector search capabilities bring art in the discovery section that aligns with the user's previous preferences (the “Discover” page).

 

You can also take a photo with your phone and find similar artworks in the application (Image Capture):

 

You can find artwork recommendations in the “More like this”  section:

 

The platform also offers text search using Astra Semantic search along with OpenSearch: 

How an art recommendation works

Use case: User search with text or image capture for similar artworks

Steps: Open App>Discover>Click on the Art Information>Similar artworks from Astra vector Search

Table:

CREATE TABLE artera.clip_vector_by_artwork ( artwork_id uuid PRIMARY KEY, clip_vector vector<float, 512>);

 CREATE CUSTOM INDEX ann_index_clip ON artera.clip_vector_by_artwork (clip_vector) USING 'org.apache.cassandra.index.sai.StorageAttachedIndex' WITH OPTIONS = {'similarity_function': 'DOT_PRODUCT’};

Find similar art:

session.execute("SELECT artwork_id FROM clip_vector_by_artwork ORDER BY clip_vector ANN OF %s LIMIT %s", (vec, 20), timeout=30)

Here's the architecture and how the OpenAI model is used to generate embeddings:

The diagram outlines the process of image-based art discovery in an application. Initially, embeddings (numerical representations of art images) are generated using CLIP for 1 million artworks (1) and stored in Astra DB (2). When a user searches for similar artworks (3), the system generates embeddings for the search query, too (4). These embeddings are then used to retrieve and recommend similar artworks (5) from the Astra database, creating a personalized and intuitive art discovery experience.

CLIP, developed by OpenAI, is an AI model that learns to understand images through natural language, unlike traditional models that rely on specifically labeled images. By studying millions of web images with their descriptions, CLIP comprehends visual concepts in a way that's similar to how humans perceive and describe the world. Its training involves "contrastive learning," where it learns to match images with their correct descriptions, setting it apart in its ability to handle various tasks by understanding the link between what we see and the words we use. This makes CLIP highly adaptable and useful for applications requiring a deep understanding of images and language together.

The image search is conducted using vector search, employing the following CQL:

query = SimpleStatement(
f"""
SELECT artwork_id
FROM clip_vector_by_artwork
ORDER BY clip_vector ANN OF {embedding} 
LIMIT 20"""
)

What’s next?

The next version of Artera’s platform brings new features to enhance the user experience and support artists. One notable addition is the Image Capture Pose Estimation, which enables users to identify similar poses in art through image capture technology. This feature adds a new dimension to art exploration and facilitates deeper engagement with artworks. 

Additionally, Artera will introduce a “Patronage System” designed to empower users to support artists directly. These updates reflect our ongoing commitment to leveraging technology to revolutionize art discovery while fostering meaningful connections between art enthusiasts and creators.

Artera showcases how technology can bridge the gap between art and digital discovery, offering a platform for viewing art and experiencing it in a way that's tailored to each user's taste. As Artera continues to evolve, it promises to keep pushing the boundaries of what's possible in the digital art world.

Get started building on Astra DB for free. The Artera Art Discovery Platform is now available for download on iOS.