Vector search is the backbone of AI systems, enabling developers to bring context into their data search. Vector search technology is rapidly advancing, with new strategies for more relevant and accurate search emerging all the time.
Here, we’ll delve into how these technologies are transforming the approach to job searching, focusing on Restworld’s pioneering platform and their use of Astra DB's vector search capabilities. We’ll take a comprehensive look at the technical aspects, challenges faced, and practical applications of these innovations.
What is vector search?
A vector is used in machine space to represent text, images, and video. An embedding maps data into vector format, and machine learning embedding models capture the semantics and context of the data.
Imagine a chatbot that can answer questions based on your unique context. A vector database serves as a knowledge store, where all context, such as PDFs, is broken into chunks, vectorized with an embedding model, represented by floating numbers in the vector format.
This brings us to vector search, also known as similarity search. In vector space, you can calculate the distance between vectors, with the distance representing the similarity of these objects.
Vector search case study: Restworld
Restworld works within the hotel, restaurant, and cafe sector, with customer success managers (CSMs) connecting with clients and candidates to make matches. Their applications are used by workers, employers, and internally by their CSMs through a platform called Lab. All communicate with the same backend, with separate data stores, including our vector store in Astra DB, which houses our knowledge representation.
Restworld provides CSMs with AI-powered tools to explore our database of workers. Their database has become large enough that finding the right candidates for a job offer is challenging. So, they developed AI tools to allow their CSMs to explore job seekers in a scalable, efficient way.
As a startup, Restworld has to be quick and iterative, launching features quickly to receive user feedback and build incrementally. This approach allows us to tailor our tools based on real-world use and maintain a foundation we can expand on as they grow.
Restworld’s vector search system
Restworld’s AI tool supports CSMs in exploring candidates. For example, the system may list a job offer for a kitchen role at a specific restaurant, with all requirements listed. Restworld also stores a list of job seekers in their database. The AI tool enables CSMs to filter candidates with semantic search, tuned to match specific parameters.
By selecting this, CSMs can extract a list of 100 candidates, each of whom has previously applied for similar roles nearby. With two clicks, they’re able to match candidates based on both role and location.
This feature is still in early release, so Restworld doesn’t have extensive quantitative data yet, but they have gathered qualitative feedback from their CSMs. Based on this feedback, they estimate a fivefold improvement in retrieval conversion rate for internal job seekers – significant added value..
How Restworld’s vector search works
Restworld creates a knowledge representation of our job positions, focusing on job data instead of worker profiles. For example, when searching for a cook looking for a cook, the search includes details such as required experience, shifts, and salary, creating a structured text description. Restworld uses a collaborative filtering algorithm.
The goal is to connect a job position with the most suitable workers based on historical data, without relying on potentially incomplete worker profiles. Restworld identifies similar past job positions (e.g., cooks) and finds workers who had been associated with these positions, assuming they might be interested in similar roles now.
The exciting part is using all this context to provide users with automatic interview questions and assessment tools. This lets CSMs tailor interview questions to each worker's profile and the specific job they applied for. By doing this, Restworld leverages the data context to see what’s been effective in similar job positions.
What Astra DB offers for vector search
Astera DB’s real-time, serverless, scalable platform was ideal for Restworld's vector search. Its compatibility with Apache Cassandra® gave them the flexibility to adapt as their data grows, without needing heavy infrastructure setup. Restworld’s partnership with DataStax was also beneficial, offering strong support and collaboration.
One significant challenge Restworld faced was filtering by multiple data fields, like job type. This filtering requires the operational database to be in sync or else certain queries slow down. With some clever design work, syncing with Astra DB solved this problem.
Conclusion
Using RAG with Astra DB enables you to store your data in a vector database and retrieve relevant context when needed, without requiring the retraining of the LLM.
We saw how these techniques enhance relevance and accuracy, especially in job searching within the hospitality sector. See what Astra DB can do for you by trying it for free today!
