Success StoriesOctober 24, 2024

How AI Agents and Humans Collaborate to Turn Unstructured Data into Action

How AI Agents and Humans Collaborate to Turn Unstructured Data into Action

Editor’s note: This guest post is from DataStax customer Skypoint, a provider of data, analytics, and artificial intelligence services for the healthcare industry. CEO Tisson Mathew explains how AI agents helped the company build a new system that helps turn the vast amounts of unstructured data in the healthcare industry into actionable data intelligence.

With the growing accessibility of building AI applications on large language models (LLMs), one thing has become abundantly clear: high-quality, unified data is now the most critical asset for enterprises looking to differentiate themselves. For highly regulated industries, like healthcare, which rely heavily on unstructured and siloed data, unlocking the power of AI requires more than just cutting-edge algorithms—it demands a solid data foundation.

At Skypoint, we’ve spent several years working on a groundbreaking solution to address this very challenge. Our journey with vertical AI agents and industrial-scale AI in healthcare has culminated in a production-level system that bridges the gap between unstructured data and actionable data intelligence. Here’s how Skypoint is transforming healthcare by leveraging the power of AI agents and its next wave, industrial AI.

The healthcare data problem: Unstructured, disconnected, and complex

Healthcare is a prime example of an industry with vast amounts of unstructured data - from EHRs like EPIC, Point Click Care (PCC), and staffing applications (UKG, Workday etc.), from medical images to contracts, policies, procedures, health plan documents, regulations, quality measures, multimedia content and more. Historically, AI’s impact has been limited in this space because these data types are difficult to organize, extract, and use. An estimated 80% of enterprise data is unstructured, and for industries like healthcare, this means valuable information often sits in silos, making it difficult to harness effectively.

What makes this issue particularly acute in healthcare is the reliance on fragmented legacy systems, making it a challenge to gain any meaningful insights or operational efficiencies. The solution? AI agents, built on industry-specific LLMs, that can process and transform unstructured data into a cohesive, reliable dataset for real-time decision-making.

Skypoint’s vertical AI agents in production: The Senior Living Market Intelligence Case Study

At Skypoint, we’ve taken on one of healthcare’s largest sectors - senior living market intelligence - as a proving ground for our vertical AI agent technology. The senior living industry, valued at over $100 billion, includes more than 6,000 operators managing more than 35,000 communities. This equates to millions of data points, ranging from community and operator information, community reviews to pricing details, state citations and healthcare services offered. Gathering, curating, and analyzing this information is an enormous task.

Skypoint relies on Astra DB, DataStax’s fully managed vector database built on Apache Cassandra, to handle the massive amounts of unstructured data generated by healthcare systems and other regulated industries. Astra DB in DataStax’s AI Platform provides the scalability, low-latency access, and high availability needed to support Skypoint’s vertical AI agents as they process and transform complex datasets. By leveraging Astra DB, Skypoint can ensure that its AI-powered platform delivers real-time, reliable insights across millions of data points, enabling faster decision-making and operational efficiency while maintaining compliance with industry regulations.

Skypoint developed vertical AI agents specifically tailored to handle the complex data environment of senior living. These agents are responsible for continuously extracting and summarizing information from multiple sources—operator websites, third-party aggregators, social media platforms, job boards, and user-generated content. This enables our platform to provide real-time, actionable data intelligence for a wide range of stakeholders, including operators, vendors, capital partners, and REITs.

For example, thousands of AI agents are built and operated using the Skypoint AI Platform for senior living market intelligence. One of those AI agents, assigned to a large operator, ensures that all relevant community data is continuously updated, categorized, and validated. By automating this process, we eliminate the need for manual data extraction and processing, significantly reducing the time and resources needed to maintain accurate data.

Human + AI agents collaboration: Data quality is priority one

While AI agents and LLMs are incredibly powerful in processing and summarizing vast amounts of data, human oversight is still crucial to ensure data quality, accuracy, and integrity. At Skypoint, we use a hybrid approach, where humans manage reference and foundational data, such as cities, zip codes, URLs, data sources, community and operators basic data, to ensure AI agents are working with verified information.

This process is built on a lakehouse medallion architecture with three layers:

  • Bronze layer - Reference data, basic data, raw data collected by AI agents.
  • Silver layer - Enrichment, aggregation, transformation, master data management and categorization of data by AI.
  • Gold layer - Final, human-validated output that drives BI, AI and apps.

This architecture enables us to blend the speed and efficiency of AI with the precision of human validation, ensuring that the final product is both scalable and accurate.

Delivering unprecedented BI reports, dashboards, and AI copilots at scale

The result of this human-AI collaboration is a platform capable of delivering predictive analytics, sentiment scoring, and real-time dashboards that users can trust. Through our Senior Living Market Intelligence solution, we provide a comprehensive view of the industry’s landscape, all powered by vertical AI agents.

Our solution offers unmatched cost-efficiency, with AI-powered dashboards and reports available at just $195 per user per month — a price point that is revolutionizing the market. By lowering the cost of intelligence, we’re enabling businesses of all sizes to leverage AI in ways that were previously unaffordable, all while maintaining the highest standards of data integrity.

Vertical AI agents: A game-changer for healthcare and beyond

What makes Skypoint’s vertical AI agents so exciting is their ability to scale across industries that have historically been underserved by technology. Healthcare and other regulated sectors are now benefiting from AI agents that can transform unstructured data into a unified, reliable asset.

As NVIDIA CEO Jensen Huang has pointed out, industrial applications of AI agents will likely surpass traditional AI Copilot use cases, particularly in industries like healthcare where the backend processing of unstructured data holds enormous potential. While AI copilots are valuable for user interactions, the real innovation lies in AI agents working behind the scenes - extracting, enriching, and summarizing data to create scalable data and AI products and services.

At Skypoint, we’re at the forefront of this revolution, delivering production-scale AI solutions that solve some of the most complex data challenges. AI agents, combined with human oversight, are unlocking new possibilities for healthcare, enabling faster, more accurate decision-making and creating a future where data-driven insights are available to all.

Experience the future of AI-driven healthcare

If you’re interested in learning more about how Skypoint’s vertical AI agents can transform your organization, we’d love to show you a demo. Whether you’re in healthcare, public sector, financial services or any other regulated industry, our AI-driven solutions can help you overcome data challenges and unlock new growth opportunities.

Learn more about Skypoint's successes with DataStax. 

Discover more
DataStax Astra DB
Share

One-stop Data API for Production GenAI

Astra DB gives JavaScript developers a complete data API and out-of-the-box integrations that make it easier to build production RAG apps with high relevancy and low latency.