Healthcare SaaS: HIPAA-Compliant RAG System for Clinical Documentation

5 min read

Client: US-based healthcare SaaS · 50+ hospital systems served

HIPAA-Compliant Clinical RAG — Outcomes280KClinical documents indexed94%Physician satisfaction (NPS +52)96.4%Answer satisfaction38Hospital systems liveCompliance & Safety PostureHIPAA BAASOC 2 Type IIPHI Redaction0 leakage incidents0 audit findings10-week deployment · scaled from 12-hospital pilot to 38 hospital systems
Outcomes from Ohveda’s HIPAA-compliant clinical documentation RAG system, deployed Q4 2025.

Engagement: 10-week RAG system deployment · October–December 2025

The Challenge

Deploy AI-assisted clinical documentation search across 280,000+ internal protocol documents, treatment guidelines, and clinical notes. Constraints: HIPAA compliance, sub-second response times, zero tolerance for hallucination on clinical content.

Approach

01

Weeks 1–2 · Compliance architecture

HIPAA BAA review. PHI redaction pipeline at the embedding layer. Data residency: pgvector on RDS in HIPAA-eligible region. SOC 2 Type II controls baseline.

02

Weeks 3–5 · Document ingestion + chunking

Hierarchical chunking with synthetic context preambles. Custom parsers for HL7 + clinical XML. Voyage-3 embeddings; document version hashes for audit.

03

Weeks 6–8 · Retrieval + generation

Cohere Rerank v3 for retrieval refinement. Claude 3.7 Sonnet for citation-disciplined generation. Grounding refusals when confidence is low.

04

Weeks 9–10 · Evaluation + rollout

400-question physician benchmark for continuous regression testing. 12-hospital pilot, expanded to 38 hospital systems by Q1 2026.

Sample: Hierarchical Chunk Embedding

def embed_with_context(document, chunk):
    """Embed each chunk with synthetic document context.
    This single change improved retrieval accuracy by 31% on our benchmark."""
    preamble = (
        f"From section {chunk.section_number} "
        f"of {document.title}, "
        f"subsection on {chunk.subsection_title}: "
    )
    text_with_context = preamble + chunk.text

    # PHI redaction BEFORE embedding (compliance requirement)
    redacted = redact_phi(text_with_context)

    # Voyage-3 embedding
    embedding = voyage_client.embed(
        texts=[redacted],
        model="voyage-3",
        input_type="document",
    ).embeddings[0]

    # Audit log: chunk hash, document version, embedding model
    audit_record = {
        'chunk_id': chunk.id,
        'doc_id': document.id,
        'doc_version_hash': document.version_hash,
        'model': 'voyage-3',
        'redaction_applied': True,
        'embedded_at': datetime.utcnow().isoformat(),
    }
    write_audit(audit_record)  # → S3 Object Lock, 7yr retention

    return embedding

Results

Metric Result
Documents indexed 280,000+
Queries / month 2,400+ (across 38 hospital systems)
Answer satisfaction 96.4% (in-app feedback)
Physician NPS +52 (94% satisfied or very satisfied)
PHI leakage incidents Zero
Q1 2026 SOC 2 audit findings Zero

“Ohveda built us something the major EHR vendors have not been able to. The audit trails were praised by our compliance officer.” — Dr. Kavita Iyer, Chief Medical Officer