Case Study Banner

CASE STUDY

i2e helps clinical data team to replace manual validation with automated, audited data-quality checks

industry-iconCLIENT :Confidential
industry-iconINDUSTRY :Pharmaceutical
industry-iconDURATION :3 months
CLIENT :
Confidential
INDUSTRY :
Pharmaceutical
DURATION :
3 months

Business case

Clinical study data doesn't get used downstream until it's earned trust, and in Unity Catalog, that meant clearing 80 SQL validation checks per study, covering control tables, metadata, operational data, clinical data, and audit data. Every one of those checks was run by hand.

At single-study scale, that was already a heavy lift. Scaled to cross-validation across 20+ studies, it became more than 1,500 manual query executions per cycle, consuming significant analyst time while leaving no audit trail of what ran, when, or by whom, and carrying a real risk of human error at every step

Our Solution

  • Streamlit-based Databricks app: enables single-study and cross-study validation runs with one click.
  • Config-driven architecture: SQL check definitions live in a centralized Delta table, sourced from NAS.
  • End-to-end audit logging: full run history, status tracking, and Excel export.
  • Automated drill-down: failed checks immediately surface root-cause data through detailed SQL outputs.
  • Automated monitoring: stale run cleanup and dynamic catalog resolution based on environment and instance.

Challenges overcome

  • Multiple environments and instances: Dev, Stage, and Prod environments, and Dev, UAT, and Exec instances, required repetitive validation with no centralized tracking.
  • Orphaned validation runs: no mechanism existed to detect, or handle runs left incomplete when users closed their browser mid-execution.
  • Check definitions scattered across NAS: SQL files required manual synchronization into Delta configuration tables.
  • No drill-down capability: failed checks forced analysts into ad-hoc queries just to find the root cause.

Benefits

  • Transformed validation from hours of manual execution into minutes through automated parallel processing.
  • Eliminated the risk of human error by standardizing execution across all validation checks.
  • Enabled full audit tracking, with run history and user details, while supporting scalable, self-service validation across studies, environments, and instances. 

Results

Results
SPM Icon
SPM maturity calculator