Open-Source Spoken Identity Transcription for Medical Record Matching

In many settings affected by low literacy, spoken communication often substitutes for written records. This work addresses matching patients’ identities typed into a centralized electronic medical record (EMR) with audio recordings of patients’ names captured at the point of care by community health workers (CHWs) with limited literacy in the Guatemalan highlands. These names are a mixture of Spanish-origin and Kaqchikel, a Mayan language, and require human transliteration and matching, introducing orthographic variation and making the process time-consuming and difficult to scale. We assembled a dataset of 228 full names (comprising two given and two family names) recorded by four Kaqchikel–Spanish-speaking CHWs, supplemented with synthetic Guatemalan name recordings generated using the GPT-4o-mini text-to-speech. We benchmarked open-source and commercial speech-to-text models and developed Jasjot AI, a task-specific Whisper model adapted using low-rank adaptation. Transcribed names were matched against candidate records using fuzzy similarity–based ranking, and the end-to-end framework was integrated into a human-in-the-loop healthcare workflow for verification. Jasjot AI achieved the strongest transcription performance with a fuzzy similarity score of 89.3, phonetic match of 62.3, and character error rate of 0.091, improving on vanilla Whisper large-v3 by 1.6%, 19.3%, and 20.2%, respectively. For downstream retrieval, our framework ranked the k most likely matches from a pool of N EMR names. For N=228, it achieved 98.7% accuracy at k=1 and 100% at k=3, with a mean reciprocal rank of 0.993. These results suggest that targeted adaptation of an open-source algorithm can enhance transcription robustness and EMR matching in low-resource, transcultural settings. Jasjot AI establishes a low-cost, scalable solution to reduce reliance on manual data reconciliation, broadening the applicability of speech technologies for communities whose languages have been largely ignored by the explosion of technology. Moreover, our approach provides a foundation for extension to other transcultural languages, tasks, and benchmarking efforts.