Machine Learning Engineer

Overview

Developed a Document AI engine to process administrative medical documents (eg. appointment forms, reports, referrals), simplifying and streamlining operations for healthcare clinics. The engine combines a finetuned multimodal transformer for intelligent visual extraction with OpenAI API for post-processing.

Key Contributions

• Developed AI engine
  • Built the startup’s machine learning framework from scratch
  • Labeled ~2000 dataset samples
  • Finetuned multimodal transformer using cloud compute
  • Modified inference logic to extract confidence scores
  • Developed evaluation procedure to measure engine performance
  • Integrated OpenAI API for post processing (semantic processing, validation, reformatting)
  • Deployed AI engine on cloud servers

• Improved backend system
  • Integrated the AI engine into backend pipeline
  • Developed mechanisms to detect duplicate documents and collect low confidence documents for review
  • Implemented interfaces with frontend

• Led Proof-Of-Concept project with new Australian client
  • Engaged client to translate their workflows and user needs into design requirements
  • Defined the POC scope, success metrics, and delivery milestones
  • Developed a custom AI engine and backend system
  • Led a team of frontend and backend engineers

Outcomes

• The AI engine demonstrated robustness with high noise, handwriting, and multi-page documents.
• Achieved accuracy of 85% on client documents, up from 60% with the previous version, and outperforming general-purpose models like GPT-4o (75%).
• End-to-end system delivers results in ~10s for a single-page document, including upload, inference, and display, with ~4s for each additional page.
• Successful POC led to an ongoing partnership with the Australian client.