AI Research Scientist (Multimodal post-training)

Overview

Job Description

Responsibilities

• - Design and execute research on multimodal model training — with a primary focus on vision-language models and, increasingly, speech-language models
• - Develop and improve models that enable our AI agents to perceive and understand patients through video, language, and speech
• - Contribute to the full model development cycle: multimodal dataset curation and annotation, architecture design, cross-modal training strategies, evaluation, and iteration
• - Collaborate across AI Engineering, Product, and Clinical teams to translate multimodal research breakthroughs into production systems that deliver patient care
• - Work towards long-term ambitious research goals — such as real-time multimodal patient state estimation, clinical memory, and safety validation — while identifying and delivering immediate milestones
• - Advance the field by publishing in top-tier AI venues and clinical journals, contributing to Sword's growing body of peer-reviewed research

Required Skills

• - A PhD in Computer Science, Machine Learning, Natural Language Processing, Computer Vision, or a closely related AI field
• - Hands-on experience fine-tuning large language models or multimodal large models (e.g., vision-language models, speech-language models)
• - Experience training or fine-tuning models that operate across multiple modalities (e.g., video + language, image + text, speech + text)
• - A strong publication track record in peer-reviewed AI conferences or journals
• - Proficiency in Python and deep experience with modern ML frameworks (e.g., PyTorch, JAX)
• - Demonstrated ability to design rigorous experiments and interpret their results

Benefits

• - Health, dental and vision insurance
• - Meal allowance
• - Equity shares
• - Remote work allowance
• - Flexible working hours
• - Work from home
• - Discretionary vacation
• - Snacks and beverages