AI Research Scientist (Multimodal post-training)

Sword Health

Europehybrid€71,000 - €110,000Added 3 days agoPermanentRemote: Hybrid

Visa Sponsor English Required Ai research Ai & algorithms

Sword Health

·Europe

Apply Now

AI Research Scientist (Multimodal post-training)

Sword Health

Requirements

What you’ll be doing:

Design and execute research on multimodal model training — with a primary focus on vision-language models and, increasingly, speech-language models — including fine-tuning, alignment, and post-training methods (SFT, RLHF) tailored for clinical domains;

Develop and improve models that enable our AI agents to perceive and understand patients through video, language, and speech, building towards unified multimodal patient understanding;

Contribute to the full model development cycle: multimodal dataset curation and annotation, architecture design, cross-modal training strategies, evaluation, and iteration;

Collaborate across AI Engineering, Product, and Clinical teams to translate multimodal research breakthroughs into production systems that deliver patient care;

Work towards long-term ambitious research goals — such as real-time multimodal patient state estimation, clinical memory, and safety validation — while identifying and delivering immediate milestones;

Advance the field by publishing in top-tier AI venues and clinical journals, contributing to Sword's growing body of peer-reviewed research.

What you need to have:

A PhD in Computer Science, Machine Learning, Natural Language Processing, Computer Vision, or a closely related AI field;

Hands-on experience fine-tuning large language models or multimodal large models (e.g., vision-language models, speech-language models), including pre-training, SFT, RLHF, or related post-training techniques;

Experience training or fine-tuning models that operate across multiple modalities (e.g., video + language, image + text, speech + text);

A strong publication track record in peer-reviewed AI conferences or journals;

Proficiency in Python and deep experience with modern ML frameworks (e.g., PyTorch, JAX);

Demonstrated ability to design rigorous experiments and interpret their results.

Original Advert

At Sword, we're building AI to heal billions and unlock humanity's full potential. In doing so, we're pioneering AI Care, a fundamentally new approach to healthcare built for medical reasoning, safety, and real-time treatment, not generic technology applied after the fact. As both a clinical-centric frontier AI lab and an applied AI platform, Sword is reimagining how care is delivered at scale, removing traditional barriers like appointments, waiting rooms, and stigma so more people can access the care they need-and ultimately get back to lives lived in full.

Since 2020, Sword has expanded across physical therapy, women's health, cardiometabolic, and mental health, and is now moving beyond the session to a fully AI-native, 24/7 care program that brings physical activity, therapeutic exercise, psychotherapy, nutrition, and behavior change into one connected experience. More than 700,000 members across three continents have completed over 10 million AI sessions, helping 1,000+ enterprise clients avoid more than $1 billion in unnecessary healthcare costs. Backed by 42 clinical studies, 44+ patents, and more than $500 million raised from leading investors including Khosla Ventures, General Catalyst, and Founders Fund, Sword is defining a new standard for healthcare.

About the Role:

At Sword, we operate as both a clinical-centric frontier AI lab and an applied AI platform - conducting the foundational research that makes clinical AI possible, then putting it into practice through platforms that treat patients directly. Our AI Research team builds the core intelligence behind Dawn and Phoenix, our continuous, always-on care agent.

Healthcare demands AI that goes far beyond answering questions or completing tasks. It requires systems capable of sustained engagement across dozens of interactions - understanding a patient's history, perceiving their present state, and guiding their recovery over weeks and months. Delivering care at this level means AI must see, hear, and understand patients across modalities. To make this possible, we are pushing the boundaries of multimodal perception - fusing video, language, and speech into unified patient understanding - alongside memory systems and multi-turn reinforcement learning for long-horizon treatment planning.

We are building a category of AI product that has never existed before - AI that provides real clinical care autonomously, at scale. That means operating at the frontier of both research and product development simultaneously, often without a playbook. You will thrive here if you are energized by uncertainty, comfortable making high-stakes decisions with incomplete information, and motivated by the outsized impact that comes with pioneering something genuinely new. It's high pressure, high reward - and the team that does it best.

Our researchers don't choose between scientific rigor and product impact. The team actively publishes in top-tier AI conferences and clinical journals while shipping the models that power care for hundreds of thousands of patients. We have the computer, the data, and the team to support your best work.

AI Proficiency at Sword Health

AI fluency is a core expectation at Sword Health. Every candidate is assessed against our three-level framework - be ready to share real examples of how AI is already part of how you work.

Explorer (Level 1) - Uses AI daily to boost personal productivity

Builder (Level 2) - Creates workflows and tools that elevate the whole team

Integrator (Level 3) - Embeds AI into products and processes at scale

Every hire must demonstrate at least Level 1. The expected level will vary depending on the seniority of the role.

What we would love to see:

First-author publications in top-tier AI conferences (e.g., NeurIPS, ICML, ICLR, CVPR, ACL, EMNLP, COLM, Interspeech);

Deep expertise in one or more of: vision-language models, video understanding, speech-language models, multimodal representation learning, or cross-modal fusion architectures;

Experience with video-based or image-based model training in applied settings (e.g., human pose estimation, action recognition, medical imaging, or biological signal processing);

Experience building or contributing to LLM-based agents, including prompt engineering, memory orchestration, or agentic workflows;

A track record of taking research ideas from conception to working systems, including developing and debugging complex multimodal ML pipelines;

Industry experience during or after the PhD (e.g., research internships at leading AI labs);

Comfort with ambiguity and a track record of delivering results in fast-moving, high-uncertainty environments where research and product development happen in parallel;

Strong communication skills and a history of effective cross-functional collaboration;

A broader record of research excellence demonstrated through grants, fellowships, patents, or impactful open-source contributions.

Location: We welcome applications from candidates across Europe and the UK. We have a preference for candidates based in London, UK, or Lisbon / Porto, Portugal, where our research team is primarily located and where we have active offices, but exceptional candidates elsewhere in Europe will absolutely be considered.

Portugal - Sword Benefits & Perks:

• Health, dental and vision insurance

• Meal allowance

• Equity shares

• Remote work allowance

• Flexible working hours

• Work from home

• Discretionary vacation

• Snacks and beverages

Note: Please note that this position does not offer relocation assistance. Candidates must possess a valid EU visa and be based in Portugal.

Sword Health complies with applicable Federal and State civil rights laws and does not discriminate on the basis of Age, Ancestry, Color, Citizenship, Gender, Gender expression, Gender identity, Gender information, Marital status, Medical condition, National origin, Physical or mental disability, Pregnancy, Race, Religion, Caste, Sexual orientation, and Veteran status.