Research Engineer - Multimodal Data

Tavus – Multimodal AI Model OptimizationResearch EngineerAt Tavus, we are pioneering the human aspect of AI technology. Our objective is to make human-AI interactions as seamless and natural as in-person conversations, allowing for a human touch in areas that were once considered unscalable.We accomplish this through groundbreaking research in multimodal AI, focusing on human-to-human communication modeling (encompassing language, audio, and video) and the development of audio-visual avatar behaviors. Our innovative models drive applications ranging from text-to-video AI avatars to real-time conversational video experiences across sectors such as healthcare, recruitment, sales, and education.By empowering AI to perceive, listen, and engage with an authentic human-like presence, we are laying the groundwork for the next generation of AI workers, assistants, and companions.As a Series B company, we are supported by renowned investors, including Sequoia, Y Combinator, and Scale VC. Join us as we shape the future of human-AI interaction.The RoleWe are seeking an accomplished Research Scientist/Engineer with expertise in model optimization to be a vital part of our core AI team.The ideal candidate thrives in dynamic startup environments, is adept at setting priorities independently, and is open to making calculated decisions. We are moving swiftly and need individuals who can help navigate our path forward.Your MissionTransform state-of-the-art research models into fast, efficient, and production-ready systems through techniques such as sparsification, distillation, and quantization.Oversee the optimization lifecycle for critical models: establish metrics, conduct experiments, and evaluate trade-offs among latency, cost, and quality.Collaborate closely with researchers and engineers to convert innovative concepts into deployable solutions.RequirementsExtensive experience in deep learning with PyTorch.Practical experience in model optimization and compression, including knowledge distillation, pruning/sparsification, quantization, and mixed precision.Familiarity with efficient architectures such as low-rank adapters.Strong grasp of inference performance and GPU/accelerator fundamentals.Proficient in Python coding and adherence to best practices in research engineering.Experience with large models and datasets in cloud environments.Capability to read ML literature, reproduce results, and modify ideas accordingly.

About TavusTavus is at the forefront of innovation in human computing. Our mission is to develop AI Humans: an advanced interface that bridges the gap between individuals and machines, eliminating the friction found in current technologies. Our state-of-the-art human simulation models empower machines to see, hear, respond, and even exhibit realistic appearances—facilitating genuine, face-to-face interactions. AI Humans integrate the emotional insight of humans with the scalability and dependability of machines, making them reliable agents accessible 24/7, in any language, on our terms.Imagine having access to an affordable therapist, a personal trainer that fits your schedule, or a team of medical assistants dedicated to providing personalized care for every patient. With Tavus, individuals, enterprises, and developers have the tools to create AI Humans that connect, comprehend, and act with empathy on a large scale.We are a Series A company supported by esteemed investors such as Sequoia Capital, Y Combinator, and Scale Venture Partners.Join us in shaping a future where machines and humans genuinely understand one another.The PositionWe are seeking an AI Researcher to join our core AI team and advance the frontiers of multimodal conversational intelligence. If you excel in dynamic environments, enjoy transforming abstract concepts into functional code, and derive motivation from pushing the boundaries of possibility, this role is designed for you.Your Responsibilities Engage in research focusing on Foundational Multimodal Models specifically in the realm of Conversational Avatars (such as Neural Avatars and Talking-Heads).Develop models for video, audio, and language sequences utilizing Autoregressive and Predictive Architectures (e.g., V-JEPA) and/or Diffusion methodologies, with a focus on temporal and sequential data rather than static images.Collaborate closely with the Applied ML team to implement your research into production systems.Remain at the forefront of multimodal learning and assist us in defining what “cutting edge” will mean in the future.Ideal Candidate ProfilePhD (or nearing completion) in a relevant field, or equivalent practical research experience.Experience in multimodal machine learning, particularly focused on conversational interfaces.

The Bot CompanyAt The Bot Company, we are on a mission to create an innovative robotic assistant for every household.Our dynamic team, composed of talented engineers, designers, and operators, is based in San Francisco. We have a rich background from industry leaders such as Tesla, Cruise, OpenAI, Google, and Pixar, and we have successfully delivered products to hundreds of millions of users, honing our ability to create exceptional products and experiences.We pride ourselves on maintaining a streamlined team structure that fosters swift decision-making and minimizes bureaucracy. Each member is considered an Individual Contributor, granted substantial autonomy, ownership, and accountability. Our culture enables us to work across the technology stack with an emphasis on rapid iteration and execution.What We Seek in CandidatesCandidates for all positions at The Bot Company must exhibit remarkable sharpness and the capacity to thrive in high-pressure environments. We expect candidates to showcase:Exceptional Cognitive Abilities: You possess quick thinking, instant learning capabilities, and the ability to reason across diverse domains.Engineering Curiosity: You demonstrate an innate desire to understand how systems function, even beyond your area of expertise.Performance-Driven Attitude: You excel in fast-paced settings, effectively navigate ambiguity, and thrive under demanding circumstances.Machine Learning: Multimodal Foundation ModelsWe are developing unified foundation models capable of reasoning across text, images, video, and kinematics to inform intelligent robotic behaviors.You will engage with large-scale multimodal networks, overseeing the complete process from data handling to model training and deployment.Your ResponsibilitiesConstruct Native Multimodal Policies: Create architectures where vision, language, and other modalities are represented in a unified manner.Enhance Cross-Modal Reasoning: Explore and implement strategies to ensure that the model not only correlates modalities but also comprehends them (e.g., linking visual physics to kinematic constraints).Manage the Training Loop from Start to Finish: Design, execute, troubleshoot, and refine large-scale training experiments; identify failure points, enhance data mixtures, and tighten evaluations to achieve measurable improvements.Deploy and Refine Real Systems: Integrate models into practical robotic frameworks, enhance robot code for model deployment, and optimize performance for edge inference.

About UsTavus is an innovative research lab at the forefront of human computing technology. Our mission is to create AI Humans—advanced interfaces that bridge the gap between individuals and machines, eliminating the friction found in current systems. Our real-time human simulation models empower machines to see, hear, respond, and appear realistic, facilitating genuine, face-to-face conversations. With AI Humans, we blend the emotional intelligence inherent in humans with the extensive reach and reliability of machines, enabling them to serve as capable and trusted agents available 24/7, capable of communicating in any language.Envision a therapist accessible to everyone, a personal trainer that tailors sessions to your schedule, or a fleet of medical assistants dedicated to providing personalized attention to every patient. Tavus enables individuals, enterprises, and developers to create AI Humans that connect, empathize, and act with understanding on a large scale.Backed by prestigious investors like Sequoia Capital, Y Combinator, and Scale Venture Partners, we are a Series A company ready to shape the future of human-machine interaction.Join us in transforming a future where humans and machines genuinely comprehend one another.The RoleWe are seeking a passionate AI Researcher to join our core AI team and advance the science of audio-visual avatar generation. If you thrive in dynamic startup environments, enjoy experimenting with generative models, and are excited to see your research translated into production, you will find a welcoming home here.Your Mission Conduct research and develop cutting-edge audio-visual generation models for conversational agents (e.g., Neural Avatars, Talking Heads).Focus on models that intricately align with conversation flows, ensuring seamless integration of verbal and non-verbal cues.Experiment with diffusion models (DDPMs, LDMs, etc.), long-video generation, and audio synthesis.Collaborate closely with the Applied ML team to transition your research into practical applications.Stay updated on the latest breakthroughs in multimodal generation and contribute to the evolution of this field.You Will Excel If You Have:A PhD (or nearing completion) in a relevant discipline, or equivalent hands-on research experience.Proficiency in applying image/video generation techniques and a solid understanding of machine learning principles.

Join Zyphra as a Research Engineer specializing in AI Performance and Kernel Optimization. In this role, you will work at the forefront of AI technologies, developing and optimizing kernel solutions that enhance the performance of our systems. You will collaborate with cross-functional teams, leveraging your expertise to drive innovation and efficiency.

Anthropic is looking for a Research Engineer focused on model evaluations. This position involves research and development to assess and strengthen the performance of AI models. Teams are based in San Francisco and New York City, and the role supports remote work with required travel. Key responsibilities Design and implement evaluations for Anthropic's AI models Collaborate with team members to enhance model performance Contribute to research that pushes the boundaries of AI systems Location Remote-friendly (travel required) San Francisco, CA New York City, NY

Zyphra is an innovative artificial intelligence company located in the heart of San Francisco, California.The Opportunity:Join our dynamic team as a Research Engineer - Audio & Speech Models, where you will play a pivotal role in advancing Zyphra’s Audio Team. You will be instrumental in developing cutting-edge open-source text-to-speech and audio models. Your contributions will span the full spectrum of the model training process, from data collection and processing to the design of innovative architectures and training approaches.Your Responsibilities:Conduct large-scale audio training operationsOptimize the performance of our training infrastructureCollect, process, and evaluate audio datasetsImplement architectural and methodological improvements through rigorous testingWhat We Seek:A strong research mindset with the ability to navigate projects from ideation to implementation and documentation.Proficiency in rapid prototyping and implementation, allowing for swift experimentation.Effective collaboration skills in a fast-paced research environment.A quick learner who is eager to embrace and implement new concepts.Excellent communication abilities, enabling you to contribute to both research and engineering tasks at scale.Preferred Qualifications:Expertise in training audio models, such as text-to-speech, ASR, speech-to-speech, or emotion recognition.Experience with training audio autoencoders.Solid understanding of signal processing, particularly in audio.Familiarity with diffusion models, consistency models, or GANs.Experience with large-scale (multi-node) GPU training environments.Strong understanding of experimental methodologies for conducting rigorous tests and ablations.Interest in large-scale, parallel data processing pipelines.Competence in PyTorch and Python programming.Experience contributing to large, established codebases with rapid adaptation.

Join worldlabs as a Research Engineer focused on scaling multimodal data. In this dynamic role, you will leverage cutting-edge technologies and methodologies to enhance data processing capabilities. You will be responsible for developing innovative solutions that integrate various data types and drive impactful research outcomes.

Our MissionAt Altos Labs, we are dedicated to restoring cell health and resilience through innovative cell rejuvenation techniques aimed at reversing diseases, injuries, and disabilities that can arise throughout life.For further insights, please visit our website at altoslabs.com.Our ValueOur singular Altos Value is: Everyone Owns Achieving Our Inspiring Mission.Diversity at AltosWe firmly believe that diverse perspectives are crucial for scientific innovation. At Altos, exceptional scientists and industry leaders collaborate globally to further our shared mission. We prioritize Belonging, ensuring all employees feel valued for their unique perspectives, and we hold ourselves accountable for maintaining a diverse and inclusive environment.Your Contributions to AltosAs a member of our team, you will accelerate and enhance our efforts in developing unified, multi-modal generative foundation models tailored for multiscale biology. You will be a key player in multidisciplinary teams that create the computational platforms essential for Altos to fulfill its mission.In this position, you will collaborate with other scientists and engineers across the Institute of Computation to design, develop, and scale cutting-edge foundation models that address biological inquiries and assist in discovering novel interventions for aging and disease. Your focus will be on synthesizing unstructured multimodal signals with structured relational data and knowledge graphs that depict biological realities.The ideal candidate will excel in a dynamic environment that values teamwork, transparency, scientific excellence, originality, and integrity.

Bland Inc. seeks a Machine Learning Researcher specializing in Multimodal Large Language Models (LLMs) to join the team in San Francisco. The focus is on advancing AI systems that integrate language with other types of data. Role overview This position centers on research and development aimed at improving how AI models process and understand information from multiple sources, such as text combined with images or other modalities. What you will do Investigate how language interacts with additional data types within multimodal LLMs Create and evaluate new methods to enhance AI model performance Work closely with colleagues on projects designed to push the boundaries of machine learning Location This role is based in San Francisco.

About Hike Medical Hike Medical is building the future of musculoskeletal care by combining advanced technology with practical healthcare solutions. Based in San Francisco’s Rincon Hill, the team develops a platform that spans three core areas: an AI-powered vision system for rapid web-based foot scans that generate custom 3D-printed orthotics, an AI agent platform that manages the entire DME workflow from intake through claims, and SoleForge, a high-scale 3D printing facility for custom medical devices. Hike Medical partners with some of the world’s largest employers and major orthotics and prosthetics organizations. Fortune 50 companies trust the platform to support employee well-being, and a broad network of clinical partners keeps the company connected to real-world needs. Custom insoles are just the starting point. The long-term goal is to reshape the industry with bionic devices: AI-designed, robotically manufactured orthotic and prosthetic products. The company aims to reach this milestone by 2040. Learn more at bionics2040.com. With $22 million raised across Seed and Series A rounds from leading investors, Hike Medical offers a results-oriented culture for those interested in the intersection of AI, manufacturing, and healthcare.

At Databricks, we are dedicated to empowering data teams to tackle the world's most challenging problems, from detecting security threats to advancing cancer drug development. We achieve this by offering the premier data and AI platform, allowing our customers to concentrate on their mission-critical challenges. The Mosaic AI organization assists companies in developing AI models and systems utilizing their own data, employing technologies that range from training large language models (LLMs) from the ground up to employing advanced retrieval methods for enhanced generation. We pride ourselves on pushing the boundaries of science and operationalizing our innovations. Mosaic AI believes that a company’s AI models hold intrinsic value, akin to any other core intellectual property, and that superior AI models should be accessible to all. Job Overview As a research engineer in the Scaling team, you will stay abreast of the latest advancements in deep learning and pioneer new methodologies that surpass the current state of the art. You will collaborate with a diverse team of researchers and engineers, sharing insights and expertise. Most importantly, you will be passionate about our customers, striving to ensure their success in implementing cutting-edge LLMs and AI systems by translating our scientific knowledge into practical applications. Your Impact Enhance performance through innovative optimization techniques, including kernel fusion, mixed precision, memory layout optimization, tiling strategies, and tensorization tailored for training-specific patterns. Design, implement, and optimize high-performance GPU kernels for training workloads, including attention mechanisms, custom layers, gradient computations, and activation functions, specifically for NVIDIA architectures. Create and implement distributed training frameworks for large language models, incorporating parallelism strategies (data, tensor, pipeline, ZeRO-based) and optimized communication patterns for gradient synchronization and collective operations. Profile, debug, and optimize comprehensive training workflows to pinpoint and resolve performance bottlenecks, utilizing memory optimization techniques such as activation checkpointing, gradient sharding, and mixed precision training.

Join Perplexity as a Research Engineering Manager, where you will spearhead a team of exceptional AI researchers and engineers dedicated to crafting the advanced models that power our innovative products. Our talented team has pioneered some of the most sophisticated models in agentic research, query understanding, and other critical domains that demand precision and depth. As we broaden our user base and expand our product offerings, our proprietary models are increasingly essential for delivering a premium experience to the world's most discerning users.You will explore our extensive datasets of conversational and agentic queries, applying state-of-the-art training methodologies to enhance AI model performance. Through proactive technical and organizational leadership, you will empower your team to create cutting-edge models for the applications that are most significant to our business and our users.

Team Overview The infrastructure team at OpenAI manages the core systems that support AI workloads worldwide. As OpenAI expands its compute capabilities across company-owned data centers, cloud environments, and strategic partnerships, the need for careful planning and resource management grows. Reliable and cost-effective compute operations depend on this foundation. The Compute Optimization group operates at the intersection of engineering, operations, finance, and infrastructure strategy. This team develops models, decision tools, and planning systems to improve how compute resources are scheduled, deployed, and scaled as global needs shift. Role Overview OpenAI is hiring a Compute Optimization Researcher/Engineer to help maximize the use of compute capacity across the organization. This role addresses complex optimization challenges related to capacity allocation, demand forecasting, cluster planning, workload placement, and infrastructure utilization. Work includes building mathematical models, developing software systems, and collaborating with other teams to improve planning and use of compute resources. Areas of focus span GPU clusters, networking, storage, and data center infrastructure. Candidates with experience in operations research, optimization, applied mathematics, infrastructure systems, or large-scale capacity planning will be well-suited for this position. Location and Work Model This position is based in San Francisco, CA. OpenAI follows a hybrid schedule with three days per week in the office. Relocation assistance is offered.

Join VOLT, a trailblazer in crafting advanced AI perception systems that enhance safety and security through real-time risk detection in the physical world.We are on the lookout for a Senior Applied AI & Machine Learning Engineer dedicated to designing, optimizing, and deploying multimodal AI models capable of functioning reliably in diverse real-world scenarios. This is a hands-on role focused on transitioning models from conceptual data to practical production, encompassing both edge devices and cloud infrastructures.In this position, you will engage with vision, video, and language-based models that interpret real-world scenes and events, ensuring their accuracy, latency, robustness, and cost-effectiveness in production systems.Reporting directly to the Head of Engineering, you will play a pivotal role in advancing VOLT AI’s core perception platform.

Overview: Join us at Spellbrush as we innovate in the world of gaming! We are developing an immersive 3D first-person adventure game where an AI companion drives the gameplay experience. Our goal is to create a game that integrates large language models (LLMs) in a way that enhances storytelling and player engagement, moving beyond simple chat interactions.About SpellbrushAt Spellbrush, we are dedicated to crafting exceptional anime games. As the leading generative AI studio, we are the creative force behind niji・journey. Our mission is clear: to harness AI in bringing vibrant characters to life and to redefine narrative-driven gaming experiences.Our ProjectWe have developed an innovative in-house LLM storytelling system that seamlessly integrates AI with narrative and gameplay, offering players a depth of interaction that transcends traditional gaming.Role OverviewAs an integral member of our small but highly skilled team, you will have the opportunity to shape the future of gaming. Collaborate with leading minds in the industry, including the creator of Warudo and a veteran from Google DeepMind behind Project Astra. Your role will afford you substantial creative and research freedom to pioneer LLM-driven storytelling.

Join Our Innovative TeamTavus is a forward-thinking research laboratory at the forefront of human-computer interaction. We are dedicated to creating AI Humans, a revolutionary interface that bridges the gap between individuals and machines, eliminating the friction commonly encountered in today's technology. Our advanced human simulation models empower machines to see, hear, respond, and even exhibit lifelike appearances, facilitating genuine, face-to-face interactions. By merging human emotional intelligence with machine efficiency, our AI Humans serve as reliable, empathetic agents available around the clock, in any language, tailored to user needs.Picture a therapist within reach for everyone, a personal trainer that fits seamlessly into your schedule, or a comprehensive team of medical assistants providing focused attention to every patient. With Tavus, individuals and organizations can develop AI Humans to foster connection, understanding, and responsiveness on a grand scale.As a Series A startup, we are backed by prestigious investors such as Sequoia Capital, Y Combinator, and Scale Venture Partners. Join us in crafting a future where humans and machines communicate effortlessly.Your RoleWe seek an AI Researcher to join our dynamic AI team and explore the frontiers of large language modeling within conversational AI. If you excel in fast-paced startup settings, enjoy experimenting with innovative ideas, and are eager to see your contributions realized in production, you will thrive here.Your Mission Conduct in-depth research on large language modeling and its adaptation for Conversational Avatars (e.g., Neural Avatars, Talking-Heads).Create methodologies to effectively model both verbal and non-verbal communication, dynamically adjusting avatar behaviors in real-time.Experiment with fine-tuning, adaptation, and conditioning techniques to enhance the expressiveness, control, and task specificity of LLMs.Collaborate with the Applied ML team to transition research from prototype to full-scale deployment.Stay informed on the latest advancements and contribute to defining the next breakthroughs.

Join Cartesia as a Model Architecture ResearcherAt Cartesia, our vision is to revolutionize AI by creating interactive intelligence that is seamlessly integrated into your daily life. Unlike current models, our goal is to develop systems capable of processing extensive streams of audio, video, and text—1 billion text tokens, 10 billion audio tokens, and 1 trillion video tokens—directly on devices.As pioneers in innovative model architectures, our founding team, which originated from the Stanford AI Lab, has developed State Space Models (SSMs)—a groundbreaking foundation for training efficient, large-scale models. Our diverse team merges deep expertise in model innovation with a design-focused engineering approach, allowing us to create and deploy state-of-the-art models and applications.Backed by leading investors such as Index Ventures, Lightspeed Venture Partners, and many others, including industry veterans and advisors, we are poised to shape the future of AI.Your ContributionIn this role, you will drive forward-thinking research in neural network architecture, focusing on alternative models like state space models, efficient transformers, and hybrid architectures.Create innovative architectures that enhance model performance, inference speed, and adaptability in various environments, from cloud infrastructures to on-device implementations.Develop advanced capabilities for models, including statefulness, long-range memory, and novel conditioning mechanisms to boost expressiveness and generalization.Analyze architectural decisions and their effects on model characteristics such as scalability, robustness, latency, and energy consumption.Create frameworks and tools to assess architectural advancements, benchmarking their performance in both research and production contexts.Collaborate with interdisciplinary teams to translate architectural insights into scalable systems that deliver real-world impact.Your QualificationsExtensive experience in architecture design with a focus on advanced models such as state space models, transformers, and RNN/CNN variants.In-depth understanding of the interplay between architectural designs and system constraints, particularly in cloud and on-device deployments.Strong proficiency in the design and evaluation of neural network architectures.

Join Achira in shaping the future of deep learning with cutting-edge generative, representational, and simulation models for molecules and materials. Our mission is to create foundational models that render the atomistic universe understandable, predictable, and designable.Why Choose Achira?Be part of an elite, cross-disciplinary team comprising ML researchers, physicists, chemists, and engineers who are redefining atomistic simulation through expansive foundation models.Advance the integration of deep learning with the principles of nature, merging generative AI, probabilistic reasoning, and molecular physics.Engage in projects at an unparalleled scale, tackling extensive datasets, computational challenges, and ambitious goals.Take full ownership of your research journey — from ideation and architecture to training, evaluation, and deployment.Flourish in a dynamic culture that values rigor, speed, creativity, and impact over bureaucracy.Position OverviewAs a Generative AI Researcher at Achira, you will contribute to the development of foundation simulation models — large-scale systems designed to learn the structure, dynamics, and energetics of the atomistic realm. These models will unite deep representation learning, generative modeling, and sophisticated simulation techniques.Your responsibilities will include:Crafting and training state-of-the-art deep generative models — including diffusion, autoregressive, flow-based, and latent-variable architectures focused on molecules, materials, and atomic systems.Creating expressive representations of molecular and atomistic structures and dynamics utilizing equivariant graph neural networks, geometric transformers, and latent encoders that respect physical symmetries and constraints.Innovating advanced sampling and simulation techniques that blend probabilistic inference, deep learning, and reinforcement learning to facilitate efficient exploration and simulation of learned energy landscapes.Developing models that comprehend, generate, and simulate the physical world, merging reasoning, simulation, and predictive capabilities.Working collaboratively with physicists and chemists to validate models against ab initio, molecular dynamics, and experimental datasets.Rapidly prototyping, benchmarking, and iterating — converting research concepts into reusable, scalable model components across Achira’s foundation model suite.