About the job
Zyphra is an innovative leader in artificial intelligence, located in the heart of San Francisco, California.
Role Overview:
As a Research Engineer specializing in Language Model Pre-Training, you will play a pivotal role in defining our language model strategy through comprehensive pretraining development. Your close collaboration with our pretraining team will ensure that your insights contribute to the advancement of our next-generation models.
Key Responsibilities:
Conduct large-scale training runs and implement model parallelization techniques.
Optimize the performance of our pretraining stack.
Oversee dataset collection, processing, and evaluation.
Research architecture and methodologies, including optimizer ablations.
Qualifications:
Demonstrated engineering prowess in developing reliable and robust systems.
A quick learner with a passion for implementing innovative ideas.
Exceptional communication and collaboration skills, capable of working effectively on both research and engineering implementations at scale.
Preferred Skills:
Profound expertise in addressing machine learning challenges and training models.
Experience training on large-scale (multi-node) GPU clusters.
In-depth understanding of model training pipelines, including model/data parallelism and distributed optimizers.
Strong methodology for conducting rigorous ablations and hypothesis testing.
Familiarity with large-scale, high-performance data processing pipelines.
High proficiency in PyTorch and Python programming.
Ability to navigate and understand extensive pre-existing codebases swiftly.
Published research in machine learning in reputable venues is an advantage.
Postgraduate degree in a relevant scientific field (Computer Science, Electrical Engineering, Mathematics, Physics).
Why Join Zyphra?
We value a research methodology that emphasizes thoughtful, methodical progress towards ambitious objectives. Both deep research and engineering excellence are given equal importance.
Join us in an environment that fosters innovation, collaboration, and professional growth.

