companyZyphra logo

Research Engineer - Language Model Pre-Training

ZyphraSan Francisco
On-site Full-time

Clicking Apply Now takes you to AutoApply where you can tailor your resume and apply.


Unlock Your Potential

Generate Job-Optimized Resume

One Click And Our AI Optimizes Your Resume to Match The Job Description.

Is Your Resume Optimized For This Role?

Find Out If You're Highlighting The Right Skills And Fix What's Missing

Experience Level

Mid to Senior

Qualifications

Strong engineering aptitude for rapidly implementing reliable and robust systems, a quick learner with an excitement for new ideas, and excellent communication and collaboration skills.

About the job

Zyphra is an innovative leader in artificial intelligence, located in the heart of San Francisco, California.

Role Overview:

As a Research Engineer specializing in Language Model Pre-Training, you will play a pivotal role in defining our language model strategy through comprehensive pretraining development. Your close collaboration with our pretraining team will ensure that your insights contribute to the advancement of our next-generation models.

Key Responsibilities:

  • Conduct large-scale training runs and implement model parallelization techniques.

  • Optimize the performance of our pretraining stack.

  • Oversee dataset collection, processing, and evaluation.

  • Research architecture and methodologies, including optimizer ablations.

Qualifications:

  • Demonstrated engineering prowess in developing reliable and robust systems.

  • A quick learner with a passion for implementing innovative ideas.

  • Exceptional communication and collaboration skills, capable of working effectively on both research and engineering implementations at scale.

Preferred Skills:

  • Profound expertise in addressing machine learning challenges and training models.

  • Experience training on large-scale (multi-node) GPU clusters.

  • In-depth understanding of model training pipelines, including model/data parallelism and distributed optimizers.

  • Strong methodology for conducting rigorous ablations and hypothesis testing.

  • Familiarity with large-scale, high-performance data processing pipelines.

  • High proficiency in PyTorch and Python programming.

  • Ability to navigate and understand extensive pre-existing codebases swiftly.

  • Published research in machine learning in reputable venues is an advantage.

  • Postgraduate degree in a relevant scientific field (Computer Science, Electrical Engineering, Mathematics, Physics).

Why Join Zyphra?

  • We value a research methodology that emphasizes thoughtful, methodical progress towards ambitious objectives. Both deep research and engineering excellence are given equal importance.

  • Join us in an environment that fosters innovation, collaboration, and professional growth.

About Zyphra

Zyphra is a cutting-edge artificial intelligence company located in San Francisco, dedicated to pushing the boundaries of technology through research and innovation.

Similar jobs

Tailoring 0 resumes

We'll move completed jobs to Ready to Apply automatically.