About the job
About AfterQuery
AfterQuery develops training data and evaluation frameworks that leading AI labs use to improve their models. The team partners with major research institutions to build datasets and run assessments that go beyond standard benchmarks. As a post-Series A company based in San Francisco, AfterQuery values contributions from every team member. Work here directly shapes the next generation of AI models.
Role Overview
The Reinforcement Learning Environment Engineer designs datasets and evaluation systems that influence how advanced AI models learn and improve. This role involves close collaboration with research teams, hands-on experimentation with new data collection methods, and the creation of metrics to track model progress. Work moves from theoretical analysis to practical experiments, feeding directly into large-scale model training efforts.
What You Will Do
- Develop data segments that expose key failure modes in sectors such as finance, software engineering, and enterprise operations.
- Refine reward signals for Reinforcement Learning from Human Feedback (RLHF) and Reinforcement Learning from Value Reinforcement (RLVR) systems.
- Define quantitative metrics for dataset quality, diversity, and their effects on model alignment and capability.
- Work closely with research teams to translate training objectives into concrete data requirements and evaluation criteria.
This position is based in San Francisco.
