About the job
About Us
At Contra Labs, we are a pioneering research lab dedicated to exploring the intersection of creativity and artificial intelligence. Our mission is to comprehend how creatives leverage AI, enhance AI tools to better meet creative demands, and uphold the diversity and originality that characterize exceptional creative work in an increasingly standardized world.
We bridge a global network of elite creative professionals with cutting-edge AI laboratories and product companies. By embedding our research in authentic creative workflows, we establish benchmarks for excellence across various mediums including design, video, and imagery. Our findings directly inform the benchmarking, evaluation, and post-training processes for leading AI models and applications.
As a subsidiary of Contra, the professional network for independent creatives, Contra Labs transforms human insights into the training and evaluation framework that empowers AI tools to serve their users effectively. Our focus remains steadfastly on the creative community.
Role Overview
As Contra Labs expands its client engagements in areas such as dataset creation, benchmarking, and evaluation partnerships, we are in search of a dedicated HCI Researcher. This role will entail overseeing the research and delivery processes from inception to completion. You will work across various projects involving AI labs, creative tool companies, and research teams, each with unique requirements and timelines. Your responsibilities will include scoping methodologies, designing evaluation protocols, assembling evaluator teams, maintaining quality standards, and ensuring the delivery of actionable insights.
Key Responsibilities
Design and lead comprehensive research studies, including annotation protocols, rubrics, and evaluation methodologies for multifaceted creative outputs.
Conduct qualitative and quantitative analyses, synthesizing results into actionable insights and recommendations for our clients.
Collaborate with creative domain experts, manage evaluator teams, and ensure consistent quality through real-time adjustments.
Establish and standardize protocols for annotation, data quality, and inter-rater reliability across various projects.
Effectively communicate research findings to stakeholders through comprehensive reports, engaging presentations, and interactive workshops.
Work closely with engineering teams to address tooling requirements and contribute to Contra Labs' visibility through case studies and benchmark publications.
Translate client objectives into meticulously crafted research plans, managing multiple concurrent projects with varying timelines and requirements.

