About the job
Join LILT as we establish a worldwide network of legal and compliance professionals dedicated to enhancing AI evaluation. Our mission is to ensure high-quality AI systems through rigorous training, benchmarking, red-teaming, and continuous model optimization. We invite experts in legal and compliance fields to lend their insights to our human-in-the-loop AI evaluation processes utilized by top enterprises and government bodies.
This position is crafted for individuals adept in interpreting legal and regulatory text within practical environments, applying their skills to evaluate and enhance multilingual AI systems.
Your expertise will play a pivotal role in shaping the quality, safety, and readiness of our multilingual AI models for deployment.
This role offers two specialized tracks tailored to experience level and responsibilities.
Track A: Legal & Compliance AI Rater
Raters will conduct structured evaluations guided by clear rubrics and instructions.
Responsibilities
Analyze AI outputs concerning legal and compliance content
Perform structured scoring, comparisons, classifications, and judgments
Evaluate accuracy, relevance, completeness, risk, and legal alignment
Identify inaccuracies, ambiguities, unsafe guidance, or misleading information
Consistently apply domain-specific guidelines across evaluations
Ideal Background
Legal practitioner, compliance expert, legal analyst, or policy advisor
Experience in interpreting laws, regulations, contracts, and policies
Exceptional attention to detail and proficiency in structured evaluations
Track B: Legal & Compliance AI Evaluator (Senior Track)
Evaluators provide strategic oversight and influence evaluation methodologies.
Responsibilities
Refine evaluation rubrics and manage edge-case scenarios
Resolve discrepancies among raters through adjudication
Conduct thorough error analyses and qualitative reviews of model outputs
Collaborate with LILT's research, product, and customer teams on evaluation strategies

