About the job
We invite you to submit your CV in English, ensuring to indicate your level of English proficiency.
Mindrift serves as a bridge connecting specialists with exciting project-based AI opportunities from top technology companies, concentrating on the testing, evaluation, and enhancement of AI systems. Please note that participation is project-based rather than permanent employment.
Key Responsibilities
In this role, you will develop challenging coding test cases designed to thoroughly assess AI coding systems:
- Evaluate and enhance realistic coding tasks based on provided production codebases, ensuring they are appropriately scoped and detailed.
- Draft comprehensive functional tests that validate true end-to-end behavior and edge cases, moving beyond superficial checks.
- Design “fair yet challenging” tasks where the AI has the necessary context but must engage in complex reasoning to succeed (information is distributed across files and external sources).
- Investigate AI failures to discern areas where the model struggles versus areas of proficiency.
- Refine your work based on feedback from expert QA reviewers who assess your contributions against seven quality criteria.
Ideal Candidate Profile
This role is well-suited for seasoned developers, software engineers, or test automation specialists who are open to part-time, non-permanent projects. The ideal candidate will possess:
- A degree in Computer Science, Software Engineering, or a related discipline.
- Over 5 years of experience in software development, specifically with Python (pytest, async/await, subprocess, file operations).
- A background in full-stack development, with balanced expertise in creating React-based user interfaces and robust back-end systems.
- Experience in writing tests (functional, integration - not merely executing them).
- Familiarity with Docker containers (running evaluations locally in containers).
- An understanding of CI/CD processes (GitHub Actions: triggers, labels, interpreting results).
- Proficiency in English at a B2 level or above.
Application Process
To apply, simply follow these steps: Apply → Pass qualifications → Join a project → Complete tasks → Get compensated.
Effort Estimation
It is estimated that tasks for this project will require approximately 20 hours to complete, depending on complexity. This is a guideline, not a strict schedule; you are free to choose your working hours. All tasks must be submitted by the deadlines and fulfill the acceptance criteria to be considered complete.
Compensation
- Compensated contributions, with rates up to $45/hour*.
- Payment may be a fixed project rate or variable, contingent on the specific project.
- Some projects may offer additional incentive payments.
*Note: Rates may vary based on expertise, skills assessment, location, project requirements, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or less critical project phases. Payment specifics will be provided per project.