Attorney Annotators for Legal LLM Evaluation
About OpenTrain
OpenTrain is the #1 platform for building careers in AI training and data labeling. Create a free account, build a profile, and apply in minutes to projects where people teach and improve AI systems.
Why this kind of AI work matters
Human labelers are the people who help AI learn to give useful, accurate answers. This role puts you on the front lines of shaping how legal LLMs respond to real legal questions.
Work is remote and flexible: many contributors choose hours that fit around other commitments while helping improve cutting-edge models.
The role
You will work with an internal legal LLM: pose legal questions and perform A/B evaluations of the model's text responses. Your judgments and labels will be used for evaluation and fine-tuning of the model.
This is a part-time contractor role (less than 20 hours/week). Pay is per label at $2.125 USD; there is no weekly cap on earnings. Current contributors on this project average $2,000 USD/week.
- Data type: text (legal questions and documents).
- Label types: evaluation/rating and fine-tuning annotations.
What you'll do day-to-day
- Pose or select legal questions and prompts for the internal LLM and review its text responses.
- Compare paired responses (A/B) and rate which response is better according to project guidelines.
- Provide structured evaluation ratings and any required fine-tuning labels for each response.
Requirements
- Legal degree from an accredited university in an English-speaking nation (example: J.D. from NYU).
- Fluency in English required.
- Entry-level experience acceptable; this role is open to contributors meeting the degree requirement.
- Available for part-time contractor work (under 20 hours per week).
- Willingness to follow annotation and evaluation guidelines precisely.
Who should apply
Licensed or academically trained attorneys who want flexible, remote work and to directly influence how legal language models behave.
People seeking part-time contractor work that leverages legal training without committing to a full-time role.
How it works and how to apply
Create a free OpenTrain account, complete your profile, and apply to this project in minutes. OpenTrain will list any project-specific qualification steps after you apply.
If accepted, you will receive the project's annotation instructions and begin labeling. Payment is per label at $2.125 USD; earnings are calculated based on completed labels and there is no weekly cap.
