Evaluate Java LLM code using public repositories, GitHub issues, and unit tests. This remote contractor project pays $25 per label and requires at least 20 hours per week.

$25/label

Contract, part-time

Remote

Open in 31 countries

- Argentina
- Bangladesh
- Belize
- Bolivia
- Brazil
- Chile
- Colombia
- Costa Rica
- Dominican Republic
- Ecuador
- Egypt
- El Salvador
- Ghana
- Guatemala
- Guyana
- Haiti
- Honduras
- India
- Kenya
- Mexico
- Nicaragua
- Nigeria
- Pakistan
- Panama
- Paraguay
- Peru
- Puerto Rico
- Suriname
- Türkiye
- Uruguay
- Venezuela

The work

This role evaluates Java code produced or reviewed for large language models. You will work with public Java repositories, GitHub issues, and unit-test codebases, then provide clear feedback that helps improve AI systems.

The duties and qualifications below describe Java LLM code evaluation, so review them before applying.

- Evaluate model outputs using project rubrics and record evidence-based judgments.
- Review source materials and model outputs against project guidelines.
- Explain your decisions clearly so feedback can improve model behavior.
- Run, modify, and test real-world Java projects locally.
- Maintain consistent, high-quality decisions across repeated review tasks.

What it pays and takes

This is a fully remote contractor assignment.



The project is open to candidates in the countries listed below and requires English fluency.

- Time: At least 20 hours per week.
- Schedule: At least four hours per day and four hours of overlap with Pacific Standard Time.
- Location: Open to candidates in Argentina, Bangladesh, Belize, Bolivia, Brazil, Chile, Colombia, Costa Rica, the Dominican Republic, Ecuador, Egypt, El Salvador, Ghana, Guatemala, Guyana, Haiti, Honduras, India, Kenya, Mexico, Nicaragua, Nigeria, Pakistan, Panama, Paraguay, Peru, Puerto Rico, Suriname, Türkiye, Uruguay, and Venezuela.
- Experience: At least three years of overall experience; previous AI training experience is not required.
- Technical skills: Strong Java experience, Git, Docker, basic software pipeline setup, and the ability to navigate complex codebases.
- Helpful background: Open-source contribution or evaluation, LLM research or evaluation, or experience building or testing developer tools and automation agents.

About AI training work

AI training is the human work behind many artificial intelligence systems. Contributors review code, rate model answers, label data, and explain their judgments so AI systems can learn from reliable examples and expert feedback.

📌 SWE Bench Engineer (Argentina)
🏢 OpenTrain AI
📍 Argentina