Description
Ciklum is looking for a Senior AI Evaluation Engineer to join our team in Argentina.
We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a integral team of over 4,000 highly skilled developers, consultants, analysts and product owners, we engineer technology that redefines industries and shapes the way people live.
About the role:
As a Senior AI Evaluation Engineer, you'll become a part of a cross-functional development team engineering experience of tomorrow.
Project Description:
Creates the quality bar every AI system must pass — golden datasets, eval harnesses, pass thresholds — before launch and continuously in production. Defines what “good enough to ship” means, and measures it monthly for the scoreboard.
Responsibilities:
- Build the central eval harness and templates every pod uses; hub-and-spoke: central standards, pod-written tests
- Create golden datasets with business and clinical reviewers, including safety and medical-accuracy suites




- Set and defend pass thresholds; audit pod evals; report eval pass rates and incidents monthly
- Stand up production monitoring for drift and regression with the pipeline engineer

Requirements:
- 4+ years in ML/LLM evaluation, QA engineering for AI systems, or applied research engineering
- Hands-on with eval frameworks, LLM-as-judge patterns and their failure modes, statistical rigor on small samples
- Independent spine: the team shipping a thing does not set its own pass bar

Desirable:
- Healthcare or safety-critical evaluation experience; red-teaming background

What’s in it for you?
- Care: your mental and physical health is our priority. We ensure comprehensive company-paid medical insurance and mental health programs, 5 undocumented sick-leave days per year
- Tailored education path: boost your skills and knowledge with our regular internal events (meetups, conferences, workshops), Udemy lice

📌 Senior AI Evaluation Engineer (Argentina)
🏢 Ciklum
📍 Argentina