15 nov
|
Web.com Canada
|
Argentina
15 nov
Web.com Canada
Argentina
Salario: $ 1.200.000 – $ 2.400.000
This role goes beyond traditional QA; the idóneo candidate should be capable of validating, evaluating, and improving AI-powered systems, agents, copilots, and LLM-driven workflows while building scalable automation frameworks and intelligent quality solutions. You will work closely with Engineering, AI/ML, Platform, Product, and DevOps teams to ensure reliability, safety, observability, scalability, and quality of modern AI-native applications and Agentic systems.Who Are We.Newfold Digital is a leading web technology company serving nearly seven million customers globally. Established in 2021 through the combination of leading web services providers Endurance Web Presence and Web.com Group, our portfolio of brands includes: Bluehost, BigRock, ResellerClub, CrazyDomains, HostGator, Network Solutions, Register.com, Web.com and many others.We help customers of all sizes build a digital presence that delivers results. With our extensive product offerings and personalized support, we take pride in collaborating with our customers to serve their online presence needs.Key ResponsibilitiesAI/LLM Quality EngineeringDesign, validate, and optimise testing strategies for AI-powered applications, copilots, chatbots, RAG pipelines, and autonomous AI agents.Develop AI evaluation (Evals) frameworks for:Hallucination detectionGroundednessToxicity & bias validationResponse qualityContext relevancePrompt regression testingDeterminism & consistencySafety and guardrail validationBuild automated validation pipelines for LLM outputs using structured assertions, semantic scoring, embeddings, and evaluation metrics.Create synthetic datasets and benchmark datasets for AI testing scenarios.Test multi-agent orchestration workflows and agent memory/context retention.Validate MCP (Model Context Protocol)
integrations and tool interoperability.Work on Context Engineering, Prompt Engineering, and Prompt Optimisation strategies.Evaluate latency, token usage, model cost optimisation, throughput, and scalability.Automation EngineeringDevelop and maintain scalable UI/API automation frameworks using:Python (mandatory)Robot Framework (preferred)Playwright/Selenium/CypressAutomate testing for:REST APIsGraphQL APIsEvent-driven systemsStreaming systemsWebhooksMicroservicesMFE (Micro Frontend) architecturesBuild reusable automation libraries, utilities, and AI-assisted testing workflows.Integrate automation into CI/CD pipelines and quality gates.AI Engineering & Platform ValidationValidate RAG architectures including:Chunking strategiesEmbedding qualityRetrieval accuracyRe-rankingVector search relevanceWork with vector databases such as:PineconeWeaviateChromaDBQdrantFAISSBuild or validate AI agents leveraging:LangChainLangGraphCrewAIAutoGenSemantic KernelOpenAI SDKsValidate AI workflows involving:Tool callingFunction callingAgent memoryAgent planning/executionMulti-modal AI systemsStructured outputsDevOps,
Observability & ReliabilityBuild and maintain quality workflows integrated with:GitHub ActionsJenkinsGitLab CI/CDAzure DevOpsWork with Docker and Kubernetes environments.Perform cloud-based testing and deployments across AWS/Azure/OCI/GCP.Validate distributed systems reliability and resiliency.Monitor AI system behaviour using observability platforms/logging/tracing tools.Contribute to shift-left and quality engineering best practices.Required SkillsMust HaveStrong programming experience in PythonStrong understanding of Software Testing & Quality Engineering principlesHands-on experience with:AI/LLM testingPrompt EngineeringContext EngineeringAI EvalsRAG systemsAI AgentsExperience with:CI/CD pipelinesDocker & KubernetesCloud platforms (AWS/Azure/OCI/GCP)API testing & automationMicroservices testingUI automationExperience with SQL and data validationStrong debugging, analytical, and troubleshooting skillsExperience working in Agile/Scrum environmentsPreferred / Good to HaveExperience with Robot FrameworkExperience with PlaywrightKnowledge of:LangChainLangGraphLlamaIndexCrewAISemantic KernelMCP (Model Context Protocol)Hands‑on experience with:CursorGitHub CopilotClaudeGeminiOpenAI APIsExperience with vector databases and embeddingsExposure to MLOps workflows and ML lifecycle testingKnowledge of performance testing and security testingExperience validating AI governance/compliance requirementsExperience with contract testing, chaos testing, or resilience engineeringFamiliarity with accessibility testing and cross-browser/device testingExposure to synthetic monitoring and production observabilityWhat We're Looking ForEngineers who actively experiment with AI tools and build side projects, prototypes, PoCs, or open-source contributions.Strong problem solvers who can combine traditional QE practices with modern AI-native engineering approaches.Passion for innovation, developer productivity, automation acceleration, and intelligent testing.Ability to rapidly learn emerging AI frameworks, tools, a#J-18808-Ljbffr
📌 AI - Software Development Engineer in Test (AI - SDET) (Argentina)
🏢 Web.com Canada
📍 Argentina