AI Quality Engineer

Tóm tắt công việc
Define and implement quality assurance strategies for AI agents built on our bbv AI Hub product, covering functional correctness, robustness, reliability, safety, and regression behavior.
Validate complete agent workflows, from user input and tool calls to integrations and final responses.
Design and maintain versioned, traceable test datasets that turn business requirements and real user scenarios into test cases, including edge cases, multi-step and tool-use tasks, and safety scenarios.
Use Langfuse to benchmark agent versions, prompts, models, and configurations against standardized datasets, tracking metrics such as task success, tool-call correctness, hallucination rate, latency, and cost.
Design evaluation rubrics and LLM-as-a-judge prompts, combining deterministic, LLM-based, and human evaluation, and validate the reliability of automated evaluators.
Establish baselines, quality thresholds, and regression criteria for agent releases, and support automated quality gates in CI/CD pipelines.
Analyze failed agent traces to find root causes and turn production failures and user feedback into permanent regression test cases.
Work closely with AI Engineers, Software Engineers, and Product Owners to define measurable acceptance criteria, resolve issues, and verify fixes.
Report evaluation results clearly and contribute to AI agent quality standards on the bbv AI Hub.
Competitive salary with full insurance coverage (medical, social, unemployment) based on net salary.
13th-month salary (paid semi-annually, with partial payment made mid-year rather than waiting until year-end).
Performance review twice a year with bonus opportunities.
Annual loyalty bonus (50% of monthly net salary from the 2nd year).
Premium health insurance and annual health check-up.
14 days of leave, plus additional leave based on tenure.
Daily perks: lunch allowance, free snacks/coffee, and sports activities.
Learning & development support: training programs and certification sponsorship.
English-speaking environment with tuition support.
Team activities and events: company trips, team building, and holiday celebrations.
Opportunity to work with international teams and join internal tech communities.