Position: AI Tester Location: Gia Lam, Hanoi Contact: Tuyen dung Vietsol ([protected info])
Job Descriptions
Review test results, test cases, and evaluation reports prepared by AI Quality Engineers; provide feedback and quality assessment conclusions.
Guide, train, and transfer AI testing methodologies and tools to the QA team.
Collaborate with Developers, Data Scientists, BAs, POs, and DevOps teams to align on quality criteria and corrective action plans.
Participate in competency assessments and propose career development paths for AI Quality Engineers.
Develop and standardize AI testing processes and guidelines.
Develop AI testing strategies and test plans: define scope, risks, evaluation methodologies (offline/online, human evaluation/LLM-as-a-judge), resources, and timelines for each AI product.
Design and develop testing tools, including evaluation harnesses, automated evaluation pipelines integrated into CI/CD, test datasets/golden datasets, and test data version control mechanisms.
Establish AI evaluation metrics: define metrics, acceptance thresholds (quality gates), and scoring rubrics for different model types and use cases.
Review AI Quality Engineers' work, including test coverage, test case accuracy, test data, and evaluation conclusions.
Establish mechanisms for monitoring model quality after deployment (drift, hallucinations, user feedback) and develop quality reporting dashboards.
Analyze the root causes of defects and collaborate with development teams to propose improvements to models, prompts, and data.
Perform other tasks as assigned by the Direct Manager.
Provide periodic reports on AI product quality to management.
Research, evaluate, and propose the adoption of new AI evaluation tools and methodologies.
Job Requirements
Bachelor's degree or higher in Information Technology, Computer Science, Data Science, Artificial Intelligence, Software Engineering, or related fields.
ISTQB certification (Advanced Level, AI Testing - CT-AI) or professional certifications in AI/ML are an advantage.
At least 5 years of experience in software testing/quality assurance, including a minimum of 2 years in testing and evaluating AI/ML/LLM products.
Experience in developing testing strategies, test plans, and leading/reviewing QA team activities.
Preferred Qualifications:
Experience in developing evaluation frameworks/harnesses for LLM, RAG, and AI Agent applications in production environments.
Proficiency in Python; experience with DeepEval, Ragas, Promptfoo, LangSmith, MLflow, etc.
Experience in integrating AI testing into CI/CD pipelines and monitoring model quality after deployment.
Technical Knowledge & Skills:
Quality-first mindset, objectivity, and data-driven decision-making in quality assessments.
Proactive in researching and staying updated on emerging AI evaluation trends and methodologies.
Strong understanding of SDLC/STLC, AI Lifecycle (AILC)/MLOps, and software testing methodologies.
In-depth understanding of LLM, RAG, AI Agent, fine-tuning, and prompt engineering mechanisms.
Strong knowledge of AI evaluation metrics, including Accuracy, Precision/Recall/F1, BLEU/ROUGE, Faithfulness, Answer Relevancy, Context Precision/Recall, Hallucination Rate, Toxicity, Bias, Latency, Cost, etc.
Knowledge of evaluation methodologies, including LLM-as-a-judge, Human Evaluation, A/B Testing, Regression Evaluation, and Red-teaming, as well as AI safety frameworks and standards such as OWASP Top 10 for LLM, NIST AI RMF, and ISO/IEC 42001.
Additional Preferred Qualifications:
Proficiency in Jira and Git.
Knowledge of statistics, data quality, API testing, and performance testing.
Benefits
Competitive salary package (including base salary, allowances, meal benefits, etc.).
13th-month salary bonus and performance-based bonuses.
Salary reviews twice a year.
Monthly lunch allowance.
Working schedule: Monday to Friday and every other Saturday, totaling 44 hours per week.
12 days of annual leave plus 1 birthday leave day.
Healthcare insurance.
Training and career development programs.
Opportunities for overseas business trips and assignments.
Opportunity to participate in electric vehicle development projects involving system-level and whole-vehicle integration.
International working environment with experts from various countries.
Clear career progression opportunities: Team Leader, Department Head, and Manager.
Vị trí AI Tester do công ty CÔNG TY TNHH VIETSOL tuyển dụng tại Hà Nội, Joboko tự động tổng hợp mức lương Negotiable, tìm thêm việc làm về AI Tester hoặc công ty CÔNG TY TNHH VIETSOL ở các link phía trên