REMOTE AI DATA LABELLER / ANNOTATOR & EVALUATOR

Company Description
TaloTrace is an AI-powered QA platform that autonomously tests web and mobile applications, discovers issues, and delivers evidence-backed findings your engineering team can act on immediately.
Unlike traditional automation tools that stop at pass or fail, TaloTrace provides a complete Trace for every issue: recorded video, logs, reproduction steps, severity, and supporting evidence.
We are looking for a Full Stack Engineer to build the product screens our customers use every day, and the standard web features around them.
The role is particularly suited to an engineer who ships quickly and improves in small steps, and gets a lot out of AI coding tools while keeping the code they produce correct and reviewable.
About the Opportunity
We are looking for careful, analytical and reliable people to help evaluate AI systems and build high-quality datasets. Most of the work involves watching recordings of an AI agent testing websites and apps, and judging whether it did the right thing.
This is a flexible way to earn extra money on the side. We welcome students, people looking for ad hoc or part-time work, freelancers, working professionals, recent graduates, career changers and anyone who wants to earn additional income around existing commitments. You do not need an AI or computer science background for general tasks; careful reading, sound judgment and attention to detail are more important.
Some projects use English and others use Vietnamese. You can apply based on the language or languages you can work in; you do not need to be fluent in both for every project. Specific projects may have additional language or subject requirements. Each project states the languages it needs, both for reading the material and for writing your reasons.
What You Will Do
Software test run review
Watch recordings of an AI agent testing a website or mobile app, step by step.
Compare what happened on screen with the expected result for each step, and mark each step as passed, failed or not reached.
Decide whether the overall test achieved its goal.
When a test fails, choose the most likely cause, such as a mistake by the agent, a problem with the app, or bad test data, and explain what you saw.
Flag test cases that are unclear, impossible to complete, or duplicated.
Bug report review
Review bugs reported by an AI system and decide whether each one is a genuine defect.
Check the bug's severity against the project's severity guidelines from critical to minor.
Match reported bugs to a list of known bugs, and note any real bugs that were missed.
Point to the exact moment in a video where a bug appears.
Test scenario and test data review
Compare original test cases with the versions imported into our system, and check that nothing was lost or changed in meaning.
Check that each test case has the accounts, data and starting conditions it needs to run.
Review conversations in which the system asked for missing information, and judge whether it asked the right questions.
Product knowledge review
Compare business rules and user flows extracted by the system against the original documents or screens.
Mark each rule as correct, partly correct or wrong, and add any important rules that were missed.
Clear and detailed feedback
Apply the evaluation rubric consistently rather than relying only on personal preference.
Explain errors precisely enough that another reviewer can understand what went wrong and why.
Check your work before submitting and maintain accuracy across repeated tasks.
Who Can Apply?
You must be based in Vietnam.
We welcome applicants from a wide range of educational and professional backgrounds. You may be a good fit if you:
Can read and follow instructions carefully.
Have strong attention to detail and can identify errors or inconsistencies.
Can make objective decisions based on stated criteria.
Can explain your reasoning clearly in English or Vietnamese, depending on the project.
Can work independently, manage your time and meet task deadlines.
Have a computer and reliable internet access.
Experience in software testing or quality assurance, or simply comfort using websites and mobile apps and noticing when something does not work as expected, is a strong plus.
Application and Qualification Process
Submit your CV and provide your contact details, language capabilities and relevant skills or experience.
CV check: we confirm that you are based in Vietnam. Eligible applicants receive a private link to our labelling portal.
Complete the Level 0 test on the portal. This is the qualification test and the single assessment round used to determine whether you can proceed for that specific benchmark queue.
Only applicants who pass Level 0 in the portal will be allowed to participate in production tasks for that queue. Receiving the link or submitting the test does not itself qualify you.
The Level 0 test generally contains 5-10 questions selected from a larger question bank. All questions must be answered correctly, with up to two attempts permitted. If you do not pass within the permitted attempts, access to that queue will be closed and you will normally need to wait six months before reattempting that same queue. You may still apply for other benchmark queues, subject to their own requirements.
Before starting paid work, you will need to:
Upload both sides of your ID card.
Provide a Vietnamese bank account for payment.
E-sign a contractor agreement, including responsibility for your own personal income tax.
Accuracy and Progression
Each benchmark has its own qualification and performance requirements, with up to five levels. Your progression depends on sustained accuracy and consistency, not simply the number of tasks completed. Higher levels may offer higher payouts per task.
Your work may be checked against reference answers and compared with independent evaluations from other annotators.
Performance reviews begin after the required initial sample of validated reference items has been completed.
The general accuracy target is 95% or higher, subject to the specific benchmark rules shown on the platform.
If accuracy falls below the target, you may receive a warning and a limited opportunity to improve. The applicable recovery tolerance and consequences are governed by the queue rules.
Persistent underperformance may result in a lower level or closure of that queue. If a queue is closed after unsuccessful performance recovery at Level 1, a six-month waiting period may apply.
Experienced, high-performing annotators may be invited to review disagreements when they meet the relevant accuracy and experience requirements.
Using AI Tools
You may use AI tools to help you work. Every answer you submit must be your own judgment, though. You must review and confirm each answer, and you are accountable for its accuracy.
Flexible Work and Task Availability
Work is released in batches. New batches are announced in our Zalo group, which you can join from the portal.
This is remote, task-based work that can fit around study, another job or other commitments. You can work from anywhere in Vietnam. Some projects may have minimum participation expectations, deadlines or estimated completion times; these will be communicated before you take on the relevant work.
Task volume is limited and varies with project demand, dataset availability and the number of qualified contributors. Tasks are offered on a first-come, first-served basis while available. There are no guaranteed hours, task volumes or earnings, so this opportunity is best treated as supplemental income rather than a fixed salary.
Compensation and Weekly Payments
You are paid according to the tasks you complete and the applicable rate for the relevant benchmark and approved performance level. Rates may differ between projects and levels. Your earnings depend on which queues you qualify for, how many tasks are available and completed, and whether your submissions satisfy the applicable quality requirements.
Payout rates will be displayed or communicated before you undertake the relevant tasks. Rate changes apply prospectively and do not automatically change the rate applicable to tasks completed previously.
Payments are made every Wednesday for work completed in the previous Monday-to-Sunday period. You will be able to view relevant task, performance and payout information through the platform.
Future Opportunities
As new projects and task queues become available, we may contact eligible applicants using the contact details they have provided. Applicants who pass the CV check may be included in our contact list for future opportunities. Joining the contact list does not guarantee work or automatic admission to a new queue; additional qualifications or tests may be required.