Python AI Engineer

Home - Careers - Python AI Engineer
Python AI Engineer
Home - Careers - Python AI Engineer
Python AI Engineer
Learn more about us
LOCATION: Hanoi, HCMC
Job Description
We are looking for a Python AI Engineer to join us in building and advancing our AI Indexing & Search Pipeline. In this role, you will be responsible for designing and implementing end-to-end pipelines for data crawling, content processing, vector embedding generation, and indexing to power high-performance semantic search capabilities.
Pipeline Development & Core Architecture:
Develop, optimize, and maintain the Python-based Batch AI Indexer system.
Build and operate an end-to-end data pipeline
Build and optimize Semantic Search / Vector Search capabilities.
Data Crawling & Processing:
Crawl and collect content from diverse web sources (Homepage, Naver Blog, and other relevant web data sources).
Optimize text chunking strategies and data preprocessing tailored for language models.
AI Models & Search Engine Integration:
Utilize OpenAI Embeddings for content vectorization.
Integrate and manage Elasticsearch 8.x for vector storage and search (working extensively with 1536-dimensional dense_vector and the Korean Nori Analyzer).
Utilize FAISS for local vector search experiments and rapid prototyping.
Batch, State & Reliability Management:
Design and implement mechanisms for Incremental Indexing and Full Re-indexing.
Manage incremental batch states using Redis.
Track batch execution history and metadata using PostgreSQL.
Implement robust retry mechanisms and error-handling logic to ensure high system reliability.
Scheduling, Deployment & Operations:
Configure automated job scheduling using APScheduler or Linux Crontab.
Deploy and operate batch applications on Linux environments using systemd and Python virtual environments (venv).
Perform logging, monitoring, and troubleshooting for batch job operations.
Collaborate closely with Backend, AI, and DevOps team members to deploy and scale system operations.
Required Skills And Experience
Core Experience & Technical Skills:
2-4+ years of strong hands-on development experience in Python.
Proficiency in web scraping and crawling libraries (e.g., BeautifulSoup, Scrapy, Playwright, or Selenium).
Practical experience with Elasticsearch 8.x (specifically Vector Search, dense_vector, kNN search, and custom analyzer configurations like Nori).
Deep understanding and hands-on experience with Embedding models (e.g., OpenAI Embeddings, HuggingFace) and text chunking/preprocessing techniques.
Databases & System Architecture:
Solid skills in Redis (for caching/state management) and PostgreSQL (or equivalent relational databases).
Strong knowledge of Batch Processing architectures, state management, idempotency, and error handling within data pipelines.
Experience with FAISS or other vector search libraries for local experimentation.
DevOps & Environment:
Proficient in Linux environments, application deployment via systemd, and Python virtual environments.
Experience with scheduling tools (crontab, APScheduler).
Strong command of Git, logging frameworks, and system troubleshooting.
Nice-to-Have Qualifications:
Experience with Korean Natural Language Processing (Korean NLP).
Experience with LLM orchestration frameworks such as LangChain or LlamaIndex.
Hands-on experience with Docker, Kubernetes, or CI/CD pipelines.
Benefits
Competitive and negotiable salary (UP TO 50M)
Probation salary is 100% of official salary
13th-month salary and performance review twice a year
Bonus for special occasions each year (Labor Day, National Day, Solar New year, Lunar New Year)
Project bonus
Employee's professional certification and training allowances subject to company regulations
Opportunity to participate in internal training programs on AI, foreign languages, and professional development
Health Care Insurance
Annual Health Assessment
Social, health and unemployment insurance following government policy
Enjoy company summer trips and other team building activities held monthly and quarterly
Professional, creative and dynamic working environment
Work five days per week with flexible check-in time
CONTACT
Send CV to: [protected info]
Hotline: +[protected info] (Recruitment Team)
Facebook GEM Careers: [protected info]
Linkedin GEM Careers: [protected info]