Senior Data Engineer

UPA Vietnam

Địa điểm làm việc: Hồ Chí Minh

Hết hạn: Còn 16 ngày

Chi tiết công việc
Giới thiệu công ty

Thu nhập: Thương lượng

Loại hình: Toàn thời gian

Chức vụ: Nhân viên

Kinh nghiệm: 5 năm

phần mềm aws architecture design

Mô tả công việc

Key Responsibilities:
1. Data Architecture and Design
• Design and implement scalable, secure, and efficient data pipelines and architectures on
AWS
• Define and manage the architecture for real-time and batch data processing.
2. Data Pipeline Development
• Build ETL/ELT processes to ingest, transform, and store structured and unstructured
data from diverse sources.
• Develop solutions for data integration, data quality, and data governance.
3. Optimization and Monitoring
• Optimize data processing workflows for performance and cost-effectiveness.
• Monitor data pipelines and systems, troubleshooting and resolving issues promptly.
4. Collaboration
• Partner with data scientists, analysts, and business teams to understand data needs and
deliver high-quality solutions.
• Collaborate with DevOps teams to implement CI/CD for data workflows.
5. Security and Compliance
• Ensure data solutions comply with organizational policies and industry security, privacy,
and governance standards.
• Implement data anonymization and encryption techniques as needed.
6. Innovation
• Stay updated with emerging AWS technologies and data engineering best practices.
• Drive continuous improvement in processes, tools, and methodologies.

Yêu cầu

Required Qualifications
1. Education
• Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or a related field.
2. Experience
• 5+ years of experience as a Data Engineer or in a similar role, focusing on AWS cloud solutions.
• Proven experience designing and building data architectures and pipelines.
3. Technical Skills
• Expertise in AWS services, including but not limited to S3, EMR, Lambda, and RDS.
• Proficiency in SQL, Python, and Java for data processing and manipulation.
• Experience with data tool stacks such as: Spark, Flink, Airflow.
• Strong knowledge of data modeling, data warehousing, and data lakes.
• Experience with data lake formats such as Delta or Iceberg.
• Familiarity with CI/CD tools and infrastructure as code (e.g., Terraform).
• Experience with data governance, security, and compliance standards.
• Containerization experience with Docker and Kubernetes (K8s).
• Real-time data integration using Debezium and Kafka.
• Experience with Linux and Bash commands.
• Experience with query engines like Trino and Presto.
4. Soft Skills
• Strong problem-solving and analytical skills.
• Excellent communication and collaboration abilities.
• Ability to work in an agile and fast-paced environment.
• Ability to communicate efficiently in English.
Nice to have
• Knowledge of machine learning workflows and integration with data pipelines.
• Knowledge of monitoring tools like CloudWatch, Datadog, or Prometheus.
• Knowledge of modern data lakehouse platforms like Apache Doris and Pinot.
• Knowledge and experience in Banking / Finance/Securities/ Fintech domains