Location: Hanoi/HCM
We are looking for an experienced Data Engineer to design, build, and operate scalable data pipelines and platforms aligned with enterprise standards. You will translate business requirements into reliable, secure data solutions and own delivery end to end, from ingestion and transformation through serving and production support, for both operational and analytical use cases. Hands-on experience with Databricks or Microsoft Fabric is essential for this role.
1. RESPONSIBILITY
● Design and implement ETL/ELT pipelines using Python, SQL, and Spark/PySpark for scalable batch and streaming data processing.
● Build ingestion from databases, APIs, files, and event streams, using change data capture (CDC) and managing schema evolution.
● Develop Data Lake/Lakehouse layers (e.g., medallion architecture) and curated datasets on Databricks or Microsoft Fabric (OneLake), using Delta Lake and Parquet in line with enterprise data architecture.
● Implement dimensional models, including star/snowflake schemas and slowly changing dimensions, to support BI and analytics requirements.
● Orchestrate pipelines with Airflow, Databricks Workflows, or Fabric Data Factory pipelines, managing dependencies, scheduling, retries, and controlled backfills.
● Implement data quality checks, reconciliation, and remediation; maintain metadata, data lineage, and catalog documentation.
● Apply security and access controls, encryption, and secrets management in line with data classification and governance policies.
● Optimize SQL queries, Spark workloads, storage layouts, and compute resources to improve performance and control platform costs.
● Own production reliability against agreed SLAs through observability, alerting, incident resolution, and tested recovery procedures.
● Use Git, CI/CD, automated testing, and infrastructure as code (IaC) to deliver repeatable deployments and maintainable engineering standards.
● Collaborate with Data Architects, BI, and application teams to agree data contracts, resolve integration issues, and communicate delivery trade-offs.
2. REQUIREMENTS
● At least 4 years of experience in data engineering or a related field, delivering and supporting production data pipelines.
● Proven hands-on experience delivering production workloads on Databricks or Microsoft Fabric (e.g., notebooks, Delta tables, Lakehouse/Warehouse, Unity Catalog or OneLake, job/pipeline orchestration).
● Strong Python and SQL proficiency, with hands-on Spark/PySpark experience in distributed processing and performance tuning.
● Practical experience with ETL/ELT, batch and streaming, and CDC, using Kafka, Azure Event Hubs, Amazon Kinesis, or equivalent services.
● Hands-on experience with at least one major cloud (Azure, AWS, or GCP); exposure to additional platforms such as Snowflake, BigQuery, or Redshift is a plus.
● Strong understanding of Data Lake/Lakehouse patterns, Delta Lake, Parquet, dimensional modeling, and schema evolution.
● Experience with Airflow, Databricks Workflows, or Fabric Data Factory, pipeline observability, failure recovery, and meeting data freshness and availability SLAs.
● Practical experience with Git, CI/CD, automated unit, integration, and data tests, and IaC tools such as Terraform.
● Working knowledge of data quality, lineage, metadata/catalog tools, and security/access control; strong communication with business and technical stakeholders.
Nice-to-have
● Certifications in Databricks or Microsoft Fabric are a strong advantage, such as Databricks Certified Data Engineer Associate/Professional, Microsoft Certified: Fabric Data Engineer Associate (DP-700), or Fabric Analytics Engineer Associate (DP-600).
● Other relevant data engineering certifications in Snowflake, Airflow, GCP, AWS, or Azure.
● Experience with dbt, reusable transformation frameworks, or Data Vault implementations.
● Advanced streaming experience with event-time processing, checkpointing, and handling late or duplicate events.
● Familiarity with data mesh, domain ownership, and implementing governed data products.
● Background in a regulated industry (telecom, finance, healthcare, insurance) with strong compliance and data governance needs.
3. BENEFITS
At SmartOSC, we offer the best to your values
● Attractive salary package:
● Salary review twice a year
● Flexible working hour
● Premium health care
● Working in One of the largest digital transformation agencies - A professional English environment
● Free English, Japanese, and professional training packages
● Firm's Certified Qualifications Sponsorship for career development
● Annual company trip inside or outside Vietnam
● Other fun activities: happy hour, quarterly team building, football club, yoga club, swimming club, charity activities, etc.
● Free entertainment parties: Birthday party, Anniversary party, Sum-up Party, Year-End Party, etc.
4. TO APPLY
Please submit your CV to [protected info] with the following information:
● Job Application Letter with "Data Engineer" in the subject line.
● Updated CV in English, including your contact e-mail, phone number and salary expectations.
Thông tin chung