Mô tả công việc
Mô tả Công việc
· Lead the design and implementation of the Data Lakehouse architecture on Azure, making critical technology choices across Azure Data Lake Storage Gen2, Delta Lake, Azure Synapse, or Azure Databricks.
· Design, develop, and optimize high-throughput data processing pipelines (ETL/ELT) for a diverse range of sources (on-premise, cloud, streaming, APIs, files, etc.), ensuring resilience and fault tolerance.
· Act as a key technical liaison with AI, BI, and DevOps teams, driving the integration of data into complex analytics applications, production-grade machine learning models, and intelligent reporting systems.
· Pioneer and implement advanced optimization strategies for large-scale data storage and query systems (big data), focusing on maximizing performance, scalability, and cost-effectiveness.
· Establish and enforce best practices for data governance, including comprehensive data quality frameworks, centralized metadata management, and automated data lineage processes.
· Collaborate with infrastructure and security teams to architect and operate secure, compliant data systems, ensuring adherence to security protocols, internal regulations, and external standards.
· Mentor junior engineers, review code/design specifications, and produce high-quality technical documentation that enables knowledge transfer and efficient operations across product and analytics teams.
Yêu cầu
Yêu Cầu Công Việc
Education & Experience
· Bachelor's degree or Master in Information Technology, Data Science, Information Systems, or equivalent.
· Minimum of 5 years of hands-on experience as a Data Engineer, with a proven track record of delivering end-to-end data solutions.
· Deep, demonstrable expertise in implementing and operating data infrastructure on Azure Cloud, with high proficiency in:
o Azure Data Lake Storage Gen2
o Architecture and implementation using Azure Synapse Analytics, Azure Data Factory, or Azure Databricks.
o Real-time/Streaming technologies like Azure Event Hub, Azure Stream Analytics, or Apache Kafka.
· Expert proficiency in SQL and a programming language such as Python or PySpark, along with a strong theoretical and practical understanding of distributed computing principles.
· Extensive experience designing and working with advanced data models (e.g., Star Schema, Snowflake Schema, Mastery of Lakehouse Architecture and Delta Lake best practices).
· Required experience in designing and managing large-scale production data systems, implementing robust real-time pipelines, and integrating data directly into AI/ML model serving layers.
Soft Skills
· Ability to work independently, proactively, and with a systematic mindset.
· Strong collaboration skills with other technical teams (AI, DevOps, Product, ...).
· Logical thinking, strong analytical, and problem-solving skills.
· Openness to learning new technologies, adapting to change, and experimenting
Quyền lợi
Laptop
Chế độ bảo hiểm
Du Lịch
Phụ cấp
Chế độ thưởng
Chăm sóc sức khỏe
Đào tạo
Tăng lương
Phụ cấp thâm niên
Nghỉ phép năm
Thông tin chung