Senior Data Engineer

Mô tả công việc
Tóm tắt công việc
We are seeking a Senior Data Engineer to play a pivotal role in building our data platform from the ground up. You will be responsible for designing, developing, and deploying a scalable, secure, and open-source based data platform on the AWS cloud. This is a unique opportunity to shape the future of our data infrastructure and empower data-driven decision making across the organization.
Design and implement a robust data platform architecture on AWS, leveraging open-source technologies whenever possible.
Evaluate and select appropriate open-source tools and services for data ingestion, transformation, storage, and analytics.
Develop and maintain data pipelines for efficient movement and processing of large datasets.
Ensure data quality and consistency throughout the data lifecycle.
Implement data security best practices to protect sensitive information.
Collaborate with data scientists, analysts, and other stakeholders to understand data needs and translate them into technical requirements.
Automate data platform tasks and processes for improved efficiency and reliability.
Monitor and troubleshoot data platform performance to identify and resolve issues proactively.
Stay up-to-date on the latest trends and technologies in the data engineering space.
Document data platform design, architecture, and processes for future reference.