Data Platform Engineer to design, build, maintain, and optimize scalable data platforms that support analytics, reporting, machine learning, and business intelligence initiatives. The ideal candidate will have strong expertise in data engineering, cloud platforms, database technologies, and data architecture, ensuring reliable and secure data solutions across the organization.
Key Responsibilities
Data Platform Development
- Design, implement, and maintain scalable data platforms and data infrastructure.
- Build and optimize data pipelines for batch and real-time data processing.
- Develop and manage data ingestion frameworks from multiple internal and external sources.
- Ensure data platforms support business analytics, data science, and operational reporting requirements.
Data Architecture & Engineering
- Define data models, storage strategies, and architectural standards.
- Implement and manage data lakes, data warehouses, and lakehouse architectures.
- Optimize database performance, data partitioning, indexing, and query execution.
- Ensure high availability, scalability, and reliability of data systems.
Cloud & Infrastructure Management
- Deploy and manage data services on cloud platforms such as Azure, AWS, or Google Cloud.
- Automate infrastructure provisioning and deployment using Infrastructure as Code (IaC).
- Monitor platform performance, capacity, and operational health.
- Implement CI/CD pipelines for data platform deployments.
Data Governance & Security
- Ensure compliance with data governance, privacy, and security standards.
- Implement data quality monitoring and validation processes.
- Manage access controls, encryption, auditing, and data retention policies.
- Support regulatory compliance requirements.
Collaboration & Support
- Collaborate with Data Engineers, Data Scientists, Analysts, and Application Development teams.
- Provide technical guidance on data platform best practices.
- Troubleshoot platform issues and support production environments.
- Document architecture, processes, and operational procedures.
Required Qualifications
- Bachelor's degree in Computer Science, Information Technology, Data Engineering, or related field.
- 3+ years of experience in data engineering, platform engineering, or related roles.
- Experience designing and supporting enterprise-scale data platforms.
- Hands-on experience with cloud data services and modern data architectures.
Technical Skills
Data Technologies
- SQL and database management (SQL Server, PostgreSQL, MySQL, Oracle, etc.)
- Data Warehousing concepts and ETL/ELT processes
- Data Lake and Lakehouse architectures
Programming
- Python, Scala, Java, or Spark
- SQL scripting and optimization
Cloud Platforms
- Microsoft Azure (Azure Data Factory, Synapse Analytics, Databricks, Fabric)
- AWS (Redshift, Glue, EMR, S3)
- Google Cloud Platform (BigQuery, Dataflow)
Big Data & Streaming
- Apache Spark
- Kafka
- Hadoop ecosystem (preferred)
DevOps & Automation
- Git
- CI/CD tools (Azure DevOps, GitHub Actions, Jenkins)
- Terraform, ARM, or other IaC tools
- Docker and Kubernetes (preferred)
Preferred Qualifications
- Experience with Microsoft Fabric, Databricks, Snowflake, or similar modern data platforms.
- Knowledge of DataOps and MLOps practices.
- Experience implementing data governance frameworks.
- Relevant certifications in Azure, AWS, GCP, Databricks, or Snowflake.