Lead Data Engineer

We are seeking a Lead Data Engineer (Python, Kubernetes, Snowflake) to own reliable data pipelines and production platforms across Cloud and on-prem. You will deploy and run data and ML workloads on OpenShift, improve CI/CD, troubleshoot complex issues and guide engineering teams as we scale an enterprise analytics and ML platform.
Apply today and be eligible for a one-month sign-on bonus.
Terms & Conditions apply.
Responsibilities
Design, build and maintain scalable data pipelines and platform components using Snowflake, Apache Spark (PySpark), SQL and Apache Airflow
Deploy, operate and support data and ML workloads on Kubernetes and OpenShift in production
Develop and maintain Python services and APIs using FastAPI or similar frameworks
Monitor, troubleshoot and optimize performance, reliability and data quality across pipelines and platforms
Build and improve continuous integration and continuous delivery (CI/CD) pipelines for safe, repeatable releases
Partner with DevOps, Platform, ML, infrastructure and application teams to deliver production-ready solutions
Drive root cause analysis for complex incidents and define preventative fixes and operational best practices
Provide technical leadership through architecture decisions, reviews and mentoring
Requirements
Proven experience in data engineering and enterprise-scale data platform delivery
Hands-on expertise with Snowflake, Apache Spark (PySpark), SQL and Apache Airflow
Strong Python development capability, including building services or APIs
Production experience operating workloads on Kubernetes or Red Hat OpenShift
Solid background in continuous integration and continuous delivery (CI/CD), Git workflows, containers and deployment automation
Experience supporting solutions across Microsoft Azure and hybrid cloud environments
Strength in monitoring, troubleshooting and production support for distributed systems
Track record of technical ownership, clear communication and mentoring within engineering teams
Nice to have
Experience with FastAPI, Databricks or Splunk
Exposure to MLOps practices, ML platform operations or LLM and GenAI enablement platforms
Infrastructure as Code experience (tooling aligned to your environment)