What you'll do
· Design, build, and maintain scalable batch and real-time data pipelines for market, fundamental, and alternative data sources.
· Develop data ingestion, cleaning, validation, enrichment, and versioning frameworks.
· Build streaming data pipelines using Kafka or equivalent technologies.
· Develop orchestration workflows with Airflow (or similar).
· Design and optimize distributed data processing jobs using Spark.
· Build data quality monitoring, anomaly detection, lineage tracking, and observability.
· Optimize storage and retrieval for large-scale time-series datasets.
· Collaborate with Quant Researchers,
AI Engineers, and Backend Engineers.
· Continuously improve pipeline performance and reliability.
Requirements
· Experience building production data platforms or large-scale data pipelines.
· Strong Python and SQL.
· Hands-on experience with Spark, Kafka, Airflow or equivalent.
· Understanding of ETL/ELT, distributed storage, and large-scale processing.
· Experience with time-series data and data versioning.
· Cloud/container familiarity.
· High ownership and attention to data quality.
Bonus points
· Financial market data experience.
· Delta Lake, Iceberg, or Hudi.
· ClickHouse, PostgreSQL, Redis.
· Kubernetes, Docker, AWS.
· Feature stores or ML pipelines.
Benefits
· Competitive salary + performance bonus.
· Ownership of institutional-grade data platform.
· Work with quant researchers and AI engineers.
· Clear growth path to Lead Data Engineer/Data Platform Architect.