ABOUT THE ROLE
Join our Technology Innovations Center of Excellence (CoE) at Crossian, a fast-growing startup that achieved a milestone of $100M in 2024. In this dynamic role, you'll check in daily with your team to stay aligned on project goals and collaborate across our ecosystem with an Agile mindset infused with a unicorn-worthy startup culture.
We are building an E-commerce Data Platform following Lakehouse Architecture, leveraging Airflow, Airbyte, dbt, and Redshift on AWS Cloud. Our system is currently in the early development stage, providing a unique opportunity for you to be involved from the ground up-shaping the architecture, defining data pipelines, and implementing best practices. Here are some examples: Storefront, Payment Gateway, Inventory, Catalog, Logistics, Marketing Insights, CRM.
WHAT YOU CAN EXPECT
We're not just about building platforms; we're about creating a workplace where talent thrives. Here's what you'll gain:
Exceptional Compensation
• Up to 30 months of salary per year through competitive salary packages and performance-based bonuses.
• Total annual compensation $40,000, reflecting your expertise and contribution.
Growth Opportunities
• Hands-on exposure to cutting-edge technologies and complex system architecture in a global-scale project.
• Clear career advancement pathways and access to continuous professional development programs.
Global Vision
• Contribute to projects that redefine how brands connect with consumers globally.
• Opportunity to work on challenging problems in data analytics, customer engagement, and operational efficiency.
In addition to development tasks, you'll collaborate closely with other CoEs to understand their unique challenges. With your technical expertise, you'll continuously propose innovative solutions to elevate their performance and contribute to the overall success of the organization.
WHAT YOU WILL DO
1. Design, develop, and maintain scalable data pipelines and ETL/ELT processes to collect, clean, and process large-scale datasets in data systems.
2. Collaborate with
Data Scientists,
Data Analysts, and other stakeholders to understand data requirements and develop Data Lake and Data Warehouse solutions for analytics, reporting, and AI/ML. Utilize BI tools like Grafana, AWS QuickSight, and Google Sheets for data visualization.
3. Build and optimize data architecture, including Lakehouse (AWS S3, Delta Lake), Data Warehouse (Redshift), and technologies like Kafka for both streaming and batch data processing.
4. Monitor and troubleshoot data pipeline issues to ensure data integrity and timely delivery. Utilize tools such as Sentry, Grafana, Prometheus, and AWS CloudWatch to track logs and send alerts via Slack.
5. Document technical processes, maintain data catalogs, and ensure compliance with data governance policies. Use the Atlassian toolset (Jira, Confluence) or Slack for work management and team collaboration.
6. Optimize the performance, reliability, and cost-efficiency of data systems. Enhance the scalability and efficiency of all components, including Kafka, dbt, Airflow, Airbyte, and storage solutions like PostgreSQL, Elasticsearch, Redis, and AWS S3, as well as data processing and orchestration frameworks.
7. Continuously research and implement new technologies to optimize the current system, applying best practices in Data Engineering and DataOps.
