Key Responsibilities
Support the design, development, and maintenance of data pipelines and ETL/ELT processes for analytics and AI/ML use cases.
Assist in building and optimizing data processing jobs using Spark (PySpark) on AWS Glue under guidance from senior engineers.
Help implement data transformation logic using dbt to produce clean and reliable datasets for analytics and machine learning.
Support development and maintenance of data workflows using AWS MWAA (Apache Airflow), including contributing to Airflow DAGs.
Write and optimize SQL queries for data extraction and basic transformation using Amazon Athena or similar services.
Assist in preparing data pipelines for AI/ML workflows such as training and inference.
Work with structured and unstructured datasets to support analytics and ML tasks.
Collaborate with
data analysts,
data scientists, and ML engineers to deliver data solutions.
Support integration with ML platforms (e.g., AWS SageMaker or similar) for model development and deployment.
Assist in monitoring and troubleshooting data pipelines to ensure reliability and data quality.
Document data pipelines, workflows, and technical processes.
1-3 years of experience in Data Engineering,
Software Engineering, or related fields.
Basic understanding of data engineering concepts, ETL/ELT pipelines, and data warehousing.
Hands-on experience with SQL (writing queries and basic optimization).
Familiarity with Python for data processing.
Exposure to Spark (PySpark preferred) is a strong advantage.
Familiarity with AWS services such as S3, Glue, Athena, MWAA, or IAM is a plus.
Basic understanding of Airflow or workflow orchestration concepts.
Understanding of ML fundamentals (data preparation, training, deployment basics) is a plus.
Experience working with datasets (structured or unstructured).
Good analytical and problem-solving skills.
Good communication skills and willingness to work in an international environment.
Preferred Qualifications (Nice to Have)
Exposure to AI/ML projects (data preprocessing, feature engineering, model training basics).
Familiarity with tools such as SageMaker, MLflow, Databricks, or similar platforms.
Awareness of Generative AI concepts (LLM, RAG, vector databases) is a plus.
Exposure to dbt or Airflow in academic/personal/work projects.
Basic understanding of Infrastructure as Code (Terraform or similar).
