Senior Data Engineer SQL, Etl, Python

Senior Data Engineer
Up to 2*** USD
Login to view salary
Your role & responsibilities:
Possess an understanding of the data structures and governance.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources.
Design and maintain optimal data and machine learning pipeline architecture.
Assemble large, complex data sets to build data mart that meet functional/non-functional business requirements.
Communicate effectively with Developers and Business users to understand and clarify requirements.
Support internal training and proper documentation ensuring the successful onboarding of new team members
Implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Develop efficient software code for multiple use cases leveraging Spark and Big Data Technologies for various use cases built on the platform.
Your skills & qualifications:
Bachelor's degree in computer science, engineering, mathematics, or a related technical discipline.
3+ years' experience working as a Data Engineer role, ETL with large amounts of data.
Experience with data warehousing architecture and data modeling, ETL data pipeline.
Experience with using the following software/tool in the big data field.
Big data tools: Apache Spark, Presto, Kafka,...
NoSQL and OLAP database: MongoDB, Elastic search, etc.
Data pipeline and workflow management tools: Airflow.
Good at programming languages: Scala, Python, Java.
Experience with SRE, Patching & Automation: Kubernetes or Docker & Containerization
Experience in the data testing process.
Experience in performing root cause analysis, identify problems and propose recommendations for improvement.
Strong organizational and multitasking skills with the ability to balance competing priorities
Fundamental knowledge of modern cloud computing platforms and concepts is a plus (AWS/GCP)