Data Engineer

1. Objective:
- The job holder is responsible for designing and developing programs, algorithms and automated processes to cleanse, integrate and evaluate large datasets from disparate sources and implement complex business logic as needed with the available data processing tools.

• Data Engineer
  • The job holder will be responsible for helping build a reliable, sustainable and scalable data processing platform, working with other tribe members to support data pipelines, convert models to machine learning codes and recommend Big Data reporting and visualization applications.
• Senior Data Engineer
  • The job holder will be responsible for integrating new data sources to increase throughput of existing systems, managing data pipelines that facilitate robust analysis, and sourcing & preparing data to ensure data completeness on metadata platforms.

2. Key accountabilities:

• Data Engineer
  • A. Data Architecture
    • Deliver functionality required for business and data analysts, data scientists and other business roles to advance the overall analytic performance and strategy of the bank
    • Assist in building best practices and strategies for data infrastructure to fulfill data analytic and utilization needs of the business with emerging latest technologies and capabilities.
    • Collaborate with relevant teams on opportunities to manage data and provide solutions for complex data feeds within the bank.
    • Maintain data integration processes into the data platform by building data pipelines and align with business representatives and business information needs.
    • Review internal and external business and product requirements for data operations and activity and suggests changes and upgrades to systems and storage to accommodate ongoing needs.
  • B. Data Integration
    • Obtain and integrate data and information from various sources into the firm's platforms, solutions and statistical models.
    • Collaborate with Data Scientists to understand data requirements and create re-usable data assets to enable data scientists to build and deploy machine learning models faster.
    • Write scripts and ingest raw data to build and maintain optimized data pipelines and ETL solutions as business support tools in providing analysis.
    • Ensure data assets are organized and stored in an efficient way so that information is easy to access and retrieve.

• A. Data Architecture
  • Deliver functionality required for business and data analysts, data scientists and other business roles to advance the overall analytic performance and strategy of the bank
  • Build the best practices and strategies for data infrastructure to fulfill data analytic and utilization needs of the business with emerging latest technologies and capabilities.
  • Proactively drive the effort of identifying opportunities to manage data and provide solutions for complex data feeds within the bank.
  • Evaluate various data architectures in the bank and utilize them to develop data solutions to meet business requirements.
  • Drive the delivery of data products and services into systems and business processes in compliance with internal regulatory requirements.
  • Oversee the review of internal and external business and product requirements for data operations and activity and suggests changes and upgrades to systems and storage to accommodate ongoing needs.
• B. Data Integration
  • Strategically obtain and integrate data and information from various sources into the firm's platforms, solutions and statistical models.
  • Lead discussion with Data Scientists to understand the data requirements and create re-usable data assets to enable data scientists to build and deploy machine learning models faster.
  • Design, build, and maintain optimized data pipelines and ETL solutions as business support tools in providing analysis and real time analytics platform for critical decision making.
  • Ensure data assets are organized and stored in an efficient way so that information is high quality, reliable, flexible, and efficient.
• C. Project Management
  • Manage project conflicts, challenges and dynamic business requirements to keep operations running at high performance.
  • Work with team leads to resolve people problems and project roadblocks, conduct post mortem and root cause analysis to help squads continuously improve their practices to ensure maximum productivity.
• D. Talent Development
  • Mentor and coach junior fellows into fully competent Data Engineers.
  • Identify and encourage areas for growth and improvement within the team.