Roles and Responsibilities

• Evaluating, developing, maintaining and testing data engineering solutions for Data Lake and advanced analytics projects.
• Implement processes and logic to extract, transform, and distribute data across one or more data stores from a wide variety of sources
• Distil business requirements and translate into technical solutions for data systems including data warehouses, cubes, marts, lakes, ETL integrations, BI tools or other components.
• Creation and support of data pipelines built on AWS technologies including Glue, Redshift, EMR, Kinesis and Athena
• Participate in deep architectural discussions to build confidence and ensure customer success when building new solutions and migrating existing data applications on the AWS platform.
• Optimize data integration platform to provide optimal performance under increasing data volumes
• Support the data architecture and data governance function to continually expand their capabilities
• Experience in development of Solution Architecture for Enterprise Data Lakes (applicable for AM/Manager level candidates)
• Should have exposure to client facing roles
• Strong communication, inter-personal and team management skills

Desired Candidate Profile

• Proficient in any object-oriented/ functional scripting languages: Java, Python, Node etc.
• Experience in using AWS SDKs for creating data pipelines ingestion, processing and orchestration.
• Hands on experience in working with big data on AWS environment including cleaning/transforming/cataloguing/mapping etc.
• Good understanding of AWS components, storage (S3) & compute services (EC2)
• Hands on experience in AWS managed services (Redshift, Lambda, Athena) and ETL (Glue).
• Experience in migrating data from on-premise sources (e.g. Oracle, API-based, data extracts) into AWS storage (S3)
• Experience in setup of data warehouse using Amazon Redshift, creating Redshift clusters and perform data analysis queries
• Experience in ETL and data modelling on AWS ecosystem components - AWS Glue, Redshift, DynamoDB

Experience in setting up AWS Glue to prepare data for analysis through automated ETL processes

• Familiarity with AWS data migration tools such as AWS DMS, Amazon EMR, and AWS Data Pipeline
• Hands on experience with AWS CLI, Linux tools and shell scripts
• Certifications on AWS will be an added plus.

Perks and Benefits