Roles and Responsibilities

As the Senior Data Engineer, you will build a world-class cloud-first data pipeline using AWS native services (S3, Redshift). You will focus on AWS native services and the development of a true serverless environment capable of delivering real-time and batch data to targets in a reliable and efficient manner. You will also provide hands-on development with tools such as AWS EMR, Hudi, Lambda, DynamoDB, EventBridge, DMS, Glue, Kinesis/Kafka, SQS, and SNS. Additionally, you will:

• Develop software using Python and Spark
• Learn different types of data integration (scheduled, near real-time, real-time)
• Collaborate with other Architects and Developers to build solutions that meet business needs
• Operate in Agile framework, creating user stories and tasks from customer requirements to track project progress
• Manage ingestion metadata as you develop the new integrations and maintain documentation for jobs that are developed
• Optimize data integration flow for increasing data volume and better latency

Desired Candidate Profile

• At least 5 years of IT experience in data integrations and data management
• At least 2 years of experience in AWS technologies
• 1 to 2 years of experience in design, architecture, and development, especially in the data integrations space
• Bachelor's degree in computer science, information technology, or another STEM field
• Experience as a data engineer
• Experience with AWS native services (S3, Redshift)
• Experience with AWS EMR, Hudi, Lambda, DynamoDB, EventBridge, DMS, Glue, Kinesis/Kafka, SQS, and SNS
• Experience with CDC-based enterprise data technologies
• Hands-on experience with streaming data architecture
• Hands-on proficiency in one or more programming languages
• Self-driven to learn new technology and build technical skill sets
• Excellent written and verbal communication skills, especially with Product Owners
• Experience with Agile project delivery frameworks and participating in Scrum ceremonies

Perks and Benefits