Job Overview

We are looking for a Data Engineer to join our growing team of Data Engineers. The hire will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow. The Data Engineer will work on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects.

Responsibilities for Data Engineer:

• Create and maintain optimal data pipeline Patterns/Architecture.
• Assemble large, complex data sets that meet functional /non-functional business requirements.
• Under the supervision of a designated lead- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the Data pipelines that extract, transform, and load data from a wide variety of data sources using Azure technologies.
• Under the supervision of a designated lead- Design and implement dimensional data models.
• Under the supervision of a designated lead- Create automated Data Quality tests to continuously monitor data quality of the data models.
• Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs.
• Keep our data separated and secure across Azure regions.

Qualifications for Data Engineer:

• Good working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
• Should have appetite to learn new technologies and identify opportunities for improvement.
• Strong analytic skills to perform root cause analysis on internal and external data and processes to answer specific business questions
• Should be able to build processes supporting data transformation, data structures, metadata, dependency and workload management.
• Should be detail oriented and possess good organizational skills.
• Should be able to work with a cross-functional team in a dynamic environment.
• We are looking for a candidate with 3 years of experience in a Data Engineer/ Data Analyst role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools:
• Experience with relational SQL and NoSQL databases.
• Experience with object-oriented/object function scripting languages: Python, C#
• Knowledge of data pipeline architecture is preferred but optional.
• Experience with Azure cloud services : Azure Data Factory, Azure SQL db, Azure Synapse Analytics, Azure Data Lake Storage Gen2, Data Bricks, PySpark