Roles and Responsibilities

• Gather and process raw data at scale.
• Design and develop data applications using selected tools and frameworks as required and requested.
• Read, extract, transform, stage and load data to selected tools and frameworks as required and requested.
• Perform tasks such as writing scripts, web scraping, calling APIs, write SQL queries, etc.
• Work closely with the engineering team and data science team to integrate your work into production systems.
• Process unstructured data into a form suitable for analysis.
• Analyse processed data.
• Monitoring data performance and modifying infrastructure as needed.

Desired Candidate Profile

• 3 7 years of recent experience in data engineering.
• Bachelors Degree or more in Computer Science or a related field. (BE, BTech, BCA, MCA, BSc(IT), MSc(IT) or any degree related to IT)
• A solid track record of data management showing your flawless execution and attention to detail.
• Programming experience, ideally in Python, Scala.
• Technology experience, ideally Spark, Spark Streaming, Kafka and a willingness to learn new technologies to meet goals and objectives.
• Database experience, ideally MongoDB, Cassandra, Postgres.
• Experience on AWS Cloud.
• Knowledge of data cleaning, wrangling, visualization and reporting, with an understanding of the best, most efficient use of associated tools and applications to complete these tasks.
• Experience in MapReduce is a plus.
• Experience on Data Lake or CDP is a plus.
• Experience processing large amounts of structured and unstructured data, including integrating data from multiple sources.
• A willingness to explore new alternatives or options to solve data mining issues, and utilize a combination of industry best practices, data innovations and your experience to get the job done.
• Experience in production support and troubleshooting.