Novotree Minds Consulting Pvt Limited

Senior Big Data Engineer

  • Job ID: PMW1T83K4J
  • Job Type: Full Time
  • Industry Type: IT Sector
  • Industry Location: chennai
  • Experience: 6-11yrs
  • No. of Positions: 5
  • Salary Range: 15-25 lac
  • Primary Skills: Big Data Spark ETL SAS Scala PySpark Hadoop Kafka Hive
  • Secondary Skills: Ab initio Data Pipeline CI/Cd MongoDB Hbase Framework Skills highlighted with ‘‘ are preferred keyskills
  • Job Location: chennai
  • Posted Date: 16 days ago
Job Description

Roles and Responsibilities

 

  • Develop Big Data applications using Spark (Scala-Spark or PySpark), Hadoop, Hive and/or Kafka, HBase, MongoDB
  • Prior exposure to ETL toolsets such as Ab Initio, SAS or Informatica
  • Understanding of Machine Learning models
  • Working knowledge on Public Cloud / Private Cloud platforms
  • At least 3 years of development experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Scala-Spark or PySpark
  • At least 3 years of development experience in the following Big Data frameworks: File Format (Parquet, AVRO, ORC), Resource Management, Distributed Processing and RDBMS
  • At least 4 years of developing applications in Agile with Monitoring, Build Tools, Version Control, Unit Test, TDD, CI/CD, Change Management to support DevOps
  • At least 6 years of development experience on Data technologies

Desired Candidate Profile

 

  • Experience in Spark (Scala-Spark or PySpark) developing Big Data applications on Hadoop, Hive and/or Kafka, HBase, MongoDB
  • Deep knowledge of Spark and Scala / Python libraries to develop and debug complex data engineering challenges
  • Prior exposure to ETL toolsets such as Ab Initio, SAS or Informatica and migration experience
  • Experience in developing sustainable data driven solutions with current new generation data technologies to drive our business and technology strategies
  • Exposure in deploying on Cloud platforms

 

Perks and Benefits



Relevant Skill Jobs
Scouting For AWS ETL Developer
Cloud Engineer Azure
Sr Backend Developer Bangalore
Experienced Data Scientist
Hadoop Developer WP 1053
Senior Software Engineer Backend Delhi