Hands-on expertise in extracting and processing large volumes of data using Big Data technologies like Hadoop, Spark, Hive etc.
Strong coding skills in Python/Java/Scala with Spark is a MUST
Handle the installation, configuration, and supporting of Hadoop.
Write MapReduce coding for Hadoop clusters; help to build new Hadoop clusters.
Converting hard and complex techniques as well as functional requirements into the detailed designs.
Proficient in writing Spark RDD/DataFrames/SQL to power data for extraction, transformation, and aggregation from multiple file formats including JSON, CSV & other compressed file formats.
Pre-processing of data using Pig ,Hive, Spark Streaming.
Must be good in writing complex SQL queries and aggregations.
Strong understanding of OLAP/data warehousing concepts , dimensional models like star schemas, snowflake schemas.
Should be able to create automated data pipelines for both batch and real-time data stream processing.
Strong hands-on working with streaming data using Flume, Kafka and other related big data tools.
Should have hands-on expertise on designing and developing reliable and robust ETL pipelines.
Preferred Skills:
Scheduling and orchestration of data work flows using Airflow etc.
Preferably should have worked on implementing CI/CD processes for deployments.
Should have worked on NoSQL Databases like Mongo, HBase etc.
Good to have skills:
Should have worked on at least one BI tool like Power BI, Tableau etc.
Hadoop developer certification is a huge PLUS.
Working knowledge of AWS technologies like Redshift, Kinesis, Lambda, RDS , S3, Glue, Athena , Dynamo DB would be an added advantage
If you are interested for the above position, pls share your updated resume with below details: