Role:- Azure ML-Big Data Engineer Location:- Plano/Dallas, TX Duration : Long Term |
- Competent ML engineer, who is independent, results driven and is capable of taking business requirements and building out the technologies to take it to production.
|
- Big Data ML Engineer with expert level experience in Hadoop ecosystem and real-time analytics tools including PySpark/Scala-Spark/Hive/Hadoop CLI/MapReduce/ Storm/Kafka/Lambda Architecture/Mongo
|
- expert with using the larger Hadoop eco system
|
- Familiar with job scheduling challenges in Hadoop
|
- Experienced in creating and submitting Spark jobs
|
- Experience in high performance tuning and scalability
|
- Experience in working on real time stream processing technologies like Spark structured streaming, Kafka, FLink + rocksDB
|
- Expertise in Python/Spark and their related libraries and frameworks
|
- Experienced with Spring Framework and Spring Boot
|
- Experience in building training ML pipeline and efforts involved in ML Model deployment
|
- Experience in other ML concepts Real time distributed model inferencing pipeline, Champion/Challenger framework, A/B Testing, Model performance Scorecard and assessment, Retraining framework, etc.
|
- Unix/Linux expertise; comfortable with Linux operating system and Shell Scripting
|
- Working Experience in Redis DB Writing to and from redis-- very critcal Also exp Azure cache
|
- Experience in creating Kubernetes open-source container-orchestration system for automating application deployment, scaling, and management; (Experience in Azure is preferable)
|
- Experience in tweaking/using Jenkins, deployment orchestration and/or Kubernetes for CI/CD pipeline
|
- Creating/modifying Dockers, microservices and deploying them via Kubernetes
|
- PL/SQL, RDBMS background with Oracle/MySQL
|
- Familiarity with Cassandra, Mongo preferred.
|
- Design, Development, Unit and Integration testing of complex data pipelines and to handle data volumes to derive insights
|
- Ability to optimize code to be able to run efficiently with stipulated SLA
|
- Excellent problem-solving skills, with attention to detail, focus on quality and timely delivery of assigned tasks.
|