Looking for strong programmers with 10 plus years of experience coming preferably from a product company or a start up
Responsibilities
Design, architect and deliver a scalable data management platform on cloud or on-premise that can handle large volumes of poly-structured data
Create data pipelines that can process different velocities of data through architectures such as lambda, kappa etc. to create different lanes of processing for streams through a single stack
Support AI/ ML teams with the right data wrangling processes
Manage lifecycle of data including separation of compute and storage
Design and develop Microservices based APIs for any data stage including collection, data processing, data storage and AI/ML
Help integrate the Microservices with a front end
Package the solution with the containerization strategy
Supporting an orchestration and distributed processing of the services through an Orchestrator
Desired Candidate Profile
Strong programming ability in Java/ J2EE including Java 1.9, Spring, Hibernate, Spring Boot
Developed Microservices using Spring Boot or other equivalent
Experience in 12 factor applications
Experience in service bus and Message Oriented Middleware with asynchronous systems such as Message Queues (ActivMQ etc.)
Experience in Big Data Stacks
File systems and resources such as Hadoop/ HDFS/ YARN or Snowflake or Parquet/ ORC
Asynchronous processing such as Kafka, MQ or AKKA frameworks
Spark (RDD, Dataframe, Datasets and SparkSQL, SparkML)
Workflow such as Airflow or Oozie
MPP databases (Mongo, Cassandra etc.)
Graph databases (Neo4J, Neptune etc.)
Worked on index everything such as Elasticsearch or other Inverted Index Search
Experience in rule engines or orchestrators such as IFTTT, Drools
Experience in containers such as Docker, OpenShift etc.
Experience in Kubernetes and orchestration
Preferred Skills
Understanding of ML algorithms and ML workflows
Experience in OLAP data modelling and BI/ Reporting