Big Data Senior Developer

Big Data Senior Developer

JOB DESCRIPTION

Understand non-functional requirements and the ability to turn them into Functional/Technical requirements. Then apply the architectural principles to leverage the Spark Framework to reproduce specific use cases and/or features inputs for models.

Passion towards automation where appropriate and beneficial

Understand Data acquisition, Develop data set processes, and knowledge of data concepts

Understand the complex transformation logic and translate them to Spark code or Spark-SQL queries

Ability to write reusable code in Scala/Java in Apache Spark

Hadoop ecosystem (Spark, Hive, HBase, YARN, and Kafka), Spark Core, Spark-SQL, and live streaming datasets using Kafka/Spark Streaming

Unix Shell Scripting and knowledge of TWS scheduling

Cloudera distribution framework, Jenkins (or any version controller)

Excellent understanding of technology life cycles and the concepts and practices required to build big data solutions

Ability to understand and build re-usable data assets

Unix Shell Scripting and knowledge of TWS scheduling and Experience in Unix environments with batch scripting and commands

Good to have experience in Distributed storage and computing technologies.