Big Data Engineer

On behalf of our client, a burgeoning software development company that specializes in Big Data Analytics, we are currently seeking an independent, resourceful Big Data Engineer to propel its software development team to new heights. A day in the life of a Big Data Engineer As a Big Data Engineer, your everyday focus is harnessing the potential of data. You will be deeply involved in the realm of Scala and Spark applications, tasked with designing, creating, and optimizing solutions for processing immense volumes of data at scale. Furthermore, you will be accountable for the deployment and monitoring of these applications in customer environments, ensuring their smooth operation. In the spirit of teamwork, you will also support other teams in the design and creation of Spark applications. Your daily tasks include - Developing analytical algorithms in Spark - Employing Spark to transform and process network data - Optimizing and testing Spark applications - Deploying and monitoring applications in customer environments - Assisting other teams with the design and development of Spark applications Requirements - A minimum of 3 years of experience in developing Spark applications in PySpark and/or Scala - Stellar abilities in deploying and monitoring distributed applications - Robust experience with Unix systems - Profound knowledge in Python - Knowledge of workflow engines, for instance, airflow - Excellent skills in teamwork, communication, and logical thinking - Familiarity with version control tools - Proficient in spoken and written English We love your personality, if you - Behave like an owner - Enjoy being a team player - Revel in startup mentality Benefits You will become part of a team that offers not just monetary rewards but also: - A stable work environment with excellent prospects - Supportive colleagues who are experts in their field.