Job expired!
We are seeking a Senior GCP Data Engineer to join our engineering team! We're searching for an individual who is confident, inquisitive, and forthright, making them an ideal fit for our empowering and driven culture. Through your determination and articulate communication, our cloud-based solutions will be effortless and clear in operation. With your dynamic convictions, you'll aid our brand to tackle issues seamlessly and efficiently in the cloud. If the prospect of being part of a team that values transparency, confidence, and productivity through cloud technology excites you, then do join us.
Responsibilities:
- Designing and developing complex and expansive data structures and pipelines to effectively organize, gather and standardize data for insights generation and reporting needs.
- Writing intricate ETL (Extract/Transform/Load) processes, designing database systems, and developing tools for real-time and offline analytics processing.
- Developing frameworks, standards, and reference materials for architecture and relevant products.
- Designing data marts and data models to support Data Science and other internal clients.
- Acting as a mentor to junior team members by providing technical advice.
- Applying knowledge of gcp-data tools and products to consult and provide advice on supplementary undertakings across numerous domains spanning a broader enterprise.
- Collaborating with the data science team to restructure data and integrate algorithms and models into highly available, production systems.
- Using comprehensive knowledge on Hadoop architecture and HDFS commands, and experience in designing & optimizing queries, to construct scalable, modular, and efficient data pipelines.
- Using advanced programming skills in Python, Java, PySpark, or any other major languages to build robust data pipelines and dynamic systems.
- Integrating data from various sources, ensuring compliance to data quality and accessibility standards.
- Experimenting with available tools and advising on new tools to identify an optimal solution considering the requirements dictated by the model/use case.
Requirements:
- At least 5 years of Data Engineering experience.
- At least 5 years of experience with PySpark (Spark/Scala).
- Minimum 3 years advanced knowledge in Hadoop architecture, HDFS commands and experience in designing & optimizing queries against data in the HDFS environment.
- Minimum 2 years' experience with Google Cloud Platform (GCP).
- Experience with bash shell scripts, UNIX utilities & UNIX Commands.
- Experience in building and implementing data transformation and processing solutions.
- Advanced knowledge in Java, Python, Hive, Cassandra, Pig, MySQL, or NoSQL or similar.
- Passion for DevOps and GCP, and ability to thrive in a collaborative and fast-paced environment.
- Inclination towards problem-solving and understanding how things work and operate.
- Thriving in an environment that consistently challenges you to learn.
