Senior/Lead Data Engineer - AWS Glue

Job Opportunity in Gurgaon: Senior/Lead Data Engineer - AWS Glue

Company: Srijan Technologies

Location: Gurgaon, Haryana, India

About Us

At Material, we transform customer challenges into growth opportunities. As a global strategy partner, we collaborate with the world’s most recognizable brands and innovative companies. Our global team thrives in designing and delivering rewarding customer experiences powered by deep human insights, design innovation, and cutting-edge technology. We accelerate engagement and growth, transforming relationships between businesses and their customers.

Srijan, a Material company, is a renowned global digital engineering firm known for solving complex technology problems using deep expertise and strategic partnerships. Join us and be part of an Awesome Tribe.

Position Overview

We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. The ideal candidate will have extensive experience with AWS Glue, Apache Airflow, Kafka, SQL, Python, and DataOps tools. Knowledge of SAP HANA and Snowflake is a plus. This critical role involves designing, developing, and maintaining our client's data pipeline architecture, ensuring efficient and reliable data flow across the organization.

Key Responsibilities

• Design, Develop, and Maintain Data Pipelines: Create robust and scalable data pipelines using AWS Glue, Apache Airflow, and other relevant technologies. Integrate various data sources, including SAP HANA, Kafka, and SQL databases.
• Data Management and Transformation: Design and implement data transformation processes using SQL and Python. Ensure data quality and integrity through rigorous testing and validation.
• Collaboration and Communication: Work closely with data scientists, analysts, and stakeholders to understand data requirements. Collaborate with cross-functional teams to implement DataOps practices.
• Monitoring and Optimization: Monitor data pipeline performance and introduce improvements. Troubleshoot and resolve data-related issues, and manage monitoring and alerting systems.
• Documentation and Best Practices: Maintain comprehensive documentation of data pipelines and processes. Adhere to best practices in data engineering, including code versioning, testing, and deployment procedures.

Required Skills and Qualifications

Technical Expertise

• Extensive experience with AWS Glue for data integration and transformation.
• Proficient in Apache Airflow for workflow orchestration.
• Strong knowledge of Kafka for real-time data streaming.
• Advanced SQL skills for querying and managing databases.
• Proficiency in Python for scripting and automation tasks.
• Experience with SAP HANA for data storage and management.
• Familiarity with DataOps tools and methodologies.

Preferred Skills

• Knowledge of Snowflake for cloud-based data warehousing solutions.
• Experience with other AWS data services like Redshift, S3, and Athena.
• Familiarity with big data technologies such as Hadoop, Spark, and Hive.

Soft Skills

• Strong analytical and problem-solving skills.
• Excellent communication and collaboration abilities.
• Detail-oriented with a commitment to data quality and accuracy.
• Ability to work independently and manage multiple projects.

Qualifications

• Bachelor's or master's degree in Computer Science, Information Technology, or a related field.
• 4+ years of experience in data engineering or a related role.
• Proven track record of designing and implementing complex data pipelines and workflows.

Why Work for Material

Material is a global company collaborating with top-tier brands worldwide. We focus on innovation and value creation, working across diverse industry sectors like technology, retail, transportation, finance, and healthcare.

Joining Material means becoming part of a talented peer group with global reach. We offer professional development and mentorship, a remote-friendly workplace, and a culture that prizes inclusion and interconnectedness. Our commitment to understanding and systems approach uniquely equips us to impact our work significantly.

What We Offer

• Professional Development and Mentorship.