Job Opportunity: Senior Data Engineer - Remote Position

Experience Level: 5-7 Years

Company: PradeepIT Consulting Services Pvt Ltd

About the Role

PradeepIT Consulting Services Pvt Ltd is looking for a talented and experienced Senior Data Engineer to join our innovative team. The successful candidate will possess a robust skill set in data engineering, including expertise in Azure Databricks, Pyspark, Medallion Architecture, and Data Lakes.

Responsibilities

• Design, develop, and optimize data pipelines using Azure Databricks and Pyspark.
• Implement and maintain Medallion Architecture to streamline data processing workflows.
• Manage and organize large datasets in Data Lakes, ensuring data quality and integrity.
• Write complex SQL queries to extract, transform, and load data from various sources.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver efficient data solutions.
• Monitor and troubleshoot data pipeline performance and resolve issues as they arise.
• Stay up-to-date with the latest advancements in data engineering and propose innovative solutions to improve data processes.

Requirements

• 5-7 years of hands-on experience in data engineering or a related field.
• Proficiency in Azure Databricks and Pyspark.
• Strong understanding of Medallion Architecture and its implementation.
• Experience working with Data Lakes and managing large datasets.
• Advanced SQL skills with the ability to write and optimize complex queries.
• Knowledge of Azure services and cloud-based data solutions.
• Strong problem-solving skills and attention to detail.
• Excellent communication and collaboration abilities.

Why Join Us?

At PradeepIT Consulting Services Pvt Ltd, we are dedicated to leveraging the power of data to drive innovative solutions. If you are passionate about working with large datasets and enjoy developing cutting-edge data solutions, we encourage you to apply for this exciting opportunity.

Interested candidates, please submit your application with a detailed resume. We look forward to welcoming a new member to our team!