Big Data Engineer

Position Overview:

NorthBay Solutions is actively seeking experienced Big Data Engineers with 2 to 3+ years of expertise to enhance our dynamic team. Ideal candidates will demonstrate a strong background in data engineering, emphasizing cutting-edge cloud-based solutions with a robust proficiency in SQL. This pivotal role involves the design, implementation, and maintenance of data pipelines for batch and streaming data processing and requires effective troubleshooting of data issues.

Location and Work Arrangement

Join us at our locations in Lahore, Karachi, or Islamabad. This position offers a hybrid work setup, accommodating both in-office and remote working arrangements to best suit your needs.

Key Responsibilities

The Big Data Engineer will:

• Design, implement, and maintain ETL/ELT processes, pivotal for both batch and streaming data processing.
• Develop and refine data pipelines using top-tier cloud data warehousing and lake solutions, such as Snowflake, BigQuery, Redshift, and Azure Synapse.
• Diagnose and resolve data issues, delivering immediate remedies and addressing underlying causes for sustainable performance.
• Leverage AWS tools including S3, DMS, Glue, and Athena to construct scalable and dependable data solutions.
• Collaborate with cross-functional teams to precisely understand data requirements, crafting tailored solutions that drive business objectives.
• Use transformation tools like dbt to effectively model and transform data for analytics and reporting.

Mandatory Requirements

Candidate must have:

• A Bachelor’s degree in Computer Science, Engineering, or a related discipline.
• Over 3 years of professional experience in data engineering.
• Advanced proficiency in SQL and extensive familiarity with modern cloud data warehousing and data lakes.
• An excellent track record in diagnosing and resolving data-related issues.
• Proficiency in AWS services including S3, DMS, Glue, and Athena.
• Experience with dbt or similar data transformation tools.

Desirable Skills

We value additional skills such as:

• Proficiency in Python programming.
• Experience with infrastructure as code tools like Terraform and Terragrunt.
• Familiarity with orchestration tools such as Dagster, Airflow, and AWS Step Functions.
• Understanding of pub-sub, queuing, and streaming frameworks like AWS Kinesis, Kafka, SQS, and SNS.
• Knowledge of CI/CD pipelines and automation practices.

NorthBay Solutions offers a stimulating opportunity for Big Data Engineers passionate about innovation and growth in the field of data processing and analytics. Apply today to become part of a team that values technology and solutions that lead the market.