Join PrePass: Lead Data Engineer Opportunity

About PrePass

PrePass® is North America's most advanced and widely utilized weigh station bypass and toll payment platform. Our cutting-edge technologies empower safe, qualified motor carriers to bypass inspection facilities at highway speeds, enhancing efficiency by saving time, fuel, and money, while also reducing emissions. Uniquely, PrePass offers both bypass and tolling solutions, allowing fleets to manage toll costs effectively, eliminate toll violations, and automatically resolve toll disputes. As the only preclearance system developed, owned, and operated in the USA, PrePass is proud to be the American Trucking Associations’ sole Endorsed Corporate Partner. More than 105,000 fleets trust PrePass, servicing over 750,000 commercial vehicles.

Position Description: Lead Data Engineer

We are seeking a Lead Data Engineer to spearhead innovative solutions that address the dynamic needs of our expanding business. In this role, you will lead the design, development, and implementation of our next-generation, cloud-native data architecture. You will also mentor and guide a team of talented data engineers, blending technical expertise with strategic vision. We are looking for a trailblazer passionate about innovation and continuous learning, equally adept at coding and conceptualizing complex solutions across various teams and domains.

Your Key Responsibilities

• Design and implement scalable, distributed architecture using Azure Databricks, Delta Lake, and other Microsoft Azure services.
• Act as a technical expert, leading projects and optimizing production systems for reliability, performance, scalability, and security.
• Partner with Product Management and Engineering leadership to architect and build solutions that align with roadmap features.
• Implement best practices in Microsoft Azure for data quality, governance, security, lineage, and cataloging.
• Mentor less experienced engineers and lead a team of approximately 5 engineers.
• Actively monitor workflows and compute resources to enhance efficiency and mitigate costs.
• Build robust solutions for orchestration and observation.
• Troubleshoot system issues, prioritize fixes, and delegate tasks effectively within the team.
• Oversee code releases to Production, ensuring smooth transitions.
• Spearhead prototyping exercises to demonstrate the feasibility and value of solutions and architectures.

Qualifications

Required

• Bachelor’s degree in computer science, a related technical field, or equivalent experience.
• Minimum 8 years of experience designing and building large-scale database solutions, with at least 2 years in a leadership role.
• A minimum of 3 years programming experience in Python and Pyspark.
• Expert SQL skills.
• Extensive experience building ETL/ELT solutions using Azure Databricks, Azure Data Factory, or similar tools.
• Experience designing data structures and models for data lakes.
• Strong understanding of distributed storage and computing.
• Proficiency with reporting and data visualization tools, specifically PowerBI.
• Proficiency with SSIS.
• Familiarity with software production engineering practices, version control, code peer reviews, automated testing, and CI/CD.
• Strong collaborative abilities, adaptable to fluid environments, and embracing change.
• Commitment to staying current with emerging technologies and industry best practices, driving continuous improvement in development methodologies.

Preferred

• Understanding of ML and AI concepts, algorithms, and techniques.
• Experience with batch and streaming data sources.
• Familiarity with No-SQL databases.
• Experience with event-driven architectures, including queues, batches, and pub/sub models.
• Experience working with data from enterprise systems like ERPs and CRMs.
• Familiarity with DevOps CI/CD implementation methodologies.

Benefits: How We Will Take Care of You

• Comprehensive benefits package including medical, dental, and vision coverage starting on your date of hire.
• Paid Time Off (PTO) including vacation, sick days, holidays, and floating holidays.
• 401(k) plan with employer match.