Azure/Databricks Data Engineer - Empower (remote/Canada-based)

Company Description

Company Overview

Hitachi Solutions is a global solution integrator with a passion for designing, developing, and delivering advanced cloud solutions to empower our clients to innovate throughout their entire business. Our company is responsible for creating the business services and technology that power some of your everyday products. We closely align with Microsoft and other leaders in the cloud computing sector.

What sets Hitachi Solutions apart is our industry focus, and the intellectual property we offer our customers. We are recognized year after year for our achievements, making every effort to be the go-to advisor for both large and medium-sized enterprises – aiding them in swiftly achieving their strategic business initiatives through superior engineering, diligent work, and compassion. With more than 3,000 team members in 14 countries, our company has seen tremendous growth and high customer satisfaction in our 18 years of operation. This success has enabled us to offer impressively competitive salaries, a matched 401k, family leave, and health benefits. We don't require a physically present office or a rigid work schedule.

Hitachi Solutions is a part of Hitachi, Ltd., a company with a rich history of innovation, solid financial stature, and a strong international presence. Since 1910, Hitachi, Ltd. has been an industry leader, producing innovative products and solutions supporting industry and social infrastructure worldwide, backed by 303,000 employees across more than 100 countries and 864 companies.

New Product Development and Innovations Team

This position is part of our New Product Development and Innovations team, established in 2021. The role provides an opportunity to advance your career quickly and to cooperate with a team of fun and enthusiastic colleagues in a disruptive environment that is well-funded, hypergrowth-focused, quick-paced, and conducive for trial and error, all in the name of innovation and engineering excellence.

Armed with an existing business portfolio, and financially secured parent company, the objective of this group is to help us introduce new products to bolster our already strong service business – aiming to simplify cloud usage for our customers, enabling us to meet our long-term financial objectives more efficiently.

Job Description

Please note: While our position offers the flexibility of remote / virtual / work-from-home, you MUST live in Canada and be authorized to work there.

DATA ENGINEER (DATABRICKS, PYTHON, SPARK) 

This is a full-time role in our product organization geared towards a highly proficient Data Engineer experienced with Big Data systems design, and especially skilled in data architecture, Spark, and Delta/Data Lake technology.

Individuals in this role will assist in designing, developing, enhancing, and maintaining complex data pipelines products that manage critical business operations and large-scale analytics pipelines. Ideally, candidates will have a proven ability to learn new concepts quickly, come with a background in data engineering, and/or boast robust software engineering expertise. 

Responsibilities

• Work with the team leadership to understand and expand platform capabilities.
• Demonstrate independence and autonomy.
• Design, develop, enhance, and maintain complex data pipeline products that handle business-critical operations and large-scale analytics applications.
• Experience leading middle and senior-level data engineers. 
• Support analytics, data science, and/or engineering teams, understanding their unique needs and problems. 
• Implement excellence in processes, methodologies, standards, and technology choices adopted by the team.
• Quickly embrace new ideas to stay up-to-date with rapidly evolving data engineering technology.
• Commit to continuous learning to keep the team informed of the latest developments in the industry.
• Commit to developing technical maturity across the company. 

Qualifications

• 5+ years of Data Engineering experience including 2+ years designing and building Databricks data pipelines are REQUIRED; Azure cloud is preferred, will consider AWS, GCP, or other cloud platform experience in its place
• Conceptual, logical and/or physical database design experience is HIGHLY DESIRED
• 2+ years of hands-on Python/Pyspark/SparkSQL experience is REQUIRED; experience with Scala is a plus
• 2+ years of experience with Big Data pipelines or DAG Tools (Airflow, Dbt, Data Factory, or similar) is REQUIRED
• 2+ years of experience with Spark, especially Databricks Spark and Delta Lake, is REQUIRED
• 2+ years of hands-on experience implementing Big Data solutions in a cloud ecosystem, specifically Data/Delta Lakes, is REQUIRED
• 2+ years of experience with source control (git) on the command line is REQUIRED
• 2+ years of SQL experience, particularly in writing complex, highly optimized queries across large data volumes, is HIGHLY DESIRED
• Strong data modeling / data profiling capabilities with Kimball/star schema methodology is HIGHLY DESIRED
• Professional experience with Kafka or other live streaming technology is HIGHLY DESIRED
• Professional experience with database deployment pipelines (i.e., dacpac’s or similar technology) is HIGHLY DESIRED
• Professional experience with one or more unit testing or data quality frameworks is HIGHLY DESIRED

#LI-CA1

#REMOTE

#DATABRICKS

#SPARK

#DATALAKEHOUSE

#BIGDATA

Additional Information

We are an equal opportunity employer. We will consider all applicants for employment without regard to age, race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status.