Azure/Databricks Data Engineer - Empower (remote/US-based)

Company Description

Company Overview

Hitachi Solutions is a global solutions integrator committed to designing, developing, and delivering cutting edge cloud solutions to assist our clients in innovating across their entire businesses. Our firm develops the business services and technology that power some of the products you use every day – and retains close alignment with Microsoft and other leaders in the cloud computing arena.  

What sets Hitachi Solutions apart is both our industry focus and the unique intellectual property we bring to our customers. Consistently recognized for our success, we strive to act as trusted advisors for both large and medium-sized enterprises, helping them move quickly to achieve strategic business aims through exceptional engineering, hard work, and compassion. With over 3,000 team members across 14 countries, over 18 years we have seen explosive growth and high customer satisfaction in our company. This success has afforded us the ability to offer highly competitive salaries, 401k match, family leave, and health benefits. And rest assured, we will not require you to commute to an office or stick to an inflexible work schedule. 

As a part of Hitachi, Ltd., our company benefits from a long and rich history of innovation, a strong financial backbone, and the international presence of one of the world's largest corporations. Since its founding in 1910, Hitachi, Ltd. has led the industry in creating innovative products and solutions that bolster industry and social infrastructure on a global scale, backed by 303,000 employees across over 100 countries and 864 companies-

New Product Development and Innovations Team

In 2021, we established a New Product Development and Innovations team within our company. Joining this group presents a chance to accelerate your career and collaborate with a team of enthusiastic and dedicated colleagues in an environment where disruption is welcomed: well-funded, rapidly growing, quick to move, and accustomed to learning from mistakes in the name of innovation and effective engineering. 

Supported by an existing client base and a stable financial parent company, this group's aim is to drive our firm's introduction of products to enhance our already robust services business – ultimately simplifying the cloud for our customers and enabling us to meet our long-term financial objectives on a greater-than-linear scale.

Job Description

Please note: Although this position allows for remote/virtual/work-from-home arrangements, you MUST reside in the United States and be authorized to work there without sponsorship. We regret we can only accept applications from US citizens and permanent US residents - no C2C or H1 transfers.

DATA ENGINEER (DATABRICKS, PYTHON, SPARK) 

We are offering a full-time role in our product organization for a highly experienced Data Engineer with expertise in Big Data systems design and skills in data architecture, particularly focused on Spark and Delta/Data Lake technologies.

Those in this role will assist in the design, development, enhancement, and maintenance of complex data pipelines products that manage business-critical operations and extensive analytics pipelines. Qualified applicants should demonstrate a rapid ability to grasp new concepts, possess a background in data engineering, and/or have substantial software engineering expertise. 

Responsibilities

• Scope and execute tasks in collaboration with team leadership. Work with the team to comprehend platform capabilities and identify how best to improve and expand these capabilities.
• Strong independence and autonomy demonstrated.
• Design, develop, enhance, and maintain complex data pipeline products that manage business-critical operations and extensive analytics applications.
• Experience leading intermediate- and senior-level data engineers. 
• Provide support to analytics, data science and/or engineering teams, understanding their unique needs and challenges.
• Introduce excellence into the team's processes, methodologies, standards, and technology choices.
• Quick adaptation to new concepts to keep pace with the fast-evolving data engineering technology.
• Continuous learning commitment to keep the team updated on the latest developments in the field.
• The commitment to enhance technical maturity throughout the company. 

Qualifications

• 5+ years of data engineering experience, including at least 2 years designing and building Databricks data pipelines, is REQUIRED; while experience with the Azure cloud is preferred, we would also consider AWS, GCP, or other cloud platform experience.
• Experience with conceptual, logical and/or physical database designs is HIGHLY DESIRED
• 2+ years of hands-on experience with Python/Pyspark/SparkSQL is REQUIRED; experience with Scala is a plus
• 2+ years of experience with Big Data pipelines or DAG Tools (like Airflow, Dbt, Data Factory, or similar) is REQUIRED
• 2+ years of Spark experience (especially Databricks Spark and Delta Lake) is REQUIRED
• 2+ years of hands-on experience implementing Big Data solutions in a cloud ecosystem, including Data/Delta Lakes, is REQUIRED
• 2+ years of experience with source control (git) on the command line is REQUIRED
• 2+ years of SQL experience, specifically in writing complex, highly optimized queries across large data volumes, is HIGHLY DESIRED
• Strong data modeling/data profiling abilities with the Kimball/star schema methodology is HIGHLY DESIRED
• Professional experience with Kafka or other live-streaming technology is HIGHLY DESIRED
• Professional experience with database deployment pipelines (for example, dacpac’s or similar technology) is HIGHLY DESIRED
• Professional experience with one or more unit testing or data quality frameworks is HIGHLY DESIRED

#LI-CA1

#REMOTE

#DATABRICKS

#SPARK

#DATALAKEHOUSE

#BIGDATA

Additional Information

We are an equal opportunity employer. We do not discriminate in the hiring process based on age, race, color, religion, sex, sexual orientation, gender identity, national origin, veteran status, or disability.

Additional Information:

All your information will be kept confidential according to EEO guidelines.

Base Salary Pay Range*: USD $97.5K – USD $175K

*This pay range is provided only as a general guideline and not as an assurance of compensation or salary. When extending an offer, other factors are considered. These include (but are not limited to) job responsibilities, education, experience, relevant skills for the role, internal equity, alignment with market data, and other legal requirements.

Other Compensation / Benefit Overview:

In addition to Base Salary, the selected candidate may be eligible to participate in the following plans / programs, once all hiring requirements are met:

· Bonus Plan

· Medical, Dental and Vision Coverage

· Life Insurance and Disability Programs

· Retirement Savings with Company Match

· Paid Time Off

· Flexible Work Arrangements including Remote Work