Azure/Databricks Data Engineer - Empower (remote/based in Costa Rica)

Company Description

Company Overview

Hitachi Solutions is a global solutions provider who takes pride in conceiving, constructing, and delivering state-of-the-art cloud solutions to stimulate innovation within our client's businesses. Our firm designs business services and technological integrations that power numerous every day products. We maintain close ties with Microsoft, and other dominating figures within the cloud computing arena.

What sets Hitachi Solutions apart is our industry focus coupled with our intellectual contributions to our customers. We consistently receive recognition for our achievements, and we aim to serve as a reliable advisor for both small and medium-size enterprises - casting speed in their pursuit towards key business objectives through exceptional engineering, robust work ethics, and understanding. With over 3,000 members in our team spanned across 14 countries, our 18 years of dedicated focus have ignited rapid growth and customer satisfaction. We offer remarkable salaries, 401k match, family leave, and health benefits to our employees. Additionally, no office presence or rigid work schedule is required.

As part of Hitachi, Ltd., our company enjoys an extensive history of innovation, financial stability, and international presence synonymous with grand corporations. Established in 1910, Hitachi, Ltd. has been manufacturing inventive products and solutions that support industries and social infrastructures worldwide, utilizing a workforce of 303,000 employees spread across more than 100 countries and 864 companies.

New Product Development and Innovations Team

This role within our company is positioned in our New Product Development and Innovations team, established in 2021. Embarking on this team offers an opportunity to accelerate your career, working with a team of fun and intellectual colleagues in a groundbreaking atmosphere: sufficiently funded, aimed at hypergrowth, progressing rapidly, and learning from mistakes to enhance innovation and solid engineering.

Armed with an existing client base, and a financially stable parent company – the goal of this group is to assist our company in launching products that can further intensify our already distinguished services, making cloud usage simpler for our customers, and aiding us in achieving our long-term financial goals with extraordinary linear scale.

Job Description

Please note: Despite this role being remote/virtual/work-from-home, the employee MUST reside, and have authorization to work, within Costa Rica.

DATA ENGINEER (DATABRICKS, PYTHON, SPARK)

This full-time role in our product organization requires a seasoned Data Engineer with comprehensive experience in the design of Big Data systems, specifically with expertise in data architecture, and Spark and Delta/Data Lake technology.

Individuals in this role will aid in the design, development, improvement, and maintenance of complex data pipelines that manage vital business operations, and vast-scale analytics pipelines. Qualified applicants must possess a quick learning ability, a background in data engineering, and/or robust software engineering expertise.

Responsibilities:

• Collaborate with team leadership to scope and execute tasks. Understand the capabilities of the platform and ways to enhance it.
• Maintain self-direction and autonomy.
• Arrange, develop, improve, and maintain complex data pipeline products that handle both business-critical operations and large-scale analytics applications.
• Experience is required in leading middle- and high-level data engineers.
• Support the unique needs and challenges of analytics, data science, and/or engineering teams.
• Ingrain excellence within the team's embraced processes, methodologies, standards, and technology choices.
• Adapt to new concepts quickly to stay up-to-date within the rapidly evolving field of data engineering.
• Proactively engage in continuous learning to keep the team aware of the latest advancements in this field.
• Commit to the development of technical maturity across the entire company.

Qualifications

• 5+ years of experience as a Data Engineer along with 2+ years of designing and constructing Databricks data pipelines is REQUIRED. Experience with Azure cloud is preferred, but experience with AWS, GCP, or other cloud platforms could be substituted.
• Experience with conceptual, logical, and/or physical database designs is HIGHLY DESIRED.
• 2+ years of hands-on experience with Python/Pyspark/SparkSQL is REQUIRED; experience with Scala is a plus.
• 2+ years of experience with Big Data pipelines or DAG Tools (Airflow, Dbt, Data Factory, or similar) is REQUIRED.
• 2+ years of experience with Spark (especially Databricks Spark and Delta Lake) is REQUIRED.
• 2+ years of hands-on experience with Big Data solutions in a cloud ecosystem, including Data/Delta Lakes, is REQUIRED.
• 2+ years of experience with source control (git) on the command line is REQUIRED.
• 2+ years of SQL experience, specifically in writing complex, highly optimized queries across large volumes of data is HIGHLY DESIRED.
• Strong data modeling/data profiling capabilities with Kimball/star schema methodology is HIGHLY DESIRED.
• Professional experience with Kafka or other live streaming technology is HIGHLY DESIRED.
• Professional experience with database deployment pipelines (i.e., dacpac’s or similar technology) is HIGHLY DESIRED.
• Professional experience with one or more unit testing or data quality frameworks is HIGHLY DESIRED.

#LI-CA1

#REMOTE

#DATABRICKS

#SPARK

#DATALAKEHOUSE

#BIGDATA

#costaricajobs

Additional Information

We provide equal opportunities for employment. All applicants will be screened for employment without regards to age, race, color, religion, sex, sexual orientation, gender identity, national origin, veteran status, or disability status.