Senior Data Engineer

Company Description

We are seeking a Senior Data Engineer to lead a small team as we transform our simple data platform into a self-service analytics project based in the cloud.

CUSTOMER
Sigma Software delivers premium software development, graphic design, testing, and support services. We have developed systems that bring distinct advantages to clients in the automotive, telecommunications, aviation, advertising, gaming, banking, real estate, and healthcare industries.

PROJECT
For an internal project at Sigma Software, our primary objectives are to automate reporting and establish a self-service data analytics system for our company. 

In this role, you will be tasked with designing and modelling data into DWH and Data Marts using Spark and MS SQL. This data will then be made available to, and you will be working closely with, BI engineers who create reports with Apache Superset, among others who use such data. You will also work with other team members who depend on this data. Furthermore, part of your responsibilities will include shifting our on-premise solution to a cloud-based environment.

Job Description

• We aim to create a robust system for collecting, processing, analysing, and visualising vast amounts of data from diverse sources.
• Lay the foundation and model the data
• Develop a platform for constructing and managing data pipelines that collect data from different sources
• Create a Data Warehouse
• Cooperate with Business analytics to analyse and prepare domain descriptions
• Create Data Marts
• Design a permission model that enables flexible control over vertical and horizontal access to data

Qualifications

• Understanding of the fundamentals of data analytics, for example, dimensional modelling, /ELT, reporting tools, data governance, data warehousing, and structured and unstructured data.
• Substantial SQL knowledge and experience with RDBMS, as well as a solid understanding of database fundamentals.
• Database development and data modelling experience, preferably with Databricks/Spark
• Python experience
• Understanding of serialization formats and their trade-offs (columnar vs. row-based) 
• Experience in debugging and optimizing Spark jobs 
• Excellent written and verbal communication skills 
• Proficient level of English 
• A degree in Computer Science or a related field 
• Experience with a Business Intelligence tool would be a plus