Job expired!
Company Description
We are a multinational group of individuals who believe that with the correct knowledge and strategy, technology can be the solution to the obstacles that businesses encounter today. Since 2016, we have been utilizing this understanding and approach to assist our clients in translating technology into success.
Our Swiss origins and in-house development teams in Lima and across the region allows us to bring together the best of both cultural spheres: the talent and enthusiasm of Latin American professionals melded with Swiss organizational capabilities and mindset.
Job Description
Bertoni Solutions is seeking a Data Engineer to work remotely for a client based in the USA.
In this role, you will be accountable for devising data solutions within the Azure cloud environment, work in harmony with diverse teams across the organization to implement data strategy, design sturdy architecture, and establish efficient governance. You should hold significant expertise in Spark, Python, and valuable familiarity with Scala, along with a comprehensive understanding of Azure Data Factory (ADF) concepts; additionally, you should demonstrate proficiency in various data access technologies, showing aptness in strong SQL and PySpark-SQL.
Qualifications
This position is only open to applicants residing in Latin America or North/Central America.
- Minimum of 6 years of experience.
- Strong experience with Scala.
- Practical experience using ADF to connect to different sources (RDBMS, Salesforce, flat files) and collect data into ADLS.
- Practical experience managing incremental load and full load methods.
- Practical experience dealing with semi-structured data such as JSON.
- Practical experience using Databricks with delta lake concepts.
- Strong SQL skills with (Slow Change Types [SCD], Joins, Common Table Expressions [CTEs], Hierarchies, etc.)
- Practical experience with PySpark.
- Practical experience using Databricks Notebooks and optimization techniques.
- Experience with Azure Cloud Services for Data Engineering.
- Understanding of Data Layers: Raw, Refined and Curated Models.
- Experience with Databricks: programming and automating notebooks using ADF.
- Performance optimization of Spark or PySpark code, including techniques such as partitioning and data caching.
- Practical experience in creating a user-defined function (UDF) in Databricks and how it can be applied to a DataFrame.
- Management and operation of DataFrames for Data Engineering activities.
- Advanced English proficiency.
- Desirable: R
Additional Information
- Excellent communication skills.
- Excellent problem-solving skills.
- Critical thinking skills.
- Attention to detail.
- Decision-making.
- Teamwork.
- Adaptability.
