Senior AWS Data Engineer

Company Description

Vitol is a leader in the energy sector with a presence across the spectrum: from oil to power, renewables and carbon. From 40 global offices, we aim to add value across the energy supply chain, using our scale and market understanding to assist in the energy transition. To date, we have committed over $2 billion in capital to renewable projects, and we are identifying and developing low-carbon opportunities worldwide.

Our people are our business. We value talent and provide an environment where individuals can reach their full potential, unrestricted by hierarchy. Our team consists of more than 65 nationalities and we are committed to fostering and maintaining diversity in our workforce. .

Job Description

As a Senior Data Engineer, you will be tasked with designing, implementing, and upkeeping large scale data processing systems on AWS, while ensuring their scalability, reliability, and efficiency.

Your role requires a high level of technical knowledge, with vast experience in MPP platforms/Spark, “big data” (e.g., weather forecasts, vessel location, satellite imagery, etc.), and resilience in developing reliable data pipelines. You will be responsible for the complete data pipelines: from acquisition, loading, and transforming, to implementing business rules/analytics and finally, delivery to the end-user (business / data science / AI).

You will also closely work with the Business and other delivery teams, as well as the Data Science team to comprehend their data needs and provide the required data infrastructure for their activities. Moreover, you will also optimize the performance of data processing systems by refining database queries, improving data access times and reducing latency.

This role necessitates strong coding abilities in SQL and Python, alongside adhering to established engineering practices.

Strong communication skills are a must. You should be able to effortlessly translate technical expressions to non-technical users and convert business requirements into technical requirements.

Qualifications

• Over 10 years of experience in data engineering
• Proficiency with MPP Databases (Snowflake, Redshift, Big Query, Azure DW) and/or Apache Spark
• Experienced in building resilient data pipelines for large datasets
• Substantial knowledge of AWS or cloud across core and extended services
• 8+ years experience working with at least three of the following: ECS, EKS, Lambda, DynamoDB, Kinesis, AWS Batch, ElasticSearch/OpenSearch, EMR, Athena, Docker/Kubernetes
• Proficiency with Python and SQL, along with experience in data modeling
• Experience with modern orchestration tools (Airflow / Dagster / Prefect / similar) and/or DBT
• Apt at working in a dynamic environment with a certain degree of uncertainty

Additional Information

Desired:

• Infrastructure as Code (Terraform, Cloud Formation, Ansible, Serverless)
• CI/CD Pipelines (Jenkins / BitBucket Pipelines / similar)
• Database/SQL tuning abilities
• Basic comprehension of data science concepts