Data Ops Engineer | Devoteam Morocco Nearshore

Company Description

At Devoteam, we are "Digital Transformakers". Respect, honesty, and passion drive our tribe every day.

Together, we help our clients win the Digital battle: from consulting to implementing innovative technologies, to the adoption of uses.

Cloud, Cybersecurity, Data, DevOps, Fullstack Dev, Low Code, RPA are no longer a secret to our tribe!

Our 10,000+ employees are certified, trained, and supported daily to take on new, innovative challenges.

As a leader in Cloud, Cybersecurity, and Data in EMEA, the Devoteam Group achieved a turnover of 1.036 billion euros in 2022 and aims to double it within the next 5 years.

Devoteam Morocco, a key player in IT expertise for over 30 years with over 350 consultants, is accelerating its growth by developing its nearshore expert activities to meet the needs of our French, European and Middle Eastern clients.

Are you ready to join us and take up this challenge together?

Job Description

Data Ops Engineer @ Devoteam Data Driven.

In a world where data sources are constantly changing, Devoteam Data Driven helps its clients transform their data into actionable information for greater business value.

Data Driven addresses the following 3 main dimensions: Data Strategy, Data for Business, and Data Foundation, supporting its clients' expertise to make them more efficient and competitive every day.

As part of the Nearshore teams at Devoteam Morocco, you will join the teams of the Data Foundation tribe: an enthusiastic team of Data Engineers, Data Ops, Tech lead architects, and project managers working on data platforms and ecosystems: designing, building and modernizing data platforms and solutions, designing data pipelines with an emphasis on agility and DevOps applied to Data.

More specifically, a community of Data Ops engineers/experts proficient in operations management, automation, reliability of components, and data platforms, even more "natively Cloud" today and their capacity and ability to collaborate with development and operations teams

You will be the essential link for:

• Setting up and configuring the necessary infrastructures for data pipelines: data processing clusters (Apache Spark, Cloudera, Databricks, Snowflake, GCP BigQuery, Kafka, Nifi, Starburst ...), databases (Elasticsearch, Redis, MongoDB, BigTable, DynamoDB ...) and other components (Airflow, Jupyter, RStudio, dbt, Minio, ...)
• Designing, developing, and maintaining the continuous integration (CI) and continuous deployment (CD) pipelines to automate the deployment process of infrastructure components
• Describing and managing infrastructure as code (IaC) using tools like Terraform, Ansible, and others to provide automated and reproducible resource provisioning
• Setting up monitoring systems to track data pipeline performance and configuring alerts for reporting in case of failure or performance issues
• Analyzing and improving the performance of data pipelines by identifying bottlenecks, adjusting configurations, and implementing optimizations
• Implementing monitoring systems to track performance metrics, detect anomalies, and trigger alerts in case of problems using tools like Prometheus, Grafana, and others
• Setting up log management systems to collect, store, and analyze application and infrastructure logs
• Collaborating with security teams to implement best practices for security, including secret management, RBAC/ABAC access methods, and data anonymization/encryption, setting security and privacy policies, including GDPR, and if necessary, participating in any potential security audits
• Guaranteeing the availability, performance, and reliability of services by designing and implementing robust and resilient architectures and load tests
• Monitoring resource usage and managing capacity planning to handle load increases while optimizing the use of existing resources
• Implementing auto-scaling mechanisms to dynamically adjust allocated resources based on workload
• Maintaining the infrastructure and deployed data components including update management, security patches, and proactive problem-solving
• Maintaining accurate documentation of configurations, data pipeline procedures, and best practices for deployment, understanding and collaboration within the team
• Working closely with development teams to integrate DevOps practices into the software development lifecycle
• Identifying, diagnosing, and rapidly resolving operational issues including preventive maintenance practices to prevent unexpected failures and ensure the continuous availability of pipelines
• Evaluating and implementing cost optimization strategies related to infrastructure and resources used
• Working closely with development teams, analysts, and data scientists to understand requirements and ensure deployed components meet their needs
• Setting up lineage mechanisms and data catalogues and deploying solutions to ensure data quality, their traceability through different processing stages in compliance with security and privacy rules

You will master your technical fundamentals, fully understand the activities of a [Site Reliability Engineer] applied to Data, the platforms, and Data components, and especially assert your willingness to understand the needs and services on which you will work.

Your playing field: distribution, energy, finance, industry, health, and transport with plenty of use cases and new data challenges to overcome together, particularly Data in the Cloud.

What we expect from you.

• That you believe in Data and DevOps
• That you help your colleague
• That you are kind to your HRs
• That you enjoy your mission
• And that Codingames don't scare you (you won't be alone: we will help you)

And more seriously:

• That you master the Data platforms and components frequently deployed in data processing, analysis, and exposure
• That you also apply on the basic expectations of a SRE engineer: automation, monitoring, reliability, and cross-functional collaboration to ensure the operational success of the services
• That you want to invest in the new paradigms of Data and DevOps SRE: Cloud, DaaS, SaaS, DataOps, AutoML and that you commit with us in this adventure
• That you like working in agile mode
• That you consistently and rigorously perform the requested activities
• That you maintain this dual Infra & Dev competence
• That you are close to the sponsors and stakeholders, that you assist them in defining their new needs and services: in workshops, by defining the user stories and by testing them through POC
• And automation is one of your drivers and you are a convinced promoter: you work your code, you commit in Open Source, you do a little competition then join us

What we will bring to you.

• A manager by your side at all times
• A Data community where you will find your place: Ideation Lab, Hackathon, Meetup ...
• A training and certification course via "myDevoteam Academy" on current and future technologies: Databricks, Spark, Azure Data, Elastic.io, Kafka, Snowflake, GCP BigQuery, dbt, Ansible, Docker, k8s ...
• Reinforcing your expertise in the field of Data to become a Tech Lead Cloud (Azure, AWS, GCP ...), an architect of future Data platforms, a DataOps expert serving businesses (Data as a Service) and Data Science (AutoML), a Data Office Manager in charge of Data Product projects, in short, many new jobs in perspective...
• The possibility to personally invest: be an internal trainer, community leader, participate in candidate interviews, help develop our offers and why not manage your own team...

Some examples of missions.

• The design, implementation, and support of data pipelines
• The deployment of data solutions in an Agile and DevOps approach
• The development of REST APIs to expose data
• The support and expertise on Data technologies and deployed solutions: Hadoop, Spark, Kafka, Elasticsearch, Snowflake, BigQuery, Azure, AWS ...

Qualifications

What assets to join the team?

• Bachelor's degree in Engineering or equivalent
• From 3 years of experience in the function
• Proven experience in SRE engineering in the Data field
• Deep understanding of orchestration tools such as Apache Oozie, Apache Airflow, Databricks Jobs
• Familiar with data processing technologies: Apache Spark, Hadoop, Kafka…
• Understanding of best practices for data security: authentication, authorization, encryption, data anonymization/obfuscation
• Mastery of automation tools Terraform, Ansible, and others
• Terraform certification is a plus to confirm practical experience
• Certifications will be a plus, especially on Azure, AWS, GCP, and Databricks Cloud platforms,
• Proven practice in managing/using secrets: HashiCorp Vault for example
• Proven practice with monitoring, alerting, log reporting tools such as Grafana, Prometheus, ELK, fluentd ... and particularly applied to data pipelines
• Ability to identify and solve performance problems in data pipelines
• A dual infrastructure (linux, ansible, k8s) dev (java, scala, python) competence
• Good knowledge of Rest APIs and microservices
• Experience with Shell, Linux systems, Docker Compose, and Kubernetes is a plus
• Experience with CI/CD integration tools (Jenkins, Gitlab) and working in agile mode
• Understanding and implementation of Service Level Objectives SLO
• Excellent interpersonal skills, you enjoy working in a team
• A strong sense of service and commitment to your activities
• The ability to communicate and listen in all circumstances and write without mistakes...
• and you are fluent in English, indeed!

Additional Information

Additional Information.

• Position based in Morocco in our offices in Rabat and/or Casablanca and only open for permanent contract
• Hybrid position with the possibility of remote work
• By joining Devoteam, you will have the opportunity to exchange with your peers, share their experience, and develop your skills by joining the Data Driven community bringing together consultants from the 18 countries of the Group

Stay connected:

• https://www.linkedin.com/company/devoteam
• https://twitter.com/devoteam
• https://www.facebook.com/devoteam