Data Engineer 2

Company Description

When you join us, you are running with the best. For several decades, we've enabled marketers from top global brands to personalize experiences for millions of individuals through our advanced technology, solutions, and services. Epsilon's top-tier identity gives brands a transparent, privacy-safe view of their customers, applicable across our suite of digital media, messaging and loyalty solutions. We process over 400 billion consumer actions daily and have multiple proprietary technology patents, including real-time modeling languages and consumer privacy advancements. Thanks to the efforts of all our employees, Epsilon India is now certified as a Great Place to Work™. Epsilon has also been consistently recognized as an industry leader by Forrester, Adweek and the MRC. As a part of the Publicis Groupe, Epsilon is a global company with more than 8,000 employees worldwide. For more information, visit epsilon.com/apac or check our LinkedIn page.

Job Description

Organization Objective/Purpose:

This position is in the Engineering team under the Digital Integration Services division. We initiate the customer experience through the personalization of offers and content. Currently, we're seeking a smart, highly motivated software engineer.

As part of a team focused on building solutions and pipelines using the latest software engineering design principles and technological stacks, you will be responsible for identifying, designing, and implementing improvements. These include redesigning infrastructure for greater scalability, optimizing data delivery and automating continuous integration and deployment processes/pipelines.

Additionally, you're expected to collaborate with various stakeholders, bring scientific rigor to design and develop high-quality software.

He/She should have excellent verbal and written communication skills and be comfortable working in a startup environment within a larger company.

Brief Description of Role:

• Develop solutions for Epsilon that deliver high quality personalized recommendations across various channels to our customers.
• Work with the Data science team to ensure seamless integration and support of machine learning models.
• Build the necessary infrastructure for optimal extraction, transformation, and loading of data from a variety of data sources using SQL and AWS 'big data' technologies.
• Develop end-to-end (Data/Dev/MLOps) pipelines based on a deep understanding of cloud platforms, AI/ML lifecycle, and business problems to ensure solutions are efficiently and sustainably delivered.
• Collaborate with other team members to ensure high-quality deliverables.
• Learn and implement the latest design patterns in software engineering.

Qualifications

Data Management:

• Experience with both structured and unstructured data, alongside technologies like Hadoop or Apache Spark.
• Good understanding of Data Modeling, Data Warehouse, Data Catalog concepts and tools.
• Experience with Data Lake architectures, and combining structured and unstructured data into unified representations.
• Aptitude for identifying, joining, exploring, and examining data from multiple disparate sources and formats.
• Capability to manage large quantities of unstructured or formless data, and transfigure it into an analyzable form.
• Competence in handling data imperfections such as missing values, outliers, inconsistent formatting, etc.
• Ability to work with large datasets (millions of rows, thousands of variables).

Software Development:

• Experience in writing code in programming languages like Python, PySpark, and shell script on Linux.
• Familiarity with software development methodologies like Agile/Scrum.
• Passion for learning new technologies and driving the organization to adapt to emerging best practices.

Architecture and Infrastructure:

• Architectural design experience on AWS.
• Experience in delivering software with AWS tools like EC2, S3, EMR/Glue, Lambda, Data Pipeline, CloudFormation, Redshift, etc.
• Good knowledge of working with UNIX/LINUX systems.

Qualifications:

• Bachelor’s Degree in Engineering or in a related field with at least 2+ years of related work experience.
• Experience working with tech stacks such as Python, PySpark, Micro services, Docker, Serverless Frameworks.
• Hands-on experience in building ETL workflows/pipelines.
• Experience in working with both relational and non-relational databases and SQL (Experience with NoSQL preferred).
• Experience working with Cloud technologies (AWS or Azure).
• Experience building data and CI/CD/MLOps pipelines.
• Familiarity with Airflow and MLFlow tools.
• Familiarity with automated unit/integration test frameworks.
• Prior experience in AdTech or MarTech technologies would be an added advantage.
• Knowledge of machine learning algorithms and concepts would be a plus.
• Strong written and verbal communication skills, team player.
• Strong analytical thought process and ability to interpret findings.

The ideal candidate should have strong business acumen, excellent interpersonal and communication skills, and the ability to work independently. He/she should be able to communicate findings and explain techniques in a way that all stakeholders, both technical and non-technical, will understand.