Senior Data Engineer

Senior Data Engineer

IQVIA's Center of Excellence in Digital Activation is hiring a senior data engineer to work remotely anywhere in Canada. Our vision paves the way for unprecedented customer engagement, next-generation omnichannel storytelling, an integrated suite of measurements, and real-time campaign optimization, all powered by a platform-independent end-to-end operating system and data for intelligent healthcare marketing and analytics. By simplifying workflows that previously took several days into seconds and integrating features that previously required multiple providers into one, we allow brands to focus their time and resources on achieving tangible results.

As a senior data engineer within the Digital Enablement Center of Excellence, you will play a crucial role in designing, developing, and maintaining the infrastructure and systems needed for efficient and effective data processing, storage, and analysis for our AIM, Email, and HCN Products. Working within an interdisciplinary scrum team, you will collaborate closely with other software engineers, data scientists, software testers, and product owners to ensure the smooth flow of data throughout the organization, thus enabling data-based decision-making and information. This role requires a solid understanding of data architecture and programming skills in Spark and Scala, along with a passion for working with large and complex datasets.

Main responsibilities:

• Spark Development: design, create and maintain scalable and robust data pipelines using Apache Spark.
• Data Pipeline Development: design, implement and optimize data pipelines to extract, transform and load data () from various sources into data storage and processing systems. Leverage Spark's data processing capabilities to perform complex transformations and aggregations on large datasets.
• Data Warehouse Management: develop and manage data warehousing solutions using Spark and Scala to ensure data availability, integrity and security. Design and implement data models, schemas, and indexing strategies to support efficient data recovery and analysis.
• Data Integration: collaborate with cross-functional teams to integrate and consolidate data from multiple sources, including databases, APIs, and external systems. Ensure seamless data integration across different platforms and applications using Spark connectors and integration capabilities.
• Data Quality and Governance: implement data quality controls and validation processes using Spark and Scala to ensure data accuracy, consistency and reliability. Establish and apply data governance standards, data security protocols, and data privacy regulations.
• Performance Optimization: identify and resolve performance issues in Spark applications, data pipelines, and databases. Optimize Spark tasks by adjusting configurations, leveraging caching mechanisms, and applying optimization techniques.
• Collaboration and Communication: collaborate with cross-functional teams, including data scientists and software engineers, to understand their data needs and provide appropriate Spark and Scala solutions. Clearly communicate complex technical concepts and solutions to both technical and non-technical stakeholders.
• Emerging Technologies and Trends: stay abreast of the latest advancements in Spark, Scala, Big Data technologies, cloud platforms, and data management tools. Evaluate and recommend new technologies and approaches that could improve data engineering capabilities.

Degrees and skills:

• Bachelor's or Master's degree in Computer Science, Information Systems or a related field or equivalent experience
• Strong programming skills in Scala and experience with Apache Spark.
• Mastery of SQL and experience working with relational databases
• Familiarity with Big Data technologies such as Hadoop, Kafka or Hive.
• Experience in data modeling, schema design, and ETL processes.
• Understanding of data warehousing concepts and experience with data warehouse solutions
• Knowledge of cloud platforms such as AWS and experience with cloud-based data services (e.g., S3, EMR, Glue).
• Strong problem-solving skills and ability to analyze and resolve complex data-related issues.
• Excellent communication and collaboration skills, with the ability to work effectively in a cross-functional team environment.
• Attention to detail and strong commitment to quality, safety, and data governance.

Senior Data Engineer

IQVIA’s Digital Enablement Center of Excellence are hiring for a Senior Data Engineer to work remotely anywhere in Canada. Our vision is paving the way for unparalleled customer engagement, next-generation omnichannel storytelling, an integrated measurement suite and real-time campaign optimization — all powered through an end-to-end data and platform-agnostic operating system for intelligent healthcare marketing and analytics. By simplifying workflows that used to take days into seconds, and integrating functionality that previously required multiple vendors into one, we allow brands to focus their time and resources on driving real-world outcomes.

As a Senior Data Engineer within the Digital Enablement Center of Excellence, you will play a crucial role in designing, developing, and maintaining the infrastructure and systems required for efficient and effective data processing, storage, and analysis for our AIM, Email and HCN products. Working as part of a cross-functional scrum team, you will collaborate closely with other software engineers, data scientists, software testers and product owners to ensure the smooth flow of data across the organization, enabling data-driven decision-making and insights. This role requires a strong understanding of data architecture and programming skills in Spark and Scala, along with a passion for working with large and complex datasets.

Key Responsibilities:

• Spark Development: Design, build, and maintain scalable and robust data pipelines using Apache Spark.
• Data Pipeline Development: Design, implement, and optimize data pipelines to extract, transform, and load (ETL) data from various sources into data storage and processing systems. Leverage Spark's data processing capabilities to perform complex transformations and aggregations on large datasets.
• Data Warehouse Management: Develop and manage data warehousing solutions using Spark and Scala to ensure the availability, integrity, and security of data. Design and implement data models, schemas, and indexing strategies to support efficient data retrieval and analytics.
• Data Integration: Collaborate with cross-functional teams to integrate and consolidate data from multiple sources, including databases, APIs, and external systems. Ensure seamless data integration across different platforms and applications using Spark's connectors and integration capabilities.
• Data Quality and Governance: Implement data quality checks and validation processes using Spark and Scala to ensure data accuracy, consistency, and reliability. Establish and enforce data governance standards, data security protocols, and data privacy regulations.
• Performance Optimization: Identify and resolve performance issues in Spark applications, data pipelines, and databases. Optimize Spark jobs by fine-tuning configurations, leveraging caching mechanisms, and applying optimization techniques.
• Collaboration and Communication: Collaborate with cross-functional teams, including data scientists, and software engineers, to understand their data requirements and provide appropriate Spark and Scala solutions. Clearly communicate complex technical concepts and solutions to both technical and non-technical stakeholders.
• Emerging Technologies and Trends: Stay up-to-date with the latest advancements in Spark, Scala, big data technologies, cloud platforms, and data management tools. Evaluate and recommend new technologies and approaches that can enhance data engineering capabilities.

Qualifications and Skills:

• Bachelor's or Master's degree in Computer Science, Information Systems, or a related field or equivalent experience
• Strong programming skills in Scala and experience with Apache Spark.
• Proficiency in SQL and experience working with relational databases
• Familiarity with big data technologies such as Hadoop, Kafka, or Hive.
• Experience with data modeling, schema design, and ETL processes.
• Understanding of data warehousing concepts and experience with data warehouse solutions
• Knowledge of cloud platforms such as AWS and experience with cloud-based data services (e.g., S3, EMR, Glue).
• Strong problem-solving skills and ability to analyze and troubleshoot complex data-related issues.
• Excellent communication and collaboration skills, with the ability to work effectively in a cross-functional team environment.
• Attention to detail and a strong commitment to data quality, security, and governance.

IQVIA is a leading global provider of advanced analytics, technology solutions and clinical research services to the life sciences industry. We believe in pushing the boundaries of human science and data science to make the biggest impact possible – to help our customers create a healthier world. Learn more at