Position Summary

The Data Engineer creates pathways for transporting data from its source to a data warehouse. These pathways are vital, allowing an organization to access and analyze its data to inform business decisions. Data pipelines transport and transform data according to established business rules or exploratory analyses the business wishes to undertake. The Data Engineer prepares and organizes the data that organizations have accumulated in their databases and other formats.

A Day in the Life of a Machine Learning Engineer

A Data Engineer's day begins with constructing and delivering high-quality data architectures and pipelines to support clients, business analysts, and data scientists. Data Engineers interact with other technical teams to extract, transform, and load [] data from a broad range of data sources. Successful Data Engineers continuously enhance ongoing reporting and processes, as well as automate or simplify self-service support for clients. Data Engineers develop, code, and deploy scripts written in Python, as Python is the primary language for data. All Data Engineers are first and foremost Software Engineers with a grasp of the Software Development Life Cycle (SDLC) process.

Key Duties and Responsibilities

• Build, test, and maintain data architectures provided by the data architect
• Analyze raw and organic data
• Create data systems and pipelines
• Design the necessary infrastructure for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS 'big data' technologies
• Create code and scripts for data architects, data scientists, and data quality engineers
• Procure data
• Identify methods to enhance data reliability, efficiency, and quality
• Create data set processes
• Prepare data for prescriptive and predictive modeling
• Automate the data collection and analysis procedures, data release and reporting tools
• Construct algorithms and prototypes
• Create analytical tools and programs
• Collaborate with data scientists and architects on various projects or initiatives

Requirements

• A Bachelor's or Master's degree in Computer Science, Engineering, or a related subject
• AWS Certified Big Data - Specialty
  • Must have this certification or be willing to obtain it within two weeks of hire
• 5+ Years of proven experience working as a Data Engineer, preferably in a professional services or consulting role
• Strong proficiency in programming languages like Python, Java, or Scala, and expertise in data processing frameworks and libraries (e.g., Spark, Hadoop, SQL, etc.)
• Comprehensive understanding of database systems (relational and NoSQL), data modeling, and data warehousing concepts
• Experience with cloud-based data platforms and services (e.g., AWS, Azure, Google Cloud), including familiarity with relevant tools and technologies (e.g., S3, Redshift, BigQuery, etc.)
• Proficiency in designing and implementing ETL processes and data integration workflows, using tools like Apache Airflow, Informatica, or Talend
• Familiarity with data governance practices, data quality frameworks, and data security principles
• Strong analytical and problem-solving abilities, with proficiency in translating business requirements into technical solutions
• Exceptional communication and collaborations skills, with the capacity to effectively engage with clients and cross-functional teams
• Self-motivated and proactive, with enthusiasm for learning and staying updated with the latest data engineering developments
• Capability to work with ambiguity and translate client wants and needs into implementable stories and epics during a sprint. This means Data Engineers understand the 'agile' progress software delivery
• A solid understanding of the SDLC process
• An understanding of object-oriented programming
• Able to work with minimal guidance
• AWS background
• Solution Engineer mindset

Essential Skills

• AWS Glue
• AWS Lake Formation
• AWS Step Functions
• Amazon Redshift
• Amazon S3

Preferred Skills and Experience

• An inquisitive attitude when addressing problems
• An attitude of 'good is not good enough' for our clients
• Snowflake or Databricks certifications and/or hands-on experience

Company Benefits

Full-time employees are eligible for our employee benefit programs:

• Healthcare (medical, dental, and vision) insurances,
• Short-term disability, long-term disability, and life insurances,
• 401k with Company match
• Accruable Paid time off (PTO) (up to 120 hours over one year)
• Paid time off for major holidays (14 days per year)
• All other offerings are subject to management’s discretion and may change any time.

Salary range for this role is $104,900-$149,800.

CA ID: IT10000584B

"The provided salary ranges are for informational purposes only, and might vary based on factors like experience, qualifications, and geographical location. The final offer will be determined based on the candidate's skills and alignment with the role's requirements."

This job description may not encompass all assigned duties, responsibilities, or aspects of the job. It may be modified any time at the sole discretion of the Employer. Tasks and responsibilities can be adjusted to reasonably accommodate individuals with disabilities. To perform this job successfully, the individuals must possess the skills, aptitudes, and abilities to perform each task proficiently. This document does not create an employment contract, implied or otherwise, other than an “at will” relationship. Effectual Inc. is an Equal Employment Opportunity (EEO) employer and does not discriminate based on any protected classification in its hiring, promoting, or any other job-related opportunities.