Position Summary

The Data Engineer is responsible for constructing pipelines that transport data from a data source to a data warehouse. These pipelines are essential as they facilitiate for an organization to access and analyze its data, which can be then used to inform business decisions. Data pipelines not only transport, but also transform data according to established business rules or a line of exploratory analysis. The duty of a Data Engineer is to prepare and organize the data that has been accumulated in the databases and other formats of the organization.

A Glimpse into the Daily Routine of a Data Software Visualization Engineer

A typical day of a Data Engineer includes creating and delivering top-quality data architectures and pipelines to support clients, business analysts, and data scientists. Additionally, a Data Engineer collaborates with other technological teams to extract, transform, and load [ETL] data from a diverse array of data sources. Skillful Data Engineers constantly improve ongoing reporting and processes and automate or simplify self-service for their clients. They also write, develop, and implement scripts in Python programming language, as Python is predominantly used in the realm of data. At their core, all Data Engineers are Software Engineers with a profound understanding of the Software Development Life Cycle (SDLC) process.

Essential Duties and Responsibilities

• Construct, test, and maintain data architectures
• Analyze raw and organic data
• Develop data systems and pipelines
• Create the infrastructure needed for optimal extraction, transformation, and loading of data from diverse sources using SQL and AWS 'big data' technologies
• Compose code and scripts for data architects, data scientists, and data quality engineers
• Collect data
• Identify ways to improve data reliability, efficiency, and quality
• Create data-set processes
• Prepare data for prescriptive and predictive modeling
• Automate processes of data collection and analysis, data releasing, and reporting tools
• Develop algorithms and prototypes
• Create analytical tools and programs
• Cooperate with data scientists and architects on various projects and efforts

Qualifications

• Bachelor's or master's degree in computer science, Engineering, or a related field
• AWS Certified Big Data - Specialty
  • Required to be obtained within the first two weeks of employment
• Prior experience as a Data Engineer, preferably in a professional services or consulting environment
• Proficient in programming languages such as Python, Java, or Scala, with expertise in data processing frameworks and libraries (e.g., Spark, Hadoop, SQL, etc.)
• Deep knowledge of database systems (both relational and NoSQL), data modeling, and data warehousing concepts
• Experience with cloud-based data platforms and services (e.g., AWS, Azure, Google Cloud) and familiarity with relevant tools and technologies (e.g., S3, Redshift, BigQuery, etc.)
• Proficiency in designing and executing ETL processes and data integration workflows using tools like Apache Airflow, Informatica, or Talend
• Familiar with data governance practices, data quality frameworks, and data security principles
• Strong analytical and problem-solving abilities, able to translate business requirements into technical solutions
• Excellent communication and collaboration skills, able to effectively work with clients and cross-functional teams
• Self-motivated and proactive, with a passion for continuous learning and staying updated on new trends and advancements in the field of data engineering
• Ability to thrive in ambiguous situations and convert client wants and needs into actionable stories and epics which can be executed during a sprint. This implies that Data Engineers thoroughly understand the 'agile' process of software delivery
• Solid understanding of the SDLC process
• A understanding of object-oriented programming
• Requires minimal direction
• Background in AWS
• A solution-oriented mindset

Must Have Skills

• Amazon QuickSight
• Amazon API Gateway
• Amazon Redshift

Nice-to-Have Skills and Experience

• An inquisitive nature and curiosity in problem-solving
• Desire to consistently exceed client expectations
• Experience or certification with Snowflake or Databricks

Company Offered Benefits

Our full-time employees are eligible to participate in our benefits programs:

• Medical, dental, and vision health insurances,
• Short term disability, long term disability, and life insurances,
• 401k with company match
• 120 hours of paid time off (PTO) that accrue over one year
• Paid time off for major holidays (14 days per year)
• These benefits, along with any other employee benefit offerings, are subject to management's discretion and may be altered at any time.

The salary range for this position is between $115,000 and $172,000.

"The indicated salary ranges are for informational purposes only and may vary depending on several factors such as experience, qualifications, and geographic location. The final salary offered will be determined based on the candidate's skills and how they align with the role's requirements."

This job description may not include all assigned duties, responsibilities, or aspects of the job described and may be revised at any time at the sole discretion of the employer. Duties and responsibilities are subject to possible modification to reasonably accommodate individuals with disabilities. To perform this job successfully, the incumbents will possess the skills, aptitudes, and abilities to perform each duty proficiently. This document does not create an employment contract, implied or otherwise, other than an “at will” relationship. Effectual Inc. is an EEO employer and does not discriminate on the basis of any protected classification in its hiring, promoting, or any other job-related opportunity.