Data Architect

Position Summary At Effectual, Data engineers are tasked with designing, building, and maintaining datasets that can be utilized in data projects. They therefore closely collaborate with both data scientists, data architects, and data governance professionals. Data engineers design, implement and maintain contemporary data pipelines. As a result, we are looking for a Senior Databricks Engineer who is committed to MLOPS as their overriding mission. Typical projects concentrate on areas such as: - Building Modern Data Pipelines - Raw Data Source Intake and ETL Methods - Feature Engineering - Change and Test Models - Deployment of Analytics Why Join Effectual? Effectual is a modern, cloud-first managed and professional services company that collaborates with commercial enterprises and the public sector to mitigate their risk and promote IT modernization. Our team is highly experienced and passionate problem solvers who apply tested methodologies to business challenges across Amazon Web Services and VMware Cloud on AWS. Effectual was named VMware Cloud on AWS Growth Partner of the Year by AWS at re:Invent 2021. The company earned Cloud Service Provider of the Year at the Channel Innovation Awards 2021. Effectual is a validated AWS MSP Partner, holding 200+ AWS Certifications. We have attained the AWS Migration Competency, AWS DevOps Competency, AWS Mobile Competency, AWS SaaS Competency, and AWS Government and AWS Nonprofit Competency designations. Effectual is a member of the AWS Well-Architected and AWS Public Sector Partner Programs as well as the AWS GovCloud (US) and Authority to Operate on AWS Programs. Additionally, Effectual is a VMware Principal Partner in VMware Cloud on AWS. Employees at Effectual benefit from a fun, fast-paced, and inclusive culture. We provide medical, dental, vision, pet, and life insurance along with paid time off, 14 holidays, and a 401k with company match. Consider joining our team today! Essential Duties and Responsibilities The essential duties and responsibilities include, but are not limited to: - Constructing, deploying, testing, and maintaining data architectures and pipelines within the Databricks (including Delta Lake and Unity Catalog) and AWS ecosystems. - Analyzing and organizing raw data. - Building the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Databricks, AWS 'big data' technologies, and SQL. - Developing code and script for data architects, data scientists, and data quality engineers. - Performing data acquisition. - Developing data-set processes. - Identifying ways to enhance data reliability, efficiency, and quality. - Preparing data for prescriptive and predictive modeling. - Automating the data collection and analysis processes, data releasing, and reporting tools. - Building algorithms and prototypes. - Developing analytical tools and programs. - Collaborating with data scientists and architects. Essential Skills and Experience - A minimum of 5 years of experience with Databricks, AWS, SQL, RDS, NoSQL, database design, and programming languages (Scala and/or Python) is required. - Advanced knowledge and technical proficiency with ETL, cloud-based data warehousing, and Apache Spark. - Profound knowledge of AWS-specific services, including Lake Formation, Amazon Aurora, Amazon Data Pipeline, Amazon Athena, Glue, Amazon S3, Amazon DynamoDB, Amazon Relational Database Service (RDS), Amazon Elastic Map Reduce (EMR), Amazon Kinesis, Database Migration Services, and Amazon Redshift. - Familiarity with Common Data Engineering tools like Apache Spark, Apache Airflow, Apache Lite, Apache Kafka, dbt, and great_expectations. - A strong desire to use the framework of the "Analytics Hierarchy of needs”: Collect/Clean/Define & Track/Analyze/Optimize & Predict. - Strong verbal and written communication skills, with the ability to work effectively across internal and external organizations. - Demonstrated ability to think strategically about business, product, and technical challenges. - The Databricks Certified Data Engineer Associate or The Databricks Certified Data Engineer Professional (Professional is preferred).