Senior Data Engineer Job Opportunity at Auctane

At Auctane, we share a profound passion for assisting sellers in delivering promises to consumers. Our mission is to empower commerce through unrivaled delivery solutions, enabling businesses to meet soaring customer expectations and making the world more accessible to consumers globally. Auctane brands facilitate the annual delivery of billions of products worth over $200 billion to customers worldwide. And we're just beginning.

About Auctane

We are a dedicated team of professionals specializing in shipping and software. Our passion lies in helping merchants bring their ideas and innovations to the global market. Our family includes brands such as ShipStation, ShipWorks, ShipEngine, ShippingEasy, Stamps, Endicia, Metapack, Shipsi, GlobalPost, and Packlink. We proudly partner with industry leaders including Amazon, UPS, USPS, eBay, BigCommerce, Shopify, WooCommerce, and Walmart.

About the Engineering Tech and Data Team

Our team’s top priority is to offer world-class production software to our customers. We ensure a fast and reliable system while providing our engineering teams with the necessary tools to deploy, monitor, and maintain our production systems.

About the Role

Data is pivotal to Auctane’s strategy, operating at scale and pace with cutting-edge architecture patterns and technologies. Our massive data set, which processes thousands of events per second, continues to grow rapidly. We're continuously enhancing our data platform and engineering stack to support growth, enable innovative solutions, and provide top-tier service to our customers.

We foster a flat and open engineering culture where data and evidence trump opinions and hierarchy. Auctane thrives on autonomy, cross-functional teamwork, energy, and a shared passion for the business and our customers.

Responsibilities

• Design, build, and manage our data lake and analytics solutions using AWS data technologies like S3, Athena, Lambda, Kinesis, and ECS.
• Utilize state-of-the-art technologies such as Airflow, dbt, and Spark for data processing.
• Develop frameworks and solutions for acquiring, processing, monitoring, and extracting value from extensive datasets.
• Support Data Analysts and Data Scientists with automation, tooling, data pipelines, and data engineering expertise.
• Deliver reliable software and data pipelines using best practices like automation, version control, continuous integration/delivery, testing, and security.
• Implement and enforce automated data security and data governance best practices.
• Mentor junior colleagues and receive mentorship from senior team members.

Qualifications

• Background in Software Engineering.
• Experience with robust, automated, and reliable data pipelines in Python and SQL.
• Knowledge of data processing frameworks such as Pandas, Polars, dbt, or Spark.
• Experience with streaming data processing.
• Proficiency in AWS, Azure, or Google Cloud.
• Passion for automation within a continuous integration/delivery environment.
• Familiarity with data orchestration solutions like Airflow, Oozie, Luigi, or Prefect.
• Understanding of both relational and non-relational database design.
• Experience designing distributed systems and understanding their trade-offs.
• Proficiency in software engineering best practices, source control systems, automated deployment pipelines like Jenkins, and DevOps tools like Terraform.

Preferred Skills

• Production experience managing large datasets.
• Experience with business intelligence and analytics tools (Tableau, Looker, Power BI).
• Understanding of PII issues and data security considerations.
• Experience with big data cloud technologies like EMR, Athena (Trino), Glue, BigQuery, Dataproc, Dataflow.
• MLOps expertise.
• Enthusiasm for adopting new technologies (Apache Iceberg, Dagster, dbt, GreatExpectations, Prefect, Snowflake).
• Readiness to attend conferences, webinars, and meet-ups and share knowledge.

What We Offer

• 💷 Clear compensation aligned with your salary expectations.
• 💻 All the necessary equipment for seamless, secure work from our London