Senior Data Engineer (London)

Senior Data Engineer (London) AND Introduction At AND, we expedite the evolution of digital capacities. Essentially, this involves supporting ambitious leaders and companies in the creation of teams, products, processes, and operational structures that they require to fill the digital skills gap within their organization today, to ensure their success tomorrow. Our clients depend on our expertise, flexibility, and skill in tech and business strategy, software development and product management to tackle some of the hardest challenges facing their businesses. We bring onboard thinkers, experimenters, passionate software specialists and inspiring technologists to assist us in solving these challenges. We are bound by pragmatism, purpose, and an ingrained belief that digital products and technology alone cannot transform a business or change the world - every individual involved is of importance. About You As a Senior Data Engineer, you will spend your time enabling clients fulfill their data engineering requirements, predominantly the creation, deployment, and maintenance of data pipelines in a rapidly paced, Agile environment. You'll notably impact the team's Agile Development practices, technical decision-making, and development, by creating valuable inputs and constantly striving to enhance the quality and reliability of our data and processes. You will primarily spend your time assisting in the delivery of client data engineering tasks and fostering the growth of our less experienced technologists, both within AND and for our clients. This will involve tasks such as: - Extracting data from varied data sources - Writing code to assimilate or transform new and existing data - Modifying data to support data scientists and product analysts - Design and build a scalable and extensible data architecture - Execute large-scale batch and real-time processing from diverse sources - Influencing the client's approach to data engineering issues - Collaborate with tech leads on system integrations To do this, it's essential you have the following: - Proficiency in modern and traditional data technologies like MongoDB, PostgreSQL, MySQL/mariaDB, Kafka, Splunk/ELK or other logging and monitoring tools - Strong cloud-native data engineering skills with AWS RDS, Redshift, Kinesis, Glue or Azure CosmosDB, DataFactory, SQL DB - Solid experience quality checking data programs, including functional and performance testing - Minimum three-years experience with Python, R, SQL or Scala - Expertise in Dimensional modeling and Data Warehouse patterns - Familiarity with Agile methods including Scrum, SAFe - Experience with modern software delivery, including CI/CD and DevOps practices - Experience with Big data storage or Data Lakes - Awareness of industry best practices in standards, quality, and continuous improvement in Data Engineering It's helpful if you also have: - Consultancy or professional service experience across multiple sectors - JavaScript / TypeScript / NodeJS experience - Experience with Hadoop, Spark, Redshift, or Parquet - Track record of coaching and fostering the career progression of junior and mid-level developers Joining AND We work hard to ensure AND is unrivaled in the way we deliver work and support our people. Here are three key things to bear in mind about us before you apply: - We're built for people, not 'resources' or 'staff'. Hence, happiness and well-being matter to us, and we detest unnecessary hierarchy and bureaucracy. - We're on a unique, rapidly evolving path that requires focus, patience, and resilience. - Diversity is key for us. To build products that appeal to a wide user base, we need a diverse team to help us build them. So for us, diversity is more than a policy or a word: it's business-critical.