Senior Data Engineer

At Lyft, our mission is to enhance people's lives with the finest transportation the world has to offer. In order to achieve this, we begin within our own ranks by fostering an environment that is open, inclusive, and representative of diverse backgrounds. The Real Time Supply Management team at Lyft is tasked with overseeing our array of real-time driver earnings products. Among their responsibilities is the promotion of key corporate objectives pertaining to financial metrics and overall marketplace performance. The team is involved with the development and refinement of algorithms which facilitate the handling of our real-time earnings products, the creation of novel earnings products, as well as managing the financial performance of said products in order to guarantee business growth. As a Data Engineer at Lyft, you'll be incorporated into an early stage team focusing on data transfer, collection, storage, and deploying services which ensure that data is a fundamental component at Lyft. We're actively seeking a Data Engineer to establish a scalable data platform. Your role will include ownership of our primary data pipeline that powers Lyft's top line metrics; You'll also apply your expertise in data to assist in refining data models across several facets of our data stack; You'll help to design and execute broad scale data pipelines which will support Lyft's burgeoning data processing and analytics needs. Your work will provide access to business and user behaviour insights, harnessing vast amounts of Lyft data to serve several teams including Analytics, Data Science, Marketplace, and numerous others. Responsibilities: - Collaborate with teams to define requirements and ensure project goals are aligned - Formulate and manage the roadmap for Lyft's Central Data team - Offer technical leadership to the team to guarantee projects are completed within prescribed scope, time, and quality constraints - Promote best practices within the team through tech talks, demos, show 'n tells etc - Proactively devise projects to improve customer experience and satisfaction by addressing prolonged problems or by developing key user facing features - Ownership of the core company data pipeline, accountable for scaling up data processing flow to meet the swift data growth at Lyft - Improve data model and data schema based on business and engineering needs - Put into place systems monitoring data quality and consistency - Create tools to support self-service data pipeline management (ETL) - Design and establish a framework to strengthen Lyft's data infrastructure, in compliance with regulatory requirementsSQL and MapReduce job tuning to enhance data processing performance Experience: - More than 10 years of applicable professional experience - Experience in mentoring junior engineers through reviewing code and breaking down, scoping, and sequencing project tasks - Familiarity with Hadoop (or similar) Ecosystem (MapReduce, Yarn, HDFS, Hive, Spark, Presto, Pig, HBase, Parquet) - Skilled in at least one SQL language (MySQL, PostgreSQL, SqlServer, Oracle) - Good understanding of SQL Engine and ability to perform advanced performance tuning - Strong scripting language skills (Python, Ruby, Bash) - Over one year's experience with workflow management tools (Airflow, Oozie, Azkaban, UC4) - Comfortable collaborating directly with data analytics to align Lyft's business objectives with data engineering Benefits: - Expanded health and dental cover options, as well as life insurance and disability benefits - Mental health benefits - Family-making benefits - Access to a Health Care Savings Account - In addition to provincially observed holidays, team members get 15 days paid time off, with an extra day added for each completed year of service - 4 Floating Holidays per calendar year (pro-rata adjustment based on date of hire) - 10 alotted paid sick day a year irrespective of province - 18 weeks of paid parental leave. All parents (biological, adoptive, and foster) are eligible Lyft proudly pursues and employs a diverse workforce. Lyft is committed to the principle that all individuals have the right to equal employment opportunities and does not discriminate on the grounds of race, ancestry, place of origin, colour, ethnic origin, citizenship, faith, sex, sexual orientation, gender identity, gender expression, age, marital status, family status, disability, pardoned record of offences or any other basis as protected by applicable law or company policy. Lyft also actively promotes a healthy and safe working environment and strictly prohibits any form of harassment. Accommodation for individuals with disabilities will be provided on request in compliance with applicable laws both throughout the application process and the hiring process. Please reach out to your recruiter if you wish to request such accommodation. This role will require in-office presence on a hybrid schedule following the establishment of a Lyft office in Toronto. Team members will be expected to be in the office for 3 days a week, specifically on Mondays, Thursdays, and a specified third day for each team. Additionally, these roles will be allowed the flexibility of working remotely for up to 4 weeks a year.