Software Engineer, Backend (Data Engineering / Growth Platform) - Merpay

This position requires English language ability. The job description is available in English only. There is no job description available in Japanese for this position. Introduction: Circulate all forms of value to unleash the potential in all people. "What can I do to help society thrive with the finite resources we have?" The Mercari marketplace app was born in 2013 out of this thought by our founder Shintaro Yamada as he traveled the world. We believe that by circulating all forms of value, not just physical things and money, we can create opportunities for anyone to realize their dreams and contribute to society and the people around them. Mercari aims to use technology to connect people all over the world and create a world where anyone can unleash their potential. For more information about Mercari Group’s mission, see Mercari's Culture Doc. Equal Opportunity Hiring: Here at Mercari, we work to realize a world in which no one’s potential is limited by their background and everyone has the opportunity to freely create value. We also firmly believe that a mindset of Diversity & Inclusion is essential for us to achieve our mission. This, of course, extends to our hiring practices as well. Mercari is committed to eliminating discrimination based on age, gender, sexual orientation, race, religion, physical disability, and other such factors so that anyone who shares our mission and values can join us, regardless of their background. For more details, please read our D&I Statement. Team’s Mission / Work Responsibilities Team mission: Develop and operate marketing tools, coupons, point rewarding systems, etc. used for the growth of not limited to Merpay but for the entire Mercari Group in the Japan Region (Mercari, Mercoin, Shops, etc.). In line with our group mission to "Circulate all forms of value to unleash the potential in all people", we aim to provide a platform for Growth teams to launch any type of campaign or growth experiment in the shortest possible time with maximum flexibility, seamlessly integrating our businesses, collaborating with other engineering teams, Data Analysts, and Machine Learning experts to implement smart solutions to large-scale problems. (Learn more about Mercari's mission and values here.) As a Backend engineer with a focus on Data Engineering, specific work responsibilities in the Marketing & Growth Domain include the following: Design, develop, and operate data infrastructure/data pipelines to support large-scale campaigns; both in streaming-based and for large batches. This work includes the development of an ecosystem enabling data collection, Data Lake, DWH, use of collected data, and other processes required by data utilization. Integrating internal and external systems. For example, in-house microservices and external marketing tools. Interact with Data Platform engineers to apply and enhance reusable solutions. Provide solutions for Data Analysts and ML experts for efficiently handling data related to the Marketing domain, including billions of data points per day generated by millions of active users. QA automation for quality improvement. Assist with operational tasks for systems, services, and campaigns. Work Responsibilities: Collecting data from microservices. Developing batch-based and streaming-based data pipelines. Constructing Data Lakes and DWH for storing collected data. Managing partial permissions and resources on DWH. Manage data utilization and implement cost-reduction measures. API/interface design for microservices using Protocol Buffers and gRPC. Improving services and measuring effectiveness using quantitative and qualitative analysis on user behavior logs, etc. Design architecture and select middleware based on application requirements. Microservice performance tuning, monitoring, and alert handling. Tools Used/Dev Environment: Programming languages: Go, Python, Java, Scala. Infrastructure: Google Cloud Platform (Compute Engine, Kubernetes Engine, Dataflow, Pub/Sub, Cloud Storage, BigQuery, Spanner, Cloud SQL, Cloud Logging etc.). Distributed processing: Apache Flink / Apache Spark. Workflow: Apache Airflow. BI tools: Looker / Data Studio. Configuration management tools: Terraform. Monitoring: Stackdriver Monitoring / Datadog / PagerDuty. CI/CD: Circle CI / Spinnaker. Knowledge Tool: Crowi / Confluence. Other: GitHub / Slack. Development Method: Agile (Scrum, Kanban, etc.) Unique and Bold Challenges: Growth Platform With campaigns distributing millions of notifications and rewards per day, it is necessary to process billions of data points from many different services (internal and external). Implement product features that increase the customer engagement and consequently contribute to the company's GMV. Collaborate with Product Managers, ML engineers, Backend Engineers, SRE, and Data Platform specialists to build solutions that scale well: robust, fast, cost-effective, and secure. Required Qualifications: Shared understanding and belief in our company’s mission and values. A degree in computer science or a related field, or else five or more years of practical experience in software development. Experience designing, developing, and operating large-scale services and/or distributed systems. At least two of the following: Experience developing in the cloud with AWS, GCP, etc. Experience developing systems utilizing container technologies such as Kubernetes. Development experience using message queues like Cloud Pub/Sub and Apache Kafka. Experience in data processing development using distributed processing frameworks like Apache Flink and Apache Spark. Experience with ETL systems using workflow engines like Airflow and Digdag. Experience designing application logs for large-scale services. Preferred Qualifications: Project management or team leadership experience. Communication skills to collaborate with stakeholders and drive product development. Experience in development and operations using Microservices architecture. Knowledge and experience in databases (RDBMS/NoSQL), networking, Linux, monitoring systems, logging, operations, and SLO/SLA. Experience in development in the cloud, particularly with GCP or AWS. Screening Criteria: Has a strong, shared belief in Mercari and Merpay’s missions and values. Capable of going bold and possessing a strong sense of responsibility in regards to their work. Possesses excellent communication skills which can push organizations and projects forward. Able to collect information independently and identify the essence of issues to solve problems through engineering. Thinks independently, acts independently, and leads operations with All for One mentality for the sake of the team while holding on to a professional mindset. (See this page for details.) Language Requirements: Either of the following is required: a. English: Independent (CEFR - B2) / Japanese: Basic (CEFR - A2). b. English: Basic (CEFR - A2) / Japanese: Independent (CEFR - B2). For details about CEFR, see here. Related Articles: https://mercan.mercari.com/en/articles/37585/. https://mercan.mercari.com/en/articles/38060/. Working Conditions Employment Status: Full-time. Probationary period: First 3 months after joining the company. (During this period your contract conditions will be the same as that of a permanent employee.) Office: Roppongi. Smoking is prohibited within our offices. Mercari has introduced a work style policy called “Your Choice.” Each member is free to choose whether they want to work in the office or work fully remote. *Exceptions made for certain kinds of work. Work Hours: Full flextime (no “core time” or “flex time”). *Does not apply to all positions. Holidays: Two days off per week (as well as national holidays, New Year's break, etc.). Paid leave, congratulatory and bereavement leave, relax days, sick leave. Salary: Annual salary paid in 12 monthly installments (including fixed overtime allowance). Based on skills, experience, and abilities. Reviewed twice a year. Benefits: Complete health and social insurance. Incentive program. Support systems, including those that benefit the employee’s family members. *See this page for details. Support: Relocation support. Language learning support. Translation/interpretation support. *See this page for details. Media Corporate Sites: Mercari, Inc. Merpay, Inc. [Japanese]. Souzoh, Inc. Mercoin, Inc. Mercari US. Owned Media: Mercari Careers. Mercari Engineering Portal. Mercan - Introduces the people that make Mercari. Mercari US Careers. Mercari US Blog.