Job expired!
At ASAPP, we're working to resolve complex and challenging issues by creating beautiful and highly beneficial products powered by machine learning. We exploit the power of artificial intelligence to tackle significant problems that possess three common aspects: immense economic size, systemic inefficiency, and large volumes of data. Our innovative and proficient teams that spearhead the development and innovation of our products are based in New York City, San Francisco, and Buenos Aires.
ASAPP's Data Engineering team designs, builds, and maintains our essential core data infrastructure and analytics platform. Precise, easily accessible, and secure data is paramount to our natural language processing customer interaction platform, which interacts with tens of millions of end users in real-time.
We're seeking to employ a Manager data engineer with an aptitude for constructing data infrastructure systems that can accommodate our ever-increasing quantities of data and the demands we have for it. Automation is a vital part of our workflow, so you'll play a role in designing and building highly available data processing pipelines that self-monitor and report anomalies. You'll need to be an expert in ETL processes and be knowledgeable about the ins and outs of various data stores that rapidly and securely serve data to all internal and external stakeholders. Being part of our rapidly expanding data engineering team, this role will also be fundamental in shaping the future of data infrastructure as it aids in enhancing our existing metric-driven development and machine learning capabilities.
Responsibilities:
Design and deploy enhancements to our essential production data pipeline, data warehouses, data systems
Identify data flow patterns and generalizations to automate as much as possible to boost productivity gains
Extend our logging and monitoring processes to identify and resolve anomalies and issues before they become problems
Develop cutting-edge automation and data solutions in Python, Spark, and Flink
Maintain, Manage, Monitor our infrastructure including Kafka, Kubernetes, Spark, Flink, Jenkins, general OLAP and RDBMS databases, S3 objects buckets, permissions
Enhance the efficiency, accuracy, and repeatability of our ETL processes
Understand how to make the trade-offs required to progress without jeopardizing quality
Requirements:
Leading, nurturing, motivating, and inspiring our Global Product Engineering teams
Promoting a healthy and collaborative culture that embodies ASAPP's values
Ensuring your teams deliver exceptional output, and constantly seeking ways to make an outsized impact
Assisting in setting direction for the teams, anticipating strategic and scaling-related challenges through careful long-term planning
Experience with CI/CD (continuous integration and deployment)
Experience with workflow management systems such as Airflow, Oozie, Luigi, and Azkaban
Experience implementing data governance, i.e. access management policies, data retention, IAM, etc.
Comfort in operating in a devops-like capacity working with AWS, Kubernetes, Jenkins, Terraform, etc., thinking about automation, alerting, monitoring, and security and other declarative infrastructure
Desired skills:
A Bachelor’s Degree in a field of science, technology, engineering, or math, or equivalent hands-on experience
Experience in maintaining and managing Kafka (not simply using)
Experience in maintaining and managing OLAP/HA database systems (not merely using)
Familiarity with handling Kubernetes clusters for various jobs, apps, and high throughput
Technical knowledge of data exchange and serialization formats such as Protobuf, Avro, or Thrift
Experience in either deploying and creating Spark Scala and/or Flink applications.
Benefits:
Competitive pay
Stock options
ICICI Lombard General Insurance LTD
On-site meal allowance
Connectivity (mobile phone & internet) allowance
Wellness perks
Mac equipment
Learning & development allowance
Parental leave, including 6 weeks paternity leave
ASAPP is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, colour, religion, gender, gender identity or expression, sexual orientation, national origin, disability, age, or veteran status. If you have a disability and need assistance with our employment application process, please email us at [email protected] to obtain assistance.
