Staff Site Reliability Engineer (Hadoop Platform, Hive, Spark, Python)

Job Title: Staff Site Reliability Engineer (Hadoop Platform, Hive, Spark, Python)

Company Description

Visa is a global leader in digital payments, facilitating over 259 billion transactions annually across more than 200 countries and territories. Our mission is to connect the world through the most innovative, reliable, and secure payment network, enhancing the capability of individuals, businesses, and economies to prosper. At Visa, we are committed to fostering a universal digital economy that uplifts everyone, everywhere. Join Visa and be part of a team that values innovation and purpose. Experience what it's like to make an impact at one of the world's most influential financial institutions – Life at Visa awaits you.

Job Description

We are currently seeking a Staff Site Reliability Engineer to provide comprehensive support and maintenance on the Hadoop platform and its associated technologies, such as Hive, SPARK, and HDFS. This role encompasses a range of responsibilities:

Responsibilities:

• Single Window Support: Apply your deep knowledge of Hadoop and related tools to perform root cause analysis for platform and user configuration/code issues.
• System Configuration: Recommend system adjustments to the DAP platform engineering teams based on user logs and system activities.
• Performance Tuning: Guide team members in writing efficient queries and apply advanced performance tuning and optimization techniques for big data solutions.
• Issue Resolution Across Tech Teams: Troubleshoot and resolve intricate technical challenges, coordinating with appropriate Tech/Data platform teams for solutions.
• Reliability Engineering: Develop reports to track performance, resolution metrics, and proactive issue detection.
• Office Hours and Liaising: Manage calls across various time zones to ensure timely delivery to clients.
• Knowledge Cataloging and Sharing: Share your knowledge and expertise with peers through Wikis and other communication channels, focusing on issues affecting multiple users.
• Develop Standards: Create standard configurations for diverse VCA workloads to enhance cluster health and operational efficiency.

This is a hybrid role requiring both remote and on-site presence, with expectations to be in the office 2-3 days per week depending on business needs.

Qualifications

Basic Qualifications:

• 7+ years relevant professional experience with a Bachelor’s degree or advanced degrees with varying experience levels.

Preferred Qualifications:

• Extensive experience in developing data pipelines using PySpark, Hive, and Airflow.
• Proven ability to troubleshoot and debug within Hadoop environments.
• Practical experience in performance tuning on Hadoop platforms and Python development.
• Familiarity with Hadoop ecosystem architecture and components such as Zookeeper, HDFS, Yarn, Hive, and Spark.
• Experience with security tools like Kerberos and Ranger, and an understanding of core system functions including Linux operations, networking, and memory storage.
• Strong communication skills and adeptness in agile work environments.

Additional Information:

Visa is an Equal Opportunity Employer. We provide equal employment opportunities to all applicants without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability, protected veteran status, or any other legally protected characteristics. Candidates with criminal histories are also considered in a manner consistent with all applicable laws and regulations.