Senior Technical Solution Engineer, Apache Spark™ Opportunity at Databricks

Job Schedule: Wednesday through Sunday

Job Code: P-993

Are you an expert in Apache Spark™ with a passion for technical consulting? Join Databricks as a Senior Technical Solution Engineer, Apache Spark™ and help resolve challenging Apache Spark™/ML/AI/Delta/Streaming/Lakehouse issues for our esteemed customers while guiding them on their Databricks journey.

Key Responsibilities

As a Senior Technical Solution Engineer, you will:

• Perform initial level analysis and troubleshooting of Apache Spark™ issues using UI metrics, DAG, and Event Logs to resolve job slowness problems reported by customers.
• Address deep code-level issues in Apache Spark™ core internals, SQL, Structured Streaming, Delta, Lakehouse, and other Databricks runtime features.
• Assist customers in setting up reproducible Apache Spark™ solutions, including SQL, Delta, Memory Management, Performance tuning, Streaming, Data Science, and Data Integration.
• Participate in the Designated Solutions Engineer program to support strategic customer's Apache Spark™ and Cloud issues.
• Collaborate with Account Executives, Customer Success Engineers, and Resident Solution Architects to coordinate on customer issues and best practice guidelines.
• Engage in screen-sharing meetings, answer Slack channel conversations, and drive significant Apache Spark™ issues at an individual level.
• Create and maintain an internal wiki and knowledge base with technical documentation and manuals for support teams and customers.
• Coordinate with Engineering and Backline Support teams to report product defects.
• Participate in weekend and weekday on-call rotation, handling escalations during runtime outages, and providing critical customer operational support.

Required Expertise

The ideal candidate will have:

• 3+ years of hands-on experience with Big Data, Hadoop, Apache Spark™, Machine Learning, Artificial Intelligence, Streaming, Kafka, and Data Science at a production scale.
• Mandatory Spark experience with expertise in performance tuning and troubleshooting Hive and Apache Spark™-based applications.
• Real-time experience in JVM and Memory Management, including Garbage Collection, Heap/Thread Dump Analysis.
• Experience with SQL databases, Data Warehousing, and ETL technologies like Informatica, DataStage, Oracle, Teradata, SQL Server, and MySQL.
• Familiarity with cloud platforms like AWS, Azure, or GCP.

Employee Benefits

We offer a comprehensive benefits package including:

• Private Medical and Dental Insurance
• Life Insurance
• Meal Allowance
• Equity Awards
• Paid Parental Leave
• Fitness Reimbursement
• Annual Career Development Fund
• Home Office/Work Headphones Reimbursement
• Childcare Reimbursement
• Business Travel Accident Insurance
• Mental Wellness Resources

#LI-DC #LI-Hybrid

About Databricks

Databricks is a global leader in data and AI. Our Data Intelligence Platform helps over 10,000 organizations, including Comcast, Condé Nast, and Grammarly, to unify and democratize data, analytics, and AI. Databricks, headquartered in San Francisco with offices worldwide, was founded by the original creators of Lakehouse, Apache Spark™, Delta Lake, and MLflow. Stay connected with us on , , and .

Our Commitment to Diversity and Inclusion

At Databricks, we embrace diversity and inclusion. We ensure fair hiring practices and consider all individuals for employment regardless of age, color, disability, ethnicity, family or marital status, gender identity or expression, language, national origin, physical and mental ability, political affiliation, race, religion, sexual orientation, socio-economic status, veteran status, or any other protected characteristics.