Spark Technical Solutions Engineer

While we encourage candidates from the listed locations for this role, we are open to remote candidates from other locations. As a Spark Technical Solutions Engineer, you will provide technical and consulting-related solutions for challenging Spark/ML/AI/Delta/Streaming/Lakehouse issues reported by our customers. You will resolve any issues involving the Databricks unified analytics platform using your extensive technical and customer-facing skills. You will assist our customers in their Databricks journey, providing them with the guidance and expertise necessary to derive value and achieve their strategic goals using our products. Impact you will have: - Performing initial level analysis and troubleshooting issues in Spark using Spark UI metrics, DAG, Event Logs for various customer job slowness issues. - Troubleshoot, resolve, and suggest deep code-level analysis of Spark to address customer issues related to Spark core internals, Spark SQL, Structured Streaming, Delta, Lakehouse, and other Databricks runtime features. - Assist customers in setting up reproducible Spark problems with solutions in Spark SQL, Delta, Memory Management, Performance tuning, Streaming, Data Science, Data Integration areas in Spark. - Participate in the Designated Solutions Engineer program and guide one or two of strategic customer's daily Spark and Cloud issues. - Plan with Account Executives, Customer Success Engineers, and Resident Solution Architects for coordinating customer issues and best practices guidelines. - Participate in screen sharing meetings, answering Slack channel conversations with our internal stakeholders and customers, helping in driving major Spark issues at an individual contributor level. - Build an internal wiki, knowledge base with technical documentation, manuals for the support team and customers, and maintain company documentation and knowledge base articles. - Coordinate with Engineering and Backline Support teams to assist in identifying and reporting product defects. - Participate in weekend and weekday on-call rotation for databricks runtime outages, incident situations, provide escalated level of support for critical customer operational issues, and plan day-to-day activities. - Provide best practices guidance around Spark runtime performance and usage of Spark core libraries and APIs for custom-built solutions developed by Databricks customers. - Be a true advocate for customers. - Contribute to the development of tools and automation initiatives. - Provide front line support on third-party integrations with Databricks environment. - Review Engineering JIRA tickets and proactively notify the support leadership team for follow-up on action items. - Manage assigned Spark cases daily and abide by committed SLA's. - Surpass expectations of the support organization KPIs. - Strengthen your AWS/Azure and Databricks platform expertise through continuous learning and internal training programs. What we look for: - Minimum 6 years of experience in designing, building, testing, and maintaining Python/Java/Scala based applications in typical project delivery and consulting environments. - 3 years of hands-on experience developing two or more of the following: Big Data, Hadoop, Spark, Machine Learning, Artificial Intelligence, Streaming, Kafka, Data Science, ElasticSearch related industry use cases at production scale. Spark experience is mandatory. - Hands-on experience in performance tuning/troubleshooting of Hive and Spark-based applications at a production scale. - Real-time experience in JVM and Memory Management techniques such as garbage collections, heap/thread dump analysis. - Hands-on experience with any SQL-based databases, Data Warehousing/ETL technologies like Informatica, DataStage, Oracle, Teradata, SQL Server, MySQL, and SCD type use-cases is preferred. - Experience with AWS, Azure, or GCP is preferred. - Linux/Unix administration skills are a plus. - Working knowledge in Data Lakes and preferably on the SCD types use cases at a production scale. - Demonstrated analytical and problem-solving skills, particularly those that apply to a "Distributed Big Data Computing" environment. Benefits: - Medical, Dental, and Vision - 401(k) Plan - FSA, HSA, and Commuter Benefit Plans - Equity Awards - Flexible Time Off - Paid Parental Leave - Family Planning - Fitness Reimbursement - Annual Career Development Fund - Home Office/Work Headphones Reimbursement - Employee Assistance Program (EAP) - Business Travel Accident Insurance - Mental Wellness Resources About Databricks: Databricks is the data and AI company. More than 9,000 organizations worldwide, including Comcast, Condé Nast, and over 50% of the Fortune 500, rely on the Databricks Lakehouse Platform to unify their data, analytics, and AI. Databricks is headquartered in San Francisco, with offices around the globe. Founded by the original creators of Apache Spark™, Delta Lake, and MLflow, Databricks is on a mission to help data teams solve the world's toughest problems.