Senior Systems Software Engineer - Performance - Accelerated Apache Spark

We are seeking experienced System Software Engineers who are skilled in Apache Spark to join our team, to help spearhead innovation for accelerated data processing on GPUs. As Nvidia is leading the way in accelerated computing, we are crafting the next generation of data processing ecosystems. Data scientists dedicate a significant amount of time delving into data and iterating over machine learning (ML) experiments. Every time-consuming compute hour required to sift through datasets, extract features, and fit ML algorithms hampers an efficient business workflow. Nvidia believes that data science workflows can hugely benefit from being faster, allowing data scientists to explore numerous more significant datasets to advance towards their business goals more swiftly and efficiently. At NVIDIA, we are enthusiastic about working on challenging problems that have a real impact. You will need to have prior experience working with Apache Spark applications, implemented big data applications for diverse customers, programming skills, as well as a familiarity with open-source big data processing frameworks. You will collaborate with an engineering team to speed up Apache Spark with GPUs using CUDA and open-source libraries. This is a critical investment for Nvidia. The code is being adopted by multiple cloud service providers and Apache Spark distributions. As Nvidia leads in providing the future data center with a CPU-GPU superchip platform, accelerating data processing will become a necessity for all enterprises. What you'll be doing: - Scaling Apache Spark acceleration for high-end GPUs and with the new Grace Hopper superchip platform. - Collaborating with select customers and partners to make Apache Spark applications faster with GPUs across on-premises and cloud environments. - Benchmarking and optimizing Apache Spark workloads on GPUs for accelerated performance. - Constructing tools to analyze data processing workloads to identify opportunities for acceleration and cost savings. - Assisting in defining a reference architecture for the next-generation computing cloud. - Working with major cloud service providers and Spark vendors globally to enable accelerated computing for data processing use cases. What we need to see: - 8+ years of experience in software development, with the majority in data processing. - 3+ years of hands-on experience with Apache Spark application design and development. - A BS/MS/PhD in computer science or a related field. - Familiarity with the modern data open source ecosystem (Apache Hadoop, Flink, Presto, Hive, Kafka, TensorFlow, PyTorch etc.) - Profound understanding of performance optimization in low-level environments. - Experience in writing high-performance distributed software at scale. - Experience working on public and private cloud platforms. - Prior experience supporting enterprise customers. - Knowledge of SQL, Python, and Scala/Java. - Excellent communication, presentation, and technical explanation skills. Ways to stand out from the crowd: - Familiar with RAPIDS and UCX. - Experience with performance optimizations in CSPs for big data use cases. - An Understanding and expertise in the hardware configuration options pertaining to performance and cost - Experience with working and developing on GPUs. We are broadly considered one of the technology world’s most desirable employers, and as a result, some of the most innovative and hardworking people in the world work for us. If you're passionate, creative, and driven, we'd love to have you on the team. With competitive salaries and a comprehensive benefits package, we are considered one of the technology world’s most attractive employers. We have some of the most creative and hardworking people globally working for us and, due to exceptional growth, our exclusive engineering teams are rapidly expanding. If you're a creative and autonomous engineer with a real passion for technology, we want to hear from you. The base salary range is 216,000 USD - 333,500 USD. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. In addition to your salary, you will also be eligible for equity and benefits. NVIDIA accepts applications on an ongoing basis.