Staff Data Engineer

NVIDIA is looking to hire senior distributed system engineers with a data engineering focus to develop and scale its AI and deep learning platforms. Our team is developing a software 2.0 developer platform with an emphasis on datasets for AI application development. Together, we'll enhance NVIDIA's capability to create and implement leading solutions for a wide array of AI-based applications like autonomous vehicles, healthcare, virtual reality, graphics engines, and visual computing. Join us to help bring autonomous vehicles to life with our NVIDIA partners! What you'll be doing: - Architect and build scalable, distributed systems for improvement, computation, and data pipelines to power our centralized data platform, the IT Data Lake. - Design and construct petabyte-sized, scalable data lakes for structured and unstructured data query interfaces and microservices to ingest, index, mine, transform, and manage large datasets. - Develop cloud cost and usage data patterns to crawl, collect, and transform terabytes of data daily. - Enable data models and views across terabytes of data that can be utilized by analytical tools like PowerBI and create PowerBI analytics for finance reporting. - Develop and implement support for versioned, traceable, and immutable datasets in a data lake in a distributed, scalable way. - Optimize efficient and insightful data selection – a crucial component for successful machine learning! - Actively write high-quality code with good design and architecture, fully tested and peer-reviewed. - Collaborate with various product and engineering teams to understand their data and computing requirements (Software, Hardware, Automobile, AI), integrating their innovations and algorithms into our production systems. - Automate everything for measuring, testing, updating, monitoring, and alerting the data platform. What we need to see: - Bachelors (or equivalent experience) or Masters in Computer Architecture, Computer Science, or a related data-intensive Engineering Degree. - 8+ years of proven experience in Data Engineering, working on designing and developing software with Big Data, Data Lake/ Lake House ecosystem, Data Analytics, backend microservices architecture, and varied data types at scale. - In-depth experience in creating ETL pipelines using Databricks, Spark, Python, SQL, Scala, Kafka, Presto, Parquet, Streaming, events, bots, AWS/cloud ecosystem. - Proficiency in developing Micro Services and using AWS frameworks such as SQS, Stream, Kubernetes, EC2, S3, Lambda, etc. - Experience with data pipelines, analysis, visualization tools like Elastic stack, Logstash, Kibana, Kafka, Grafana, Splunk, Pandas, Message brokers, Data modeling. - Expertise in Data Lakehouse architecture and end-to-end Databricks techniques, including Data Science components. - Experience with data lifecycle from Data Ingestion, Data Transformation, to Data Consumption layer. Familiar with API and its applicability. - Knowledge of Cloud solutions like Kendra, SageMaker, Auto-ML, Big Query, RedShift, Glue, Athena. Ways to stand out from the crowd: - Understanding and experience with Cost and Usages analytics are a plus. - Expertise in Spark, Parquet, streaming, events, Kafka, telemetry, MapReduce, Hadoop, Hive, Presto, Spark, data query methods, and dashboarding. - Have implemented Enterprise use cases like CMDB, Governance, time series classification, telemetry anomaly detection, logs, and real-time data ingestion through APIs. - Experience with structured data like Avro, Parquet, Protobuf, Thrift, and concepts like schema evolution. - Working knowledge of Amazon Web Services, Kubernetes, Docker is a plus. NVIDIA is regarded as one of the tech world’s most sought-after employers. We have some of the most innovative and hardworking individuals on the planet working for us. If you're creative and autonomous, we want to hear from you! The base salary range is 160,000 USD - 304,750 USD. Your base salary will be determined according to your location, experience, and the pay of employees in similar positions. You will also be eligible for equity and benefits.