Senior Research Scientist, Multimodal Foundation Models and Robotics Opportunity at NVIDIA

Are you an experienced Research Scientist with a focus on Multimodal Foundation Models and Robotics? NVIDIA is actively seeking a talented individual to join the Generalist Embodied Agent Research (GEAR) group. Our mission is to develop general-purpose embodied agents capable of mastering complex skills in both virtual and physical realms.

About the Role

As part of an innovative and collaborative research team, you will focus on creating humanoid robot foundation models and systems. Our team has previously worked on high-impact projects including Eureka, VIMA, Voyager, MineDojo, MimicPlay, Prismer, and our latest milestone, Project GR00T.

Your work will significantly influence our groundbreaking research projects and product development roadmaps.

Key Responsibilities

• Design and implement advanced AI algorithms and models for humanoid robots and embodied agents.
• Develop large-scale AI training and inference methods for foundation models.
• Optimize and deploy AI models in physical simulations and on robot hardware.
• Collaborate with teams across NVIDIA to transition research into products and services.

Qualifications

General Requirements

• Ph.D. in Computer Science/Engineering, Electrical Engineering, or equivalent research experience.
• 5 years of relevant work or research experience in at least one of the following fields:
  • Multimodal Foundation Models
  • Hands-on training experience and publications in LLMs, large vision-language models, video generative models, diffusion algorithms, or action-based transformers.
• Proficiency in rapid prototyping and model training frameworks like PyTorch, Jax, or TensorFlow. Python is required; C++ and CUDA are a plus.

Robotics-Specific Requirements

• Experience and publications in robot learning techniques such as reinforcement learning, imitation learning, and classical control methods.
• Skilled in Python, C++, ROS, and machine learning frameworks like PyTorch.
• Deep understanding of robot kinematics, dynamics, and sensors.
• Ability to safely operate robot hardware, lab equipment, and tools.
• Knowledge of control methods, including PID, model predictive control, and whole-body control.
• Experience with physics simulation frameworks like MuJoCo and Isaac Sim.
• Hands-on experience in robot hardware design and building.

Why Join NVIDIA?

NVIDIA is consistently recognized as one of the world's most desirable technology employers. Our forward-thinking and dynamic team is among the most productive globally. This is your chance to be part of a trailblazing effort to develop general-purpose robots and embodied agents.

Compensation and Benefits

We offer a base salary range of 180,000 USD to 345,000 USD, determined by your location, experience, and similar positions' pay levels. You will also be eligible for equity and comprehensive benefits.

Commitment to Diversity

NVIDIA is dedicated to fostering a diverse workforce and is proud to be an equal opportunity employer. We highly value diversity and do not discriminate in our hiring and promotion practices based on race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status, or any other protected characteristic.

Apply now and join NVIDIA's mission to revolutionize the world of robotics and AI!