Open Position: Software Engineer - vLLM (Machine Learning)

Location: Somerville, Massachusetts

About Neural Magic

Neural Magic, a Series A startup located in Somerville, MA, is revolutionizing the AI landscape with the backing of esteemed investors such as Andreessen Horowitz, NEA, Pillar, VMware, Verizon Ventures, Comcast Ventures, and Amdocs. We are dedicated to an open-source AI future, striving to empower enterprises with advanced LLM and VLLM capabilities. As a pioneer in AI acceleration and operational simplicity for GenAI deployments, we lead the development of the vLLM project and innovative model quantization and sparsification techniques.

Our Mission

Our mission is to democratize AI by bringing the power of open-source LLMs and vLLM to enterprises worldwide.

Your Role

As a Software Engineer focused on vLLM, you will drive innovation by collaborating with our team to address critical challenges in model performance and efficiency. Your contributions in machine learning and high-performance computing will be instrumental in advancing our software platform and shaping the future of AI deployment and utilization.

Responsibilities

• Develop robust Python and C++ code, focusing on vLLM systems, high-performance machine learning primitives, performance analysis, modeling, and numerical methods.
• Review code and contribute to developing best practices for the team.
• Work closely with machine learning teams to optimize neural network performance in the engine.

Requirements

• Extensive experience in writing high-performance code for GPUs, with a deep understanding of GPU hardware.
• BS, MS, or PhD in Computer Science.
• Experience with mathematical software, particularly linear algebra or signal processing.
• Proficiency in modern C++, Python, and Pytorch.
• Expertise in tensor computations and deep neural network models and techniques.
• Ability to work independently and learn quickly.
• Strong communication skills for interaction with both technical and non-technical team members.
• A strong sense of project ownership and personal responsibility.
• A genuine interest in continuous learning.

Benefits

• Competitive compensation and stock option plan.
• Comprehensive health care (medical, dental, vision).
• Retirement plan (401k, IRA).
• Generous paid time off (vacation, sick leave, holidays).
• Family leave (maternity, paternity).
• Disability coverage.
• Professional development opportunities.
• Flexible work arrangements (remote options).
• Wellness resources.
• Free food and snacks (in the office).

Neural Magic is an equal-opportunity employer committed to fostering a diverse and inclusive workplace. All applicants will be considered for employment without attention to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran, or disability status.