Software Engineer, Inference

About the role Our Inference team creates the service that generates outputs from our models in production. This service is primary in driving our efficiency, latency, and reliability. As an engineer on this team, you'll work to enhance these metrics by tackling complex distributed-systems problems across all layers of our stack. About Anthropic Anthropic aims to design reliable, interpretable, and manageable AI systems. We strive for AI to be safe and beneficial for our users and society as a whole. Our team is rapidly expanding with devoted researchers, engineers, policy experts, and business leaders working together to create useful AI systems. You may be a suitable candidate if you: - Have substantial software engineering experience - Are results-driven, with a bias towards flexibility and impact - Are willing to pick up slack, even if it goes beyond your job description - Enjoy pair programming (we embrace it!) - Are eager to learn more about machine learning research - Care about the societal impacts of your work Strong candidates may also have experience with: - High performance, large-scale distributed systems - Kubernetes - Python - Machine learning Representative projects: - Improving how inference requests are routed to model servers to maximize compute efficiency - Building a performance model to predict the impact of future architecture and hardware enhancements - Implementing inference for a new model architecture - Utilizing observability data to adjust performance based on production workloads - Implementing inference on a new hardware platform - Creating instruments to identify and eliminate Python GIL contention - Increasing the efficiency of our GPU kernels - Preparing our inference engine for deployment Annual salary (USD) - The predicted salary range for this position is $250k - $375k. Logistics Location-based hybrid policy: At present, we expect all staff to be in our office at least 25% of the time. Deadline to apply: None. Applications will be reviewed on a rolling basis. US visa sponsorship: We are a sponsor for visas! However, we aren't able to successfully sponsor visas for every role and candidate; operations roles are particularly difficult to support. But if we make you an offer, we will do everything possible to bring you to the United States, and we retain an immigration lawyer to assist with this. We urge you to apply even if you don't believe you meet every single qualification. Not all strong candidates will fulfill every single qualification as listed. Research demonstrates that individuals who identify as being from underrepresented groups are more prone to experiencing imposter syndrome and doubting the strength of their candidacy. We urge you not to rule yourself out prematurely and to submit an application if you're interested in this work. The AI systems that we are building pose significant social and ethical implications. This means representation is even more important, and we strive for a diverse range of perspectives in our team. Compensation and Benefits Anthropic’s compensation package comprises salary, equity, and benefits. We commit to pay fairness and aim for these three elements to collectively be highly competitive with market rates. Equity - On top of this position's salary (listed above), equity will be a substantial component of the total compensation. We aim to provide higher-than-average equity compensation for a company of our size, and reveal equity amounts at the time of offer issuance. Benefits - Benefits we provide include: - Optional equity donation matching at a 3:1 ratio, up to 50% of your equity grant. - Comprehensive health, dental, and vision insurance for you and all your dependents. - 401(k) plan with 4% matching. - 21 weeks of paid parental leave. - Unlimited PTO – most staff take between 4-6 weeks each year, sometimes more! - Allocations for education, home office enhancements, commuting, and wellness. - Fertility benefits via Carrot. - Daily meals and snacks in our office. - Support for those relocating to the Bay Area. Compensation and benefits info is based on Anthropic’s good faith estimate for this position in San Francisco, CA, as of the date of publication and may change in future. The level of pay will depend on various job- related factors, including previous work experience, relevant education, and performance on our interviews or during a work trial. How we're different We anticipate that the most impactful AI research will be big science. At Anthropic, we collaborate as a single unit on extensive research efforts. We value impact — promoting our long-term goals of adaptable, reliable AI — over working on smaller and more defined puzzles. We see AI research as an empirical science, containing elements of physics and biology, as well as traditional computer science methods. We're a highly collaborative group, conducting frequent research discussions to ensure our work is high-impact. Communication skills are essential. We attempt to accumulate knowledge and intuition about the most effective AI innovations. Therefore, well-documented null results are nearly as valuable as successful discoveries. We don't differentiate between engineering and research, expecting all our technical staff to contribute as needed. To understand our research, read our recent research. This research continues many directions our team pursued before Anthropic, including: GPT-3, Circuit-Based Interpretability, and a variety of other projects.