Senior Machine Learning Engineer, Inference

Accountabilities: You will lead end-to-end inference optimization, ensuring that advanced machine learning models perform efficiently and reliably in production environments. Your responsibilities will include: Own the optimization process from research models to production deployment. Improve latency and throughput for real-time voice and LLM inference at consumer scale. Work directly with inference engines such as vLLM, TensorRT-LLM (TRT-LLM), and SGLang to enhance performance. Implement model acceleration techniques, including quantization, distillation, speculative decoding, caching, continuous batching, and paged attention. Develop high-performance GPU workloads using C++, CUDA, Rust, or optimized Python. Build and scale distributed multi-GPU and multi-node inference systems using Kubernetes, Ray, and custom load-balancing solutions. Collaborate with research and product teams to bring new models into production. Requirements The ideal candidate is a senior-level machine learning inference or systems engineer with strong hands-on experience optimizing production inference engines and building scalable GPU infrastructure. You should bring: At least 5 years of experience in ML inference or systems engineering, at a senior level equivalent to Google L5 or Meta E5. Practical experience optimizing inference engines such as vLLM or TRT-LLM at the engine level, beyond basic model deployment. A proven background in developing backend and machine learning systems and serving models in production. Strong systems programming and GPU optimization skills using C++, CUDA, Rust, or optimized Python. Hands-on knowledge of model acceleration techniques, including quantization, distillation, speculative decoding, caching, continuous batching, and paged attention. Experience with distributed systems, Kubernetes, Ray, load balancing, multi-GPU and multi-node inference, and high-concurrency environments. Excellent English communication skills. Residence in the United States, Canada, the United Kingdom, Germany, Switzerland, or Serbia. Additional advantages include a track record of rapid career progression, contributions to open-source inference engines or technical publications, and a PhD in Computer Science, Physics, or Mathematics, or a degree from a leading computer science program. The ability to commute to Mountain View, California, is also a plus. Benefits Salary: $250,000–$300,000 annual base salary, plus competitive equity. Employment: Full-time, permanent position. Work arrangement: Onsite in Mountain View, California, preferred; remote opportunities available in the United States, Canada, the United Kingdom, Germany, Switzerland, and Serbia. Visa support: H-1B and OPT transfers supported. Relocation assistance: Available for candidates relocating to the San Francisco Bay Area. Language: English-speaking working environment.