Software Engineer II - AI/ML, AWS Neuron

The Annapurna Labs team at Amazon builds AWS Neuron, the software development kit used to accelerate deep learning and GenAI workloads on Amazon’s custom machine learning accelerators, Inferentia and Trainium.The Basic qualifications: - 3+ years of non-internship professional software development experience - Bachelor's degree in computer science or equivalent - 3+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience - Fundamentals of Machine learning and LLMs, their architecture, training and inference lifecycles along with work experience on optimizations for improving the model execution. - Software development experience in C++, Python (experience in at least one language is required). - Strong understanding of system performance, memory management, and parallel computing principles. - Proficiency in debugging, profiling, and implementing best software engineering practices in large-scale systems. Preferred: - Familiarity with PyTorch, JIT compilation, and AOT tracing. - Familiarity with CUDA kernels or equivalent ML or low-level kernels. - Candidates with performant kernel development such as CUTLASS, FlashInfer etc., would be well suited. - Familiar with syntax and tile-level semantics similar to Triton. - Experience with online/offline inference serving with vLLM, SGLang, TensorRT or similar platforms in production environments. - Deep understanding of computer architecture, operation systems level software and working knowledge of parallel computing. Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status. Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner. The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits . USA, WA, Seattle - 143,700.00 - 194,400.00 USD annually