AI Language Engineer II, Alexa for Shopping Lang-Tech

The Conversational Shopping team is looking for an AI Language Engineer to drive innovation and scalable solutions as it delivers a delightful AI-assisted shopping experience. This role supports Alexa for Shopping, helping customers find and discover the best products through recommendations, comparisons, product Q&A, and more. The role is cross-functional and requires collaboration across global product, design, science, and engineering teams.

The candidate should be passionate about the intersection of language and technology and able to develop automated, scalable solutions in the Large Language Model (LLM) space. Responsibilities include applying expertise in LLMs, coding, and natural language to address challenges in model evaluation, automation, and context engineering for agentic systems. The role contributes to an evaluation-driven product development strategy and works with Product Managers, Applied Scientists, Software Engineers, UX Researchers, and Editors to drive quality, speed, and consistency in AI-driven shopping experiences across mobile and web apps.

Key responsibilities and activities focus on authoring, optimizing, and managing system prompts for customer-facing AI-driven experiences, defining requirements for internal tooling via prototypes, evaluating model performance, and producing actionable insights to inform product decisions. The role also involves creating and standardizing editorial workflows to meet quality standards, and supporting onboarding and upskilling of Editors and AI Tutors.

Team context: The CMX-Lang-Tech team is a technical sub-team of the AI Shopping Content team responsible for AI response quality in evaluating and prompting AI models.

• Develop LLM-as-a-judge systems to support human-in-the-loop evaluations
• Automate operations and perform data analysis using scripting languages (e.g., Python)
• Author, optimize, and manage system prompts for customer-facing LLM systems
• Integrate API calls into Retrieval Augmented Generation (RAG) systems
• Evaluate model performance and annotation quality to produce reports for stakeholders
• Produce, process, and manipulate different types of language data
• Contribute to defining platform requirements for internal tooling by developing prototypes
• Raise the quality bar on editorial workflows and SOPs through standardization, documentation, and periodic audits
• Support processes to onboard and upskill Editors and AI Tutors
• Design, implement, and refine control mechanisms, metrics, and methodologies to ensure editorial and annotation quality
• Collaborate with editors, applied scientists, engineers, and product managers to deliver an optimal customer experience by defining metrics, guidelines, and workflows
• Deliver across parallel workstreams, balancing timelines, impact, and stakeholder requirements

The CMX-Lang-Tech team is a technical sub-team of the AI Shopping Content team, responsible for AI response quality in evaluation and prompting of AI models.

• Experience with Unix tools
• Strong analytical skills, attention to detail, and effective communication abilities
• Experience in a fast-paced, dynamic organization
• Experience prioritizing and handling multiple assignments while meeting deadlines
• Master's Degree in Applied Linguistics, Computational Linguistics, Natural Language Processing (NLP), or related field
• Experience with Large Language Models, NLP, or Machine Learning
• Experience with Python libraries for data analysis (e.g., pandas, scikit-learn)
• Familiarity with AI coding assistants

• PhD in Applied Linguistics, Computational Linguistics, NLP, or related field
• Experience with SQL and Git
• Experience building RAG or agentic systems
• Experience conducting quantitative analysis
• Experience building data pipelines
• Experience with AWS services (Bedrock, S3, EC2, etc.)
• Knowledge of user experience concepts and methods
• Familiarity with online retail (e-commerce)

Amazon is an equal opportunities employer. We value diversity and hire based on experience and skills. We are committed to protecting privacy and the security of data. Please review our privacy practices to understand how we collect, use, and transfer personal data of candidates. We do not discriminate on protected veteran status, disability, or other legally protected characteristics. If you require a workplace accommodation during the application and hiring process, please notify us for support.