ApplyBlast uses AI to match you with the right jobs, tailor your resume and cover letter, and apply automatically so you can land your dream job faster.

Features

Auto Apply Resume Optimizer Cover Letter Optimizer Interview Prep

Company

Social Media

Need Help?

team@applyblast.com

AI Quality Engineer at Momentive Software - ApplyBlast

AI Quality Engineer

Momentive Software

5 months ago

Remote

Full Time

Senior

0 applicants

View Job Listing

Apply to 100+ jobs

About this role

This role focuses on designing and implementing evaluation frameworks for Large Language Models and agentic AI systems to assess quality, safety, and accuracy. The incumbent will build automated test pipelines, develop regression detection tooling, and collaborate with engineering teams to define quality metrics for AI features. Success involves establishing robust testing standards that enable confident release decisions while identifying edge cases in multi-step agentic workflows.

Responsibilities

Design and implement evaluation frameworks for LLM and agentic AI quality
Build and maintain automated test pipelines for AI features
Develop tooling to detect regressions in model behavior and prompts
Define and track quality metrics including hallucination rates and latency
Collaborate with engineers to identify edge cases in agentic pipelines

Skills

Qualifications

Bachelor's degree in Computer Science or Engineering3-5 years of software or quality engineering experienceExperience with LLMs or agentic AI systemsProficiency in Python for scripting and test automationFamiliarity with agentic frameworks like LangChain or LangGraph