ApplyBlast uses AI to match you with the right jobs, tailor your resume and cover letter, and apply automatically so you can land your dream job faster.

Features

Auto Apply Resume Optimizer Cover Letter Optimizer Interview Prep

Company

Social Media

Need Help?

team@applyblast.com

AI Evaluation Engineer at Dialpad - ApplyBlast

AI Evaluation Engineer

Dialpad

2 months ago

4.0 on Glassdoor

Vancouver, British Columbia, Canada

Onsite

Full Time

Medior

0 applicants

View Job Listing

Apply to 100+ jobs

About this role

The AI Evaluation Engineer will join Dialpad's AI Evaluation team to own evaluation coverage for the company's agentic AI systems. The role focuses on designing validation strategies, building regression and A/B evaluations, and co‑owning LLM‑judge metric development and calibration. The engineer will also create and manage annotation pipelines, develop QA tooling, and collaborate with applied science, engineering, and product QA to ensure release‑readiness of voice and chat AI solutions.

Responsibilities

Design and execute validation strategies for agentic AI workflows
Build and improve regression evaluations, A/B comparisons, and red teaming analyses
Co‑own LLM‑judge metric development, calibration, and prompt refinement
Create and monitor data annotation jobs to maintain evaluation datasets
Develop and maintain QA tooling, notebooks, and pipeline components

Skills

Qualifications

Bachelor's degree in Computer ScienceMaster's degree in Computer Science

About Dialpad

dialpad.com

Headquarters

San Ramon, California, United States

Company Size

1001-5000 employees

Founded

2011

View Profile

Dialpad is an AI-powered cloud communications platform that provides business phone systems, contact center, messaging, and video to help teams connect, support, and sell. Its built-in AI—real-time transcription, Voice Intelligence, coaching, and automated call summaries—reduces manual work, surfaces conversational insights, and boosts agent and sales productivity. Cloud-native architecture and integrations with CRMs and collaboration tools make deployment and workflows seamless for distributed teams. Companies use Dialpad to improve customer experience, speed resolution, and gain analytics-driven visibility into conversations.