Senior Software Engineer, AI Eval

As a Senior Software Engineer on Sentry’s AI/ML team, you will build evaluation infrastructure to assess the accuracy and reliability of AI systems, ensuring correct and predictable performance. This role includes designing datasets and benchmarks, creating automated test harnesses, and collaborating with AI engineers and product leaders to establish measurable criteria for AI behavior. With a focus on high-quality evaluation processes, you will oversee the lifecycle of major AI initiatives from experimentation to production monitoring.