Observability & Evaluation Engineer

The Observability & Evaluation Engineer role focuses on building telemetry, tracing, dashboards, and evaluation suites to ensure production AI systems are monitored, evaluated, and supported with clear operational visibility. This position involves implementing observability for LLM-powered applications, developing automated evidence collection for release readiness, and creating runbooks to support priority agent releases. The role requires collaboration with platform engineers and Product Owners to define monitoring requirements and analyze production behavior for continuous improvement.