AI Product Analyst

Salary: £80,000 - 85,000 per year Requirements: • Substantial experience as a product, data, or commercial analyst, having owned end-to-end product measurement. • Strong hands-on SQL and Python skills, or close equivalents, with the ability to construct automated analysis and dashboards independently. • Comfort working with AI or ML systems, with a firm understanding of why measuring probabilistic models differs from measuring deterministic products. • Exceptional stakeholder communication skills, including the ability to draw structured data requirements out of subject-matter experts and hold evidence-based recommendations in front of senior leaders. • Familiarity with advanced AI concepts is desirable, including experience with RAG systems, golden datasets, scoring, regression testing, or A/B experiment design. • Domain exposure is desirable, including prior experience in tax, HR, health and safety, care, or another regulated professional services environment. Responsibilities: • Measure how our AI products perform across retrieval quality, correctness of output, and user engagement. • Own production-quality observability, tracking metrics like thumbs-down rates, regeneration rates, task abandonment, and usage drift. • Build and maintain the daily analysis and dashboards the AI team relies on, working with engineers to instrument and capture missing data points. • Interrogate data using cohort comparison, trend analysis, and segmentation to identify high-material patterns and outliers. • Define what gets tested, analyse evaluation outputs for regressions, and collaborate with engineers to integrate testing harnesses into the build pipeline. • Build, version, and maintain curated sets of questions and known-good answers, running elicitation sessions with subject-matter experts to capture professional edge cases. • Translate complex findings into actionable recommendations for the Product Manager and Director to shape priorities and inform the broader roadmap. Technologies: • AI • Product Manager • Python • RAG • SQL More: Our client is building cutting-edge AI products for customers and colleagues across the group in tax, HR, health and safety, and care. This is a full-time role based in our London office, working alongside a highly technical engineering team to shape what ships and help define the metric framework for a scaling AI portfolio from scratch. The salary range is £80,000 to £85,000 per annum, with potential to increase depending on experience, and the role offers full-time Monday-to-Friday working hours. We work directly with in-house professional experts on high-stakes products where structural accuracy and correctness matter, and this is a strong opportunity for an ambitious analyst who wants to drive evidence-based product decisions. last updated 37 week of 2026