# Product Engineer, Agent Experience

Source: https://realanalystjobs.com/job/product-engineer-agent-experience-avante-ec3e10d8
Product Engineer, Agent Experience at Avante, an on-site role based in Seattle, Washington, USA. Posted on 11 November 2026, read from Avante's own careers page (Greenhouse), which RealAnalystJobs re-checks daily and removes the role when it closes.

Updated 22 November 2026.

## Key facts

- Company: [Avante](https://realanalystjobs.com/companies/avante)
- Location: Seattle, Washington, USA
- Work mode: On-site
- Posted: Nov 11, 2026
- Employment type: Full time
- Source: Avante's careers page (Greenhouse), re-checked daily

Skills: Agents, Evals, LLM, TypeScript, Customer-facing, [SQL](https://realanalystjobs.com/skills/sql)

## About this role

Avante is the enterprise AI platform for benefits teams. Benefits is one of the most complex, high-stakes functions in any company, and Avante’s purpose-built agents take on end-to-end benefits work, from checking vendor contracts against claims to preparing leadership reports to helping employees use and understand their benefits. Samsung Semiconductor, Real Chemistry, Zscaler and News Corp use Avante today to keep costs in check, support employees across the globe, and get more from their benefits investment.

The Role

We are hiring a Product Engineer to own the quality of our Agents end to end. Carly delivers personalized benefits guidance to employees, grounded in plan designs, coverage rules, and each employee's context. Ava is the agentic intelligence that benefits teams use to run their benefits operations. In this role, you will review evals, collect qualitative and quantitative feedback, define what a high-quality outcome looks like, ship improvements, and measure their impact.

The role combines product judgment with hands-on implementation. You will own decisions such as whether a problem calls for a prompt change, a new skill, or a code change; whether an evaluation is rigorous enough to trust; and whether a new model is ready for production. You will then implement those changes yourself across evaluations, prompts, skills, model configuration, and our TypeScript codebase, using AI development tools as a core part of your workflow, and partner with our agent engineers on deeper changes to retrieval, tooling, or the platform.

You will join our Agent Experience team, embedded with agent engineering and working closely with product leadership, design, and Customer Success. We welcome candidates from either direction: product-minded engineers, and engineering-minded product managers who are already building. We place more weight on what you have shipped and how quickly you learn than on years of experience, and we are open to candidates early in their careers who show high agency, curiosity, and sound judgment.

Success in this role means evaluation results consistently translate into shipped improvements, model and prompt changes are validated against representative customer tasks, recurring failures become regression cases, and the team can clearly explain how each change affects usefulness, reliability, latency, and cost.

What You'll Do

• Define what good looks like for our agents. Review production evals, customer feedback, usage patterns, and evaluation results. Find where an answer or action falls short, including cases where it's factually correct but still doesn't help the person asking.

• Build evaluations the team can trust. Turn customer needs into representative scenarios, clear rubrics, and regression checks. Test whether automated scores agree with human judgment, and use the results to decide what to fix first.

• Ship improvements yourself. Update prompts, skills, and agent behavior. Build prototypes and targeted product changes, and contribute reviewed code. Partner with engineers when the problem sits in retrieval, tools, or the underlying platform.

• Make the call on new models. Compare models against real tasks for quality, safety, latency, and cost. Decide with engineering whether we upgrade, run controlled rollouts, inspect production results, and follow through on regressions.

• Own the feedback loop with customers. Work with CX and product teammates to understand what people are trying to get done. Turn recurring friction into product changes, then check whether those changes improve task completion, engagement, and trust.

• Use AI tools as a core part of how you work. Build with AI coding tools and agents every day, and be ready to explain how you checked what they produced.

• Keep the team current. Try new AI tools and approaches, form a view on where they help, and share what you learn. Turn the experiments that work into changes that hold up in real use, with clear evidence and appropriate guardrails.

What We're Looking For

• Experience shipping digital products in product management, product engineering, software engineering, or a similarly hands-on role. You can point to customer outcomes you personally improved.

• Strong product judgment. You can look at an agent interaction, explain what the user needed, and tell a useful result from one that only looks good in a demo or scores well.

• Active, practical use of AI coding tools and agents. You've built prototypes, tools, or product changes with them and can explain how you checked their behavior and handled mistakes.

• Enough coding fluency to read an unfamiliar codebase, make targeted changes, use version control, and contribute reviewed code. You're comfortable learning our TypeScript stack.

• Comfort with data and experiments: querying or scripting an analysis, defining metrics that mean something, and recognizing when a sample or automated evaluation gives false confidence

• Systems thinking and customer empathy. You spot where a skill, an automation, or a workflow change would remove friction, and you can talk with customers about it.

• High agency in an ambiguous environment. You pick a useful next step, ship, learn quickly, and communicate clearly with engineers, designers, and customer-facing teams.

Nice to Have

• Experience with LLM evaluations, prompt and context design, or agent tools and orchestration

• Running controlled experiments on a live product

• Working knowledge of SQL and TypeScript

• Familiarity with benefits, healthcare, HR technology, or enterprise software

What to Expect from Us

Interview Process:

• Initial screen with our recruiters

• Video Interview with our Engineering Manager

• Technical interview focused on real-world problem solving (no whiteboard algorithms)

• On-site interview loop:

• Team behavioral interview (60 min)

• Session with our Engineering Manager (60 min)

• Final conversation with our CTO (30 min)

The base salary for this r

Description from Avante's public careers feed, reproduced so you can read the role here. Apply on the company's own site; Real Analyst Jobs never submits anything for you.

## How to apply

Open [this role on Real Analyst Jobs](https://realanalystjobs.com/job/product-engineer-agent-experience-avante-ec3e10d8) and use Apply now there to reach the employer's application. The listing is re-checked against Avante's careers page daily and removed when it closes.

## Avante

Analyst roles on Real Analyst Jobs. [More roles at Avante](https://realanalystjobs.com/companies/avante)

## More like this

- [Data analyst jobs in the USA](https://realanalystjobs.com/jobs/data-analyst-jobs-in-usa)
- [Analyst job market report](https://realanalystjobs.com/analyst-job-market)
- [What analyst roles pay](https://realanalystjobs.com/blog/data-analyst-salaries)

## Sources and links

- [This role on Real Analyst Jobs](https://realanalystjobs.com/job/product-engineer-agent-experience-avante-ec3e10d8)
- [Avante on Real Analyst Jobs](https://realanalystjobs.com/companies/avante)
- [All analyst jobs](https://realanalystjobs.com/jobs)
