# Member of Technical Staff, Inference Infrastructure

Source: https://realanalystjobs.com/job/member-of-technical-staff-inference-infrastructure-noeri-b6951f40
Member of Technical Staff, Inference Infrastructure at Noeri, an on-site role based in Palo Alto, California, USA. Pay stated in the posting: $200K to $350K. Posted on 2 November 2026, read from Noeri's own careers page (Ashby), which RealAnalystJobs re-checks daily and removes the role when it closes.

Updated 22 November 2026.

## Key facts

- Company: [Noeri](https://realanalystjobs.com/companies/noeri)
- Location: Palo Alto, California, USA
- Work mode: On-site
- Pay stated in the posting: $200K to $350K
- Posted: Nov 2, 2026
- Employment type: Full time
- Source: Noeri's careers page (Ashby), re-checked daily

Skills: Fine-tuning, LLM, CUDA, Kubernetes, [Python](https://realanalystjobs.com/skills/python), Go, C++, Rust

## About this role

About Us
We are a small team of researchers and engineers focused on pushing the frontier of what agents are capable of. We believe the next leap in agent performance will come not from bigger general models, but from specialization and continual learning: agents that get better at their specific job the more they do it. We build the systems that make this possible, turning real-world usage into lasting improvement.

Summary
We are looking for a senior engineer to design and own the systems that serve our models in production. Our customers run many specialized models that are updated often as they learn from real-world use. You will build inference infrastructure that is fast, reliable, and cost-efficient, and that makes it easy to deploy new model versions safely. You will also work closely with the RL team, since fast inference is at the heart of RL training. You will have a large influence on our technical direction.

Responsibilities

• Design, build, and operate our production inference platform for serving many specialized models at scale.

• Improve latency, throughput, and GPU utilization using techniques such as batching, caching, scheduling, quantization, and speculative decoding.

• Build systems to serve many fine-tuned model variants efficiently on shared hardware.

• Build safe, fast rollout systems for new model versions, including canary releases, A/B tests, and quick rollbacks.

• Support deployments in customer cloud environments and VPCs, with strong security and isolation.

• Work with the RL infrastructure team to make inference fast and efficient for rollout generation during training.

• Capture production traces and feedback in a way that can feed back into training.

• Build monitoring, alerting, and on-call practices that keep the platform reliable.

• Evaluate and adopt new models, hardware, and inference engines as they appear.

• Set engineering standards and mentor other engineers as the team grows.

Qualifications

• Experience building or operating ML inference systems in production.

• Deep experience with LLM serving, including KV caching, continuous batching, request scheduling, and parallelism strategies.

• Hands-on experience with inference engines such as vLLM, SGLang, or TensorRT-LLM, ideally including working on their internals.

• Strong Python skills. C++, Rust, or Go is a plus.

• Experience with Kubernetes, autoscaling, load balancing, and running GPU fleets across cloud providers.

• Strong understanding of transformer architectures and what drives inference performance.

• A track record of owning critical production systems, including incident response.

• Comfortable with ambiguity and able to own large problems end to end.

Nice to Have

• Experience building inference infrastructure at a frontier lab or AI infrastructure company.

• Contributions to open-source inference engines.

• Experience with GPU kernel programming (CUDA, Triton) and profiling.

• Experience with multi-LoRA serving, disaggregated prefill and decode, or Mixture of Experts models.

• Experience integrating inference with RL training systems.

Description from Noeri's public careers feed, reproduced so you can read the role here. Apply on the company's own site; Real Analyst Jobs never submits anything for you.

## How to apply

Open [this role on Real Analyst Jobs](https://realanalystjobs.com/job/member-of-technical-staff-inference-infrastructure-noeri-b6951f40) and use Apply now there to reach the employer's application. The listing is re-checked against Noeri's careers page daily and removed when it closes.

## Noeri

Analyst roles on Real Analyst Jobs. [More roles at Noeri](https://realanalystjobs.com/companies/noeri)

## More like this

- [Data analyst jobs in the USA](https://realanalystjobs.com/jobs/data-analyst-jobs-in-usa)
- [Analyst job market report](https://realanalystjobs.com/analyst-job-market)
- [What analyst roles pay](https://realanalystjobs.com/blog/data-analyst-salaries)

## Sources and links

- [This role on Real Analyst Jobs](https://realanalystjobs.com/job/member-of-technical-staff-inference-infrastructure-noeri-b6951f40)
- [Noeri on Real Analyst Jobs](https://realanalystjobs.com/companies/noeri)
- [All analyst jobs](https://realanalystjobs.com/jobs)
